[HN Gopher] The Hardware Lottery (2020)
___________________________________________________________________
The Hardware Lottery (2020)
Author : ahamez
Score : 30 points
Date : 2023-11-22 10:06 UTC (12 hours ago)
(HTM) web link (hardwarelottery.github.io)
(TXT) w3m dump (hardwarelottery.github.io)
| winterismute wrote:
| I am looking at solving this challenge in a specific way: using
| high-perf, GPU-accelerated HW simulators and ML algorithms to
| tune a new HW architecture automatically. Best ML HW => run on it
| the best ML models => produce new best HW (arch) => build new
| best HW => GOTO 10.
|
| Reach out if you are interested in any way.
| heyitsguay wrote:
| How would you define "new best HW"? Seems challenging,
| particularly if it's for generation N+1 ML models that haven't
| been created yet. Also, while there is work underway to use ML
| models to guide HW design, it's not clear to me that the best
| ML models for that == the best ML models for more general
| purpose tasks in audio, visual, and natural language
| processing, i.e. is HW circuit design done using transformers?
| What are the inputs, latent space, and outputs?
|
| GPU development seems driven by more general computational
| principles that might be summed up glibly as "We're hitting
| fundamental physics limits for single-core processors, what is
| the maximum amount of data we can move per second through the
| maximum number of cores?" Perhaps there is a way to extend this
| with a view toward the challenges of current ML model training
| and inference, but I imagine trying to approach it as a black-
| box optimization problem could be quite difficult.
| shae wrote:
| I wish for graph reduction hardware, we could build amazing
| things with that.
| LargoLasskhyfv wrote:
| https://www.cs.york.ac.uk/fp/reduceron/
|
| https://www.cambridge.org/core/journals/journal-of-functiona...
|
| https://github.com/tommythorn/Reduceron
|
| also: https://en.wikipedia.org/wiki/Graph_reduction_machine
| voiceblue wrote:
| Related: The Bitter Lesson [0], and previous discussion [1].
|
| [0] http://www.incompleteideas.net/IncIdeas/BitterLesson.html
|
| [1] https://news.ycombinator.com/item?id=19393432
| dang wrote:
| Thanks! Macroexpanded:
|
| _The Bitter Lesson: General Computation Methods Are the Most
| Effective_ - https://news.ycombinator.com/item?id=37893468 -
| Oct 2023 (1 comment)
|
| _The Bitter Lesson (2019)_ -
| https://news.ycombinator.com/item?id=37139223 - Aug 2023 (1
| comment)
|
| _The Bitter Lesson (2019) [pdf]_ -
| https://news.ycombinator.com/item?id=37129921 - Aug 2023 (60
| comments)
|
| _The Bitter Lesson - Rich Sutton (2019)_ -
| https://news.ycombinator.com/item?id=36017857 - May 2023 (8
| comments)
|
| _The Bitter Lesson (2019)_ -
| https://news.ycombinator.com/item?id=34006388 - Dec 2022 (33
| comments)
|
| _The Bitter Lesson (2019)_ -
| https://news.ycombinator.com/item?id=30889873 - April 2022 (37
| comments)
|
| _The Bitter Lesson (2019)_ -
| https://news.ycombinator.com/item?id=23781400 - July 2020 (85
| comments)
|
| _The Bitter Lesson_ -
| https://news.ycombinator.com/item?id=19393432 - March 2019 (53
| comments)
| dang wrote:
| Related:
|
| _The Hardware Lottery_ -
| https://news.ycombinator.com/item?id=29409415 - Dec 2021 (1
| comment)
|
| _Ride the Hardware Lottery_ -
| https://news.ycombinator.com/item?id=25043592 - Nov 2020 (4
| comments)
|
| _The Hardware Lottery_ -
| https://news.ycombinator.com/item?id=24515035 - Sept 2020 (16
| comments)
___________________________________________________________________
(page generated 2023-11-22 23:01 UTC)