[HN Gopher] GradIEEEnt half decent
___________________________________________________________________
GradIEEEnt half decent
Author : notmysql_
Score : 245 points
Date : 2023-05-01 23:42 UTC (23 hours ago)
(HTM) web link (tom7.org)
(TXT) w3m dump (tom7.org)
| dirtyid wrote:
| two tom7 videos in as many months, what a treat.
| ironbound wrote:
| Best start to a video this year, really wish more people would
| use a hook to get people engaged in learning.
| sva_ wrote:
| > I think this is a fractal in the sense that it is chaotic, has
| a color gradient, and could be on the cover of an electronic
| music album
|
| I didn't know of this criterion
| pjc50 wrote:
| It's on my album "Hausdorff-Besicovitch Criterion"
| eur0pa wrote:
| Two videos so far this year and we're still in May. What a bliss.
| sethaurus wrote:
| Tom7 our beloved. How might we make this creator publish more
| videos? Money? Is it money? Can we give him money somehow?
| UncleMeat wrote:
| Tom has a pretty good day job, so I suspect that money isn't
| the thing.
|
| The sigbovik conference seems to be the motivating goal
| behind these videos. I think the key thing would be to make
| sure that sigbovik continues and that people are excited
| about the papers.
| shadowgovt wrote:
| Unfortunately, Tom's a lot more creativity and concept-driven
| than money driven. So the best strategy is probably to spend
| the money on some kind of near-C sublight vehicle and take a
| one-hour jaunt in it, then return to Earth in a year and see
| what's tickled Tom's fancy since you last saw him.
|
| this solution may have side-effects that are left as an
| exercise for the Rocketeer.
| tysam_and wrote:
| I believe that this floating point imprecision was something that
| David Page used (I think it may have been accidental as he was
| originally doing something else with it?) to achieve a world
| record training speed on CIFAR10 by summing a bunch of loss
| values together instead of taking the average. The sum
| effectively reduced the precision of the loss value and seemed to
| have a regularizing impact on network training as best as I
| personally understand. :) <3
| chrisldgk wrote:
| Always delighted to see tom7 content. Dude just thinks out of the
| box like no one else.
| newswasboring wrote:
| It has been a long standing dream to make videos like this. I
| have no idea where to start though. I think everyone has these
| out of the box ideas, but we (or at least I) disregard them as
| "who will be interested in this".
| airstrike wrote:
| In addition to picking a subject that interests people, it
| helps to be good at storytelling. Fortunately, that skill can
| also be learned
| physicles wrote:
| His videos also cover a super wide range of content areas. Just
| this latest one has a refresher on neural nets, ieee floating
| point (and explained the accuracy tradeoffs more clearly than
| I've ever seen), fractals, chess engines, and more.
| newswasboring wrote:
| Not just that, he is also very good at making videos. His
| story lines are good and everything flows like a deep river.
| It's mesmerizing to look at.
| airstrike wrote:
| and he's also great at naming things, at least based on
| this one submission
| fugalfervor wrote:
| And running the length of every street in Pittsburgh, of all
| things
| x86x87 wrote:
| Not only that but they are highly entertaining.
|
| This one is probably my all time favorite:
| https://youtu.be/LA_DrBwkiJA
| waynecochran wrote:
| Dude is way out of the box. Using IEEE floating point rounding
| error as his neural network transfer / activation function --
| never thought to go there.
| unlikelymordant wrote:
| Reminds of some earlier openai work
| https://openai.com/research/nonlinear-computation-in-deep-li...
|
| "Neural networks consist of stacks of a linear layer followed by
| a nonlinearity like tanh or rectified linear unit. Without the
| nonlinearity, consecutive linear layers would be in theory
| mathematically equivalent to a single linear layer. So it's a
| surprise that floating point arithmetic is nonlinear enough to
| yield trainable deep networks."
| [deleted]
| alexb_ wrote:
| This is one of the few tom7 videos that I see and am utterly
| confused by in every way. It feels like he's speaking a different
| language than I am.
| shadowgovt wrote:
| I lucked out a bit in having only recently learned about the
| need for a nonlinear transfer function (although it was taught
| to me as a "That's just how we do it," not with the associated
| underlying explanation that you can't represent all functions
| with only linear functions).
| VikingCoder wrote:
| Tom7 is kind of like the electronic version of the Primitive
| Technology Youtuber.
|
| It's fascinating watching someone using some of the worst tools
| ever to make something in about the most labor-intensive way
| imaginable - and it's just beautiful. It's practically
| meditative.
| hoosieree wrote:
| I know what you mean by Primitive Technology Youtuber, but I
| would encourage you to not dismiss the underlying concepts out
| of hand. Building a chair from green wood with hand tools may
| not work for Ikea, but it is much more energy-efficient than
| power tools [1].
|
| There's also something incredibly empowering about working with
| tools you understand, both in software and woodworking.
|
| [1]:
| https://www.mortiseandtenonmag.com/blogs/blog/podcast-12-the...
| VikingCoder wrote:
| Oh absolutely. I'm going to confess that I read this book,
| "Replay" by Ken Grimwood, which is roughly about a guy who
| keeps having a heart attack on the same day, and wakes up in
| his younger body, with all of his memories. Again and again
| and again. Kind of like Groundhog Day, but for an entire
| adult life. I kind of became obsessed for a while with the
| idea that I would like to live my life over again. It was a
| really unhealthy obsession. It took a while for me to live in
| the moment, try to not hold on to my regrets, etc.
|
| Back to the topic... I kind of watch things like Primitive
| Technology out of a hope/fear that I'll either get thrown
| back in time to when these skills were relevant, or that I'll
| live in a post-apocalyptic world where they're relevant
| again. (Which was probably inspired by reading Stephen King's
| "The Stand," and worrying about how my life would be if there
| was no electricity.)
|
| But yeah, I love watching people show their enthusiasm... I
| love watching people demonstrate mastery over tools I don't
| know how to use... I love watching someone build something...
| Tom7 and Primitive Technology check all those boxes. Adam
| Savage's Tested often does, too.
| wyager wrote:
| > but it is much more energy-efficient than power tools
|
| Only if you forget to amortize the cost of supporting a human
| life for the extra marginal hours it takes
| yetanotherloser wrote:
| sure, let's all maximise the time we have available to
| spend on a couch consuming mindless tv.
|
| The "but it takes tiiiiime" argument against making your
| own things (and achieving a kind of satisfaction, control
| and learning that's hard to beat) is simply dreadful.
| VikingCoder wrote:
| It's nice to have the luxury of spare time. Many people
| don't. Or they have to budget it extremely tightly.
| wyager wrote:
| Don't project your use of spare time on everyone else.
| Some of us use spare time towards productive ends. I
| would personally prefer not to spend all my time living
| like a subsistence farmer from 1200 years ago.
| kelnos wrote:
| Who are you to decide what people do and don't want to do
| with their spare time? Some people don't achieve any
| satisfaction from making their own things; they just see
| it as a chore. Your disappointment in that is just your
| subjective opinion, not any reflection of objective
| truth.
|
| You also seem to think that there's nothing "productive"
| anyone could do with the time they save from not having
| to make their own things, which seems obviously false.
___________________________________________________________________
(page generated 2023-05-02 23:03 UTC)