[HN Gopher] The Little Book of Deep Learning [pdf]
___________________________________________________________________
The Little Book of Deep Learning [pdf]
Author : dataminer
Score : 463 points
Date : 2023-05-01 00:18 UTC (22 hours ago)
(HTM) web link (fleuret.org)
(TXT) w3m dump (fleuret.org)
| sreeramvenkat wrote:
| Any recommendations for a set of hands on exercises that go well
| with this book ?
| _giorgio wrote:
| Probably his course, all in pytorch.
|
| https://fleuret.org/dlc/
| przem8k wrote:
| Great writing! I love how this book gets to the point right from
| the start, answering one of my questions about deep learning (how
| did this start?) in the very first sentence:
|
| "The current period of progress in artificial intelligence was
| triggered when Krizhevsky et al.[2012] showed that an artificial
| neural network (..) could beat complex state-of-the-art image
| recognition methods by a huge margin (..)"
| kristopolous wrote:
| Usually I'm a harsh critic of text like this because casual
| language and what look like casual words but are actually
| strictly defined domain specific technical definitions are
| utterly indistinguishable.
|
| However! This book back-references its appendix with grey
| underlines thus signaling that something has a technical
| definition and is jargon.
|
| For instance, "loss" is really common English. In ML it's a loss
| function. You can clearly see it's a special word in this world
| in the text. Other examples include weights, capacity and
| channel.
|
| I don't have to sit there confused trying to guess which English
| words are being used in special ways.
|
| This discipline is fantastic. In some texts, such terms might be
| italicized but that behavior has seem to fallen out of practice.
|
| As someone who isn't a professional mathematician, hints like
| these help greatly.
| tpoacher wrote:
| You say this about exactness, but my experience with much of
| science and ML in particular is the opposite.
|
| People still haven't decided what the difference between AI and
| ML is, for instance, but somehow still insist to treat the two
| as obviously separate in conversations.
|
| My experience has been that the naming problem is alive and
| well in the sciences; and it'sfar more problematic than in
| programming and variable naming.
| bjornasm wrote:
| >People still haven't decided what the difference between AI
| and ML is, for instance, but somehow still insist to treat
| the two as obviously separate in conversations.
|
| Strange, in most of the papers and the books about this they
| define AI and ML and their relation.
| tpoacher wrote:
| Yes, but the definitions tend not to be consistent.
|
| The best operational definition I got to was, it's ML when
| it involves a "machine" (possibly defined in software) that
| tries to 'optimise' some objective on behalf of a user;
| whereas AI involves some sort of Agent, who needs to
| demonstrate intelligence, that involve particular
| environments / states contexts, etc.
|
| But, not everyone defines it like this. To many people AI
| is rapidly becoming "Neural Networks, particularly Deep
| ones", with ML becoming "Anything that is in scikit learn
| that isn't a neural network". Which isn't really a useful
| definition if you ask me.
| mdp2021 wrote:
| > _AI and ML ... still insist to treat the two as obviously
| separate_
|
| Automated problem solving does not imply "learning". E.g.
| clustering does not. Also, expert systems are pretty static,
| and it is not really the "machine" that learns (a flowchart
| can hardly be said to "learn").
|
| > _decided_
|
| It is not a matter of deciding (" _de-cidere_ " (cutting) and
| fuzziness do not really marry), it is just looking at the
| terms with some historic awareness.
|
| The "brain" works through fuzzy patterns, and of course the
| concepts somehow relevant to said model also do. You don't
| cut over a blurry line, but separate points in space remain
| distant.
| rsfern wrote:
| The way you present the distinction is really clear, but in
| the physical sciences it's very common to use the terms AI
| and ML interchangeably, and I think it's partly because
| people in those fields are not quite sure of the
| distinction between the two terms
| woodson wrote:
| It's ML when you're doing research and AI when you ask for
| funding ;).
| rwoerz wrote:
| I fully agree with your general critique. It is a common bad
| habit in science to use common words with an uncommon meaning
| without any explanation. I remember me sitting in a lecture
| about cryptology and wondering how that "I can prove to know a
| SECRET without revealing it" is supposed to work. I just did
| not realize that SECRET just meant "random-looking string"
| instead of something like "I know where the money is hidden".
| chaxor wrote:
| If it's a map to the hidden treasure (as in the exact bits of
| the image of the map) then it's kind of the same thing right?
| But yes, the non-uniqueness of expressing similar ideas can
| make things difficult there.
| abricq wrote:
| I had a course named `Deep Learning` by Francois Fleure at EPFL,
| and he is a really an amazing professor, able to share his
| passion alongside explaining very advanced topics. It does not
| surprise me to see that his books are of very high quality !
|
| The course was focusing on all of the mathematical aspects of
| Deep Learning, starting from the simple understanding of the
| gradient descent algorithm to (trying to) understand how
| transformers work. There was also quite a lot of computer science
| involved and lots of practical assignments. One of the 2 projects
| of this course was to design from scratch in Python or C++ a DNN
| framework (roughly an API like pytorch or tensorflow) which
| required to really think properly about which architecture to use
| for your code. The minimum requirements only asked to implements
| a few activation, normal layers and convolutional layers but you
| go beyond that and implements all kind of layers. Lots of fun.
| This course remains as one of my favorite courses.
| cinntaile wrote:
| He runs a very similar course at the University of Geneva (the
| practicals are available). https://fleuret.org/dlc/
| mdp2021 wrote:
| > _practicals are available_
|
| Also the videos (slides + lesson). Partytime! ;)
| sidcool wrote:
| For my level of knowledge, this is a bit advanced book. Anything
| more basic?
| Version467 wrote:
| You haven't specified in which way this is too advanced for
| you.
|
| Still, I'll take a stab at recommending some other resources.
|
| 1: Practical Deep Learning by fast.ai - a (free) hands on
| course that's designed to get you to making something useful as
| quickly as possible.
|
| 2: Neural Networks Zero to Hero from Andrej Karpathy - A Series
| of Youtube Lectures that requires only basic python knowledge
| and takes you all the way up to a simplified implementation of
| the tech inside gpt models.
|
| 3: Neural Networks from Scratch by Sentdex. A book that's
| explicitly written to teach you the basics. Doesn't cover stuff
| like Transformers and other advanced concepts, but _really_
| takes its time with the basics.
|
| Take a look at all three of them. They all have a different
| teaching style and I'd guess that at least one of them will gel
| with you.
| sidcool wrote:
| This helps. Thanks.
| totetsu wrote:
| Careful not to swallow this.
| abhayhegde wrote:
| What a neat little book! I suppose this was typeset in TeX. How
| did they optimize for smaller width though?
|
| Edit: I found the style templates on author's webpage [1].
|
| [1]: https://fleuret.org/cgi-
| bin/gitweb/gitweb.cgi?p=littlebook.g...
| hota_mazi wrote:
| I know La/TeX is the standard for this but I'm still irritated
| that it forces all diagrams to appear at the top of the page,
| which really gets in the way of explaining things clearly.
|
| This books suffers a lot from this limitation, with figures
| sometimes appearing 5 pages before they are actually referenced
| in the text.
| abhayhegde wrote:
| I am not sure if that is entirely correct. For e.g., see
| p.no. 14-15 where the diagrams appear at the bottom of the
| page.
|
| But, I agree that usually it is not easy to place images
| where you want in LaTeX.
| abhayhegde wrote:
| If you have texlive-extra installed, this mobile-friendly PDF
| can be converted to a desktop-readable format using:
| pdfxup -o lbdla4.pdf -ow -im 10 -m 40 -is 0 -fw 0 --portrait
| -nup 2x2 lbdl.pdf
| lamontcg wrote:
| Are there arguments to that to tile it by columns first
| instead of rows? (but not one long column all the way down
| the left to the end and then one long column on the right,
| just want to shuffle the order on each page).
|
| It reads kind of r/nosafetysmokingfirst to me.
|
| (And its `brew install texlive` on mac to get pdfxup for
| anyone else wondering)
| stevesimmons wrote:
| His website also has a version for printing out on 36 sheets of
| paper and folding into a real book:
|
| https://fleuret.org/public/lbdl-a5-booklet.pdf
| ksd482 wrote:
| By "little" I thought it would be brief, which I think it sort of
| is. But what it really means by little is that it is optimized
| for little screens such as a smartphone.
|
| Go ahead and open it in your phone. You'll be delighted to read
| it.
|
| Question for HN: how can I convert my existing PDFs and eBooks
| that I can easily read from my phone?
|
| For e.g., I have a lot of Math textbooks in PDF format and I
| would like to convert them into a format similar to this deep
| learning book. How can I go about doing that?
| wodenokoto wrote:
| Your comment prompted me to look around. There is also an A5
| version of the book
|
| https://fleuret.org/public/lbdl-a5-booklet.pdf
|
| Which again, is a small paper size!
| driscoll42 wrote:
| Good find! Though the inversion of every other page makes it
| difficult to read unless printing out
| MacTea wrote:
| Reads great on Kindle too!
|
| To answer your question: Calibre might be able to help you out!
| sthatipamala wrote:
| Look into k2pdfopt. The term you're looking for is you want to
| "reflow" your PDFs
| lnyan wrote:
| https://github.com/koreader/koreader
|
| koreader has a pdf reflow mode
| DeusCodex wrote:
| This is such a amazing little book. I love it, AI is hard for me
| to understand but this book makes it very easy to grasp some of
| the concepts.
|
| Thanks for sharing
| mdp2021 wrote:
| > _AI is hard for me to understand_
|
| Periodic reminder that the lessons (of classical AI - the base)
| of the late Prof. Patrick Winston, MIT, are freely available
| (at MIT OpenCourseware, also on YT).
| hzay wrote:
| Andrew Ng's courses are great too btw
___________________________________________________________________
(page generated 2023-05-01 23:03 UTC)