[HN Gopher] An Introduction to Statistical Learning with Applica...
___________________________________________________________________
An Introduction to Statistical Learning with Applications in Python
Author : alexmolas
Score : 372 points
Date : 2023-07-08 13:04 UTC (2 days ago)
(HTM) web link (www.statlearning.com)
(TXT) w3m dump (www.statlearning.com)
| sabertoothed wrote:
| HN hug of death. Download appears to be down.
| __rito__ wrote:
| You can download the whole book (legally) from here [pdf]:
| https://hastie.su.domains/ISLP/ISLP_website.pdf
| Exuma wrote:
| That is out of date. The link you put is literally in OP's link
| but 2nd edition.
| bshipp wrote:
| I'm not certain, but looking at that site it appears that the
| 2nd edition is the R version, whereas there is only one
| edition of the Python version.
| Exuma wrote:
| Hmm. right you are it seems
| loehnsberg wrote:
| ISL is the best intro-level textbook of classic ML methods. It's
| theory-oriented yet simple enough to appeal to a wide audience of
| students (with basic knowledge in stats, linear algebra, and
| coding).
|
| Having the examples only in R was a pain when teaching with it
| while using Python. I hope they'll now turn this into a series of
| Jupyter notebooks and distribute it through Colab or similar.
| nemoniac wrote:
| > I hope they'll now turn this into a series of Jupyter
| notebooks
|
| You can find them here: https://www.statlearning.com/resources-
| python
| loehnsberg wrote:
| Thanks for sharing. That's exactly what I was looking for.
| fensu wrote:
| Good ol days of ML
| thumbuddy wrote:
| I don't think most people realize this but the "old" stuff
| often works better, has less churn, and has far lower overhead
| costs for deployment than the "new" stuff. Depends on the
| domain and the goal.
| randcraw wrote:
| It's also a LOT more explainable and interpretable.
| craigching wrote:
| To your point, I replaced an LSTM that required ~$100k of
| infrastructure with XGBoost that required no more
| infrastructure (we created and used the model at query time
| on existing infrastructure we already had for query loads)
| and only lost about 2% accuracy (LSTM: 98%, XGBoost: 96%).
| This was two years ago and it's still in use.
| graycat wrote:
| Might consider, say,
|
| Sidney Siegel, N. John Castellan, Jr. 'Nonparametric Statistics
| for the Behavioral Sciences, Second Edition', ISBN 0-07-057357-3,
| McGraw-Hill, New York, 1988.
|
| So, "nonparametric" means make no assumptions about a probability
| distribution based on _parameters_. Or, call the material
| _distribution-free_.
|
| E.g., get to see about _resampling_ plans -- tiny assumptions,
| really simple, darned cleaver, quite generally useful, especially
| appropriate for computing. Might use _resampling_ to get more
| information from the data from "A - B" tests.
| laichzeit0 wrote:
| Huh? I have that book, and it's nothing like ISLR, at all. It's
| a good book, but ISLR covers topics such as gradient boosted
| trees, survival analysis, GLMs, etc. Nothing at all like the
| book you mentioned. If forced, you could say ISLR is more
| focused on prediction, not inference or hypothesis testing.
| graycat wrote:
| For _statistical learning_ , the book I mentioned
|
| _Nonparametric Statistics for the Behavioral Sciences_ ,
|
| should make a good contribution to _statistical learning_.
| Some of the techniques are so _robust_ , i.e., need such
| meager assumptions, that they should be especially welcome in
| _automatically applied_ AI (artificial intelligence)
| applications.
|
| Actually the book ISLR, _Introduction to Statistical
| Learning_ , does claim to cover
|
| "Resampling methods"
| wodenokoto wrote:
| I've been meaning to do a comparison of lab zero between the two.
|
| I've only had the chance to look over the Python lab for a few
| minutes, but compared to what I remember from the R labs, it is
| much, much more involved and longer.
|
| I know HN likes to complain about how difficult and confusing R
| is, but I think that it is an easier language for beginners or
| stat inclined people to start doing statistical work in.
|
| /two cents
| rossdavidh wrote:
| Python is more natural for programmers learning statistics/ML.
| R is more natural for statisticians learning programming. Which
| is not too surprising, since those were the audiences each
| language was intended for. I think it's good to be accepting of
| both, and use the one that works better for a given task.
| defphysics wrote:
| Personally, I prefer Bishop (Pattern Recognition and Machine
| Learning) or Murphy (Probabilistic Machine Learning: An
| Introduction).
| StefanWestfal wrote:
| I heard good things about Bishop however I am a SE that would
| like do know more about what the ML team is doing and maybe
| work on some ML side projects. Would you recommend Bishop here
| or is it considerer to theoretical for such a case?
| craigching wrote:
| Bishop is going to be more theoretical than ISL. It is true
| that Bishop is taught as an introduction to ML in many
| universities, but if you want more hands on to start with,
| ISL is an excellent option. There is another text called
| "Elements of Statistical Learning" that pairs well with ISL
| for a more theoretical treatment. I haven't looked at ESL in
| a long time, the only concern I'd have is if they aren't
| covering some introductory deep learning topics. Most of ISL,
| ESL, and Bishop are more traditional machine learning,
| covering a wide variety of algorithms, so bear that in mind.
| dunefox wrote:
| Those books are in a completely different ballpark.
| antegamisou wrote:
| Those are what most universities around the world use for
| their introductory machine learning courses.
| dunefox wrote:
| Press X to doubt.
| lysecret wrote:
| If you want more depth have you looked at the Elements from the
| same author ? This book is intended for beginners.
| PheonixPharts wrote:
| It's worth noting that neither of those books contain any code
| at all.
|
| I suppose that's what makes the ISLA being translated such a
| big deal. A sufficiently advanced student in ML/Statistical
| modeling doesn't really need code at all since it should be
| fairly trivial to translate the mathematical models into
| computational ones, and the ability to do so is a prerequisite
| to understanding these models in the first place.
| craigching wrote:
| ISL is a more introductory book than Bishop or Murphy. There's
| no reason not to read all of them, they're all excellent books
| that cover different topics. I'd also throw in Elements of
| Statistical Learning from the same authors as ISL(R/P). I've
| read ISL, ESL, and Bishop, started Murphy but didn't finish it
| (no real reason, just lost track of it when I got busy). I
| highly recommend any and all of these texts.
| wiz21c wrote:
| I understand some may say "classical" ML but to me, those "few
| parameters" methods are _very_ helpful in many cases and much
| easier to interpret than RNN :-)
| srvmshr wrote:
| This is an update to a very popular text which was originally in
| R. Professors Hastie & Tibshirani are leading educators in
| statistical learning. They also have a video course following
| these notes in Stanford Online. Very highly recommended if
| learning theoretical aspects of classical ML
| Exuma wrote:
| For anyone looking for the videos:
| https://www.youtube.com/playlist?list=PLoROMvodv4rOzrYsAxzQy...
| KRAKRISMOTT wrote:
| Any plans for a Julia version?
| srvmshr wrote:
| I actually like this book by Yoni Nazarathy
|
| https://statisticswithjulia.org/
|
| They have a book on Mathematics of DL too which has several
| of the concepts covered here.
|
| (Disclosure: I am slightly biased towards this since I've
| known the author by online interactions)
| tndoan wrote:
| Honestly, I dont think they will have Julia version soon. I
| converted all the code in this book to Julia sometimes ago:
|
| https://github.com/tndoan/ISLR.jl
| [deleted]
| TheAlchemist wrote:
| Off topic, but it's very interesting to observe the ratio of
| upvotes / comments.
|
| On any given chatGPT topic, there are hundreds of comments
| usually. Here, so far 100 upvotes, only 7 comments.
|
| The book looks great - and given the authors, it almost certainly
| is (I will buy it for sure). It makes me think though about the
| state of 'ML / AI / Data Science' - and the cynic part of me
| thinks that this upvotes / comments ratio kind of reflects the
| fact that most people interested in AI hype have not really
| touched a lot of underlying concepts and don't have any deeper
| understanding of maths / stats behind.
|
| PS. That being said, I didn't do a meaningful comment on the link
| topic neither.
| melling wrote:
| It's a well-known machine learning book. I've read through it
| and done the exercises in R.
|
| https://github.com/melling/ISLR
|
| There's an edX course from the authors:
|
| https://www.edx.org/course/statistical-learning
| suchagain wrote:
| I also went through this book for a GIS course in grad school
| and put together some Python notebooks.
| https://github.com/gbrunner/islr-for-gis
| godelski wrote:
| As a ML researcher, I don't think you're far off the point.
|
| w.r.t HN, there's almost all hype and no "science". People have
| strong convictions but not strong evidence. They happily cite
| papers, but only read the abstracts and miss the essential
| nuance. Especially in a field where suggesting limitations puts
| you at high risk of rejection (reviewers just copy paste that
| and thank you for the work).
|
| w.r.t academia, it is a bit better, but I find that in general
| there are a lot of researchers missing math fundamentals. I
| know or have met people at top universities or top labs that
| don't know the difference between likelihood and probability.
| Similarly ones that don't understand probability density. Even
| ones working on diffusion. But I will say, that in general the
| most prominent researchers do have these skills. But you'll
| notice that they aren't publishing as fast and their works
| might not even be as popular. A lot of research right now goes
| into parameter tuning and throwing compute at the problem. I've
| been a bit vocal about this though. Mostly due to it being a
| barrier to other types of research (because I'll admit that the
| tuning is needed, but we need to be honest that it isn't high
| innovation either and that it is hard to prove these are better
| given that we haven't tuned other models/architectures to the
| same degree).
|
| tldr: You're pretty spot on. There's a shit ton of noise in
| ML/AI. Especially on HN
|
| Edit:
|
| I thought I should also suggest Richard McElreath's Statistical
| Rethinking (https://xcelab.net/rm/statistical-rethinking/),
| which is a more enjoyable read than ISLR and will also
| introduce you to Bayesian stats (Lectures are also on youtube).
| I'd also suggest Gelman's Regression and Other Stories
| (https://avehtari.github.io/ROS-Examples/).
| mathisfun123 wrote:
| >don't know the difference between likelihood and
| probability. Similarly ones that don't understand probability
| density.
|
| I'm a phd student in a "top university", in a research group
| primarily focused on data science (NLP, LLMs, blah blah
| blah). I'm 100% sure I am the only person in the group of ~25
| (including profs/postdocs) that knows the difference between
| f(th|x) and f(x|th). In fact I'm pretty sure I'm the only
| person that has ever even seen f(th|x) (because I took a
| stats sequence out of casella+berger). This group puts out
| dozens of papers a year. My research focus is not data
| science (compilers).
| godelski wrote:
| Well your name says something about who you are (and might
| mean you can guess at the roots of mine :). I often find PL
| people are more likely to have good math chops because it
| is taken seriously in their field.
|
| Fwiw, at CVPR last year I asked every author of a diffusion
| paper about likelihood or score and only 2 gave me
| meaningful answers (1 compared their model's density
| against the data's density which was estimated through an
| explicit density method. Yeah, parametric vs parametric,
| but diffusion is not a tractable density method). It is
| really impressive that people who are working with
| probability and likelihood every day do not understand the
| difference (I see many assume they are the same, not just
| not know the difference).
| mathisfun123 wrote:
| >and might mean you can guess at the roots of mine :)
|
| your services are required on the busy beaver thread!
|
| >It is really impressive that people who are working with
| probability and likelihood every day do not understand
| the difference
|
| i think i come away from the whole experience (the phd,
| even though i'm not done yet) with a deep
| skepticism/cynicism of very many things. but it's
| probably not what you expect. i just don't think the math
| is at all relevant/important epistemically as long as you
| can run the experiments efficiently. which is exactly
| what you see happening - people with access to gobs of
| compute develop good intuition that leads them towards
| breakthroughs, and people that don't have access to
| compute struggle and make do with the formalisms. it's
| not much different in physics, where the good
| experimentalists aren't born that way, they're made in
| the well-funded labs.
|
| i firmly believe that in ML, the math does not matter at
| all, beyond the tiny bit of calculus and linear algebra
| you need to kind of understand forwards and backwards. of
| course everytime i say this on here i'm skewered/debated
| to death on it, as if i don't know what i'm talking about
| :shrug:
| wodenokoto wrote:
| I imagine a lot of people like this book and are happy to see
| an update.
|
| In that case it is hard to say anything of value other than an
| upvote.
| NeutralForest wrote:
| I know about the book, I just think it's cool there's an actual
| Python version now so I upvote.
| DeathArrow wrote:
| Wait, so there's more in AI than ChatGPT?
| Eigenwert wrote:
| [dead]
| lysecret wrote:
| This and elements where the intro to ML for me as well.
|
| I understand your sentiment but we also have to accept that for
| a lot of Ml usecases just calling ChatGPT api is 100x better
| approach than creating your own Ml model, and thus there is
| really no need to understand any math.
|
| As an example I am building an Ai nutrition counting app. And I
| use ChatGPT function calling. I can just add a field that has
| say an emoji of the good and it automatically classifies any
| food to the right emoji. There is absolutely no need to know
| gradient descent or any fundamental property to be able to do
| that.
| xapata wrote:
| > for a lot of Ml usecases
|
| Have you forgotten the No Free Lunch Theorem?
| A4ET8a8uTh0 wrote:
| There is, naturally, a reason for this. GPT is effectively a
| nice wrap on an otherwise complicated set of issues. I almost
| think of it as gui instead of console. Yeah, you lose some of
| the functionality and control, but a lot of people will take it
| and run with it simply because it is just so much easier.
|
| Case in point, Google AML AI, which promises to do away with
| pesky model validation and such ( because it will do everything
| in a closed box you will not have a reason to investigate ). I
| am already looking forward to the conversations with
| regulators.
| adamsmith143 wrote:
| This is pure gatekeeping. The math behind LLMs, that is, the
| math behind Neural Nets, is undergrad freshman level Calculus
| and some linear algebra. Not really complex at all. Can you
| deal with derivatives, the chain rule and matrix
| multiplications? Great you know all the "math" behind Deep
| Learning.
| posterboy wrote:
| > Can you deal with derivatives
|
| Linear Algebra is a huge ... space?
|
| And then there's all the other algebras. I'm not sure what
| you are thinking, like, _do you not know matrix
| multiplication you had better get a clue_.
| adamsmith143 wrote:
| What does that have to do with ML/DL? You don't need Lie
| Algebra to understand how to do a matrix dot product...
| godelski wrote:
| That's the beginning math, but definitely not "the math
| behind LLMs". That includes probability theory, metric
| theory, topology, and more. But most people don't even
| acknowledge this, but then again, unless you're deep in a
| subject you don't really know the complexities of that
| subject. Red flags should go off whenever anyone says "it's
| just <x>" or calls something simple. It's like the professor
| saying the proof is trivial, when that's the hardest part of
| the entire problem.
| adamsmith143 wrote:
| People like you are hilarious. You're sitting high in your
| ivory tower thinking that no one without a PhD in CS from
| Stanford/Berkeley/MIT can do what you do. Meanwhile people
| will take the Fast.ai course and be training full llm's
| from scratch in 6 months all the while you moan that "they
| don't even understand the REAL math". Yawn.
| sudosysgen wrote:
| If you want, you can get all of that math at any Top500
| university, even in undergrad. I agree that people will
| be having impact and deploying these models without that
| understanding, but you don't need to be at Stanford to
| gain that understanding and it's something desirable if
| you want to do research instead of deployment.
| godelski wrote:
| People like you are funny because you don't realize I'm
| actually also calling out a lot of ivory tower people.
|
| Also, I'm not saying you need the math to train a model.
| I'm not sure you even need linear algebra to do that,
| mostly just programming. That's why I called it the
| beginning. I was directly responding to your claim that
| this is all you need __to understand__. Because let's be
| real, you don't need to know backprop (and thus
| derivatives and chain rule) to train models. The math is
| about how to analyze your models. You know, specifically
| what academia is supposed to be doing. Research and
| engineering overlap but they aren't necessarily the same
| thing.
|
| Besides, math education is notorious for being
| essentially free. Who needs Stanford/Berkeley/MIT when
| textbooks exist widely. (Btw, CS doesn't typically
| produce mathematicians)
| thumbuddy wrote:
| There's a meme where it shows someone stepping over all the
| steps to understand how to think about and analyze data
| directly to BERT. Well now people are stepping past BERT to
| stable diffusion and ChatGPT. It's been like this for years.
| Most work environments suffer from it in a bad way. I don't
| envy practicing data scientists managing expectations.
| adamsmith143 wrote:
| Interesting seeing job postings wanting 5+ years of LLM
| experience. Like unless you were at OpenAI working on GPT-1
| or Google on BERT there's no one else in the world with that
| much experience and your shitty Startup/Fortune 500 company
| can't afford them anyway.
| vram22 wrote:
| >Interesting seeing job postings wanting 5+ years of LLM
| experience.
|
| Not really interesting.
|
| Similar to seeing job postings some years back (and even
| recently), wanting n+ years of Rails experience when DHH
| had created it significantly less than n years before.
|
| It's almost a meme on HN at this point.
| lisasays wrote:
| _Interesting seeing job postings wanting 5+ years of LLM
| experience._
|
| Since they're probably using ChatGPT to generate these job
| reqs - this is only to be expected.
| __rito__ wrote:
| What you are saying is true.
|
| This kind of books are key to getting started with Machine
| Learning/AI, and this particular book is a very good one. I
| started my ML journey with this book.
|
| There is a lot of hype around AI and it is going to be like the
| dotcom bubble.
|
| Unlike Crypto, AI has real uses right now and I am saying this
| not taking into account _any_ LLM products. But there is also a
| lot of hype and wishful thinking, and this bubble is going to
| burst and hurt a lot of people. But that won 't stop people
| making real money in the short term. Many hypers I know
| understand this well.
|
| But AI is here to stay. And even after the bubble bursts, there
| will be real uses of AI all around us.
| godelski wrote:
| > and this bubble is going to burst and hurt a lot of people
|
| > But AI is here to stay. And even after the bubble bursts,
| there will be real uses of AI all around us.
|
| This is, as a ML researcher, exactly where I'm at (in
| belief). The utility is high, but so is the noise. Rather,
| the utility is sufficient. The danger of ML is not so much
| X-risk or malevolent AGI, but dumb ML being used
| inappropriately. And in general, that is using ML without
| understanding the limitations and having checks on it to
| ensure that when hallucinations happen that they don't cause
| major problems. But we're headed in a direction where we're
| becoming more reliant upon them and then once we have a few
| big issues with hallucinations the bubble will burst and can
| end up setting us back a lot in our progress to creating AGI.
| Previous winters were caused by lack of timely progress, but
| the next winter will happen because we shoot ourselves in the
| foot. Unfortunately, the more people you give guns to, the
| more likely this is to happen -- especially when there's no
| safety training or even acknowledgement of danger (or worse,
| the only discussion is about being shot by others).
| basedbertram wrote:
| > This is, as a ML researcher, exactly where I'm at (in
| belief).
|
| As an ML researcher myself, I'm glad to see other
| researchers maintaining a level-headed approach amidst the
| noise.
| godelski wrote:
| Well if you care more about the study than the money
| (which is nice -- though I'm at the tail of grad school),
| it makes more sense to chase knowledge than chase
| metrics. Then again, I don't come from a computer science
| background, so maybe not having that momentum helps.
| Borrible wrote:
| Probably just needs a ChatGPT code interpreter plugin.
| kingkongjaffa wrote:
| I have been using this and other resources to learn linear and
| logistic regression .
| cschmidt wrote:
| The python version is great news. I get asked fairly frequently
| to recommend an intro ML book. I would have suggested this,
| except they usually only knew python and not R. Now it a perfect
| first book!
| craigching wrote:
| I used to introduce people new to machine learning with a
| python-converted version of ISL that I was developing. I never
| finished converting all of ISLR so this is very welcome!
| isaacfung wrote:
| Chapter 13 comes in handy. Always wanted to learn the basics of
| testing but didn't want to take a full statistics course.
| semajian wrote:
| A bit too wordy for me, I wish Landau was around to rewrite books
| like these, they'd be much shorter. Can we get a LLM to make it
| happen?
| rossdavidh wrote:
| Sure, as long as it's ok if it makes up stuff that isn't true a
| few times per chapter. :)
| iaw wrote:
| I've read through ISLR once and will likely go through it again
| at some point, this is a great text.
| wklm wrote:
| is there a python variant of "The Elements of Statistical
| Learning"?
| disgruntledphd2 wrote:
| There's no code in ESL so no Python version is necessary.
| verbify wrote:
| I loved the youtube series that accompanied earlier versions of
| this book -
| https://www.youtube.com/watch?v=5N9V07EIfIg&list=PLOg0ngHtcq...
|
| It can be watched without the book. The coding parts can be
| skipped. It has some insights missing in the book, and they've
| got an amazing mix of incredible technical talent and a great
| ability to distill and explain concepts.
| barrenko wrote:
| It's shocking how often it does not occur to me to check for
| video lectures of a great book.
___________________________________________________________________
(page generated 2023-07-10 23:02 UTC)