[HN Gopher] Show HN: Natural Language Processing Demystified
___________________________________________________________________
Show HN: Natural Language Processing Demystified
Link: https://www.nlpdemystified.org/ Hi HN: After a year of
work, I've published my free NLP course. The course helps anyone
who knows Python and a bit of math go from the basics to today's
mainstream models and frameworks. I strive to balance theory and
practice, so every module consists of detailed explanations and
slides along with a Colab notebook putting the ideas into practice
(in most modules). The notebooks cover how to accomplish everyday
NLP tasks including extracting key information, document search,
text similarity, text classification, finding topics in documents,
summarization, translation, generating text, and question
answering. The course is divided into two parts. In part one, we
cover text preprocessing, how to turn text into numbers, and
multiple ways to classify and search text using "classical"
approaches. And along the way, we'll pick up valuable bits on how
to use tools such as spaCy and scikit-learn. In part two, we dive
into deep learning for NLP. We start with neural network
fundamentals and go through embeddings and sequence models until we
arrive at transformers and the mainstream models of today. No
registration required: https://www.nlpdemystified.org/
Author : mothcamp
Score : 212 points
Date : 2022-12-01 12:00 UTC (11 hours ago)
| yannis wrote:
| Thanks excellent course, watched about an hour or two. Very well
| made. Deserves more exposure.
| carvking wrote:
| dang wrote:
| Could you please stop posting unsubstantive and/or flamebait
| comments? We have to ban accounts that do this. It's not what
| this site is for, and destroys what it is for.
|
| https://news.ycombinator.com/newsguidelines.html
| jumasheff wrote:
| OMG! Can't thank you enough!
| santiagobasulto wrote:
| Great content! And thank you for making it open and free. I
| recommend adding a License to your Github repo.
| posharma wrote:
| Who is the intended audience for this course? Is it application
| developers looking to use NLP in their apps? Or machine/deep
| learning devs?
| mothcamp wrote:
| It's for anyone who wants to learn NLP such that they get (a)
| an understanding of what's going on under the hood and (b)
| knowledge of how to get stuff done.
|
| So the ideal outcome is someone who gets an end-to-end view
| from theory/concept to implementation.
|
| If someone just wants to learn how to use tools/frameworks, I'd
| stick to the Colab notebooks. If someone's already experienced
| in ML and wants to learn something NLP-specific, I'd skip
| around to see what's interesting.
| posharma wrote:
| Thanks. Excellent course.
| yupis wrote:
| I wish there where written notes to study. Anyways great video.
| reichardt wrote:
| I love your course for being very comprehensive and technical
| while not getting lost in mundane details. Like the opposite of
| the following quote:
|
| "I didn't have time to write a short letter, so I wrote a long
| one instead." [1]
|
| [1] https://www.goodreads.com/quotes/21422-i-didn-t-have-time-
| to...
| mothcamp wrote:
| Really appreciate that. Finding that balance was one of the
| hardest parts of building this course.
| reichardt wrote:
| Yes, it's easy to see you put a lot of thought into that. I
| hope your course receives much more exposure. When I first
| found your videos a few weeks ago, I was surprised how few
| views they have given to the quality of the course.
|
| Do you record the voice track of your videos yourself?
|
| Glad to see you published the final lesson about
| transformers. Was looking forward to that!
| mothcamp wrote:
| I did record all voice tracks, yeah. If I do this again,
| I'll probably use a lot of generative tools now. :-D
|
| Hope you find the transformers module useful!
| mharig wrote:
| "Je n'ai fait celle-ci plus longue que parce que je n'ai pas eu
| le loisir de la faire plus courte."
|
| Blaise Pascal, 1656
|
| FYI
| reichardt wrote:
| Ah, thanks! I didn't know that and shouldn't have used the
| first result that came up with a google search.
| wazoox wrote:
| This looks awesome. No signup, that's a dream :)
| culanuchachamim wrote:
| Link: https://www.nlpdemystified.org/
| clueless wrote:
| thank you. Free ytube videos, with link to google colabs, , this
| is incredible...
| account-5 wrote:
| I like that your site runs properly with only first party scripts
| enabled in ublock, very rare these days.
|
| Secondly kudos for not requiring a sign up and for making it
| free!
|
| Looks like a brilliant resource, thank you.
| brooksbp wrote:
| Thank you for sharing this! I am currently studying NLP..
|
| Along the way, I've been struggling with a question and I hope
| someone can help me understand how to go about this: how would
| you build a model that does more than one NLP task? For a simple
| classifier like input: text (a tweet) and output: text (an
| emotion), you can fine-tune an existing classifier on such a data
| set. But, how would you build a model that does NER and sentiment
| analysis? E.g. input: text (a Yelp review of a restaurant) and
| output: list of (entity, sentiment) tuples (e.g. [("tacos",
| "good"), ("margaritas", "good"), ("salsa", "bad")]). If you have
| a data set structured this way, and want to fine-tune a model,
| how does that model know how to make use of a Python list of
| tuples?
| gattilorenz wrote:
| Yours is an example of aspect-based sentiment analysis.
| Typically it has been tackled in two steps: first extract the
| aspects, then classify them as positive/negative. GPT or T5 are
| possible options for doing both in one go, but splitting the
| task seems to be still a good option [1].
|
| [1] http://essay.utwente.nl/91778/1/Middelraad_BA_EEMCS.pdf
| axiom92 wrote:
| If you have the dataset, you can try to train a model like T5
| [1], notebook [2].
|
| You just need to create [(input, output)] examples in the
| format you want.
|
| For example
|
| [(a Yelp review of a restaurant, [("tacos", "good"),
| ("margaritas", "good"), ("salsa", "bad")]].
|
| With enough data, the model should be able to learn to generate
| the output in the right format.
|
| > Python list of tuples
|
| Things get interesting if you want to generate actual Python
| code. You can use a large language model with just a few
| examples of the task to generate such code. For example, see
| https://reasonwithpal.com/.
|
| Happy to answer more questions!
|
| [1] https://huggingface.co/docs/transformers/model_doc/t5
|
| [2]
| https://colab.research.google.com/github/huggingface/noteboo...
| mothcamp wrote:
| You could start by looking into either multitask transformers
| or really general seq2seq models like T5. With T5, for example,
| it just learns to transform one _text_ sequence into another.
| So you could fine-tune T5 to produce your target sequence, but
| rather than outputting an explicit Python list of tuples, it
| would output a string that looks like a sequence of tuples.
|
| Or maybe skip all that and outsource it to GPT:
| https://imgur.com/a/BQv6C3K
| brooksbp wrote:
| Ah, so if the model is just converting input text into output
| text, it can really learn how to do just about anything? But,
| there may be certain aspects of model design that make it
| better at some types of conversions ("tasks") than others?
| And there may be certain data sets that you want to train a
| base model on to get base learning of such as general
| language comprehension, and then build on top of that for
| your specific use case?
| mothcamp wrote:
| Yeah, I can see that being the case for specialized
| domains. With state-of-the-art models widely available to
| the public, knowledge of the domain and its workflows, and
| fine-tuning models to suit the domain will probably be your
| edge.
| trenchgun wrote:
| It is kind of like a very opaque but trainable Turing
| machine.
| insane_dreamer wrote:
| 10/10 for making this freely available
| strumyktomira wrote:
| Oh! That interesting me a lot :) I wanted to learn it in next
| months. Thank You very much! :)
| toolslive wrote:
| off topic: When did the default semantics for "NLP" change from
| Nonlinear programming [0] to Natural Language Processing?
|
| [0] https://en.wikipedia.org/wiki/Nonlinear_programming
| echobear wrote:
| thank you!! as a CS student interested in ML I will 100% be
| taking a look at this when I get some free time
| fuzzythinker wrote:
| Part 1 thread 6 mos ago:
| https://news.ycombinator.com/item?id=31421232
| ddtaylor wrote:
| What is the cost?
| mothcamp wrote:
| Your time. That's it.
___________________________________________________________________
(page generated 2022-12-01 23:01 UTC)