[HN Gopher] 2023: The Year of AI
___________________________________________________________________
2023: The Year of AI
Author : talboren
Score : 87 points
Date : 2023-12-25 18:52 UTC (4 hours ago)
(HTM) web link (journal.everypixel.com)
(TXT) w3m dump (journal.everypixel.com)
| Flux159 wrote:
| I think that it would also make sense to have a diagram that has
| open source achievements in Ai for 2023 (at least open weight &
| inference code). A lot of the announcements in the chart are
| behind an api so can't be run locally.
|
| Just thinking off the top of my head, Segment Anything, Llama 1
| and 2, Mistral, Stable diffusion XL, ControlNet, Whisper are all
| open source AI releases this year.
| rightbyte wrote:
| This article seems very corporate centric. Like, I am able to run
| a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just
| like this: wget
| https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-
| GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf
| git clone https://github.com/ggerganov/llama.cpp cd
| llama.cpp make ./main -ngl 32 -m
| ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048
| --temp 0.7 --repeat_penalty 1.1 -n -1 -i -ins
|
| Haven't people realized what they can run themself?
| swsieber wrote:
| I appreciate the sentiment. It definitely seems possible, but
| it doesn't ever seem as easy as copy + paste if you ever want
| performance or to go outside the same generic tutorial.
|
| > Just like this:
|
| Just is doing a lot of heavy lifting there.
|
| Is this the first model you came across?
|
| We're there any dependencies you had to install?
|
| Did you have to check video card compatibility?
|
| Where did you get the command arguments. It looks awfully
| complicated for a "just", as if there's a lot of options that
| aren't straightforward to use.
|
| Etc
| rightbyte wrote:
| Ye ... with "just like this" I actually mean "proficient with
| compiling C projects on an Unixy system" which is like years
| of dev, "power user" or admin experience (not being sarcastic
| here). For the audience here I would say that "just" is about
| right though.
|
| I have no clue about anything LLM related. I just made it run
| after reading some comment on HN pointing in its direction.
|
| My point is that these locally run LLMs seems way
| "underreported". I even tried to make a "Show HN" post about
| it but it got zero interest.
|
| But maybe I am missing something?
| ilaksh wrote:
| The Makefile for llama.cpp is really good and doesn't
| require hacking to make it work.
| mitthrowaway2 wrote:
| Even on Windows?
| marcosdumay wrote:
| It's probably a hell to run it on Windows. But again, for
| the audience here, that's not expected to be a large
| roadblock.
| ruune wrote:
| With almost every AI story I read on here (granted, I don't
| read that many, I find them quite boring), the top comment
| seems to be something like "you can do the same on your
| machine, see ollama". Case in point, your comment was the
| first one listed for me. So the interest for running LLMs
| on here seems quite high. Don't know why your post didn't
| catch anyone.
| ilaksh wrote:
| The Makefile for llama.cpp is really good, it will work the
| way he said.
|
| You can also use ollama which makes it into a one line
| install and then a simple command to run any popular model.
| yieldcrv wrote:
| check out LM Studio
|
| no command line involved, has a huggingface browser built in
| to load in the model of the day, has a chat-like interface
| for chatGPT like use, and can create a local server to run
| your local model for your programs to interact with it using
| an API that is identical to OpenAI's
| 23B1 wrote:
| No they haven't. It's actually quite weird how the whole hype-
| cycle is working this time; I think tech journalists are so
| caught up in the drameh of OAI and whatever they haven't a clue
| that most people can run most of these novelty UCs on their
| own. You don't even need a speedy rig!
| rightbyte wrote:
| Ye it is strange. And in the tail around Sam Altman there
| seems to be these semi-religious lunatics believing that only
| some government mandated big tech monopoly can save mankind
| from the LLMs.
|
| Meanwhile amateurs and uni researchers post really
| interesting work on a site named after a hugging emoji ...
| thefz wrote:
| > Haven't people realized what they can run themself?
|
| Lots, lots of people do not care.
| Kiro wrote:
| Why would I run it myself when I can run GPT-4 for less than
| the electricity cost? Running these yourself is not free.
| OpenAI is heavily subsidized.
| rightbyte wrote:
| Ye well at some point getting a sausage from Deliveroo or
| Uber Eats was a no-brainer since it was cheaper than in the
| actual fast food joint, due to some VC selling a dollar for
| pennies.
|
| Should limit the stock price prospects though.
| tempodox wrote:
| I followed your instructions, and it works without a hitch.
| macOS-13.6 on an Intel iMac with a Radeon Pro 5700 GPU. My
| first direct contact with an LLM, and I don't understand most
| of the command line options yet, but it's certainly
| interesting.
| phillipcarter wrote:
| A key difference is that for basic use cases, yes, this is a
| comparable experience.
|
| Where it stops being comparable is general application across
| literally millions of use cases. The ChatGPT system has proven
| itself a valuable utility across industries, people, and use
| cases. No open model I know of can match it yet.
| simbolit wrote:
| They don't have to match the market leader, they just have to
| be "good enough".
|
| There are oodles of use-cases where sending your data to an
| outside provider is a complete no-go. In these cases
| OpenAI/Google/whoever-products aren't relevant competition.
| guitarlimeo wrote:
| This is generating answers a lot slower on my machine (X1
| carbon) than GPT-3.5. :(
| forevernoob wrote:
| Would you say that this could run (smoothly) on my X220 with
| i7-2640M?
| rightbyte wrote:
| Imagine some old sci-fi movie where the computer write out
| the answer to a query, as if it actually was some scene
| worker typing it to some terminal. On this:
| NVIDIA GeForce GTX 1050 Ti Intel(r) Core(tm) i5-8300H
| x 8 32,0 GiB ram
|
| So I guess your computer would be a faster typist?
| vasco wrote:
| A few years ago peak HN comment was complaining about wget |
| sh.
|
| Nowadays we just run full on opaque models directly from
| huggingface without thinking twice about reading anything.
| Interesting how times change. I wonder what supply chain
| attacks will come from huggingface, must not be long now.
| fragmede wrote:
| That is, by the by, why models are no longer distributed as
| Python Pickle files, which _can_ root your box if the model
| being loaded is malicious.
| andy99 wrote:
| Something I find interesting is that in the last year or so, the
| talk around AI shifted from the model architecture to the trained
| model. People talk about Mistral 7B e.g., not transformer with
| rotary position embedding and gelu feed forward network (I don't
| know Mistral's architecture).
|
| Contrast this to a few years ago we'd talk about Resnet or
| Retina-net or whatever, not so much about the facebook pertain on
| Image-net when describing the model.
|
| In 2018 I remember hearing "architecture is the new feature
| engineering" (mostly meaning over-fitting I undetstood). Now it's
| all (mostly) about the dataset and training, the architecture, in
| 2023, was a minor detail. I personally think architecture will
| make a comeback soon.
| dartos wrote:
| It's because weights are where the magic lies, for the most
| part.
|
| Annnd nowadays tools like transformers can automatically select
| the architecture based on the name of the weights, so when
| interacting with LLMs, you generally just name the weights.
|
| Also MANY models are just using the llama or llama2
| architecture.
| kleiba wrote:
| _> It's because weights are where the magic lies, for the
| most part._
|
| What's that supposed to mean? Weights are not independent of
| the underlying architecture.
| bugglebeetle wrote:
| IIRC Mistral's architecture is llama-2. They just trained it
| from scratch with undisclosed data and techniques.
| ilaksh wrote:
| I think it did with Mixtral and MoE.
|
| I think people talk about the models because the architectural
| details are generally inaccessible for us since we don't have
| sufficient training (which it requires quite a lot to have an
| intuitive understanding I believe). Whereas the models are
| easily downloaded and tested.
| xiphias2 wrote:
| It probably means that AI is getting more use outside technical
| circles.
|
| I would definitely include Mamba state space models and of
| course would prefer a technical review over a corporate review
| of the year.
| greatpostman wrote:
| TLDR: openai still crushing the competition. Competing models
| beat benchmarks, but are borderline worthless on real tasks.
| dmezzetti wrote:
| Going to call BS on this. I've delivered multiple projects this
| year using open models.
| apwell23 wrote:
| > delivered multiple projects
|
| Very curious about the type of projects you've delivered.
| dmezzetti wrote:
| You can look at https://github.com/neuml/txtai. Biggest
| thing of 2023 was RAG with models like Mistral.
| benrow wrote:
| Also curious.
|
| I'm a sample of 1 and also relatively inexperienced. But I
| felt I quickly reached the limits of what was possible when
| I tried doing sentence classification with OSS sentence
| embedding models. The issue was with negation. I'd
| attributed too much magic to embedding models - they don't
| really understand language.
|
| Not to say there isn't very capable tech out there. Just to
| add a datapoint that "sentiment analysis"-like approaches
| in blogs don't always scale to your particular use-case.
|
| Edit: conscious I've drifted from the topic of chatbot type
| models, but felt relevant somehow.
| ewweezdsd wrote:
| How do you define "real tasks"? Even small models that can be
| run on consumer CPU can produce a coherent summary of an email,
| as an example. Isn't that a real task?
| ilaksh wrote:
| I think that might have been more true several 8-12 months ago.
| But now it feels like the momentum has swung towards open
| source. Multiple models are close to or exceeding GPT 3.5 now.
| They can do a lot of useful things.
|
| I have come to the conclusion that as much as possible I should
| try to wean myself off of OpenAI immediately. Because it's just
| not necessary or desirable to be tied to a single vendor
| anymore for many tasks. And in 2024 the open source
| capabilities will continue to increase. Soon everyone with a
| relatively new computer will be running things like LLMs
| locally. Within a couple of years it will be integrated into
| every OS or browser.
| Kiro wrote:
| Why are people benchmarking against 3.5? To me, the real race
| started with GPT-4. 3.5 and 4 are completely different beasts
| and that open models are catching up to 3.5 doesn't mean
| much. I've yet to see anything come close to 4.
| JanSt wrote:
| Because 3.5 is a very useful model. Getting there with an
| open model already is pretty cool.
| api wrote:
| AI has been gradually improving for decades, but this is the year
| we finally noticed.
|
| The big thing was huge progress in natural language
| understanding. 2023 was the year the Turing test was smashed.
|
| Seeing computers win games, drive cars, optimize systems, even
| design things wasn't as _subjectively_ impressive to most of us
| as being able to talk to them. This was the year we first saw AI
| that could sort of communicate with us the way we can with each
| other.
| UrineSqueegee wrote:
| not only talk to them, but ask them to perform tasks, sometimes
| obscure and they do it.
| Nullabillity wrote:
| People were amazed by ELIZA back in '67. That doesn't mean that
| it did anything useful...
| steveoscaro wrote:
| What's your point? Because I can't imagine it's to suggest
| that GPT4 doesn't do anything useful.
| bamboozled wrote:
| Their point is, we've been here before.
| hiddencost wrote:
| I dunno. The year of AI hype? LLMs are gonna change the world but
| the real incredible stuff is gonna take a few years before it
| really hits.
| chime wrote:
| Things are already changing for early adopters. Yesterday, I
| used a PDF GPT to review an 80+ page contract and GPT4 to
| explain specific parts of it in depth using layman's terms. I
| could have hired a lawyer like I have done in the past but that
| itself is a slow, laborious process and I often feel like my
| questions don't get thoroughly answered. I feel very confident
| of my understanding of the contract now.
|
| I also used StableDiffusion to mock-up landscaping designs for
| our back patio and make ceramic art ideas for my wife last
| month.
|
| I was finally able to use super resolution to improve the
| quality of an old video CD of a VHS tape of a theater play my
| dad starred in with his friends in the 80s. I tried without
| super resolution many times in the past but it was never good
| enough.
|
| You are right that it will take a few years before it hits but
| AI in 2023 is not hype if you know which tool to use when.
| bamboozled wrote:
| When we use it to solve real problems, make some real
| progress on climate change, then it will be something else
| from hype and it will be the year of AI.
|
| When you as an individual use it to sort PDFs or something
| it's really hype.
___________________________________________________________________
(page generated 2023-12-25 23:00 UTC)