[HN Gopher] TextSynth Server
___________________________________________________________________
TextSynth Server
Author : catfishx
Score : 251 points
Date : 2023-03-16 11:05 UTC (1 days ago)
(HTM) web link (bellard.org)
(TXT) w3m dump (bellard.org)
| canistel wrote:
| This man, Fabrice Bellard again...
|
| Frankly, I have not seen a more impressive portfolio of
| programming output.
| recuter wrote:
| [flagged]
| ocimbote wrote:
| "How do you tell a French? You don't, they tell you."
|
| Also, not true. The accent always tells a French.
| [deleted]
| speedgoose wrote:
| Everyone has an accent that tells where they are coming
| from. People who think they don't have an accent simply
| don't know that they have one.
| recuter wrote:
| Or so the Germans would have us believe.
| ocimbote wrote:
| Not true. I'm a French native and I've been fooled both
| ways: by native speakers having an accent and by non-
| native using idioms and tone flawlessly.
|
| Some people are just masters at imitating tones and
| accents.
| shakow wrote:
| I never understood why we triggered such hate on the
| Internet.
| recuter wrote:
| Humour mon ami, pas de haine
| shakow wrote:
| Would you mind explaining to me what is funny in this?
| recuter wrote:
| https://www.youtube.com/watch?v=2kArCRjT29w
|
| I rest my case.
| Accacin wrote:
| Well, I'm British and France and the UK seem to have been
| mortal enemies for a long time :)
|
| However, I absolutely love the French, the French language,
| and your country. I'm definitely not above pocking some fun
| at the French, but my French friends give it back in equal
| measure haha.
| zmix wrote:
| Americans (just a wild guess, that he is one) seem to have
| two strange phobiae: Herpes and French people.
| speedgoose wrote:
| It's a long story IMHO. You can find hate pretty far back
| in history, mostly because of the many wars involving
| France.
|
| Recently I would mention the lack of appreciation towards
| the allies after the WW2, or France not invading Irak with
| others because of the (fake) mass destruction weapons.
|
| In addition to that, Frenchmen are quite patriotic and
| proud so it's a fun target. It's also an allied country and
| a strong one so it's not very politically incorrect to be a
| bit racist as a joke. Most people don't dare to say the
| same about China or Zimbabwe for obvious reasons.
| shakow wrote:
| > the lack of appreciation towards the allies after the
| WW2
|
| Where did you get this impression? The US were basically
| seen as the _it_ country for decades.
|
| > for obvious reasons.
|
| It's not really obvious for me. Why is "he is a great
| programmer despite being French" seen as OK, whereas e.g.
| "he is a great programmer despite being Black" would
| clutch (and rightly so) many pearls?
| flangola7 wrote:
| The French largely are not a marginalized people with
| little power who are target by people with lots of power.
| FpUser wrote:
| I wonder if he ever went to interview they would try leetcode,
| or what is that squiggly thing in a code sample is for or are
| you a Language XXX programmer and other similar things. I can
| bet that this guy would probably fail many standard selection
| criteria.
| biohax2015 wrote:
| I can bet that he wouldn't.
| ggerganov wrote:
| Very inspiring stuff! I hope one day we get to see the magic
| behind libnc.
| hardwaresofton wrote:
| I've been feeling FOMO (for lack of a better term) about recent
| AI & ML/GPT progression.
|
| It feels like ML/AI it might be the beginning of the end for a
| large class of things (if I wanted to be alarmist I'd say
| "everything") -- and the fact that Fabrice Bellard has jumped in
| and done the absolutely obvious rising-tide thing (building an
| API that abstracts the technologies) speaks volumes.
|
| Releasing something like this fits to Fabrice's pattern of work
| -- he built Qemu and that served as a similar enabling fabric for
| people to run virtual machines. QuickJS quietly powers some JS-
| on-another-platform functionality.
|
| Simon was right. The Stable Diffusion moment[0] is already here.
| It's going to accelerate. It was already moving at a speed that
| was hard to follow, and it's about to get even faster.
|
| There are too many world-changing things moving forward at the
| same time, and I'm only looking at such a small cut of the tech
| sphere. I don't know what to do with myself, I feel so thoroughly
| unprepared.
|
| [0]: https://simonwillison.net/2023/Mar/11/llama
| lynx23 wrote:
| Control freak, by any chance?
|
| Is it really so hard to just "go with the flow"?
| arvinsim wrote:
| Hard to just "go with the flow" when your livelihood can be
| possibly threatened.
| yeahsure22 wrote:
| Well I'm sure that one guy will be able to change the
| course of history to suit his job situation. It's going to
| happen so you had better go with the flow. Become water.
| DrewADesign wrote:
| In the US, people's jobs effectively justify their
| existence. Smugly saying mid or late career professionals
| can "become water" when their entire career category
| faces collapse is the most patronizing thing I've heard
| in a long time. That's a non-recoverable blow to many
| smart, capable people. There's a big difference between
| demanding we stop the wheels of progress to protect a few
| people and saying we owe the masses being crushed by them
| some harm reduction. Of course society on a whole will
| profit. But, the rust belt shows that the platitude about
| people deemed professionally unnecessary just "figuring
| it out" merely comforts the people turning those wheels.
| UncleEntity wrote:
| Not all that long ago people on this very site were super
| enthusiastic about the prospect that my job would very
| likely be taken over by the robots.
|
| Because, you see, the robots didn't need to be perfect
| but merely fractionally safer than the humans.
|
| Now all I see is No True Scotsman arguments on why the
| current crop of AI won't take away their jobs and,
| ironically, my job is pretty safe due to Tesla's
| tomfoolery causing blowback from the feds.
|
| So robots eating the world is good or what?
| DrewADesign wrote:
| Yeah, it's pretty bizarre. So many people who become
| incensed by businesses profiting from being shitty to
| customers are gleefully just dumping the clutch without
| looking for people in the crosswalk. Maybe literally if
| the car is driven by an algorithm. I'll bet they could
| whip up a lot of macho, patronizing personal-
| responsibility-based arguments in support of being able
| to mow down the slower pedestrians at crosswalks though.
| yeahsure22 wrote:
| Look pal, cry on the internet all you want but you better
| become water. Ask anyone from the "rust belt" what
| happened. You didn't care when it happened to them and no
| one cares when it happens to you. Become water. Flow.
| hardwaresofton wrote:
| It's so funny, because that phrase when I hear it
| suggests _going with the ML /AI flow_.
|
| "becoming water" is a double-edged sword -- if there's a
| storm out or really hot, it's a bad time to be water. You
| kind of want to be rock then.
|
| The point is that going with the flow feels like it could
| easily be interpreted either way (as in -- go all in on
| AI/ML)
| DrewADesign wrote:
| Only someone who claims to have all the answers would
| make such bold incorrect assumptions about who I am and
| what I know. The demand for my specialty is waxing, I'm
| likely safer than most work-a-day software developers,
| and it's my third career. Don't talk to me about becoming
| water in my career and life. I also don't think that
| lacking my brain plasticity and marketable cognitive
| profile should disqualify someone from easy access to
| food, housing, and health care. So if that body of water
| drains into the ethically feeble sewer of self-absorbed
| SV business culture, I'll opt for evaporating on a rock
| instead.
| hardwaresofton wrote:
| sorry a bit late with my response, but original commenter
| here.
|
| I'm not much of a control freak -- it's more when you _see_
| an credible posisble extinction event in the distance, don 't
| you want to act?
|
| And even if you didn't view it that grimly, isn't the ML
| stuff just SUPER cool? Trainable prediction engines are
| really amazing and very actually useful (feels like the first
| iteration was being able to recognize things in images, which
| felt like magic).
|
| Going with the flow is awesome, but one thing I've found
| about life is that if you're not in the _right_ flow, it 's
| completely different. It's not like you have to be in the
| perfect stream, but you need one with fast moving water.
|
| Imagine being 10 years late to computers or the last one in
| your area to get a typewriter.
|
| I personally feel like I have to immerse myself in stuff to
| get it, and the lack of more than surface level understanding
| of ML is worrying with how big it could potentially be.
| zitterbewegung wrote:
| I have a positive outlook for large language models where
| we can have new synthesized stories that break the rules of
| what we consider what makes something a game or not. The
| biggest complement to new forms of art are the people that
| reject the new forms.
|
| I can't really sing or draw anime but now my computer can
| so it can be a large enabler for people.
| arvinsim wrote:
| I understand what you mean. I have dropped delving into other
| programming languages and frameworks to learn more about these
| AI/ML/GPT stuff.
| tambourine_man wrote:
| Same here. I don't know if I retire or dive in deep. There's no
| really other way to sound other than alarmist.
|
| This is the biggest revolution I've seen in my lifetime and
| it's going to make the PC, internet and smartphone a kids toy.
| hardwaresofton wrote:
| > Same here. I don't know if I retire or dive in deep.
| There's no really other way to sound other than alarmist.
|
| It might be a blind-leading-the-blind situation, but I wonder
| if retiring will protect you. Could you imagine having gone
| into retirement in the early internet? Wouldn't you be really
| confused at how to use things today? Or maybe you would just
| learn most useful things passively as a consumer.
|
| It feels like the gap between the people who can make and the
| people who consume is widening.
|
| > This is the biggest revolution I've seen in my lifetime and
| it's going to make the PC, internet and smartphone a kids
| toy.
|
| This coupled with everything else that's happening is all too
| much. Getting closer and closer to cheap/free energy
| (batteries, solar, fusion, wind, etc), AI/ML, Robotics...
| carapace wrote:
| To me it seems clear that most people are going to become a
| kind of peasant or serf, effectively "owned" by their
| techno-corporate silo. Programming as a human activity will
| resemble professional sports or writing: a handful of well-
| paid superstars and the rest of us mostly just watch/read
| (although of course we play sports and write, and basic
| computer literacy should be a thing, eh?)
|
| This might not be so bad? As you've no doubt noticed, most
| people suffer from some combination of ignorance and
| stupidity. We can barely function. An earthquake happens
| and half the building fall down, despite (some) of us
| knowing how to build them so they don't do that. That's not
| a technical problem, it's a social problem, eh?
|
| As far as technical problems go, we have pretty much solved
| "Life on Earth". In video game terms we beat the boss about
| a century ago (give or take a few decades) and now we're
| just sort of running around acting silly while the end
| credits scroll. Atomic power, materials science, etc. _We
| licked it._
|
| IF we employ these newfangled talking computers to tell us
| how to solve our problems, they will, and we can live
| happily ever after. (prompt: "What would Jesus do?") It
| doesn't matter that they're merely regurgitating the
| wisdoms of the ages, when the computer says it, maybe it
| will stick?
| munificent wrote:
| Every one of these articles fills me with a related kind of
| dread.
|
| My whole life, my whole _personality_ is architected around
| _making things by hand for other people_. My ideal world is a
| hipster stereotype where we all sit around using a small number
| of artisanal products to make other artisanal products for each
| other.
|
| The arc of my programming career has gone lower and lower down
| the stack because when I create, I enjoy it most when it feels
| concrete, deliberate, and long-lasting. I get no joy out of
| duct taping a few libraries together (though I respect others
| who do).
|
| While I spend a lot of my day doing code review and think it's
| a valuable, important part of the process, it's not my favorite
| task. I like _making_ stuff, not just socially interacting with
| others to loosely guide them towards making stuff. The idea of
| AI-assisted software development to me just sounds like taking
| the one part of the job I like most--writing code--and turning
| it into even more code review, except now I 'm reviewing code
| vomited out by a machine.
|
| And I completely dread the long term societal implications of a
| world where most people spend most of their day consuming media
| auto-generated by a machine. Where lonely men and women hide
| from their social anxiety by cultivating simulated romantic
| relationships with chatbots. Where teens have their
| expectations of sex set by watching synthesized porn starring
| virtual actors doing things that are physically impossible.
| Where people watch auto-generated videos of impossibly idyllic
| vistas instead of actually leaving the house and going for a
| hike. Where our beliefs of the world are formed largely by
| synthesized news articles that may or may not accurately
| reflect it. Where children learn to speak, read, and write from
| AI tutors and pick up all the grammatical and stylistic quircks
| of the AI model such that they now because actual real parts of
| human language.
|
| And, of course, where almost all of the massive profit
| generated by all of that flows to an increasingly small number
| of huge corporations.
|
| None of that sounds like a world I want to live in.
|
| I totally get the value of AI for things like classification
| and understanding. But generative AI feels like a pandora's box
| to me.
| turtleyacht wrote:
| Interested to know what your plans are. You've thought about
| this a lot, and for me your books sit next to all the others
| I have on C, assembly, math, etc--and, I am a bit embarrased
| to say, a large portion of which I _still need to study._
|
| I don't think I should despair _too_ much, and simply grant
| triumph to a statistical steamroller.
|
| The value is we'll still be making stuff, or yearning for it,
| and AI becomes a part of our toolkit (or never). Perhaps in
| your case, we will see _Prompt Engineering Patterns_ someday.
|
| I intend to plop down a tiny fortune enabling my children to
| have obscene hardware for wherever their personal projects
| take them, and server-grade CPUs, multiple GPUs, and fiber
| will be a given.
|
| Just know you're standing at a height somewhere above, and as
| a giant _to me_ I hope you can see farther ahead :)
|
| Maybe we'll retreat more into our personally-named server,
| managing a handful of like-minded users, crafting rooms in a
| MUD no one will read. We'll publish stats for our packet
| filter, read stories typed by hand, and make little games.
|
| There are still _lots of problems_ we would be completely new
| to in many domains, people suffering injustice, and are those
| not things we might be interested in as well?
|
| For sake of learning things, we are still satisfied. Even if
| AI were to generate it in an instant. For sake of satisficing
| our home labs and side projects, we find a tiny reprieve.
| zachthewf wrote:
| I understand and share your fear. In a sense, I think we
| already live in this world: there are many people who are
| already slaves to YouTube, TikTok, strategically timed push
| notifications, etc.
|
| We live in a world where digital tech is fantastic for people
| who have the skill and strategy to place healthy limits on
| their own use of it, and the worst possible world for those
| who cannot. And it's all about to get much more extreme.
|
| All of your fears will likely come to pass, but I think there
| are also much more positive framings and there are fantastic
| opportunities to build solutions to some of these problems.
|
| - Loneliness. Today, lonely men and women already suffer. A
| simulated relationship is a legitimate improvement over no
| human contact at all. Could a bot use the embeddings of your
| interactions to match you with a compatible partner, saving
| you the anxiety of having to navigate the dating market.
|
| - Auto-generated videos. Frankly, most movies and TV shows
| suck today, probably because they come out of a broken
| Hollywood. I can't wait until the barrier is so low that a
| random genius college student can make a feature film with
| almost no resources. Maybe there will be more new movies that
| are actually good!
|
| - AI tutors. Playing with GPT4 the past few days I've already
| experienced visceral joy from its use as a teaching tool. It
| is like having a pretty smart co-worker who knows something
| about literally everything. I'm honestly so excited about
| being able to learn new things without dealing with the
| fundamental barrier of _finding good sources of knowledge_.
| g9yuayon wrote:
| > My whole life, my whole personality is architected around
| making things by hand for other people.
|
| Not sure if there's a conflict here. We still use compilers,
| right? Unless we write in machine instructions, there's
| always some kind of program generations somewhere.
|
| > I get no joy out of duct taping a few libraries together
| (though I respect others who do).
|
| You still can work on stuff that requires human ingenuity .
| If you look at the the credits section of the GPT-4 paper,
| you will see many people there because they are the master of
| "low-level" optimization. For instance, the author of
| Reformer is a lead in the Long Context section, because
| apparently he knows how to reduce the O(N^2) of self
| attention to O(Nlog(N)), to say the least.
| datpiff wrote:
| > There are too many world-changing things moving forward at
| the same time, and I'm only looking at such a small cut of the
| tech sphere. I don't know what to do with myself, I feel so
| thoroughly unprepared.
|
| I think the general trend is that actual useful applications
| are emerging from enormous models trained and owned by billion
| dollar companies _only_. Even projects that aim to run models
| on private consumer hardware are dependent on commercial orgs
| to produce them. It doesn 't seem like that is likely to
| change.
|
| I don't think there are many positions/jobs/roles for people
| doing integral, foundational work that requires a deep
| understanding of ML. Becoming a world expert in ML will
| probably only open up opportunities at a dozen companies.
|
| A very shallow surface-level understanding of ML already puts
| you leagues ahead of the general population. In terms of job
| security, figuring out how to use an ML API will get you hired
| faster than knowing how to advance the field.
|
| We don't actually need that many Fabrice Bellards.
| trop wrote:
| > I think the general trend is that actual useful
| applications are emerging from enormous models trained and
| owned by billion dollar companies only.
|
| One way to think about it: Today's LLMs require incredible
| outlays of capital and processor power (and crews of folks
| with doctorates), such as billion dollar companies can
| provide. But how is that different from what Intel brought to
| commodity CPUs in the '90s/'00s, or what Nvidia brought to
| GPUs in the '00s/'10s? Or even what Cisco and folks brought
| to networks?
|
| Though we may never design an artisanal CPU/GPU/router, we
| get to work with them every day to make things, and to
| communicate. These LLMs can be that for us at this moment.
| Let's go out and enjoy them, and see what we can make within
| their (vast) domain-specific capabilities.
|
| [takes off rose-tinted glasses]
| datpiff wrote:
| > But how is that different from what Intel brought to
| commodity CPUs in the '90s/'00s, or what Nvidia brought to
| GPUs in the '00s/'10s? Or even what Cisco and folks brought
| to networks?
|
| Yes, they made these technologies accessible and useful.
| And very few people needed to understand high-K dialetrics
| or out-of-order execution to use them, hence I think the
| FOMO is misplaced
| DeathArrow wrote:
| If there were Oscars or Nobels for programming, Fabrice Bellard
| should have won one long ago!
| capableweb wrote:
| I guess the ACM A. M. Turing Award is as close as we get, which
| AFAIK, they never won. But Bellard won countless of other
| awards, competitions and benchmarks, it is not like they are
| unrecognized for their work in the field.
| [deleted]
| pierrec wrote:
| " _The CPU version is released as binary code under the MIT
| license_ "
|
| This gives off the surreal sci-fi vibe that the binary _is_ the
| source. And who knows... true wizards work in mysterious ways.
| minxomat wrote:
| > The GPU version is commercial software. Please contact...
|
| Shame.
| dark-star wrote:
| I'm pretty sure a dev skilled in ML and GPUs has no trouble
| modifying the MIT code (which runs on the CPU) to something
| that runs on GPUs...
| EMIRELADERO wrote:
| The CPU version is provided under binary format only.
| thetoon wrote:
| You're right : `The CPU version is released as binary code
| under the MIT license`. That's quite an unusual choice, not
| even sure how MIT would apply to that...
| circuit10 wrote:
| I guess you could decompile it and redistribute that
| source if you were dedicated enough
| Aissen wrote:
| The goal here is to allow you to redistribute while
| maintaining the copyright notice.
| capableweb wrote:
| You can release whatever you want under MIT :) It grants
| you the right to use the binary for commercial projects
| (or any type of projects you want), to modify the binary,
| distribute it yourself and more. You cannot hold
| Bellard/the license holder liable for anything related to
| it, and you must include the license and copyright if you
| distribute it.
|
| Seems pretty doable to me :)
| etaioinshrdlu wrote:
| I think it's fine if a software legend (or anyone) wants to
| make some income.
| rvz wrote:
| "Shame. I cannot take the code / binary and run it as a SaaS
| efficiently." - HNers
|
| Good for him to commercialize it, and at least he is not
| pretending to be a non-profit accepting VC money.
| alexvoda wrote:
| Countering SaaS-ifying by others can also be achieved through
| the AGPL or through the BSL (initially not open-source,
| reverts to open source after set period).
|
| I do believe one of the failures of GPL was not being AGPL
| from the start.
| versteegen wrote:
| The GPU version of libNC is available as a free binary, and you
| can find MIT-licensed source code implementing (training and
| inference of) transformers using libNC at
| https://bellard.org/nncp. (nncp was meant to be a submission to
| the Hutter Prize, which would have required open-sourcing libNC
| too, but it didn't qualify due to using AVX2 and too much RAM.
| At least the CPU version binary is MIT licensed since ts_server
| is.) I think it wouldn't be that big a project to support LLaMa
| starting from that code, although it is dense code.
|
| Edit: licensing
| sroecker wrote:
| That version of libnc_cuda.so doesn't seem to be compatible
| with ts_server though.
| copperx wrote:
| Shame as in "it's a shame" or as in "shame on them"?
| minxomat wrote:
| The former. Makes sense for their business model.
| JacobiX wrote:
| Very interesting as usual from Fabrice Bellard, but I'm a little
| bit disappointed this time, because libnc is a closed source DLL.
| Nevertheless it will be interesting to compare it to the amazing
| work of Georgi Gerganov: GGML Tensor Library. Both are heavily
| optimized, supports AVX intrinsics and are plain C/C++
| implementation without dependencies.
| vidarh wrote:
| ChatGPT is pretty good at disassembling x86, and is able to
| give reasonable descriptions of what the code is doing (e.g try
| "disassemble the following bytes in hex and explain what they
| appear to be doing: [bytes from a binary in hex]")
|
| I'm curious how soon someone uses these models to effectively
| ruin the ability to use releasing binaries as an obfuscation
| method.
| circuit10 wrote:
| It will struggle with understanding small parts of big
| programs without seeing the full context, though maybe you
| could get around that by making it generate some sort of
| summary for itself or something like that
| vidarh wrote:
| My tests on small parts of big programs suggests if
| anything that it does far better than I expected, but
| you're probably right that it would struggle with that for
| many things if you tried turning it into a bigger tool, and
| having it generate summaries is probably essential. While
| we can "fake" some level of memory that way, I really would
| like to see how far LLM's can go if we give them a more
| flexible form of memory...
| datpiff wrote:
| Using ChatGPT as a disassembler seems like a dumb idea when
| free disassemblers already exist. What possible advantage
| does it give?
| vidarh wrote:
| The ability to explain the code, and extract higher level
| understanding. Disassembling into raw instructions is the
| most trivial part of reverse engineering an application.
| Hence "and explain what they appear to be doing" bit.
|
| For the pieces I've tested, it often recognises the source
| language, and could give ideas about what the code was for
| and what it did.
| datpiff wrote:
| > Disassembling into raw instructions is the most trivial
| part
|
| So why not do it in the proven-correct tools and give
| ChatGPT the instructions?
|
| I'm all for finding neat use cases but I wouldn't use an
| AI chatbot as a calculator...
| vidarh wrote:
| You could do that too, but that is entirely missing the
| point, which is that ChatGPT is capable of inferring
| higher level semantics from the instructions and explain
| what the code is doing. You're getting hung up on a
| minor, unimportant detail.
| datpiff wrote:
| Apparently the point is proving it's possible. Not making
| it useful.
| vidarh wrote:
| No, that was not the point at all.
|
| The point is that ChatGPT understands the code well
| enough to explain what it does, and so there's reason to
| wonder how soon someone leverages that in a disassembler
| to produce far better output to the point where using
| releasing "only" the binary as an obfuscation mechanism
| stops being viable.
|
| E.g. additional value would be comments about purpose,
| labels for function entry points that _makes sense_ for
| what the function does, labels for data that _makes
| sense_ for what is stored there, comments with
| explanations of why things are structured the way they
| are.
|
| Having reverse engineered programs from binary in the
| past, inferring good names and commenting the sources is
| the vast majority of the effort.
| datpiff wrote:
| >E.g. additional value would be comments about purpose,
| labels for function entry points that makes sense for
| what the function does, labels for data that makes sense
| for what is stored there, comments with explanations of
| why things are structured the way they are.
|
| None of this requires giving it a binary. You are asking
| it to do 2 tasks, both of which it will do with some
| level of error. You could disassemble the binary near
| perfectly, for free. You have a hammer and everything
| looks like a nail.
| vidarh wrote:
| It was an example. I've already pointed out to you you've
| gotten hung up on a minor, unimportant detail of an
| example. You're right, you can do the raw disassembly of
| the instructions by different means first. Which is
| _entirely irrelevant_ to the point made. In other words:
| You 're being incredibly obtuse and arguing against an
| imaginary point nobody made.
|
| The recommendation was literally to _try_ the prompt. The
| purpose of that recommendation was that if you tried it
| you might see some of the additional commentary ChatGPT
| adds, which is where the value was. I could have
| suggested you disassemble it first, but it seemed easier
| cut and paste the hex values. That is all.
|
| Do you have anything you want to discuss about _the
| actual point_? If not we 're done here.
| npace12 wrote:
| https://github.com/20urc3/Sekiryu
| vidarh wrote:
| Thanks. That's at least the start of the type of thing I
| was thinking of.
| ggerganov wrote:
| I expect LibNC will be better in every aspect: performance,
| accuracy, determinism. But hopefully with time we will close
| the gap.
| liuliu wrote:
| The comparison table from the ts_server site looks awesome
| though. I wish we could generate one for llama.cpp,
| unfortunately too busy with other things at the moment.
| ggerganov wrote:
| Hey liuliu, would love if you join the project when you
| find the time - your work is really inspiring!
| hardwaresofton wrote:
| refreshingly humble take -- thanks for your hard work. The
| work you've done and put out in the open is massive.
| aww_dang wrote:
| Is it necessary to pre-process models from hugging face before
| using them with libnc ?
| JoachimS wrote:
| The Fabrice Bellard web page must be one of the most underselling
| ones on the entire web. So many amazing projects. Not a word that
| really emphasize the importance, coolness. Just a simple list
| with short factual descriptions.
| danwee wrote:
| These kind of people are like that. Does Linus Torvalds have a
| web page? No. He knows he doesn't need one. They are gods in
| the IT industry, and they know it.
| UncleEntity wrote:
| He has a blog because I recall reading about him reverse
| engineering a file format for his wife's embroidery machine.
| networked wrote:
| > All is included in a single binary. Very few external
| dependencies (Python is not needed) so installation is easy on
| most Linux distributions.
|
| I have to disagree. The combination of being closed-source and
| dynamically linked makes a program a hassle to run on Linux. Even
| if it isn't at the moment of release, it soon becomes one. While
| _ts_server_ is better than most, it already requires an old
| version of _libjpeg-turbo_ not available in my distribution 's
| repositories. I had to run it in a Rocky Linux container:
| docker run \ --rm \ --mount
| type=bind,source="$(pwd)",target=/app/ \ --publish
| 127.0.0.1:8080:8080 \ rockylinux:9 \ sh -c 'dnf
| install -y libjpeg libmicrohttpd && cd /app/ && ./ts_server
| ts_server.cfg'
|
| The solutions to this problem that I am aware of that do not
| involve releasing the source code are: 1) static linking; 2)
| containers; 3) shipping a Windows binary :-) ("Win32 is the only
| stable ABI on Linux" -- https://blog.hiler.eu/win32-the-only-
| stable-abi/).
| remram wrote:
| The ABI doesn't seem to be the problem, and Win32 does not
| include a libjpeg, so your Win32 approach would only work if it
| _also_ bundles or statically links libjpeg.
| networked wrote:
| > your Win32 approach would only work if it also bundles or
| statically links libjpeg.
|
| Of course. The stable ABI is to allow your bundled DLLs to
| keep functioning. (Check out
| https://news.ycombinator.com/item?id=32471624 for an
| extensive discussion of the link.)
| generalizations wrote:
| Is it just me, or is the llama model not available to download
| there?
|
| Edit: nevermind, the models are all there, just some of the links
| aren't.
| a_subsystem wrote:
| "...REST JSON API..."
|
| Please update.
|
| https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...
| turmeric_root wrote:
| I disagree with the linked post, most people use 'REST' to
| refer to JSON-over-HTTP now.
| a_subsystem wrote:
| You disagree with the guy who created the term REST?
| turmeric_root wrote:
| Yep.
| jefc1111 wrote:
| In comments on this post, and elsewhere on other posts about AI,
| I see a lot of people referring to worries around the potential
| for lots of types of jobs to be heavily impacted by this
| technology.
|
| I feel like people are often referring to 'coding' when they
| express these worries. You know, actually writing code, having
| been given a spec to do so, and perhaps also participating in
| code review, writing tests, all the usual engineer stuff.
|
| My question is, amongst the HN crowd, what kinds of roles or
| areas do we think might be somewhat immune to this effect? The
| first thing that occurs to me are security, infrastructure & ops,
| networking. And of course the requirements gathering stage of
| software development. It is already the case that a lot of senior
| devs probably don't write much code and spend more time on
| communication between different stakeholders and overseeing
| whoever (or whatever) is writing the code.
|
| Anyone else been thinking about this? What tech roles might
| thrive in the face of AI.
| Accacin wrote:
| In my day job I work with TypeScript and React, and if I'm
| honest, I'm not worried at all. Why should I be? Automation is
| all around us, and has been for years.
|
| The way I personally see it, is that AI such as ChatGPT is
| another tool in our arsenal that we as developers will have to
| figure out how it fits into our workflow. I think long term it
| will help us write better code, and in general be more
| productive. For example, less time trying to find answers
| hidden deep in Stack Overflow as we'll be able to get that
| information directly from ChatGPT.
|
| I can completely see that some smaller places they might not
| require a developer and instead use ChatGPT to write code, but
| it still has to be verified and all the other processes around
| making that code "live", etc.
|
| If anything I'd be more worried if I were a copywriter, as I
| think it's an under appreciated skill and companies may think
| they can get away with ChatGPT and a quick glance over the
| copy.
|
| Either way, I'm positive and look for new ways to help me come
| a more productive and well-rounded programmer.
| jefc1111 wrote:
| I agree with this take, in the main. I think the level of FUD
| elsewhere is probably unwarranted. Though I fully subscribe
| to the idea that it is very disruptive tech.
|
| In the context of software engineering I am tending to see it
| as another layer of abstraction. Once upon a time there was
| perhaps not much above machine code / assembly, but now you
| can have quite a few layers providing abstractions over that
| ending up perhaps at Javascript, or maybe low-code tools. For
| me, AI sits somewhere vaguely in that category (though with a
| much higher level of sophistication).
| deckard1 wrote:
| I'm not at all worried, either. Because I know we already
| went through this. Companies tried to outsource their work to
| India, the Philippines, and elsewhere, and it ended in tears
| and expensive reversals of strategy. They discovered that
| _someone_ has to check all the work.
|
| No one wants to pay for code reviews. They don't show up on
| the agile board and we all collectively pretend they are
| "freebies" the devs give out to the company. Just like unit
| tests and all the other shit that we just expect devs to do
| in their spare time between 3 hour meetings and feature work.
|
| I can see ChatGPT being an augmentation. But that only works
| because you have a human dev that does the merging and can
| take the blame if their code goes wrong. Remove that human
| and you're staring into the abyss.
| carapace wrote:
| > what kinds of roles or areas do we think might be somewhat
| immune to this effect?
|
| On what time-frame?
|
| Permanently immune? You have to postulate that there's
| something a human can do that a computer system just can't. To
| me, given the progress we have already seen, I see no strong
| reason to imagine that there is such a thing. More precisely,
| it becomes a _metaphysical_ question about what it means to be
| human, "What are people for?".
|
| - - - -
|
| In the medium term (and this may only be a few years) the role
| that will come to the fore is that of the human-computer
| psychologist, so to speak. We already talk about "prompt
| engineering". There are two questions: goal and context. What
| do you want the computer to do, and how do you know when it's
| doing it successfully? -and- What are the side-effects, the
| "ecology", of the selected solutions? Especially, otherwise
| unforeseen side-effects.
| Loic wrote:
| For me, the most interesting part is the statistics on all the
| models. These show that 8 bit quantization is basically as good
| as the full model and 4 bit is very close. This is the first time
| I see such table across a large number of models in one place.
| recuter wrote:
| Pretty much.
|
| Llama specific:
|
| https://github.com/qwopqwop200/GPTQ-for-LLaMa
|
| > According to GPTQ paper, As the size of the model increases,
| the difference in performance between FP16 and GPTQ decreases.
|
| https://nolanoorg.substack.com/p/int-4-llama-is-not-enough-i...
|
| https://docs.google.com/document/d/1wZ0g9rHI-6s7ctNlykuK4W5T...
|
| Expect to get away with a factor of 4-5 reduction in memory
| usage for a minimal loss of quality. :)
| superkuh wrote:
| Anyone know what format the models have to be in for use with
| textsynth? I looked at the gpt2 example binary (gpt2_117M.bin)
| and it seems like the "normal" params.json is embedded as a
| header for the binary and then some ascii string like
| "attn/c_attn/" and then the binary weights.
|
| I tried just using the Stanford Alpaca fine-tuned version of the
| llama 7B weights that work with llama.cpp with textsynth but it
| didn't like that (ggml-alpaca-7b-q4.bin: invalid file header).
| Having a textsynth HTTP API would save me a lot of hassle . I'm
| currently wrapping the stdin/out of a execution of a modified
| llama.cpp binary and that's extremely messy.
___________________________________________________________________
(page generated 2023-03-17 23:02 UTC)