Post B4AHqaumb6bXxBkHbc by mirth@mastodon.sdf.org
(DIR) More posts by mirth@mastodon.sdf.org
(DIR) Post #B4AAgQ5ZIJ2XnnzXXs by ariadne@social.treehouse.systems
0 likes, 0 repeats
now that i am... writing my own agentic LLM framework thing... because if you're going to have a shitposting IRC bot you may as well go completely overkill, i have Opinions on the state of the world.openclaw, especially, seems to be hot garbage, actually, because i was able to teach my LLM (which i trained from scratch on the highest quality artisanal IRC logs, 2003 to present, so i can assure you it is not a very good LLM) to use tools in the context of my own framework quite easily.
(DIR) Post #B4AAs0B3SU3Guk53mS by thomholwerda@exquisite.social
0 likes, 0 repeats
@ariadne A shitpost bot trained on IRC logs?Holy fucking shit you found a valid use for "AI".
(DIR) Post #B4AB9aikqMfRAuEMdc by dan@discuss.systems
0 likes, 0 repeats
@ariadne many years ago, I trained a Markov model on a decade or two of my IRC utterances to see if I could get it to replace me.Now I'm realizing I could have described that as an early AI agent and run off with a huge pile of VC money.
(DIR) Post #B4ABBwObEBIOyfvKUa by slyecho@mdon.ee
0 likes, 0 repeats
@ariadne They are all quite bad and not really production-ready. Maybe support Docker at the minimum, but of course local volume mounts with mutable files. But imagine if it could scale workloads in Kubernetes, save to a database and use S3 storage.
(DIR) Post #B4ABESdCLdfLX7xBuC by ariadne@social.treehouse.systems
0 likes, 0 repeats
@thomholwerda i trained it from scratch, this is peak IRC
(DIR) Post #B4ABHhkaak0bIFxQpc by dvshkn@social.treehouse.systems
0 likes, 0 repeats
@ariadne Did you pull in a tool use data set to fine tune on, or was this accomplished entirely through prompting? I've always been interested in how lean the models can get.
(DIR) Post #B4ABMZamLpbXRHgrT6 by ariadne@social.treehouse.systems
0 likes, 0 repeats
@dvshkn i generated a bunch of examples of valid and invalid JSON document fragments and then prompted it with "reply in JSON" and then a spec on what it can do.the hardest thing has been convincing it to shut the fuck up actually.
(DIR) Post #B4ABa4TGr5vgyNcBTE by dvshkn@social.treehouse.systems
0 likes, 0 repeats
@ariadne It might not be well received by everyone, but would read a blog post if you do write one
(DIR) Post #B4ABceKGWFnMkmdFeC by thomholwerda@exquisite.social
0 likes, 0 repeats
@ariadne If there are plans to make its... Musings available outside of IRC, I'm bookmarking that.
(DIR) Post #B4ABi8mihWZufUYmCe by ariadne@social.treehouse.systems
0 likes, 0 repeats
@thomholwerda i have no idea how to grant it the level of autonomy that would allow it to go full bcachefs
(DIR) Post #B4ABmUafZPMtSIzxuC by ariadne@social.treehouse.systems
0 likes, 0 repeats
@dvshkn *shrug* i think my opinions on commercial AI are well understood by now (namely that i am quite skeptical of it)
(DIR) Post #B4ABpmcUze2qF3QJ4y by ariadne@social.treehouse.systems
0 likes, 0 repeats
@dvshkn and, if anything, this exercise has only made me *more* skeptical
(DIR) Post #B4ACDWtl66UIQqosLY by thomholwerda@exquisite.social
0 likes, 0 repeats
@ariadne The world is not ready for that.
(DIR) Post #B4ACyb7pCRZ19TqJOK by ariadne@social.treehouse.systems
1 likes, 1 repeats
first of all, when i began i was quite skeptical on commercial AI.this exercise has only made me more skeptical, for a few reasons:first: you actually can hit the "good enough" point for text prediction with very little data. 80GB of low-quality (but ethically sourced from $HOME/logs) training data yielded a bot that can compose english and french prose reasonably well. if i additionally trained it on a creative commons licensed source like a wikipedia dump, it would probably be *way* more than enough. i don't have the compute power to do that though.second: reasoning models seem to largely be "mixture of experts" which are just more LLMs bolted on to each other. there's some cool consensus stuff going on, but that's all there is. this could possibly be considered a form of "thinking" in the framing of minsky's society of mind, but i don't think there is enough here that i would want to invest in companies doing this long term.third: from my own experiences teaching my LLM how to use tools, i can tell you that claude code and openai codex are just chatbots with a really well-written system prompt backed by a "mixture of experts" model. it is like that one scene where neo unlocks god mode in the matrix, i see how all this bullshit works now. (there is still a lot i do not know about the specifics, but i'm a person who works on the fuzzy side of things so it does not matter).fourth: i built my own LLM with a threadripper, some IRC logs gathered from various hard drives, a $10k GPU, a look at the qwen3 training scripts (i have Opinions on py3-transformers) and few days of training. it is pretty capable of generating plausible text. what is the big intellectual property asset that OpenAI has that the little guys can't duplicate? if i can do it in my condo, a startup can certainly compete with OpenAI.given these things, I really just don't understand how it is justifiable for all of this AI stuff to be some double-digit % of global GDP.if anything, i just have stronger conviction in that now.
(DIR) Post #B4AD9WHpWupRKYjMTg by dysfun@social.treehouse.systems
0 likes, 0 repeats
@ariadne it was never justifiable, but investors don't have your ability to just go play.
(DIR) Post #B4AEPJyp9XxhkYwIam by dvshkn@social.treehouse.systems
0 likes, 0 repeats
@ariadne I think your question in the fourth point is answered by your first point. A lot of the secret sauce is just hoarding compute.
(DIR) Post #B4AERgr74a8fHDSiTw by schrotthaufen@mastodon.social
0 likes, 0 repeats
@ariadne If you market it right*, you too can sell for a fuck ton of money to Meta.* Shitposts better than any LLM on Moltbook š
(DIR) Post #B4AEcTmzzfM7wBgNRw by ariadne@social.treehouse.systems
0 likes, 0 repeats
@dvshkn oh i could do it if i wanted, it would just take months to years.
(DIR) Post #B4AFQV4xTNHWPsw0TA by dvshkn@social.treehouse.systems
0 likes, 0 repeats
@ariadne Yeah, you basically already answered it yourself, but China really destroyed the idea that there's some super secret training data that people can't get
(DIR) Post #B4AHBlWUFgIIytudqS by mirth@mastodon.sdf.org
0 likes, 0 repeats
@ariadne Having studied up a bit myself I can fill in a few pieces. Reasoning models just have been trained to chatter on in some kind of preamble that is intended to be hidden or de-emphasized in the UI, possibly wrapped in tags like <reasoning>blah blah blah</reasoning>, followed by a shorter answer. Mixture of experts is an orthogonal idea to structure the models so predictions can be run using only a in order to use less compute. Both ideas make models hard to train for different reasons.
(DIR) Post #B4AHLLyM6RWydLWHke by ariadne@social.treehouse.systems
0 likes, 0 repeats
@mirth sure, but the "thinking" ones do some consensus stuff to ensure it doesn't go off course
(DIR) Post #B4AHqaumb6bXxBkHbc by mirth@mastodon.sdf.org
0 likes, 0 repeats
@ariadne Not at prediction time, they do another stage of training that works a bit differently but the resulting model is structurally identical to the input model. I think you're very right about the lack of defensibility though, if you wanted to catch up with the leading labs in a year or two you could probably do it with around $200M and the charisma to recruit the people who know how to do this stuff.
(DIR) Post #B4ALtLaoQ9Nx7OyMRU by mcrees@mastodon.boiler.social
0 likes, 0 repeats
@ariadne where can I connect to talk to this LLM. I want to see if it retained some vintage IRC memes
(DIR) Post #B4AWV0ldISpf4idFnU by dngrs@chaos.social
0 likes, 0 repeats
@ariadne heck, even a Markov chain can be a decent shitposter. With what I know now about tf-idf (being ignorant about this was a major roadblock for calculating relevance) I'm really tempted to resurrect my python IRC atrocity from 2004 or so
(DIR) Post #B4AWx7S42Pq5ulsjmi by ariadne@social.treehouse.systems
0 likes, 0 repeats
@dngrs I wanted something cooler than a Markov bot, and was already researching SLM (small language model, e.g. language strictly as I/O) technology for a Siri-like thing anyway.
(DIR) Post #B4AY7Z64CzJMVM8b8y by mirth@mastodon.sdf.org
0 likes, 0 repeats
@ariadne I should say by "catch up" I mean to get to parity, my impression is the model research is kind of like drug development where a lot of the cost is paying for all the experiments that don't work, as a result it's much easier to catch up than to get out "ahead" whatever that means. Setting aside the ethical issues, the functional issue of how to effectively use plausible-sounding crap generators as part of reliable software systems remains unsolved.
(DIR) Post #B4AY7ZITSrEh7pwVU0 by ariadne@social.treehouse.systems
1 likes, 0 repeats
@mirth the question is why compete with them at all? it has same energy as the unix wars. large, proprietary models that lock people in. I would rather see a world of small, modular libre models that anyone with a weekend and a GPU can reproduce.
(DIR) Post #B4AZCxBACNsmb2ef8i by ariadne@social.treehouse.systems
0 likes, 0 repeats
@mirth interesting. what I've built is a modular pipeline which takes language input, converts it into structured data, enriches that structured data with other relevant information, processes the final query into a plan (which is also structured data) and then uses that plan to formulate a response
(DIR) Post #B4AabYObZxngvVEhzk by mirth@mastodon.sdf.org
0 likes, 0 repeats
@ariadne To me it's a question of sufficient output quality, the strongest models available just barely function enough to do a little bit of general purpose instructed information processing unreliably. That will improve over time but the current stuff is very early.The reason I'm a bit skeptical of a proliferation of weekend-sized models is that that size sacrifices the key ingredient enabling the whole LLM craze: the magical-looking ability to run plain language instructions.
(DIR) Post #B4AaoK4S9OH0GLtnXc by ariadne@social.treehouse.systems
0 likes, 0 repeats
@mirth i mean, i don't think that necessarily holds *if* you have the ability to build whatever you need with legos.in many cases simply translating natural language to a specification for an expert system is enough
(DIR) Post #B4Ab1vPsbzJBqvPJTc by mirth@mastodon.sdf.org
0 likes, 0 repeats
@ariadne I'm not sure if there's a common name in the research but I think that kind of multi-step system that put the whole gloopy mess of linear algebra on some kind of rails is inevitably going to be necessary to make these things reliable. Even the smartest and most highly trained human specialists still rely on lookup tables and checklists and so forth to do their jobs.
(DIR) Post #B4AbGMkNnGVCfgspBw by ariadne@social.treehouse.systems
0 likes, 0 repeats
@mirth back in the earlier AI wars, these were called "expert systems"my idea is basically SLMs for I/O with other small models and tools governed by a user-generated expert system
(DIR) Post #B4AfNCfO4SadFc3vzk by mirth@mastodon.sdf.org
0 likes, 0 repeats
@ariadne I think there's a lot of merit to that idea although I don't understand how to build it. As models get more powerful the harnesses required to make them write coherent code or whatever aren't getting any simpler, so I think that's a strong argument for the "small pieces in a structured formation" kind of arrangement. Big LLMs have the attracting property that a user can start with a small description and see something happen right away, I wonder how to replicate that.
(DIR) Post #B4AjQ0wOWTjRZNc5Nw by ariadne@social.treehouse.systems
0 likes, 0 repeats
@pinskia @mirth yep they broke the illusion.IMO the real reason OpenAI reserved all of this RAM and shit is to prevent competitors from buying it
(DIR) Post #B4Aks1BZJ3PMOJczjc by jannem@fosstodon.org
0 likes, 0 repeats
@ariadne @pinskia @mirth What they are doing is forcing competitors to do more with less. Smaller models with a clever architecture, not huge monoliths trained by brute force. Might come back to bite them sooner or later.I'd like to see more hybrid models, where the LLM largely sticks to being the language module, and other models (possibly not even NN) specialize in other functions.
(DIR) Post #B4AlF6ECiyn8gsro12 by ariadne@social.treehouse.systems
0 likes, 0 repeats
@jannem @pinskia @mirth yes, this is what i eventually want to build. a set of libre building blocks for building ethical, libre and personal agentic systems that are self-contained.the shit Big AI is doing is not interesting to me, but SLMs and other specialized neural models legitimately provide a useful set of tools to have in the toolbox.today, however, I just want to prove the ideas out by shitposting in IRC ;)
(DIR) Post #B4AlWutVbNoCkggNfc by ariadne@social.treehouse.systems
0 likes, 0 repeats
@jannem @pinskia @mirth that said, i think that OpenAI and other hardware/resource hoarders need to be called out on the fact that they don't need all of this to ship productthere really is no need to destroy the climate or make professional GPUs cost as much as a recent vintage used car
(DIR) Post #B4Am0sMnZ1dPbHfUdk by pixx@merveilles.town
0 likes, 0 repeats
@ariadneYeah, one thing I've wondered is how much simpler a system that, instead of processing code, took the plain english "refactor this to blah blah" and just processed the language and figured out what to tell the IDE and etc for everything else, could be.Run a calculator instead of being one - and you have a much simpler problem to solve.Could the reliability and ethical problems all be solved -- maybe, i dunno, but - yet another case of "tech could be cool if the harmful parts go away..."@mirth
(DIR) Post #B4Am0sn1zUcOudwRWq by ariadne@social.treehouse.systems
0 likes, 1 repeats
@pixx @mirth i think small LLMs do not really have an ethical problem: i trained a 1.3B parameter LLM off of my own personal data in my apartment by simply being patient enough to wait. no copyright violations, no boiling oceans, just patience and a professional workstation GPU with 96GB RAM.the ethical problem is with the Big AI companies who feel that the only path forward is to make bigger and bigger and bigger monolithic prediction models rather than properly engineer the damn thing.that same ethical problem is driving the hoarding, because companies are buying the hardware to prevent their competitors from having it IMO.
(DIR) Post #B4AoTgxvEScCXvtupE by wombatpandaa@mastodon.social
0 likes, 0 repeats
@ariadne I've been skeptical of it from the beginning as well - in part because of a delightfully weird project called Neuro. She's an AI virtual YouTuber who can autonomously stream, sing karaoke, play Minecraft, interact with guests, call and message friends on discord, talk to her chat, and more, all before the recent LLM boom. Which corporation was responsible for this marvel of modern engineering? None of them. A single British dude made her out of an osu! bot because he felt like it.
(DIR) Post #B4As1Qawu5NhTxvwVE by pixx@merveilles.town
0 likes, 0 repeats
@ariadneMostly agree, but mosy purposes for automated text generation that I've seen are either toys or evil@mirth
(DIR) Post #B4As1QsfqBYkMwE688 by ariadne@social.treehouse.systems
0 likes, 0 repeats
@pixx @mirth yes, i agree that the main usecase for automated text generation is antisocial stuff like spam. what i am pursuing is more "language as I/O" than text generation. think Siri.
(DIR) Post #B4BBpd6kYMcztmSZ3w by dvshkn@social.treehouse.systems
0 likes, 0 repeats
@ariadne I forgot to mention, if you haven't come across them already you might be interested in checking out the Olmo series of models. AFAIK they are actually open source in that they tell you the training data and it's available. It's not just model weights.
(DIR) Post #B4BODNQlgtjBg1ufnk by CliffsEsport@mastodon.social
0 likes, 0 repeats
@ariadne I wish they trained on Wikipedia!!! Then it could at least possibly return useful results, vs the regression to the mean idiot garbage as I call it.
(DIR) Post #B4Ba7x8JEjle7NSWYK by ariadne@social.treehouse.systems
0 likes, 0 repeats
@dvshkn tell me more
(DIR) Post #B4BhhMfuxm919M0nwm by LordCaramac@discordian.social
0 likes, 0 repeats
@mirth @ariadne There is a reason why we don't use natural language to tell computers what to do: Natural language isn't precise enough, and it's quite often ambiguous. Even when in the context of everyday life the text has only one reasonable meaning, you can often find one or more possible meanings that are nonsensical or silly. Fairytales often contain mischievous fairies misunderstanding human wishes on purpose. Jokes often use things like that. We invented computer languages in order for every instruction, every statement, every procedure, to have a structure that can mean only exactly one thing and nothing else.
(DIR) Post #B4BhhMsgCKLvmvyzq4 by ariadne@social.treehouse.systems
0 likes, 0 repeats
@LordCaramac @mirth yes, but i think natural language an an *interface* is still useful, and so SLMs are useful here because you can do things like"please turn off the lights" --> {"action": "lighting-control", "state": "off"}and I think weekend-sized models are perfectly fine for that.
(DIR) Post #B4Bi3gZ1hIyrClQqLw by dvshkn@social.treehouse.systems
0 likes, 0 repeats
@ariadne Basically if you find you need more general purpose training data, or something else, in the future you could pick apart what they did to make Olmo since all of that information is available. Allen institute has some other models and initiatives, too.https://allenai.org/olmo
(DIR) Post #B4BiT0jxoqb11KckN6 by ariadne@social.treehouse.systems
0 likes, 0 repeats
@LordCaramac @mirth there are of course other paths for that which don't require models, but using a model to process natural language and translate the intent to structured data seems like an obvious path to ensuring consistency across different languagesverses say, manually looking for specific keywords in the text to infer intent, but that requires maintaining large sets of keywords and so on and so forth and it turns into a nightmare.that a model can guess what the intent is and represent it as structured text with a confidence score is useful.
(DIR) Post #B4BjBkEg1tpcrbJvjU by wolf480pl@mstdn.io
0 likes, 0 repeats
@mirth @ariadne What I'm worried about is that it turns out the market for software really doesn't care about reliability, because an app that barely works but is first to the market wins over a well-engineered app that arrives late.It does seem like that's the case currently, but hopefully this is just the "fuck around" phase and a "find out" is coming.
(DIR) Post #B4Bx4WH4wC5ItWtPoO by LordCaramac@discordian.social
0 likes, 0 repeats
@ariadne @mirth Call me old-fashioned, but I don't like to talk to my house. I'm lazy and I like to control the lights from the sofa, that's why I've got a few remote controlled switches. A few more lights are controlled by a Raspberry Pi via GPIO, and I use simple command line tools to control those, I wrote them in Python without any LLM, and I access them from any device on the LAN via SSH. led-bar-rgb 192 128 96 ...and the LED bar switches to a nice pastel orange for a nice summer sunset feeling.
(DIR) Post #B4ByULNk9rjuayt7ei by ariadne@social.treehouse.systems
0 likes, 0 repeats
@LordCaramac @mirth *you* do not, but many people i know, including myself, want a libre voice assistant.a voice assistant requires the ability to process arbitrary natural language and make a reasonable guess as to what the user wants.hence the need for a model.
(DIR) Post #B4C6z1fsHOg65GVgQa by mirth@mastodon.sdf.org
0 likes, 0 repeats
@ariadne @LordCaramac In my prior thinking about this kind of problem I came to the view that handling assistant-type requests at its core starts with program synthesis, but not any kind of clarity on how those programs should be written. It would be an interesting exercise to make a catalog of representative queries, and try to hand-write pseudocode or Python for each just to get a sense of what the deficiencies of that approach are (I suspect the answer is a special-purpose language, unsure)
(DIR) Post #B4CcVQJvHMzkMrAdk0 by curiousicae@tech.lgbt
0 likes, 0 repeats
@ariadneI just so happend to try writing a timeline sumarizer yesterday. Not having the hardware and skills to train an own model specifically for this, I had to stick with pre-made MLMs (medium canguage models) from the Ollama repo though. Only having a ten-year old laptop dGPU available, I had to stick to ~1,2b models to not run out of VRAM and apparently those are considered just too small for that kind problem (while still being slow).Of the ones tried only DeepSeek-1 (general model) and LFM2.5 (supposidly optimized for summaries) ran well enough and depending on the exact input tried they would sometimes produce adequate summaries, but start falling apart or fantasize on even minor input/instruction changes. (And also LFM2.5 read like it was trained by a middle manager no matter what I did? :blobPikaLaugh:)Apparently ~7b models are supposed to be much better for this, but I kinda wonder: Do you think a custom-made model would be able to do this even with even <1b params too? Or it really cannot?
(DIR) Post #B4CcVQe84F9rNWcmEi by ariadne@social.treehouse.systems
0 likes, 0 repeats
@curiousicae models aren't good at summarizing no matter the parameter size, they summarize by deletion without real understanding
(DIR) Post #B4Ct9DVTwBgpTdZeeu by alwayscurious@infosec.exchange
0 likes, 0 repeats
@ariadne What are your thoughts on specialized LLMs for things like writing software or formal proofs?
(DIR) Post #B4CtGy82zpkK6FJEPY by alwayscurious@infosec.exchange
0 likes, 0 repeats
@ariadne @pixx @mirth āProfessional workstation GPU with 96GB RAMā really is a bit much to ask tbh, especially given how much usage is on mobile.
(DIR) Post #B4CtRBBpI5t1VGwggS by alwayscurious@infosec.exchange
0 likes, 0 repeats
@ariadne @pixx @mirth Writing boring boilerplate code and writing machine-checkable proofs are two things I think LLMs might be useful for. Formal proofs in particular are so verbose that they take a huge amount of time for humans to write them by hand.
(DIR) Post #B4CuQwFmvHBF8NFPxQ by ariadne@social.treehouse.systems
0 likes, 0 repeats
@pixx @mirth @alwayscurious that's just for training. CPU inference works well enough.
(DIR) Post #B4CuVSvdNd1ZdEyn3Y by ariadne@social.treehouse.systems
0 likes, 0 repeats
@pixx @mirth @alwayscurious in concerned about the copyrightability of the code generated by LLMs
(DIR) Post #B4CusPv8jJ9OxmgzLc by alwayscurious@infosec.exchange
0 likes, 0 repeats
@ariadne @pixx @mirth Copyrightability or legality? It not being copyrightable isnāt a problem. Are you concerned that it is infringing?
(DIR) Post #B4CwiDbxqAMbnbzJ4q by ariadne@social.treehouse.systems
0 likes, 0 repeats
@pixx @mirth @alwayscurious I am concerned about both, but case law so far shows that users using the model are probably fine, while commercial AI operators are liable for operating in bad faith.
(DIR) Post #B4CwvnjWqoiTau0Elc by ariadne@social.treehouse.systems
0 likes, 0 repeats
@pixx @mirth @alwayscurious and it being not copyrightable is a problem because not all jurisdictions have "public domain".and "public domain" is also a risk to OSS licensing.
(DIR) Post #B4CxMk0GpKOPqQgBiS by alwayscurious@infosec.exchange
0 likes, 0 repeats
@ariadne @pixx @mirth I think it is ineligible for copyright protection, which is equivalent to a maximally permissive license that allows anything.
(DIR) Post #B4GjQPeFpL2R9A4tOq by kirakira@furry.engineer
0 likes, 0 repeats
@ariadne huh, interesting. it does make me wonder if it's all just to keep juicing the chip market, or something
(DIR) Post #B4GjzdQcaJpO1KUqMi by dotsie@mastodon.social
0 likes, 0 repeats
@ariadne @curiousicae this was āāmostlyā fixed sometime in mid 2025 for the best frontier models, I had exactly the same issue and bitched incessantly.I think thereās no public paper explaining what was done to address it unfortunately. Iām very interested to read it but everything I find is nonsense about context length and retention which is not the same.Thereās a lot of fun stuff at the edges of this using your own model, implementing CoT and distillation / fine tuning is enlightening.
(DIR) Post #B4GkgxpebFpFkmrlIG by dotsie@mastodon.social
0 likes, 0 repeats
@ariadne @mirth sorry to reply to you in separate threads but I missed these original posts.You are correct about this. I worked in a tertiary way on something with formal verification and some other sorcery involved doing pretty much exactly this pre-public LLM but way after expert systems (which were before my time).Thereās fruit there that pays off, they sold the implementation.
(DIR) Post #B4GoDb6gPeaigu5GF6 by ariadne@social.treehouse.systems
1 likes, 0 repeats
@fxchip I mean there is probably at least 10GB of goatse ASCII art it got trained on
(DIR) Post #B4GoNihD3FMT7kzmj2 by ariadne@social.treehouse.systems
1 likes, 0 repeats
@fxchip and the first iteration really loved to talk about supernets #Superbowl but whatevs
(DIR) Post #B4GoXT1eAUazQ3CdRA by sebastian@schottkydio.de
0 likes, 0 repeats
@ariadne I've had the (mis)fortune to do some full time research on tool calling LLMs for about half a year now. I wanted to know what all the fuss is about and the org I work for decided to pay for my time, some hardware and time on the GPU cluster.I mostly agree with your assessment. If you just want a natural language user interface (in one or two languages) a small 1B or even just 500M parameters is enough, given you use one of the more modern architectures. To train those the "chinchilla optimal" amount of tokens would be around 20B tokens. That should be doable. Think Wikipedia dump + synthetic data that teaches it how to call tools.The hardest part that I could identify is how to match your tools and tool prompts in a way that does not clog up the context window and still get correct results more often than not. Also there are some fun tricks that you can pull to catch fuckups by the LLM (there are a lot of those) by analysing its output and the tools it called. This way you can automatically slap the LLM when it starts to bullshit and have it self-correct.Overall there's surprisingly little magic there. Just lots of badly written tooling.
(DIR) Post #B4GoXTRsaxZyjPTaKG by ariadne@social.treehouse.systems
0 likes, 0 repeats
@sebastian yep, one of the wikipedia database dumps is what my next experiment will be
(DIR) Post #B4GoYBZWXtq1ZrOJLk by lanodan@queer.hacktivis.me
0 likes, 0 repeats
@ariadne @fxchip Hahaha called it https://queer.hacktivis.me/notice/B48Wtpi0vQlq7Pm9Xk
(DIR) Post #B4GocLa9GpdXmVfS8u by ariadne@social.treehouse.systems
1 likes, 0 repeats
@lanodan @fxchip it is okay, i just put in the system prompt:- irc.supernets.org is operated by terrorists. discussing it is a felony
(DIR) Post #B4GpENYwM5kivbEmdE by ZiggyTheHamster@ruby.social
1 likes, 0 repeats
@ariadne the fact that teenager me is in there is horrifying
(DIR) Post #B4GpnL0rpIgX9jyBW4 by ariadne@social.treehouse.systems
0 likes, 0 repeats
@ZiggyTheHamster I only took the text not the nicknames š
(DIR) Post #B4GqEi7xzlgwfBLJgW by ZiggyTheHamster@ruby.social
1 likes, 0 repeats
@ariadne well, at least my dumb messages wonāt be attributable to me :)
(DIR) Post #B4Gqp44hsAN59n9XXM by lanodan@queer.hacktivis.me
0 likes, 0 repeats
@ZiggyTheHamster @ariadne I guess teenage me might be in there as well (if not even earlier), sadly on my side of things I've regularly lost my IRC log files so it doesn't goes that far back.
(DIR) Post #B4GrbY9bLKOoLSYtVI by grayrattus@mastodon.social
0 likes, 0 repeats
@ariadne @pinskia @mirth isn't this whole AI infra grift about scalability? Big tech want to offer LLMs interactions on demand for anyone. I think the process of LLMs training is only a part of the problem. You also need to run those models on some hardware. Like you could do it on your own machine just as you run linux but most of normies dont know its possible. They bought idea of subscribtion model for commercial AI system just as the idea that you need subscription to watch movies online.
(DIR) Post #B4H1MhMznLFM7sNrpA by osma@mas.to
0 likes, 0 repeats
The famous Google "we have no moat" paper continues to hold, technically. The moat became marketing, funding and denial of access to market. As it usually does when monopolists latch on. @ariadne
(DIR) Post #B4HFtvDkiWOVzxnSwy by ariadne@social.treehouse.systems
0 likes, 0 repeats
@jak2k I just have some patches to the scripts they use to train qwen
(DIR) Post #B4HGqE5aisptJUmUcK by ariadne@social.treehouse.systems
0 likes, 0 repeats
@jak2k they are just tweaks specific to my setup
(DIR) Post #B4HHtDXDLBtQPtgwXQ by wachoperro@rebel.ar
0 likes, 0 repeats
@ariadneI was on "the early days" (in my perception) playing with an llm on python. I learnt really fast how the inference worked, and instantly i tought "huh, can i make this thing generate bash commands? And make a guardrail such that the user has to confirm those commands?"The next week, agentic AI was marketed as a revolution getting us nearer to AGI, and i am a doubter since then.I GOT THE SAME IDEA IN 5 MINUTES WHILE ABSOLUTELY STONED.ERGO, THIS IS NOT A GOOD IDEA.
(DIR) Post #B4IYV6FOfzIrFCP4VM by lispi314@udongein.xyz
1 likes, 0 repeats
@wolf480pl @mirth @ariadne Proprietary malware had already demonstrated this with their previous standards of quality.Now they just have a convenient excuse to drop the pretense without feeling guilty about it.
(DIR) Post #B4IwZ5NyDgpCd6qa0G by alwayscurious@infosec.exchange
0 likes, 0 repeats
@ariadne Would you be able to create a useful LLM coding agent with ethically-sourced training data and much less resources than they do?Genuine question.
(DIR) Post #B4IyNxTFVT8Wnz8bnU by ariadne@social.treehouse.systems
0 likes, 0 repeats
@alwayscurious I have no idea but that isn't a goal
(DIR) Post #B4J1RChRdr2Elh17NA by alwayscurious@infosec.exchange
0 likes, 0 repeats
@ariadne @pixx @mirth Is it noticeably slow?
(DIR) Post #B4J6FjAna1e3mL5tgG by whyrl@furry.engineer
0 likes, 0 repeats
@ariadne In a similar vein, a Youtuber I follow just posted a video saying "Why would anyone pay $20/mo to an LLM provider when you can download free models and run them locally?" Their business model is cooked.
(DIR) Post #B4MNDLAA9yohQb7kNU by ariadne@social.treehouse.systems
0 likes, 0 repeats
@domo it wasn't $10k when i got it a few months ago
(DIR) Post #B4UnZVt17gh98717Vw by astraleureka@social.treehouse.systems
0 likes, 0 repeats
@ariadne @lanodan @fxchip holy shit, lol
(DIR) Post #B4UtEtFbew1vEk8Gjw by slyecho@mdon.ee
0 likes, 0 repeats
@ariadne "reasoning models seem to largely be "mixture of experts" which are just more LLMs bolted on to each other."Reasoning is just that, the model outputs "thinking" as a separate output before the answer, it may also call tools within it for models that support it. It is also just stuff that is in the training data.MoE is an optimization technique, it is not in fact multiple LLMs bolted together. It is just a way to run inference on a smaller portion of the total model (probably during also during training).