Posts by mala@fosstodon.org
(DIR) Post #At38CKMl2XBOuGJVAm by mala@fosstodon.org
0 likes, 0 repeats
#Llamafile is a Mozilla tool that packages a language model in a single executable file that will run on most platforms. It is 100% local and has been optimized to run on slower hardware, from Raspberry Pis to my 8yo laptop. It is based on llama.cpp which supports a plethora of models, not just LLMs: I chose all-minilm because it’s tiny (50MB) and has open code, research papers, and datasets. https://github.com/Mozilla-Ocho/llamafile https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
(DIR) Post #At38CLHTdeYLkAqna4 by mala@fosstodon.org
0 likes, 0 repeats
In #BYOTA, we use #llamafile to calculate *sentence embeddings*.If you don’t know what embeddings are, just think about them as numerical descriptors of your Mastodon statuses, which are closer (as in two cities’ coordinates being close on a map) the more semantically similar their respective statuses are. We’ll get back later to this with a more visual description. If you are interested in embeddings and wanna delve deeper, see https://vickiboykis.com/what_are_embeddings/ by @vicki.
(DIR) Post #At38CMJzlmA4yH2K92 by mala@fosstodon.org
0 likes, 0 repeats
Marimo is a reactive notebook for Python, that is also sharable as an application. It mixes code, markdown and UI widgets in the same interface so you can (1) develop as you’d do with other notebook environments, (2) share it as an application, by hiding the code and only displaying the UI components, (3) allow people to use it as they want, customizing both code and interface within a grid-based UI. Above all, marimo relies on WASM to run everything inside one’s browser.
(DIR) Post #At38CN9OgfHJXh5NGS by mala@fosstodon.org
0 likes, 0 repeats
What does having a WASM-powered notebook mean? Consider #BYOTA: you can download it from its repo, pip-install its deps, and run it locally as any python notebook. But you can also deploy it as HTML+Javascript files, host it somewhere super cheap (coz the server won’t run any of your code), and people will be able to run it in their browser with no need to install anything else! Plus, this will work both with “my” algorithm and whatever alternative you might develop starting from BYOTA’s code.
(DIR) Post #At38CO0DWHWsBVnYau by mala@fosstodon.org
0 likes, 0 repeats
So, what can you do with #BYOTA? The first thing is embeddings visualization. In these pictures you can see a 2D plot of embeddings calculated on four different timelines: home (blue, only people I follow), local (orange, all posts from my instance, which is fosstodon.org), public (red, federated posts from people followed by users of my instance), and the timeline that I got searching for the #gopher hashtag (light blue).
(DIR) Post #At38CP8lHzxTiIntYG by mala@fosstodon.org
0 likes, 0 repeats
You can also plot all embeddings together! In this picture I have selected, analyzed, and annotated a few areas of this map. What i find cool is that semantically similar statuses will always be close to each other regardless of which timeline they appeared in, so you can follow content across different timelines. This is simple to grasp and easy to interpret, as it is based on the actual text present in statuses. And this way of encoding semantic similarity allows us to do many other things.
(DIR) Post #At38CQnD95uMq2jiEq by mala@fosstodon.org
0 likes, 0 repeats
Semantic search is one of them! If you provide a status ID (which you can get by selecting a post in the map) you can look for the statuses (default is top 5) which are most similar, that is closest, to it. You can also just make up a sentence that describes what you are interested in (e.g. the figure shows the closest statuses to “I am a fan of free software and everything opensource”). Note that the results come from different timelines, in this case public, local, and tag/gopher.
(DIR) Post #At38CRWwP4UJ7s8EW8 by mala@fosstodon.org
0 likes, 0 repeats
How could we have a timeline algorithm without post re-ranking? As you see in the figures, by relying on sentence embeddings you can have a content-based re-ranking of a timeline. You basically apply the “style” of a set of statuses to another list, by putting on top those statuses which are overall closer to the ones you provided. For privacy purposes I did not show a re-ranked public timeline: I did it with my own messages, re-ranking them according to the “style” of #gopher and #bookstodon.
(DIR) Post #At38CSMLJxbXhIBHdY by mala@fosstodon.org
0 likes, 0 repeats
“This looks nice, but how heavy is the calculation of these embeddings?”.I embedded 800 statuses using four different models, from the 22M-parameters all-MiniLM to the 7B e5-mistral. I tested them with two different local servers, llamafile and ollama, and two different laptops, my 2016 MacBook Pro with Intel CPU and my 2024 one with M3 Max. The results are shown in the picture. The summary is: it can work on older hardware, and on recent one you will barely notice the calculation overhead.
(DIR) Post #At38CSvnC9xdTElatM by mala@fosstodon.org
0 likes, 0 repeats
All-miniLM took 11 sec on M3, 52 sec on my 8yo laptop. And embeddings are already good! Larger models might provide extra perks (which I have not investigated yet) but at the price of higher compute. Interestingly, despite the fact that both ollama and llamafile are based on llama.cpp, ollama seems to be faster on newer hardware / smaller models, while llamafile becomes a better choice for larger models on older hardware. I used default params for both servers so there is space for improvement.
(DIR) Post #At38CTQHMoLazn1wPY by mala@fosstodon.org
0 likes, 0 repeats
“What data do I have to share to run this?”The answer is: none. The embedding servers run completely offline. And for the marimo notebook, the only remote connections you’ll do are those related to the marimo and WASM dependencies at bootstrap time, and those required to download posts from the mastodon instance. You are always in control of which and how many messages you download before running the algorithm. From then on (embeddings, plots, search, re-ranking) everything runs on your device.
(DIR) Post #At38CTz1He8WjXHgYq by mala@fosstodon.org
0 likes, 0 repeats
This was #BYOTA, thanks for following this long thread! What you see here are some of the next steps I planned: in addition to natural ML extensions, I’d like to see it grow as a tool for people to experiment with different algorithms and easily share them, and for less tech-savvy people to use as easily as possible. For this to be true, I will invest some time in understanding how to bring this to fruition at protocol level, rather than a single application. Stay tuned!
(DIR) Post #Atn3kgtWDxr07iMOf2 by mala@fosstodon.org
0 likes, 0 repeats
@lohang TY for sharing, I am looking to set up my own fediverse instance and I wanted to give #YunoHost a try… it’s great to know you are so happy with it!How I got here: @_elena boosted @tio ‘s message, @yunohost you have a lot of friends 💙(ah btw, I donated regardless of how things will turn out for me, as I want to support #SelfHosting 🙏)
(DIR) Post #AuFeeSjVc93rSaTxZ2 by mala@fosstodon.org
0 likes, 0 repeats
@_elena Yay, count me in! 💙
(DIR) Post #AutHNBmKs4AhKaznuK by mala@fosstodon.org
0 likes, 0 repeats
@_elena Thank you and your team for making this video! Yesterday it was a magical moment: my 12yo kid came back home exactly when the video was about to start, I called him and we watched it together 💙 I am not sure he understood all of it (he’s not a native English speaker) but he definitely got the vibes -and my enthusiasm- right!I am now looking forward to showing the video to my eldest (15yo) son too on Tuesday. I hope this will help both realize how great this community is!
(DIR) Post #AutJjnKe838q51x2ky by mala@fosstodon.org
0 likes, 0 repeats
@_elena Italian, so… yaaaay!!! 🤗
(DIR) Post #AzDqK2PU8zS5XTHl7g by mala@fosstodon.org
0 likes, 0 repeats
@_elena JellyFin rocks! My setup is JellyFin and Tailscale on my home server, JellyFin app (+ Finamp or Monet for music) on mobile devices
(DIR) Post #AzzynjEk7TkPyy0QTI by mala@fosstodon.org
0 likes, 0 repeats
Today is #sfscon25 opening day! 🤩(and I am there in case you want to chat 🙂)#sfscon
(DIR) Post #B0ryGZYXaDbawDIAkq by mala@fosstodon.org
0 likes, 0 repeats
I am quite sure that if you haven’t donated to @bonfire ‘s campaign yet it’s just because you did not know about it yet, so here is a link: https://www.indiegogo.com/en/projects/bonfire/community 🙂In case you are not convinced yet, here’s a slide for you about (and against!) #Enshittification 💩In this moment, I think every project that fights enshittification should deserve attention and support. I myself contributed, and got me an awesome print (limited edition number 19, which as y’all know is binary for 🤟😜)
(DIR) Post #B26mgVTbgf2xs80xdY by mala@fosstodon.org
0 likes, 0 repeats
I myself don’t believe my own plans until they actually happen but…I’ll be at #FOSDEM !I won’t bring a talk (unless someone invites me to improvise one on the spot 😛) but I’ll be there to listen, meet friends, and talk to folks interested in #OpenSource #AI, #agents, and what @MozillaAI is doing.If you are around and want to chat just reach out, or look for me in the #SociaWeb devroom or at the Mozilla stand.