Post AVpSpNmcv5xm9RMw76 by simon@fedi.simonwillison.net
(DIR) More posts by simon@fedi.simonwillison.net
(DIR) Post #AVn83AveDDcJswSGjQ by simon@fedi.simonwillison.net
2023-05-18T21:07:14Z
0 likes, 0 repeats
llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMsI've been putting together a fun suite of CLI tools for working with the OpenAI APIs - here's what I've got so far:https://simonwillison.net/2023/May/18/cli-tools-for-llms/
(DIR) Post #AVnkTtZCfUeO7B7oAq by rvalue@infosec.exchange
2023-05-19T04:17:37Z
0 likes, 0 repeats
@simon Interesting how the framing of the stories is kinda highlighted by the LLM summary.“Nobody at the Pentagon knew where this $3b was going, but it was definitely going” sounds like a story of its own imo.
(DIR) Post #AVpKZgKeZv0AJVL0ZU by StuartGray@mastodonapp.uk
2023-05-19T22:36:23Z
0 likes, 0 repeats
@simon If you haven't already, check out Guidance from Microsoft. Several fun little tools for boosting LLM performance, accuracy, and formatting.I was a slightly gutted to see their acceleration feature as it was an idea I'd also thought of to boost commonly used prompt speed, but I'm glad it exists.Their Token Healing featuer was a new class of problem for me though, and a likely indication that there's a lot of issues with LLMs we've yet to fully explore.https://github.com/microsoft/guidance
(DIR) Post #AVpO0exusV8ir5Cvqq by simon@fedi.simonwillison.net
2023-05-19T23:13:26Z
0 likes, 0 repeats
@StuartGray I have to admit I've been having real trouble figuring out what guidance is doing beyond offering prompt templates
(DIR) Post #AVpOKypIkEv0FErqM4 by simon@fedi.simonwillison.net
2023-05-19T23:16:21Z
0 likes, 0 repeats
@StuartGray is part of it the same basic idea as jsonformer? https://simonwillison.net/2023/May/8/jsonformer/
(DIR) Post #AVpRwZK6JU740xtAtE by StuartGray@mastodonapp.uk
2023-05-19T23:57:08Z
0 likes, 0 repeats
@simon Partly, that's one aspect but it's wider in scope in that it uses a variable templating approach, so it's not limited to json, more like a paramterised prompt, where placeholders are swapped out with param values.The main readme isn't all that clear tbh except on templating. For the other 3 features, I found the notebook links were more useful;https://github.com/microsoft/guidance/blob/main/notebooks/guidance_acceleration.ipynbhttps://github.com/microsoft/guidance/blob/main/notebooks/token_healing.ipynbandhttps://github.com/microsoft/guidance/blob/main/notebooks/pattern_guides.ipynb
(DIR) Post #AVpSpN6RRwDe2bdFMO by StuartGray@mastodonapp.uk
2023-05-20T00:07:03Z
0 likes, 0 repeats
@simon * Acceleration is essentially caching encoded prompts, which is most useful when you reuse a specific prompt a lot. * Token Healing is a fix for an issue with encoding, which can change the underlying meaning in unhelpful ways. The notebook explains this better than I could.* Regex is essentially Token Healing, but guided by a regex instead to constrain the possible LLM outputs for a given paramter to match a regex format.
(DIR) Post #AVpSpNmcv5xm9RMw76 by simon@fedi.simonwillison.net
2023-05-20T00:08:05Z
0 likes, 0 repeats
@StuartGray yeah I still don't get it - I think I need to run it with a hack to intercept calls it makes out to the OpenAI APIs to understand what it's actually doing
(DIR) Post #AVpT13DZCw6fqMX9yy by simon@fedi.simonwillison.net
2023-05-20T00:10:25Z
0 likes, 0 repeats
@StuartGray Aah - it looks like those more advanced features are more like jsonformer in that they don't work against most API-driven LLMs, they're for when you are running one locally instead such that you can customize the token logic
(DIR) Post #AVpWnJFBE5ItJGzWvw by StuartGray@mastodonapp.uk
2023-05-20T00:53:57Z
0 likes, 0 repeats
@simon Yes, and it seems that's what makes the docs most unclear.There's a number of features, all provided through a "unified API", but some of those features are dependant on LLM API support, and unless you're familiar with specific LLM APIs, it's not immediatly obvious what's usable with a given LLM.It seems that in general, most locally hosted LLMs will support all features, whereas OpenAI appears to support only the templating & the control-flow DSL aspects.
(DIR) Post #AWmy3zdp8dHY6hcpIO by simon@fedi.simonwillison.net
2023-06-17T17:07:20Z
0 likes, 0 repeats
I released a major update to my LLM CLI tool today - version 0.4, which adds conversation mode and prompt templates so you can store and re-use interesting prompts: https://llm.datasette.io/en/stable/changelog.html
(DIR) Post #AWnTMtDgKpusGut7NA by simon@fedi.simonwillison.net
2023-06-17T22:58:02Z
0 likes, 0 repeats
You can now install LLM using Homebrew like this: brew install simonw/llm/llmSee docs here: https://llm.datasette.io/en/stable/setup.html#installation
(DIR) Post #AWnecfo14gVInYc0HI by leafmuncher@mastodon.online
2023-06-18T01:04:09Z
0 likes, 0 repeats
@simon brilliant tool, installed it today and will definitely be a part of my daily tookbox.
(DIR) Post #AWnicejhS1bhIVq9dQ by jamwil@fosstodon.org
2023-06-18T01:48:51Z
0 likes, 0 repeats
@simon This is a super handy tool—thank you for making it!
(DIR) Post #AWnue55IAncKkrNygC by shajith@mastodon.social
2023-06-18T04:03:50Z
0 likes, 0 repeats
@simon started using this last week, and loving the flow of updates, thank you! Have you considered an “llm server” type command to spin up a local chatbot/copilot?
(DIR) Post #AWoIDAqF3dNngx34zI by simon@fedi.simonwillison.net
2023-06-18T08:27:43Z
0 likes, 0 repeats
strip-tags 0.4 is now out, with a set of features to make it easier to turn large HTML pages into content suitable for feeding to a LLMhttps://github.com/simonw/strip-tags/releases/tag/0.4I wrote more about strip-tags here: https://simonwillison.net/2023/May/18/cli-tools-for-llms/