Post B5RVTyuVxphyYoZ8RE by wren6991@types.pl
 (DIR) More posts by wren6991@types.pl
 (DIR) Post #B5RVTyuVxphyYoZ8RE by wren6991@types.pl
       0 likes, 0 repeats
       
       Psst, want to see a funny GitHub issue? https://github.com/anomalyco/opencode/issues/18100
       
 (DIR) Post #B5RVTzAozCkhNOC9r6 by wren6991@types.pl
       0 likes, 0 repeats
       
       If you have heard the buzzword "agentic AI" but avoided finding out what it meant until now:1. Someone figured out an LLM can do JSON RPCs by typing out the JSON token by token.2. The LLM is run in a harness that regexes out the JSON from its output and executes the RPC.3. The response is catted into the LLM's context window, also in the form of JSON that the LLM just reads.4. People connect these harnesses to system shells on their dev machines.5. Fast forward, this is a trillion-dollar industry held together by markdown files asking the LLM to please not curlbash from the internet.
       
 (DIR) Post #B5RVTzR80ZnQBxpBGy by astraleureka@social.treehouse.systems
       0 likes, 0 repeats
       
       @wren6991 this absurdity is on-par with the "claude re-emits vaguely json-shaped output repeatedly until the linter says it's valid" discovery from the recent source leak
       
 (DIR) Post #B5RXnSRMb4EnYXLT4i by wren6991@types.pl
       0 likes, 0 repeats
       
       @astraleureka I don't know what Anthropic do but llama-cpp (open-source inference) apparently does masked decoding for tool calls. It recognises a magic token indicating the start of a tool call and from that point it forces probability to 0 for tokens that don't match an FSM for JSON syntax + tool call schema. This is done at inference level and might not be visible in the Claude leak, which afaik was just the harness.So it's not quite as dumb as I made it sound because the LLM is constrained to only produce syntactically and schematically correct JSON during tool calls. It's still funny that it just... types the JSON though
       
 (DIR) Post #B5RXnSgxf4iMKudvO4 by wren6991@types.pl
       0 likes, 0 repeats
       
       @astraleureka There's a little bit of info here: https://github.com/ggml-org/llama.cpp/blob/master/grammars/README.mdThere is some plumbing to make this match whatever the model is post-trained to emit for tool calls. No idea where that is. The whole file format situation is absolutely fucked in general
       
 (DIR) Post #B5RXnStMuwdgxORpj6 by astraleureka@social.treehouse.systems
       0 likes, 0 repeats
       
       @wren6991 frankly, this is quite as dumb as you made it sound. "in-band signalling is bad" was a lesson learned by telecom developers more than 40 years ago before the advent of Modern Development as we know it. developers of all walks have understood this very basic concept after much blood, sweat and tears, and yet here we are with ANTHROPIC_MAGIC_STRING_TRIGGER_REFUSAL_1FAEFB6177B4672DEE07F9D3AFC62588CCD2631EDCF22E8CCC1FB35B501C9C86 and second-person prompts begging the model to Please Don't Do The Dangerous Thing as if it was self-aware