[HN Gopher] SymbolicAI: A neuro-symbolic perspective on LLMs
___________________________________________________________________
SymbolicAI: A neuro-symbolic perspective on LLMs
Author : futurisold
Score : 212 points
Date : 2025-06-27 18:49 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| sram1337 wrote:
| This is the voodoo that excites me.
|
| Examples I found interesting:
|
| Semantic map lambdas S = Symbol(['apple',
| 'banana', 'cherry', 'cat', 'dog']) print(S.map('convert all
| fruits to vegetables')) # => ['carrot', 'broccoli',
| 'spinach', 'cat', 'dog']
|
| comparison parameterized by context # Contextual
| greeting comparison greeting = Symbol('Hello, good
| morning!') similar_greeting = 'Hi there, good day!'
| # Compare with specific greeting context result =
| greeting.equals(similar_greeting, context='greeting context')
| print(result) # => True # Compare with different
| contexts for nuanced evaluation formal_greeting =
| Symbol('Good morning, sir.') casual_greeting = 'Hey,
| what\'s up?' # Context-aware politeness comparison
| politeness_comparison = formal_greeting.equals(casual_greeting,
| context='politeness level') print(politeness_comparison) #
| => False
|
| bitwise ops # Semantic logical conjunction -
| combining facts and rules horn_rule = Symbol('The horn only
| sounds on Sundays.', semantic=True) observation = Symbol('I
| hear the horn.') conclusion = horn_rule & observation # =>
| Logical inference
|
| `interpret()` seems powerful.
|
| OP, what inspired you to make this? Where are you applying it?
| What has been your favorite use case so far?
| futurisold wrote:
| That's gonna be a very, very, long answer. What's funny is that
| not much changed since 2022 (eoy) when the projected started;
| the models just got better, but we had a good chunk of the
| primitives since gpt-3.
|
| What's more recent is the DbC contribution which I think is
| unique. It literally solved anything agent related I've thrown
| at it -- especially because I can chain contracts together and
| the guardrails propagate nicely.
|
| I've built most the custom tools myself. For instance, not only
| perplexity was rendered useless by openai's web search, but
| openai's web search itself is not good enough for what you can
| customize yourself. To this end, I've built my own deep
| research agent. Here's a thread with some results from the
| first day it was working:
| https://x.com/futurisold/status/1931751644233945216
|
| I'm also running a company, and we've built an e2e document
| generation just from contracts (3 contracts chained together in
| this case). Here's an output (sorry about the PDF rendering,
| that's not what we serve, it's just what I've quickly hacked
| for local dev): https://drive.google.com/file/d/1Va7ALq_N-
| fTYeumKhH4jSxsTrWD...
|
| This was the input:
|
| ---
|
| Prompt:
|
| > I want the files to be analyzed and I am interested in
| finding patterns; feel free to make suggestions as well. I want
| to understand how different providers use their system prompts,
| therefore things like: what kind of tags do they use - are they
| XML, markdown, etc, are they prone toward sycophancy or trying
| to manipulate the user, are they using tools and if so how,
| etc. I want the tech report to deconstruct and synthesize and
| compare the information, find interesting patterns that would
| be hard to spot.
|
| Generated instructions:
|
| (a) Query: Conduct a comparative analysis of system prompts
| across major AI providers (OpenAI, Google, Anthropic, xAI,
| etc.) to identify structural patterns, linguistic frameworks,
| and operational constraints that shape AI behavior and
| responses.
|
| (b) Specific Questions:
|
| 1. What syntactic structures and formatting conventions (XML,
| markdown, JSON, etc.) are employed across different AI system
| prompts, and how do these technical choices reflect different
| approaches to model instruction?
|
| 2. To what extent do system prompts encode instructions for
| deference, agreeability, or user manipulation, and how do these
| psychological frameworks vary between commercial and research-
| focused models?
|
| 3. How do AI providers implement and constrain tool usage in
| their system prompts, and what patterns emerge in permission
| structures, capability boundaries, and function calling
| conventions?
|
| 4. What ethical guardrails and content moderation approaches
| appear consistently across system prompts, and how do
| implementation details reveal different risk tolerance levels
| between major AI labs?
|
| 5. What unique architectural elements in specific providers'
| system prompts reveal distinctive engineering approaches to
| model alignment, and how might these design choices influence
| downstream user experiences?
|
| ---
|
| Contracts were introduced in March in this post:
| https://futurisold.github.io/2025-03-01-dbc/
|
| They evolved a lot since then, but the foundation and
| motivation didn't change.
| futurisold wrote:
| Btw, besides the prompt, the other input to the technical
| report (the gdrive link) was this repo:
| https://github.com/elder-plinius/CL4R1T4S/tree/main
| futurisold wrote:
| One last comment here on contracts; an excerpt from the
| linked post I think it's extremely relevant for LLMs, maybe
| it triggers an interesting discussion here:
|
| "The scope of contracts extends beyond basic validation. One
| key observation is that a contract is considered fulfilled if
| both the LLM's input and output are successfully validated
| against their specifications. This leads to a deep
| implication: if two different agents satisfy the same
| contract, they are functionally equivalent, at least with
| respect to that specific contract.
|
| This concept of functional equivalence through contracts
| opens up promising opportunities. In principle, you could
| replace one LLM with another, or even substitute an LLM with
| a rule-based system, and as long as both satisfy the same
| contract, your application should continue functioning
| correctly. This creates a level of abstraction that shields
| higher-level components from the implementation details of
| underlying models."
| dogcomplex wrote:
| Anyone interested in this from a history / semiotics /
| language-theory perspective should look into the triad
| concepts of:
|
| Sign (Signum) - The thing which points Locus - The thing
| being pointed to Sense (Sensus) - The effect/sense in the
| interpreter
|
| Also known by: Representation/Object/Interpretation,
| Symbol/Referent/Thought, Signal/Data/User,
| Symbol/State/Update. Same pattern has been independently
| identified many many times through history, always ending
| up with the triplet, renamed many many times.
|
| What you're describing above is the "Locus" essential
| object being pointed to, fulfilled by different
| contracts/LLMs/systems but the same essential thing always
| being eluded to. There's an elegant stability to it from a
| systems design pov. It makes strong sense to build around
| those as the indexes/keys being pointed towards, and then
| various implementations (Signs) attempting to achieve them.
| I'm building a similar system atm.
| futurisold wrote:
| Thanks for bringing this up. I'm fairly familiar with
| Peirce's triadic semiotics and Montague's semantics, and
| they show up in some of my notes. I haven't turned those
| sketches into anything applied yet, but the design space
| feels *huge* and quite promising intuitively.
| VinLucero wrote:
| Agreed. This is a very interesting discussion! Thanks for
| bringing it to light.
|
| Have you read Escher, Bach, Godel: the Eternal Golden
| Braid?
| haileys wrote:
| Why is carrot the vegetablefication of apple?
| herval wrote:
| Also if you run it twice, is it gonna be a carrot again?
| futurisold wrote:
| It's subjected to randomness. But you're ultimately in
| control of the LLMs's hyperparams -- temperature, top_p,
| and seed -- so, you get deterministic outputs if that's
| what you need. However, there are downsides to this kind of
| LLM deterministic tweaks because of the inherent
| autoregressive nature of the LLM.
|
| For instance, with temperature 1 there *could be* a path
| that satisfies your instruction which otherwise gets
| missed. There's interesting work here at the intersection
| of generative grammars and LLMs, where you can cast the
| problem as an FSM/PA automaton such that you only sample
| from that grammar with the LLM (you use something like
| logits_bias to turn off unwanted tokens and keep only those
| that define the grammar). You can define grammars with libs
| like lark or parsimonious, and this was how people solved
| JSON format with LLMs -- JSON is a formal grammar.
|
| Contracts alleviate some of this through post validation,
| *as long as* you find a way to semantically encode your
| deterministic constraint.
| d0100 wrote:
| Since these seem like short prompts, you can send as
| context data that was correct on past prompts
|
| You can create a test suite for your code that will compile
| correct results according to another prompt or dictionary
| verification t.test(
| Symbol(['apple', 'banana', 'cherry', 'cat',
| 'dog']).map('convert all fruits to vegetables'),
| "list only has vegetable and cat,dog" )
| pfdietz wrote:
| Are you asking for the root cause?
| HappMacDonald wrote:
| I think it's interpreting the command as "replace each fruit
| with a vegetable", and it might intuit "make the resulting
| vegetables unique from one another" but otherwise it's not
| trying to find the "most similar" vegetable to every fruit or
| anything like that.
| futurisold wrote:
| This is the correct view. Since the instruction was
| ambiguous, the LLM did its best to satisfy it -- and it
| did.
| lmeyerov wrote:
| You might enjoy Lotus: https://github.com/lotus-data/lotus
|
| It takes all the core relational operators and makes an easy
| semantic version of each as a python dataframe library
| extension . Each call ends up being a 'model' point in case you
| also want to do fancier things later like more learning based
| approaches. Afaict, snowflake and friends are moving in this
| direction for their cloud SQLs as well.
|
| We ended up doing something similar for louie.ai , where you
| use AI notebooks/dashboards/APIs (ex: MCP) to talk to your data
| (splunk, databricks, graph db, whatever), and it'll figure out
| symbolic + semantic operators based on the context. Super
| helpful in practice.
|
| My 80% case here is:
|
| - semantic map: "get all the alerts from splunk index xyz, add
| a column flagging anything suspicious and another explaining
| why" <--- generates an enriched dataframe
|
| - semantic map => semantic reduce: "... then summarize what you
| found" <--- then tells you about it in natural text
| robertkrahn01 wrote:
| Probably linking the paper and examples notebook here makes sense
| as they are pretty explanatory:
|
| https://github.com/ExtensityAI/symbolicai/blob/main/examples...
|
| https://arxiv.org/pdf/2402.00854
| futurisold wrote:
| Wanted to do just that, thank you
| futurisold wrote:
| I didn't expect this -- I was supposed to be sleeping now, but I
| guess I'll chat with whoever jumps in! Good thing I've got some
| white nights experience.
| b0a04gl wrote:
| this works like functional programming where every symbol is a
| pure value and operations compose into clean, traceable flows.
| when you hit an ambiguous step, the model steps in. just like IO
| in FP, the generative call is treated as a scoped side effect.
| this can engage your reasoning graph stays deterministic by
| default and only defers to the model when needed. crazy demo
| though, love it
| futurisold wrote:
| Yes, pretty much. We wanted it be functional from the start.
| Even low level, everything's functional (it's even called
| functional.py/core.py). We're using decorators everywhere. This
| helped a lot with refactoring, extending the framework,
| containing bugs, etc.
| nbardy wrote:
| I love the symbol LLM first approaches.
|
| I built a version of this a few years ago as a LISP
|
| https://github.com/nbardy/SynesthesiaLisp
| futurisold wrote:
| Very nice, bookmarked for later. Interestingly enough, we share
| the same timeline. ~2yo is when a lot of interesting work
| spawned as many started to tinker.
| jaehong747 wrote:
| great job! it reminds me genaiscript.
| https://microsoft.github.io/genaiscript/
|
| // read files
|
| const file = await workspace.readText("data.txt");
|
| // include the file
|
| content in the prompt in a context-friendly way def("DATA",
| file);
|
| // the task
|
| $`Analyze DATA and extract data in JSON in data.json.`;
| futurisold wrote:
| Thank you! I'm not familiar with that project, will take a look
| krackers wrote:
| Some of this seems a bit related to Wolfram Mathematica's
| natural language capabilities.
|
| https://reference.wolfram.com/language/guide/FreeFormAndExte.
| ..
|
| It can (in theory) do very similar things, where natural-
| language input is a first class citizen of the language and
| can operate on other objects. The whole thing came out almost
| a decade before LLMs, I'm surprised that they haven't
| revamped it to make it really shine.
| futurisold wrote:
| > I'm surprised that they haven't revamped it
|
| No worries! I can't find it right now, but Wolfram had a
| stream (or short?) where he introduced "Function". We liked
| it so much we implemented it after one day. Usage: https://
| github.com/ExtensityAI/symbolicai/blob/main/tests/en...
| futurisold wrote:
| Wolfram's also too busy running his TOE exps to focus on
| LLMs (quite sadly if you ask me).
| bjt12345 wrote:
| How did you sort out mapping python constructs to their semantic
| equivalents?
|
| I hope you keep at this, you may be in the right place at the
| right time.
|
| It's getting to the point where some of the LLMs are immediately
| just giving me answers in Python, which is a strong indication of
| what the future will look like with Agents.
| futurisold wrote:
| I'm struggling to understand the question. I'll revisit this
| when I wake up since it's quite late here.
| VinLucero wrote:
| Nice! But have you considered a Neurosymbolic AI that can Evolve?
|
| https://deepwiki.com/dubprime/mythral/3.2-genome-system
|
| Or feel Emotion? https://deepwiki.com/search/how-do-emotives-
| work_193cb616-54...
|
| Have you read Marvin Minsky's Society of Mind?
| afxjzs wrote:
| But is it also explainable or a magic black box?
| futurisold wrote:
| Shortly, yes to all. We actually had an experiment going from
| theory of mind to emotion, but it's hanging right now since I
| feel the models aren't quite there yet and it yields diminish
| returns relative to effort. But could easily be revived. Minsky
| isn't my fav though, I'm leaning more toward Newell/Simon and
| friends from that generation.
| VinLucero wrote:
| @futurisold, would love to collaborate with your team on
| running experiments. We have $300k of GPU credits to burn in
| the next 2 months.
|
| There's only so many cat videos my Agentic AI Army can
| create:
|
| https://youtu.be/upVY1QioDeY?si=H566-_JIm7FmW4u0
| futurisold wrote:
| That's very kind of you, thank you. Let's sync and see if
| we can align on something. You can find me on X, or shoot
| me an email at leo@extensity.ai
| xpitfire wrote:
| We've been working on some exciting things with SymbolicAI and
| here a few things which might interest the HN community.
|
| Two years ago, we built a benchmark to evaluate multistep
| reasoning, tool use, and logical capabilities in language models.
| It includes a quality measure to assess performance and is built
| on a plugin system we developed for SymbolicAI.
|
| - Benchmark & Plugin System:
| https://github.com/ExtensityAI/benchmark
|
| - Example Eval:
| https://github.com/ExtensityAI/benchmark/blob/main/src/evals...
|
| We've also implemented some interesting concepts in our
| framework: - C#-style Extension Methods in Python: Using
| GlobalSymbolPrimitive to extend functionalities.
| - https://github.com/ExtensityAI/benchmark/blob/main/src/func.py#
| L146
|
| - Symbolic <> Sub-symbolic Conversion: And using this for quality
| metrics, like a reward signal from the path integral of multistep
| generations. -
| https://github.com/ExtensityAI/benchmark/blob/main/src/func....
|
| For fun, we integrated LLM-based tools into a customizable shell.
| Check out the Rick & Morty-styled rickshell:
|
| - RickShell: https://github.com/ExtensityAI/rickshell
|
| We were also among the first to generate a full research paper
| from a single prompt and continue to push the boundaries of AI-
| generated research:
|
| - End-to-End Paper Generation (Examples):
| https://drive.google.com/drive/folders/1vUg2Y7TgZRRiaPzC83pQ...
|
| - Recent AI Research Generation: - Three-Body
| Problem: https://github.com/ExtensityAI/three-body_problem
| - Primality Test: https://github.com/ExtensityAI/primality_test
| - Twitter/X Post:
| https://x.com/DinuMariusC/status/1915521724092743997
|
| Finally, for those interested in building similar services, we've
| had an open-source, MCP-like API endpoint service available for
| over a year:
|
| - SymbolicAI API:
| https://github.com/ExtensityAI/symbolicai/blob/main/symai/en...
| GZGavinZhao wrote:
| not to be confused with symbolica.ai
| futurisold wrote:
| +
| bionhoward wrote:
| FYI, there's a correctness issue in the part about correctness
| contracts: valid_opts = ['A', 'B', 'C'] if v not in valid_sizes:
|
| valid_sizes is undefined
| futurisold wrote:
| Oh, thank you; refactoring artifacts. Fixed now.
| nickysielicki wrote:
| I spent some time toying around with LLM-guided "symbolic
| regression", basically having an LLM review documents in order to
| come up with primitives (aka operators) that could be fed into
| github.com/MilesCranmer/PySR
|
| I didn't get very far because I had difficulty piping it all
| together, but with something like this I might give it another
| go. Cool stuff.
| futurisold wrote:
| Oh, definitely. I recommend you go for contracts. I've used
| something similar for a contract that iteratively "stitched
| together" a broken ontology graph. Here's some of the data
| models for inspiration -- you could have something similar for
| your ops, and write the contract to solve for one op, then
| apply the op, etc.
|
| --- class Merge(LLMDataModel):
| indexes: list[int] = Field(description="The indices of the
| clusters that are being merged.") relations:
| list[SubClassRelation] = Field( description="A
| list of superclass-subclass relations chosen from the existing
| two clusters in such a way that they merge." )
| @field_validator("indexes") @classmethod
| def is_binary(cls, v): if len(v) != 2:
| raise ValueError( f"Binary op error:
| Invalid number of clusters: {len(v)}. The merge operation
| requires exactly two clusters." )
| return v class Bridge(LLMDataModel):
| indexes: list[int] = Field(description="The indices of the
| clusters that are being bridged.") relations:
| list[SubClassRelation] = Field( description="A
| list of new superclass-subclass relations used to bridge the
| two clusters from the ontology." )
| @field_validator("indexes") @classmethod
| def is_binary(cls, v): if len(v) != 2:
| raise ValueError( f"Binary op error:
| Invalid number of clusters: {len(v)}. The merge operation
| requires exactly two clusters." )
| return v class Prune(LLMDataModel):
| indexes: list[int] = Field(description="The indices of the
| clusters that are being pruned.") classes:
| list[str] = Field(description="A list of classes that are being
| pruned from the ontology.")
| @field_validator("indexes") @classmethod
| def is_unary(cls, v): if len(v) > 1:
| raise ValueError( f"Unary op error:
| Invalid number of clusters: {len(v)}. The prune operation
| requires exactly one cluster." )
| return v class
| Operation(LLMDataModel): type: Merge | Bridge |
| Prune = Field(description="The type of operation to perform.")
|
| ---
| pkkkzip wrote:
| what are the implicaitons and actual real world application of
| this? better agents? more accurate, debuggable LLM answers?
| futurisold wrote:
| I'd argue it's all of them. Contracts simply make better
| agents. I believe it also gives a very nice bias on how to talk
| about agents -- as apps obeying contracts. If you find time,
| please read this blog post; it gives the underlying motivation
| for using contracts in agent design:
| https://futurisold.github.io/2025-03-01-dbc/
| Aynur4 wrote:
| Hey, I think I am not using this framework to its full
| potential, but I have been using it as my main framework for 2
| years now. Advantages for me are, that I can easily develop
| integrations myself. I like the concept of contracts, that I
| can first make a detailed output model using their LLMDataModel
| class (an extension of pydantic), I can validate every field
| both syntactically and semantically using their semantic
| operations and raise exceptions in case of failures with
| meaningful messages (and those will be considered via the LLM
| for retries). So I like this clean separation of actual
| business logic and all things validation, hallucinations
| checking and I don't have to worry about retries and error
| handling, because that is done by framework as long as I
| provide a good output data model. It makes my code much more
| clean and secure and it also makes it easier for me to analyse
| by just looking at the output model how secure it is. Also I
| liked that I can basically inject any python logic,be
| multimodal without overcomplicating everything with DAGs etc. I
| like having low level control of what is going on. Also once I
| started out building my main issue with most frameworks was
| speed. I wanted to create complex multistep workflows using
| LLMs and still be very fast and I can't really stand when my
| code looks messy. I had a data set of 50 million news and a
| very small compute budget. After some experiments, decided to
| go with symbolicai, because it was easier for me to speed it up
| with batching my agent workflows.
| krackers wrote:
| One question, OP, how does cost for this work? Do you pay the LLM
| inference cost (quite literally if using an external API) every
| time you run a line that involves natural language computation?
| E.g. what happens if you call a "symbolic" function in a loop.
| futurisold wrote:
| Yes, that's correct. If using say openai, then every semantic
| ops are API calls to openai. If you're hosting a local LLM via
| llama.cpp, then obviously there's no inference cost other than
| that of hosting the model.
| demarq wrote:
| This will need a cache of some sort
| thallukrish wrote:
| Since code is generated by LLM these days, how does specific
| syntactic constructs like a Symbol which essentially carries the
| context and can be manipulated with python operators help when
| compared to a normal python code generated by LLM with all the
| checks and balances instructed by a human? For example, I can
| write in this syntax to convert all fruits to vegetables or I can
| simply prompt an LLM to construct a program that takes a list of
| fruits and calls a LLM in the background to return the vegetables
| equivalent. I am trying to understand the difference.
| ijustlovemath wrote:
| Hallucination obstruction, I'd imagine. When you have an LLM
| create a formal system, it can be verified way easier than a
| general purpose one
| thallukrish wrote:
| Yes. That seems to be the case. While it may not be saving
| any time compared to generating general python code vs.
| specific symbolic code, the real value could be that it has
| an engine to enforce the contract on LLM responses with the
| library or even do the calls to the LLM as a common piece of
| code making it less error prone and bringing consistency in
| the interactions with the LLM.
| alexgunnarson wrote:
| Very cool. Being able to use semantic (as opposed to syntactic)
| operators like `==`, `+`, etc. feels like fertilizer for some
| novel ideas. Sort of like when word embeddings first came out and
| there was a loose concept algebra introduced with it ("King - Man
| + Woman = Queen").
|
| That said the neuro + symbolic integration here is, like most
| systems, pretty shallow/firewalled (taxonomically, Type 3 /
| Neuro;Symbolic -- https://harshakokel.com/posts/neurosymbolic-
| systems). I think the real magic is going to come when we start
| heading toward a much more fundamental integration. We're
| actually working on this at my company (https://onton.com). How
| do we create a post-LLM system that: 1) features an integrated
| representation (neither purely symbolic nor dense floating point
| matrix); 2) can learn incrementally from small amounts of noisy
| data, without being subject to catastrophic forgetting; 3) can
| perform mathematical and other symbolic operations with
| bulletproof reliability; and 4) is hallucination-free?
|
| The cobbling together of existing systems hot-glue style is
| certainly useful, but I think a unified architecture is going to
| change everything.
| fhd2 wrote:
| What bums me out a bit - "Symbolic AI" is already well defined:
| https://en.m.wikipedia.org/wiki/Symbolic_artificial_intellig...
| futurisold wrote:
| We hear you. We might end up renaming it. In the paper we have
| a footnote about the name choice -- it's meant to credit the
| foundational work of Newell and Simon that inspired this
| project.
| fhd2 wrote:
| I'd appreciate it! It's cool and I wish you success. Just
| hope that when someone says "We're using Symbolic AI" a year
| from now, it won't be even more ambiguous than today :D
| futurisold wrote:
| Thank you to everyone for participating in the discussion and for
| your overall support! As I said, I didn't expect this. I'm always
| just an email or tweet away, so you know how to reach me. It was
| great talking to you all!
___________________________________________________________________
(page generated 2025-06-28 23:01 UTC)