[HN Gopher] Scallop - A Language for Neurosymbolic Programming
___________________________________________________________________
Scallop - A Language for Neurosymbolic Programming
Author : andsoitis
Score : 180 points
Date : 2025-03-22 04:45 UTC (18 hours ago)
(HTM) web link (www.scallop-lang.org)
(TXT) w3m dump (www.scallop-lang.org)
| FloorEgg wrote:
| Ever since I learned about category theory and its relationship
| with symbolic reasoning I've suspected that AGI will come from
| elegantly combining symbolic reasoning and probabilistic
| reasoning. This is the first project I've seen that seems to be
| positioned that way. Very cool.
| dr_dshiv wrote:
| When LLMs code in order to reason, isn't that a combination of
| probabilistic reasoning and symbolic reasoning?
| sigmoid10 wrote:
| Neural networks are actually somewhere in between. They don't
| directly operate on symbolic expressions or explicit logical
| rules. And while they rely on probabilistic aspects for
| training (and sometimes for inference), they rely more on
| continuous-valued transformations in extremely high
| dimensional spaces. But if your goal is human-like
| intelligence, they are a pretty good bet, because we know the
| human brain also doesn't perform symbolic reasoning at its
| core and these things only emerge as high-level behaviour
| from a sufficiently complex system. But it also makes neural
| networks (and us too) prone to failure modes that you would
| not see in strictly symbolic reasoning processes.
| mountainriver wrote:
| Yes, this seems to be what the symbolists always forget. We
| don't use symbolism like this, we just have very dense
| neural connections that emerge from scale and approximate
| it
| suddenlybananas wrote:
| That's not really what symbolists actually argue. Read
| Fodor and Pylylshyn 1988 to better understand how
| symbolists view the relationship between symbolic
| representation and connectionist models. What you're
| saying is akin to saying there's no point in doing
| neuroscience or trying to understand neural-networks as
| its just particle physics deep down.
| WhitneyLand wrote:
| So if inventing the airplane how long should one stick
| with a flapping bird wing design?
| nextos wrote:
| That is not a good analogy. Symbolism has given us lots
| of useful things, including SAT/SMT and theorem provers.
| WhitneyLand wrote:
| The point is that while biology has been a great source
| of inspiration toward technical advances it's only a
| guide, it can't guarantee there isn't a better way.
| Xmd5a wrote:
| https://www.brunogavranovic.com/posts/2024-03-13-my-thesis-i...
| nextos wrote:
| http://forestdb.org is quite old and includes some toy examples
| that IMHO elegantly combine symbolic and probabilistic
| reasoning.
| noduerme wrote:
| I'm really confused. Is this metaprogramming in the sense that,
| add_relation and add_rule are using an LLM to make an educated
| guess about what to do based on what went before it? Or is it
| using some deterministic method or heuristic to evaluate those
| terms?
| sgt101 wrote:
| an llm or other nn construct
| revalo wrote:
| This is amazing. I've been looking forward to such a thing for a
| while now.
| awestroke wrote:
| Icky name
| VinLucero wrote:
| If you like scallop, you are gonna love lobster:
|
| https://liby99.github.io/res/papers/lobster.pdf
| sgt101 wrote:
| Thank you.
|
| you seem to be more in the know than me :) Please could you
| just sketch out a few bullets and explain the relationship
| between Scallop and Lobster and what you think is going on?
| alankarmisra wrote:
| I read the paper on Lobster a little bit. Scallop does its
| reasoning on the CPU - whereas Lobster is an attempt to move
| that reasoning logic to the GPU. That way the entire
| neurosymbolic pipeline stays on the GPU and the whole thing
| runs much faster.
| versteegen wrote:
| Unfortunately it doesn't seem to be available yet. Scallop and
| Lobster are both from UPenn, and the Scallop website says "We
| are still in the process of open sourcing Scallop," so I assume
| it's a matter of time.
| integralof6y wrote:
| The scallop source code is in github,
| https://github.com/scallop-lang/
| LoganDark wrote:
| I wish this website explained what neurosymbolic means.
| alankarmisra wrote:
| It's a combination of neural networks and symbolic reasoning.
| You can use a neurosymbolic approach by combining deep learning
| and logical reasoning:
|
| A neural network (PyTorch) detects objects and actions in the
| image, recognizing "Jim" and "eating a burger" with a
| confidence score.
|
| A symbolic reasoning system (Scallop) takes this detection
| along with past data (e.g., "Jim ate burgers 5 times last
| month") and applies logical rules like:
| likes(X, Food) :- frequently_eats(X, Food).
| frequently_eats(Jim, burgers) if Jim ate burgers > 3 times
| recently.
|
| The system combines the image-based probability with past
| symbolic facts to infer: "Jim likely likes burgers" (e.g., 85%
| confidence).
|
| This allows for both visual perception and logical inference in
| decision-making.
| f1shy wrote:
| Also can be used to verify NN decisions. In autonomous
| driving, a NN can make "instinctive" decisions, and a GOFAI
| system can verify they work and don't break civil or physical
| laws. You can have many parallel NN giving recommendations,
| and let a symbolic system take the final decision.
| eternauta3k wrote:
| Is the reasoning strictly downstream of the image
| recognition? Or can prior knowledge impact how objects are
| recognized? E.g. I'm driving on the road at night so the two
| incoming lights are probably a car.
| alankarmisra wrote:
| In your specific example, time of day, weather (foggy,
| sunny, over-cast) along with images of cars with different
| colors, models, makes, from different angles will all be
| training parameters to begin with so the neural net can do
| this on its own without needing specific symbolic
| processing apriori or downstream. Training data input into
| neural nets is usually sanitized and transformed to some
| extent but whether this sanitization / preprocessing
| requires symbolic programming depends on the use case. For
| example, with the car example, you preprocess car images to
| color them differently, hide random sections of it, clip it
| in different ways so only partial sections are showing,
| turn them upside down, introduce fake fog, darken, lighten,
| add people, signs, fire, etc and use each of these images
| for training so that the neural net can recognize cars
| under different situations (even after accidents where they
| are upside down and on fire). Eventually the neural net
| will recognize a car in most circumstances without symbolic
| programming/intervention.
|
| So when would you use symbolic programming? To generate
| quality data for the neural network. For example, maybe the
| neural net reports it read the speed limit to be 1000 km/h
| on a sign because of someone's shenanigans. A symbolic
| programming aid which knows potential legal limits will
| flag this data as potentially corrupt and pass it back to
| the network as such allowing the neural network to take
| more sensible decisions.
| andoando wrote:
| Is this really all they different from writing some functions
| in any language that use a neural net to make these
| predictions?
|
| Why is this a language and not just some say, Java/Rust
| library?
|
| It's interesting but doesnt seem like fundamentally anything
| new.
| JFingleton wrote:
| A bit over my head - but can't Prolog achieve similar results?
| sgt101 wrote:
| Anything can do anything else given enough time and power, but
| I think: no, not without shenanigans. This has primitives for
| interfacing to nn's including foundation models so you can ask
| it (for example) to label images of cats and dogs using clip,
| then you can reason over the results.
|
| So it's intended to combine nn reasoning and logical reasoning
| cleanly.
| nurettin wrote:
| Scallop's examples have syntax for probabilistic programming,
| so probably not.
| gregjw wrote:
| I honestly have no idea what any of this means.
|
| It seems like schizo ramblings to me. But I'm sure there's some
| merit to it.
| sgt101 wrote:
| Let's say you want to program a system that uses a variety of
| neural nets for reasoning about a problem, and you also want to
| use more traditional programmatic reasoning - for example to
| score and rank results. I think this is the kind of language
| that could allow you to do that.
|
| But - it's time to do the tutorials and try and see.
| versteegen wrote:
| Wow, I'm currently reading the Scallop paper, so funny to see it
| posted here!
|
| I really love the concept. This isn't just differentiable
| neurosymbolic declarative probabilistic programming; Scallop has
| the flexibility of letting you use various (18 included) or
| custom provenance semirings to e.g. track "proofs" why a
| relational fact holds, not just assign it a probability. Sounds
| cool but I'm still trying to figure out the practicality.
|
| Also worth pointing out that it seems that a lot of serious
| engineering work has been done on Scallop. It has an interpreter
| and a JIT compiler down to Rust compiled and dynamically loaded
| as a Python module.
|
| Because a Scallop program (can be) differentiable it can be used
| anywhere in an end-to-end learning system, it doesn't have to
| take input data from a NN and produce your final outputs, as in
| all the examples they give (as far as I can see). For example you
| probably could create a hybrid transformer which runs some
| Scallop code in an internal layer, reading/writing to the
| residual stream. A simpler/more realistic example is to compute
| features fed into a NN e.g. an agent's policy function.
|
| The limitation of Scallop is that the programs themselves are
| human-coded, not learnt, although they can implement
| interpreters/evaluators (e.g. the example of evaluating
| expressions).
| alankarmisra wrote:
| I'm wondering if this is a limitation though. If it can be
| learnt from training data, would it not be part of the neural
| network training data? I imagine we use Scallop to bridge the
| gap where we can't readily learn certain rules based on
| available data or perhaps we would prefer to enforce certain
| rules?
| daveguy wrote:
| I'm pretty sure "differentiable" isn't necessary or sufficient
| to create valid and useful code.
| tempodox wrote:
| The tutorial claims that fib(0) = 1, which is wrong.
|
| https://en.wikipedia.org/wiki/Fibonacci_sequence
|
| This one was easy to spot and would have been easy to get right.
| Makes me wonder...
| ginko wrote:
| From the first paragraph in the article you linked:
|
| > Many writers begin the sequence with 0 and 1, although some
| authors start it from 1 and 1[1][2] and some (as did Fibonacci)
| from 1 and 2.
| fib0equal1 wrote:
| From the first paragraph on Wiki:
|
| > Many writers begin the sequence with 0 and 1, although some
| authors start it from 1 and 1[1][2] and some (as did Fibonacci)
| from 1 and 2.
| mark_l_watson wrote:
| I looked at Scallop a year ago and decided that it was not a
| replacement for Prolog - for me.
|
| I may re-evaluate now, thinking of smoother LLM integration as
| well as differentiability.
|
| Has anyone here used Scallop for a large application? I ask
| because in the 1980s I wrote a medium large application in Prolog
| and it was a nice developer experience.
| anonzzzies wrote:
| Not Scallop related but did you try Mercury? It is prolog with
| types and flagging of deterministic functions; the Prolog we
| ported had a very large (... vague, I know) performance boost
| and that is a lot of code. Porting is gradual mostly.
| mark_l_watson wrote:
| I tried Mercury for a short while.
| light_hue_1 wrote:
| The problem with scallop is that no one has ever shown a single
| program that wasn't easier to write without it. Their papers
| usually contain also no examples, and the ones that do are a heck
| of a lot more complicated than asking the model to do it
| directly. The programming languages world let's them get away
| with this because they're excited to be relevant in the LLM
| universe. But I wouldn't accept any of this work if I was their
| reviewer; they need to write real programs and compare against
| baselines. Many of them. And show that their language provides
| any value.
|
| Just look at the examples on their website. All 3 are lame and
| far easier without their language.
|
| It's like publishing that you have a new high performance systems
| language and never including any benchmark. They would be
| rejected for that. Things just haven't caught up in the ML+PL
| world.
| mentalgear wrote:
| I think you misunderstand what a neuro-symbolic programming
| language (like Scallop) is for.
|
| It's not about performance, but safety.
|
| Making safe decisions becomes exponentially more important as
| ML / agents evolve, to avoid "performant" but ultimately
| inefficient/dangerous/wasteful inferences.
| meltyness wrote:
| The SCC example is interesting, I wonder what behavior that gens
| to. Reminds me of Lean, have to suspect it may make the processor
| quite spicy, like Lean. Also don't see clear indication that this
| benefits from heterogeneous compute resources.
|
| Oh, boy, it's written in Rust!
| hnax wrote:
| How does Scallop compare to PyReason
| (https://neurosymbolic.asu.edu/pyreason/)? Are they by and large
| the same, or tailored towards different use cases?
| Xmd5a wrote:
| If you encode a business on top of this, you get differentiable
| management. Metrics disappear behind obscure activation
| vectors/embeddings, cargo cult is not possible anymore,
| everything traced back to measured economic efficiency.
| johnisgood wrote:
| I love its logo (the color scheme, too).
|
| By the way I wish there were more real-life examples, both basic
| and advanced to show what it may be especially useful for, maybe
| even compare it to other languages like Prolog. I expected the
| tutorial to have examples for what "neurosymbolic" means, because
| I am not entirely sure what it means in practice.
| slopeloaf wrote:
| Love to see this! I'm a huge fan of Neurosymbolic methods, but
| more advanced examples might be needed to help convince folks to
| adopt or try Scallop. The three on the page feel very toy. An
| example rooted in NLP or working with an LLM front and centered
| might help
|
| Very pleasant branding though. Great work! :)
| xabush wrote:
| How does Scallop scale on large knowledge bases (KBs) for doing
| probabilistic reasoning? I'm currently working on large KB with ~
| 12M facts and trying to do probabilistic inference on it. So far
| I've been using [cplint](https://friguzzi.github.io/cplint/_build
| /html/index.html) which is based on SWI-Prolog. It works fine for
| toy examples, however, it doesn't finish running for the large KB
| - even after waiting for it for more than a week. Does know any
| Probabilistic Logic Programming (PLP) libraries that are fast and
| scale to large KBs? Preferably in Prolog ecosystem, but not a
| hard requirement.
___________________________________________________________________
(page generated 2025-03-22 23:01 UTC)