[HN Gopher] Testing Generative AI for Circuit Board Design
___________________________________________________________________
Testing Generative AI for Circuit Board Design
Author : DHaldane
Score : 206 points
Date : 2024-06-21 16:16 UTC (6 hours ago)
(HTM) web link (blog.jitx.com)
(TXT) w3m dump (blog.jitx.com)
| bottlepalm wrote:
| It'd be interesting to see how Sonnet 3.5 does at this. I've
| found Sonnet a step change better than Opus, and for a fraction
| of the cost. Opus for me is already far better than GPT-4. And
| same as the poster found, GPT-4o is plain worse at reasoning.
|
| Edit: Better at chain of thought, long running agentic tasks,
| following rigid directions.
| stavros wrote:
| Opus is better than GPT-4? I've heard mixed experiences.
| imperio59 wrote:
| That's because the sample size is probably small and for
| niche prompts or topics.
|
| It's very hard to evaluate whether a model is better than
| another, especially doing it in a scientifically sound way is
| time consuming and hard.
|
| This is why I find these types of comments like "model X is
| so much better than model Y" to be about as useful as
| "chocolate ice cream is so much better than vanilla"
| r2_pilot wrote:
| And both flavors have a base flavor of excrement... Still,
| since I started using Claude 3 Opus (and now 3.5 Sonnet) a
| couple of months back, I don't see myself switching from
| them nor stopping use of LLM-based AI tech; it's just made
| me feel like the computer is actually working for and with
| me and even that alone can be enough to get me motivated
| and accomplish what I set out to do.
| skapadia wrote:
| "it's just made me feel like the computer is actually
| working for and with me and even that alone can be enough
| to get me motivated and accomplish what I set out to do."
|
| This is a great way to describe what I've been feeling /
| experiencing as well.
| stavros wrote:
| True, I just tried it for generating a book summary, and
| Sonnet 3.5 was very bad. GPT-4o is equally bad at that ,
| gpt-4-turbo is great.
| netsec_burn wrote:
| This more likely has to do with context length?
| stavros wrote:
| No, all the information is there, but gpt-4o tends to
| produce bullet points
| (https://www.thesummarist.net/summary/the-making-of-a-
| manager...), whereas gpt-4-turbo tends to produce much
| more readable prose (https://www.thesummarist.net/summary
| /supercommunicators/the-...).
| DHaldane wrote:
| It really depends on the type of question, but generally I'm
| between Gemini and Claude these days for most things.
| DHaldane wrote:
| That's an interesting question - I'll take a few pokes at it
| now to see if there's improvement.
| DHaldane wrote:
| Update: Sonnet 3.5 is better than any other model for the
| circuit design and part finding tasks. Going to iterate a bit
| on the prompts to see how much I can push the new model on
| performance.
|
| Figures that any article written on LLM limits is immediately
| out of date. I'll write an update piece to summarize new
| findings.
| CamperBob2 wrote:
| That name threw me for a loop. 'Sonnet' already means
| something to EEs ( https://www.sonnetsoftware.com/ ).
| cjk2 wrote:
| Ex EE here
|
| _> The AI generated circuit was three times the cost and size of
| the design created by that expert engineer at TI. It is also
| missing many of the necessary connections._
|
| Exactly what I expected.
|
| Edit: to clarify this is even below the expectations of a junior
| EE who had a heavy weekend on the vodka.
| shrimp_emoji wrote:
| It's like a generated image with an eye missing but for
| circuits. :D
| cjk2 wrote:
| AI proceeds to use 2n3904 as a thyristor.
|
| AI happy as it worked the first 10ns of the cycle.
| jeffreygoesto wrote:
| Every natural Intelligence knows that you need to reach out
| to a 2N3055 for heavy duty. ;)
| FourierEnvy wrote:
| Why do people think inserting an LLM into the mix will make it
| better than just an evolutionary or reinforcement model
| applied? Who cares if you can talk to it like a human?
| Terr_ wrote:
| Yeah, when the author was writing about that initial query
| about delay-per-unit-length, I'm thinking: "This doesn't tell
| us whether an LLM can apply the concepts, only whether
| relevant text was included in its training data."
|
| It's a distinction I fear many people will have trouble
| keeping in-mind, faced with the misleading eloquence of LLM
| output.
| rzzzt wrote:
| I read an article on evolutionary algorithm-based designs a
| long time ago -- they are effectively indecipherable by humans
| and rely on the imperfections of the very FPGA that they are
| synthesized on, but work great otherwise.
|
| - https://www.damninteresting.com/on-the-origin-of-circuits/
|
| -
| https://www.sciencedirect.com/science/article/abs/pii/S03784...
| dindobre wrote:
| Using neural networks to solve combinatorial or discrete problems
| is a waste of time imo, but I'd be more than happy if somebody
| could convince me of the opposite.
| utkuumur wrote:
| There are recent papers based on diffusion that perform quite
| well. Here's an example of a recent paper
| https://arxiv.org/pdf/2406.01661. I am also working on ML-based
| CO. My approach has a close 1% gap on hard instances with
| 800-1200 nodes and less than 0.1% for 200-300 nodes on Maximum
| Cut, Minimum Independent Set, and Maximum Clique problems. I
| think these are very promising times for neural network-based
| discrete optimization.
| dindobre wrote:
| Thanks, will try to give it a read this weekend. Would you
| say that diffusion is the architectural change that opened up
| CO for neural nets? Haven't followed this particular niche in
| a while
| HanClinto wrote:
| This feels like an excellent demonstration of the limitation of
| zero-shot LLMs. It feels like the wrong way to approach this.
|
| I'm no expert in the matter, but for "holistic" things (where
| there are a lot of cross-connections and inter-dependencies) it
| feels like a diffusion-based generative structure would be
| better-suited than next-token-prediction. I've felt this way
| about poetry-generation, and I feel like it might apply in these
| sorts of cases as well.
|
| Additionally, this is a highly-specialized field. From the
| conclusion of the article:
|
| > Overall we have some promising directions. Using LLMs for
| circuit board design looks a lot like using them for other
| complex tasks. They work well for pulling concrete data out of
| human-shaped data sources, they can do slightly more difficult
| tasks if they can solve that task by writing code, but eventually
| their capabilities break down in domains too far out of the
| training distribution.
|
| > We only tested the frontier models in this work, but I predict
| similar results from the open-source Llama or Mistral models.
| Some fine tuning on netlist creation would likely make the
| generation capabilities more useful.
|
| I agree with the authors here.
|
| While it's nice to imagine that AGI would be able to generalize
| skills to work competently in domain-specific tasks, I think this
| shows very clearly that we're not there yet, and if one wants to
| use LLMs in such an area, one would need to fine-tune for it.
| Would like to see round 2 of this made using a fine-tuning
| approach.
| DHaldane wrote:
| My gut agrees with you that LLMs shouldn't do this well on a
| specialty domain.
|
| But I think there's also the bitter lesson to be learned here:
| many times people say LLMs won't do well on a task, they are
| often surprised either immediately or a few months later.
|
| Overall not sure what to expect, but fine tuning experiments
| would be interesting regardless.
| cjk2 wrote:
| I doubt it'd work any better. Most of EE time I have spent is
| swearing at stuff that looked like it'd work on paper but
| didn't due to various nuances.
|
| I have my own library of nuances but how would you even fine
| tune anything to understand the black box abstraction of an
| IC to work out if a nuance applies or not between it and a
| load or what a transmission line or edge would look like
| between the IC and the load?
|
| This is where understanding trumps generative AI instantly.
| DHaldane wrote:
| I doubt it too, but I notice that I keep underestimating
| the models.
|
| Do you have a challenge task I can try? What's the easiest
| thing I could get an LLM to do for circuit board design
| that would surprise you?
| cjk2 wrote:
| Make two separate signals arrive at exactly the same time
| on two 50 ohm transmission lines that start and end next
| to each other and go around a right hand bend. At 3.8GHz.
|
| Edit: no VSWR constraint. Can add that later :)
|
| Edit 2: oh or design a board for a simple 100Mohm input
| instrumentation amplifier which knows what a guard ring
| is and how badly the solder mask will screw it up :)
| bmicraft wrote:
| It would seem to me that the majority of boards would be
| a lot more forgiving. Are you saying you wouldn't be
| impressed if it could do only say 70% of board designs
| completely?
| AdamH12113 wrote:
| Not the GP, but as an EE I can tell you that the majority
| of boards are not forgiving. One bad connection or one
| wrong component often means the circuit just doesn't
| work. One bad footprint often means the board is
| worthless.
|
| On top of that, making an AI that can regurgitate simple
| textbook circuits and connect them together in reasonable
| ways is only the first step towards a much more difficult
| goal. More subtle problems in electronics design are all
| about context-dependent interactions between systems.
| nurple wrote:
| I hate that this is true. I think ML itself could be
| applied to the problem to help you catch mistakes in
| realtime, like language servers in software eng.
|
| I have experience building boards in Altium and found it
| rather enjoyable; my own knowledge was often a constraint
| as I started out, but once I got proficient it just
| seemed to flow out onto the canvas.
|
| There are some design considerations that would be
| awesome to farm out to genai, but I think we are far from
| that. Like stable-diffusion is to images, the source data
| for text-to-PCB would need to be well-labeled in addition
| to being correllated with the physical PCB features
| themselves.
|
| The part where I think we lose a lot of data in pursuit
| of something like this, is all of the research and
| integration work that went on behind everything that
| eventually got put into the schematic and then laid out
| on a board. I think it would be really difficult to
| "diffuse" a finished PCB from an RFQ-level description.
| cjk2 wrote:
| No because it's hard enough picking up an experienced
| human's designs and work with them. A 70% done board is a
| headache to unwrap. I'd start again.
| nurple wrote:
| This is how I am with software. There's usually a reason
| I'm arriving at 70% done, and it's not often because it's
| well designed and documented...
| DHaldane wrote:
| Right - LLMs would be a bit silly for these cases. Both
| overkill and underkill. Current approach for length
| matching is throw it off to a domain specific solver.
| Example test-circuit:
| https://x.com/DuncanHaldane/status/1803210498009342191
|
| How exact is exactly the same time? Current solver
| matches to under 10fs, and I think at that level you'd
| have to fab it to see how close you get with fiber weave
| skew and all that.
|
| Do you have a test case for a schematic design task?
| cjk2 wrote:
| Yeah. But you need $200k worth of Keysight kit to test
| it.
|
| The point is there's a methodology to solve these
| problems already. Is this better? And can it use and
| apply it?
| LeifCarrotson wrote:
| Really? Most of the time?
|
| I find I spend an enormous amount of time on boring stuff
| like connecting VCC and ground with appropriate decoupling
| caps, tying output pins from one IC to the input pins on
| the other, creating library parts from data sheets, etc.
|
| There's a handful of interesting problems in any good
| project where the abstraction breaks down and you have to
| prove your worth. But a ton of time gets spent on the
| equivalent of boilerplate code.
|
| If I could tell an AI to generate a 100x100 prototype with
| such-and-such a microcontroller, this sensor and that
| sensor with those off-board connectors, with USB power, a
| regulator, a tag-connect header, a couple debug LEDs, and
| break out unused IO to a header...that would have huge
| value to my workflow, even if it gave up on anything analog
| or high-speed. Presumably you'd just take the first pass
| schematic/board file from the AI and begin work on anything
| with nuance.
|
| If generative AI can do equivalent work for PCBs as it can
| do for text programming languages, people wouldn't use it
| for transmission line design. They'd use it for the
| equivalent of parsing some JSON or making a new class with
| some imports, fields, and method templates.
| scld wrote:
| "Looks like you forgot pullups on your i2c lines" would
| be worth a big monthly subscription hahaha.
| oscillonoscope wrote:
| There are schematic analysis tools which do that now just
| based on the netlist
| DHaldane wrote:
| I've found that for speeding up design generation like
| that, most of the utility comes from the coding approach.
|
| AI can't do it itself (yet), and having it call the
| higher level functions doesn't save that much time...
| sweezyjeezy wrote:
| Some research to the contrary [1] - tldr is that they didn't
| find evidence that generative models really do zero shot well
| at all yet, if you show it something it literally hasn't seen
| before, it isn't "generally intelligent" enough to do it
| well. This isn't an issue for a lot of use-cases, but does
| seem to add some weight to the "giga-scale memorization"
| hypothesis.
|
| [1] https://arxiv.org/html/2404.04125v2
| HanClinto wrote:
| > But I think there's also the bitter lesson to be learned
| here: many times people say LLMs won't do well on a task,
| they are often surprised either immediately or a few months
| later.
|
| Heh. This is very true. I think perhaps the thing I'm most
| amazed by is that simple next-token prediction seems to work
| unreasonably well for a great many tasks.
|
| I just don't know how well that will scale into more complex
| tasks. With simple next-token prediction there is little
| mechanism for the model to iterate or to revise or refine as
| it goes.
|
| There have been some experiments with things like speculative
| generation (where multiple branches are evaluated in
| parallel) to give a bit of a lookahead effect and help avoid
| the LLM locking itself into dead-ends, but they don't seem
| super popular overall -- people just prefer to increase the
| power and accuracy of the base model and keep chugging
| forward.
|
| I can't help feeling like a fundamental shift something more
| akin to a diffusion-based approach would be helpful for such
| things. I just want some sort of mechanism where the model
| can "think" longer about harder problems. If you present a
| simple chess board to an LLM or a complex board to an LLM and
| ask it to generate the next move, it always responds in the
| same amount of time. That alone should tell us that LLMs are
| not intelligent, and they are not "thinking", and they will
| be insufficient for this going forward.
|
| I believe Yann LeCun is right -- simply scaling LLMs is not
| going to get us to AGI. We need a fundamental structural
| shift to something new, but until we stop seeing such insane
| advancements in the quality of generation with LLMs (looking
| at you, Claude!!), I don't think we will move beyond. We have
| to get bored with LLMs first.
| pton_xd wrote:
| > If you present a simple chess board to an LLM or a
| complex board to an LLM and ask it to generate the next
| move, it always responds in the same amount of time.
|
| Is that true, especially if you ask it to think step-by-
| step?
|
| I would think the model has certain associations for
| simple/common board states and different ones for
| complex/uncommon states, and when you ask it to think step-
| by-step it will explain the associations with a particular
| state. That "chattiness" may lead it to using more
| computation for complex boards.
| HanClinto wrote:
| > > If you present a simple chess board to an LLM or a
| complex board to an LLM and ask it to generate the next
| move, it always responds in the same amount of time.
|
| > Is that true, especially if you ask it to think step-
| by-step?
|
| That's fair -- there's a lot of room to grow in this
| area.
|
| If the LLM has been trained to operate with running
| internal-monologue, then I believe they will operate
| better. I think this definitely needs to be explored more
| -- from what little I understand of this research, the
| results are sporadically promising, but getting something
| like ReAct (or other, similar structures) to work
| consistently is something I don't think I've seen yet.
| visarga wrote:
| > I just want some sort of mechanism where the model can
| "think" longer about harder problems.
|
| There is such a mechanism - multiple rounds of prompting.
| You can implement diverse patterns (chains, networks) of
| prompts.
| anonymoushn wrote:
| We have 0 y/o/y progress on Advent of Code, for example.
| Maybe we'll have some progress 6 months from now :)
| https://www.themotte.org/post/797/chatgpt-vs-advent-of-code
| DHaldane wrote:
| Have you tried using more 4000x more samples?
|
| https://redwoodresearch.substack.com/p/getting-50-sota-on-
| ar...
| eimrine wrote:
| I like how you called it holistic, it is maybe the first time I
| see this word not in a "bad" context.
|
| What about the topic, it is impossible to synthesize STEM
| things not in the manner an engineer does this. I mean thou
| shalt to know some typical solutions and have all the
| calculations for all what's happening in the schematic being
| developed.
|
| Textbooks are not a joke and no matter who are you - a human or
| a device.
| omgJustTest wrote:
| I asked this question of Duncan Dec 22!
|
| If you are interested I highly recommend this + your favorite
| llm. It does not do everything but is far superior to some
| highly expensive tools, in flexibility and repeatability.
| https://github.com/devbisme/skidl
| HanClinto wrote:
| This tool looks really powerful, thanks for the link!
|
| One thing I've been personally really intrigued by is the
| possibility of using self-play and adversarial learning as a
| way to advance beyond our current stage of imitation-only
| LLMs.
|
| Having a strong rules-based framework to be able to be able
| to measure quality and correctness of solutions is necessary
| for any RL training setup to proceed. I think that skidl
| could be a really nice framework to be part of an RL-trained
| LLM's curriculum!
|
| I've written down a bunch of thoughts [1] on using games or
| code-generation in an adversarial training setup, but I could
| see circuit design being a good training ground as well!
|
| * [1] https://github.com/HanClinto/MENTAT
| hoosieree wrote:
| I agree diffusion makes more sense for optimizing code-like
| things. The tricky part is coming up with a reasonable set of
| "add noise" transformations.
| HanClinto wrote:
| > The tricky part is coming up with a reasonable set of "add
| noise" transformations.
|
| Yes, as well as dealing with a variable-length window.
|
| When generating images with diffusion, one specifies the
| image ahead-of-time. When generating text with diffusion,
| it's a bit more open-ended. How long do we want this
| paragraph to go? Well, that depends on what goes into it --
| so how do we adjust for that? Do we use a hierarchical tree-
| structure approach? Chunk it and do a chain of overlapping
| segments that are all of fixed-length (could possibly be
| combined with a transformer model)?
|
| Hard to say what would finally work in the end, but I think
| this is the sort of thing that YLC is talking about when he
| encourages students to look beyond LLMs. [1]
|
| * [1] https://x.com/ylecun/status/1793326904692428907
| surfingdino wrote:
| > This feels like an excellent demonstration of the limitation
| of zero-shot LLMs. It feels like the wrong way to approach
| this.
|
| There is one posted on HN every week. How many more do we need
| to accept the fact this tech is not what it is sold at and we
| are bored waiting for it get good? I am not say "get better",
| because it keeps getting better, but somehow doesn't get good.
| makk wrote:
| That's a perception and the problem isn't the AI it's human
| nature: 1. every time AI is able to do a thing we move the
| goalposts and say, yeah, but it can't do that other thing
| over there; 2. We are impatient, so our ability to get bored
| tends to outpace the rate of change.
| slg wrote:
| I don't think the problem is moving the goalposts, but
| rather there are no actual goalposts. Advocates for this
| technology imply it can do anything either because they
| believe it will be true in the near future or they just
| want others to believe it for a wide range of reasons
| including to get rich of it. Therefore the general public
| has no real idea what the ideal use cases are for this
| technology in its current state so they keep asking it to
| do stuff it can't do well. It is really no different than
| the blockchain in that regard.
| goatlover wrote:
| The other side of this coin is everyone overhyping what AI
| can do, and when the inevitable criticism comes, they
| respond by claiming the goal posts are being moved.
| Perhaps, but you also told me it could do XYZ, when it can
| only do X and some Y, but not much Z, and it's still not
| general intelligence in the he broad sense.
| refulgentis wrote:
| I appreciate this comment because I think it really
| demonstrates the core problem with what I'll call the
| "get off my lawn >:|" argument, because it's avowedly
| about personal emotions.
|
| It's not "general intelligence", so it's over hyped, and
| They get so whiny about the inevitable criticism, and
| They are ignoring that it's so mindnumbingly boring to
| have people making the excuse that "designed a circuit
| board from scratch" wasn't something anyone thinks or
| claims an LLM should do.
|
| Who told you LLMs can design circuit boards?
|
| Who told you LLMs are [artificial] general intelligence?
|
| I get sick of it constantly being everywhere, but I don't
| feel the need to intellectualize it in a way that blames
| the nefarious ???
| ben_w wrote:
| > Who told you LLMs are [artificial] general
| intelligence?
|
| *waves*
|
| Everyone means a different thing by each letter of AGI,
| and sometimes also by the combination.
|
| I know my opinion is an unpopular one, but given how much
| more general-purpose they are than most other AI, I count
| LLMs as "general" AI; and I'm old enough to remember when
| AI didn't automatically mean "expert level or better",
| when it was a _surprise_ that Kasparov was beaten (let
| alone Lee Sedol).
|
| LLMs are (currently) the ultimate form of "Jack of all
| trades, master of none".
|
| I'm not surprised that it failed with these tests, even
| though it clearly knows more about electronics than me.
| (I once tried to buy a 220 k resistor, didn't have the
| skill to notice the shop had given me a 220 O resistor,
| the resistor caught fire).
|
| I'd still _like_ to call these things "AGI"... except
| for the fact that people don't agree on what the word
| means and keep objecting to my usage of the initials as
| is, so it would't really communicate anything for me to
| do so.
| derefr wrote:
| ML scientists will tell you it can do X and some Y but
| not much Z. But the public doesn't listen to ML
| scientists. Most of what the public hears about AI comes
| from businessmen trying to market a vision to investors
| -- a vision, specifically, of what _their business_ will
| be capable of _five years from bow_ given _predicted
| advancements in AI capabilities in the mean time_ ; which
| has roughly nothing to do with what current models can
| do.
| exe34 wrote:
| how long does it take for a child to start doing surgery?
| publishing novel theorems? how long has the humble
| transformer been around?
| ben_w wrote:
| Wall-clock or subjective time?
|
| I think it would take a human about 2.6 million (waking)
| years to actually _read_ Common Crawl[0]; though obviously
| faster if they simply absorb token streams as direct
| sensory input.
|
| The strength of computers is that transistors are
| (literally) faster than synapses to the degree to which
| marathon runners are faster than continental drift; the
| weakness is they need to, too -- current generation AI is
| only able to be this good due to this advantage allowing it
| to read far more than any human.
|
| How much this difference matters depends on the use-case:
| if AI were as good at learning as we are, Tesla's FSD would
| be level 5 autonomy years ago already, even with just
| optical input.
|
| [0] April 2024: 386 TiB; assuming 9.83 bits per word and
| 250 w.p.m: https://www.wolframalpha.com/input?i=386+TiB+%2F
| +9.83+bits+p...
| TeMPOraL wrote:
| Subjective time doesn't really matter unless something is
| experiencing it. It could be 2.6 million years, but if
| the wall-clock time is half a year, then great - we've
| managed to brute-force some degree of intelligence in
| half a year! And we're at the beginning of this journey;
| there surely are many things to optimize that will
| decrease both wall-clock and subjective training time.
|
| As the saying goes - "make it work, make it right, make
| it fast".
| echelon wrote:
| I'm in awe of the progress in AI images, music, and video.
| This is probably where AI shines the most.
|
| Soon everything you see and hear will be built up through a
| myriad of AI models and pipelines.
| slg wrote:
| > Soon everything you see and hear will be built up through
| a myriad of AI models and pipelines.
|
| It is so bizarre that some people view this as a positive
| outcome.
| goatlover wrote:
| I sincerely hope not. Talk about a dystopian future. That's
| even worse than what social media has become.
| ben_w wrote:
| They already are, when using the meaning of "AI" that I
| grew up with.
|
| The Facebook feed is AI; Google PageRank is AI; anti-spam
| filters are AI; A/B testing is AI; recommendation systems
| are AI.
|
| It's been a long time since computers took over from humans
| with designing transistor layouts in CPUs -- I was hearing
| about the software needing to account for quantum mechanics
| nearly a decade ago already.
| refulgentis wrote:
| There's this odd strain of thought that there's some general
| thing that will pop for hucksters and the unwashed masses,
| who are sheep led along by huckster wolves who won't admit
| LLMs aint ???, because they're profiting off it
|
| It's frustrating because it's infantalizing, it derails the
| potential of an interesting technical discussion (ex. Here,
| diffusion), and it misses the mark substantially.
|
| At the end of the day, it's useful in a thousand ways day to
| day, and the vast majority of people feel this way. The only
| people I see vehemently arguing the opposite seem to assume
| only things with 0 error rate are useful or are upset about
| money in some form.
|
| But is that really it? I'm all ears. I'm on a 5 hour flight.
| I'm genuinely unclear on whats going on that leads people to
| take this absolutist position that they're waiting for ??? to
| admit ??? about LLMs.
|
| Yes, the prose machine didnt nail circuit design, that
| doesn't mean whatever They you're imagining needs to give up
| and accept ???
| ben_w wrote:
| > But is that really it? I'm all ears. I'm on a 5 hour
| flight. I'm genuinely unclear on whats going on that leads
| people to take this absolutist position that they're
| waiting for ??? to admit ??? about LLMs.
|
| Irony: humans think in very black-and-white terms, one
| could even say boolean; conversely LLMs display subtly and
| nuance.
|
| When I was a kid, repeats of Trek had Spock and Kirk
| defeating robots with the liar's paradox, yet today it
| seems like humans are the ones who are broken by it while
| the machines are just going "I understood that reference!"
| refulgentis wrote:
| Excellent point, it really is what it comes down to.
| There's people getting hoodwinked and people hoodwinking
| and then me, the one who sees them for what they are.
| goatlover wrote:
| And yet we still don't have Data or the Holographic
| Doctor.
| ben_w wrote:
| You're demonstrating my point :)
|
| When we get to that level, we're all out of work.
|
| In the meantime, LLMs are already basically as good as
| the scriptwriters made the TNG-VOY era starship computers
| act.
| shermantanktop wrote:
| So what should we make of the presence of actual hucksters
| and actual senior execs who are acting like credulous
| sheep? I see this every day in my world.
|
| At the same time I do appreciate the actual performance and
| potential future promise of this tech. I have to remind
| myself that the wolf and sheep show is a side attraction,
| but for some people it's clearly the main attraction.
| wruza wrote:
| Why should we even?
|
| The problem with everything today is not only that it's
| hype-centric, but that that carries away those who were
| otherwise reasonable. AI isn't any special in this
| regard, it's just "crypto" of this decade.
|
| I see this trend everywhere, in tech, socio, markets.
| Everything is way too fake, screamy and blown out of
| proportion.
| refulgentis wrote:
| The wolves/sheep thing was to indicate how moralizing and
| infantalizing serves as a substitute for actually
| explaining what the problem is, because surely, it's not
| that the prose machine isn't doing circuit design.
|
| I'm sure you see it, I'd just love for someone to pause
| their internal passion play long enough to explain what
| they're seeing. Because I refuse to infantalize, I refuse
| to believe it's just grumbling because its not 100%
| accurate 100% of the time, and doesn't do 100% of
| everything.
| shermantanktop wrote:
| I am literally right now explaining to a senior exec why
| some PR hype numbers about developer productivity from
| genAI are not comparable to internal numbers, because he
| is hoping to say to his bosses that we're doing better
| than others. This is a smart, accomplished person, but he
| can read the tea leaves.
|
| The problem with hype is that it can become a
| pathological form of social proof.
| photonthug wrote:
| I'll play along. The thing that's annoying me lately is
| that session details leaking between chats has been
| enabled as a "feature", which is quickly making ChatGPT
| more like the search engine and social media echo
| chambers that I think lots of us want to escape. It's
| also harmful for the already slim chances of having
| reproducible / deterministic results, which is bad since
| we're using these things for code generation as well as
| rewriting emails and essays or whatever.
|
| Why? Is this naive engineering refusing to acknowledge
| the same old design flaws? Nefarious management fast
| tracking enshittification? Or do users actually want
| their write-a-naughty-limerick goofs to get mixed up with
| their serious effort to fast track circuit design? I
| wouldn't want to appear cynical but one of these
| explanations just makes more sense than the others!
|
| The core tech such as it is is fine, great even. But it's
| not hard to see many different ways that it's already
| spiraling out of control.
| yousif_123123 wrote:
| One downside for diffusion based systems (and I'm very noob in
| this) is that the model won't be able to see it's input and
| output in the same space, therefore wouldn't be able to do
| follow-up instructions to fix things or improve on it. Where as
| an LLM generating html could follow instructions to modify it
| as well. It's input and output are the same format.
| guidoism wrote:
| This reminds me of my professor's (probably very poor)
| description of NP-complete problems where the computer would
| provide an answer that may or may not be correct and you just had
| to check that it was correct and you do test for correctness in
| polynomial time.
|
| It kind of grosses me out that we are entering a world where
| programming could be just testing (to me) random permutations of
| programs for correctness.
| moffkalast wrote:
| Well we had to keep increasing inefficiency somehow, right?
| Otherwise how would Wirth's law continue to hold?
| cushychicken wrote:
| I'm terrified that JITX will get into the LLM / Generative AI for
| boards business. (Don't make me homeless, Duncan!)
|
| They are already far ahead of many others with respect to next
| generation EE CAD.
|
| Judicious application of AI would be a big win for them.
|
| Edit: adding "TL;DRN'T" to my vocabulary XD
| DHaldane wrote:
| I promise that we want to stay a software company that helps
| people design things!
|
| Adding Skynetn't to company charter...
| AdamH12113 wrote:
| The conclusions are very optimistic given the results. The LLMs:
|
| * Failed to properly understand and respond to the requirements
| for component selection, which were already pretty generic.
|
| * Succeeded in parsing the pinout for an IC but produced an
| incomplete footprint with incorrect dimensions.
|
| * Added extra components to a parsed reference schematic.
|
| * Produced very basic errors in a description of filter
| topologies and chose the wrong one given the requirements.
|
| * Generated utterly broken schematics for several simple
| circuits, with missing connections and aggressively-incorrect
| placement of decoupling capacitors.
|
| Any one of these failures, individually, would break the entire
| design. The article's conclusion for this section buries the lede
| slightly:
|
| > The AI generated circuit was three times the cost and size of
| the design created by that expert engineer at TI. It is also
| missing many of the necessary connections.
|
| Cost and size are irrelevant if the design doesn't work. LLMs
| aren't a third as good as a human at this task, they just fail.
|
| The LLMs do much better converting high-level requirements into
| (very) high-level source code. This make sense (it's
| fundamentally a language task), but also isn't very useful.
| Turning "I need an inverting amplifier with a gain of 20" into
| "amp = inverting_amplifier('amp1', gain=-20.0)" is pretty
| trivial.
|
| The fact that LLMs apparently perform better if you literally
| offer them a cookie is, uh... something.
| lemonlime0x3C33 wrote:
| thank you for summarizing the results, I feel much better about
| my job security. Now if AI could make a competent auto router
| for fine pitch BGA components that would be really nice :)
| neltnerb wrote:
| I think the only bit that looked handy in there would be if it
| could parse PDF datasheets and help you sort them by some
| hidden parameter. If I give it 100 datasheets for microphones
| it really should be able to sort them by mechanical height.
| Maybe I'm too optimistic.
|
| The number of times I've had to entirely redo a circuit because
| of one misplaced connection, yeah, none of those circuits
| worked for any price before I fixed every single error.
| DHaldane wrote:
| Agree that PDF digesting was the most useful.
|
| I think Gemini could definitely do that microphone study.
| Good test case! I remember spending 8 hours on DigiKey in the
| bad old times, looking for an audio jack that was 0.5mm
| shorter.
| hadlock wrote:
| As I understand it, PDF digestion/manipulation (and
| particularly translation) has long been a top request from
| businesses, based on conversations I've had with people
| selling the technology, so it doesn't surprise me that
| Gemini excels at this task.
| robxorb wrote:
| Anyone looking for an idea for something potentially
| valuable to make: ingest PDF datasheets and let us
| search/compare etc, across them. The PDF datasheet is
| possibly one of the biggest and most unecessary hurdles to
| electronics design efficiency.
| doe_eyes wrote:
| Yes, this seemed pretty striking to me: the author clearly
| _wanted_ the LLM to perform well. They started with a problem
| for which solutions are pretty much readily available on the
| internet, and then provided a pretty favorable take on the
| model 's mistakes.
|
| But the bottom line is that it's a task that a novice could
| have solved with a Google search or two, and the LLM fumbled it
| in ways that'd be difficult for a non-expert to spot and
| rectify. LLMs are generally pretty good at information
| retrieval, so it's quite disappointing.
|
| The cookie thing... well, they learn statistical patterns.
| People on the internet often try harder if there is a quid-pro-
| quo, so the LLMs copy that, and it slips past RLHF because
| "performs as well with or without a cookie" is probably not one
| of the things they optimize for.
| oscillonoscope wrote:
| I don't know enough about LLMs to understand if its feasible or
| not but it seems like it would be useful to make certain tasks
| hard-coded or add some fundamental constraints on it. Like when
| making footprints, it should always check that the number of
| pads is never less than the number of schematic symbol pins.
| Otherwise, the AI just feels like your worst coworker
| sehugg wrote:
| How does this compare to Flux.ai?
| https://docs.flux.ai/tutorials/ai-for-hardware-design
| built_with_flux wrote:
| flux.ai founder here
|
| Agree with OP that the raw models aren't that useful for
| schematic/pcb design.
|
| It's why we build flux from the ground up to provide the models
| with the right context. The models are great moderators but
| poor sources of great knowledge.
|
| Here are some great use cases:
|
| https://www.youtube.com/watch?v=XdH075ClrYk
|
| https://www.youtube.com/watch?v=J0CHG_fPxzw&t=276s
|
| https://www.youtube.com/watch?v=iGJOzVf0o7o&t=2s
|
| and here a great example of levering AI to go from idea to full
| design https://x.com/BuildWithFlux/status/1804219703264706578
| shrubble wrote:
| Reminds me of this, an earlier expert-system method for CPU
| design, which was not used in subsequent designs for some reason:
| https://en.wikipedia.org/wiki/VAX_9000#SID_Scalar_and_Vector...
| surfingdino wrote:
| Look! You can design thousands of shit appliances at scale! /s
| Terr_ wrote:
| To recycle a rant, there's a whole bunch of hype and investor
| money riding on a very questionable idea here, namely:
|
| "If we make a _really really good_ specialty text-prediction
| engine, it could be able to productively mimic an imaginary
| general AI, and if it can do that then it can productively mimic
| _other_ specialty AIs, because it 's all just intelligence,
| right?"
| ai4ever wrote:
| investor money is seduced by the possibilities and many of the
| investors are in it for FOMO.
|
| few really understand what the limits of the tech are. and if
| it will even unlock the usecases for which it is being touted.
| seveibar wrote:
| I work on generative AI for circuit board design with tscircuit,
| IMO it's definitely going to be the dominant form of
| bootstrapping or combining circuit designs in the near future (<5
| years)
|
| Most people are wrong that AI won't be able to do this soon. The
| same way you can't expect an AI to generate a website in
| assembly, but you CAN expect it to generate a website with
| React/tailwind, you can't expect an AI to generate circuits
| without having strong functional blocks to work with.
|
| Great work from the author studying existing solutions/models-
| I'll post some of my findings soon as well! The more you play
| with it, the more inevitable it feels!
| HanClinto wrote:
| I'd be interested in reading more of your findings!
|
| Are you able to accomplish this with prompt-engineering, or are
| you doing fine-tuning of LLMs / custom-trained models?
| seveibar wrote:
| No fine tuning needed, as long as the target language/DSL is
| fairly natural, just give eg a couple examples of tscircuit
| React, atopile JotX etc and it can generate compliant
| circuits. It can hallucinate imports, but if you give it an
| import list you can improve that a lot.
| maccard wrote:
| > The same way you can't expect an AI to generate a website in
| assembly, but you CAN expect it to generate a website with
| React/tailwind
|
| Can you? Because last time I tried (probably about February) it
| still wasn't a thing
| blueyes wrote:
| See Quilter: https://www.quilter.ai
| kristopolous wrote:
| Just the other day I came up with an idea of doing a flatbed scan
| of a circuit board and then using machine learning and a bit of
| text promoting to get to a schematic
|
| I don't know how feasible it is. This would probably take low
| $millions or so of training, data collection and research to get
| not trash results.
|
| I'd certainly love it for trying to diagnose circuits.
|
| It's probably not really that possible even at higher end
| consumer grade 1200dpi.
| cmbuck wrote:
| This would be an interesting idea if you were able to solve the
| problem of inner layers. Currently to reverse engineer a board
| with more than 2 layers an x-ray machine is required to glean
| information about internal routing. Otherwise you're making
| inferences based on surface copper only.
| kristopolous wrote:
| Maybe not. I scanned a bluetooth aux transceiver yesterday as
| a test of how well a flatbed can pick up details. There's a
| bunch of these on the market and the cheap ones, they are
| more or less equivalent. It's a CSR 8365 based device, which
| you can read from the scan. The industry is generally
| convergent on the major design decisions for some hardware
| purpose for some given time period.
|
| And the devices, in this case, bluetooth aux transceivers,
| they all do the same things. They've even more or less
| converged on all being 3 buttons. When optimizing for cost
| reduction with the commodity chips that everyone is using to
| do the same things, the manufacturer variation isn't that
| vast.
|
| In the same way you can get 3d models from 2d photos because
| you can identify the object based on a database of samples
| and then guess the 3d contours, the hypothesis to test is
| whether with enough scans and schematics, a sufficiently
| large statistical model will be good enough to make decent
| guesses.
|
| If you've got say 40 devices with 80% of the same chips doing
| the same things for the same purpose, a 41st device might
| have lots of guessable things that you can't necessarily
| capture on a cheap flatbed
|
| This will _probably_ work but it 's a couple million away
| from becoming a reality. There's shortcuts that might make
| this a couple $100,000s project (essentially data contracts
| with bespoke chip printers) but I'd have to make those
| connections. And even then, it's just a hobbyist product. The
| chances of recouping that investment is probably zero
| although the tech would certainly be cool and useful. Just
| not "I'll pay you money" level useful.
| amelius wrote:
| Can we have an AI that reads datasheets and produces Spice
| circuits? With the goal of building a library of simulation
| components.
| klysm wrote:
| That's the kind of thing where verification is really hard, and
| things will look plausible even if incorrect.
| ncrmro wrote:
| I had it generate some opencad but never looked into it further.
___________________________________________________________________
(page generated 2024-06-21 23:00 UTC)