[HN Gopher] Raspberry Pi's New AI Hat Adds 8GB of RAM for Local ...
___________________________________________________________________
Raspberry Pi's New AI Hat Adds 8GB of RAM for Local LLMs
Author : ingve
Score : 234 points
Date : 2026-01-15 08:23 UTC (14 hours ago)
(HTM) web link (www.jeffgeerling.com)
(TXT) w3m dump (www.jeffgeerling.com)
| dwedge wrote:
| > In practice, it's not as amazing as it sounds.
|
| 8GB RAM for AI on a Pi sounds underwhelming even from the
| headline
| agent013 wrote:
| A good illustration of how "can run LLM" [?] "makes sense to run
| LLM". A prime example of how numbers in specs don't translate
| into real UX.
| phito wrote:
| Sounds like some PM just wanted to shove AI marketing where it
| doesn't make sense.
| buran77 wrote:
| I think Raspberry lost the magic of the older Pis, they lost that
| sense of purpose. They basically created a niche with the first
| Pis, now they're just jumping into segments that others created
| and are already filled to the brim with perhaps even more
| qualified competition.
|
| Are they seeing a worthwhile niche for the tinkerers (or
| businesses?) who want to run local LLMs with middling performance
| but still need full set of GPIOs in a small package? Maybe. But
| maybe this is just Raspberry jumping on the bandwagon.
|
| I don't blame them for looking to expand into new segments, the
| business needs to survive. But these efforts just look a bit
| aimless to me. I "blame" them for not having another "Raspberry
| Pi moment".
|
| P.S. I can maybe see Frigate and similar solutions driving the
| adoption for these, like they boosted Coral TPU sales. Not sure
| if that's enough of a push to make it successful. The hat just
| doesn't have any of the unique value proposition that kickstarted
| the Raspberry wave.
| ulnarkressty wrote:
| That niche was long taken over by cheap Chinese SBCs, so they
| have to innovate somehow. Their only advantage that remains is
| the community.
| mschuster91 wrote:
| The thing with the Chinese SBCs is that (like _every_ other
| player in the embedded world) they don 't give a flying fuck
| about upstreaming their code to the Linux kernel.
|
| Of course, Raspberry Pi just like everyone else has their
| custom patches, but at least to my knowledge you can use a
| straight Linux kernel and still have a running system.
| ChromaticPanic wrote:
| Or just updates in general. Need documentation? Good luck.
| joe_mamba wrote:
| Yep. RPi foundation lost the plot a long time ago. The original
| RPi was in a league of its own when it launched since nothing
| like it existed and it was cheap.
|
| But now if I want some low power linux PC replacement with
| display output, for the price of the latest RPi 5, I can buy on
| the used market a ~2018 laptop with a 15W quad core CPU, 8GB
| RAM, 256 NVME and 1080p IPS display, that's orders of magnitude
| more capable. And if I want a battery powered embedded ARM
| device for GPIO over WIFI, I can get an ESP32 clone, that's
| orders of magnitude cheaper.
|
| Now RPi at sticker price is only good for commercial users
| since it's still cheaper than the dedicated industrial embedded
| boards, which I think is the new market the RPI company caters
| to. I haven't seen any embedded product company that hasn't
| incorporate RPis in its products they ship, or at least in
| their lab/dev/testing stage, so if you can sell your entire
| production stock to industrial users who will pay top dollar,
| why bother making less money selling to consumers, just thank
| them for all the fish. Jensen Huang would approve.
| haunter wrote:
| Not just laptops but the used enterprise micro PCs from Dell,
| HP, and Lenovo. All the same small form factor with very low
| TDP You can have up to 32 or 64 GB RAMs depending on the CPU,
| dual or even triple disks if you want a NAS etc.
| joe_mamba wrote:
| yeah, depends on what the used market looks like where you
| live. Here I see way more laptops for sale for cheap than
| those enterprise thin clients.
|
| And the thin clients when they are for sale tend to have
| their SSDs ripped out by IT for data security, so then it's
| a hassle to go out and buy and extra SSD, compared to just
| buying a used laptop that already comes with display ,
| keyboard, etc.
| Mashimo wrote:
| I have seen quite a few the size of a Mac Studio / Intel
| Nuc, what are the device called that are the size of a pi?
| qsera wrote:
| >I can buy on the used market a ~2018 laptop with a 15W quad
| core CPU, 8GB RAM, 256 NVME and 1080p IPS display, that's
| orders of magnitude more capable..
|
| But it won't be as reliable, mostly motherboards won't last
| long.
| joe_mamba wrote:
| Don't know what your source is for that, but that's not my
| experience, and i've had dozens of laptops through my hands
| due to my hobby.
|
| The ticking timebomb lemons with reliability or design
| issues, will just die in the first 2-4 years like
| clockwork, but if they've already survived 6+ years without
| any faults, they'll most likely be reliable from then on as
| well.
| ForHackernews wrote:
| Bathtub curve is extremely common
| https://en.wikipedia.org/wiki/Bathtub_curve
| qsera wrote:
| >survived 6+ years without any faults, they'll most
| likely be reliable from then on as well
|
| Ok, let us say they ll last 4 more years, so 10 years
| total lifespan.
|
| A PI would last a lot longer.
| gambiting wrote:
| What makes you think so? Just a feeling? A Vibe?
| qsera wrote:
| About what exactly..
| gambiting wrote:
| That if a laptop is 6 years old it will only last 4 more
| years. Or that a Pi will last more than 10.
| qsera wrote:
| If it is a generic laptop, yes. 10 years is a stretch.
| Components used in the motherboard are probably not high
| quality enough to last more than 10 years. A manufacturer
| does not have an incentive to put high quality stuff
| (that is probably costlier) in a laptop who's only
| selling point is cheap for the "features", and not
| reliability or longevity..
|
| One might get lucky with such a laptop, but I won't count
| on it.
| gambiting wrote:
| Again, is that just a feeling, or do you have some data
| to actually show this. In my experience even old basic
| Acer laptops easily last more than 10 years, probably
| without the battery and married to the charger forever
| now, but they will work fine. But I don't go on the
| internet and tell everyone laptops last most than 10
| years just because I know of a few Acers lasting that.
| Likewise, do you have any statistics on longevity of
| Raspberry Pis.
| boomlinde wrote:
| What makes you assume that the Raspberry Pi is using
| higher quality components?
| qsera wrote:
| One thing is that Raspberry PI have a fewer of them. So
| less chance of one becoming faulty.
|
| Regarding higher quality components, I think the for the
| usecase (I mean the kinds of thing it is supposed to be
| used for) of Raspberry PI, reliability is more important.
|
| This also matches with my experience.
| boomlinde wrote:
| _> Regarding higher quality components, I think the for
| the usecase (I mean the kinds of thing it is supposed to
| be used for) of Raspberry PI, reliability is more
| important._
|
| That you think that reliability is more important for a
| Raspberry Pi usecase than a laptop doesn't somehow
| magically make it a fact that its components are of
| higher quality than your average laptop. You only
| speculate and then speculate further on the basis of your
| original speculation. That's not how you arrive at a
| basis for a factual claim or an estimate.
| jasomill wrote:
| Sure, but $200 on eBay will get you something along the
| lines of a Dell Latitude, with decent build quality,
| cost-optimized more than a flagship workstation-class
| laptop, but certainly not designed to squeeze out the
| last penny at the expense of reliability or repairability
| like the cheapest consumer models.
|
| And if you buy a 5-year-old corporate laptop in very good
| condition with minimal visible wear on the keyboard and
| touchpad, it was likely only used as a desktop
| replacement connected to a dock, so unlikely to have
| suffered abuse not apparent from visual inspection alone.
|
| If you're planning to use it as an actual laptop, price
| out a replacement battery before purchase, as battery
| capacity will degrade over time, even if the laptop is
| exclusively used on AC, so will always be something of a
| crapshoot.
|
| Otherwise, I'd expect the rate of component failure to be
| no higher than for any other lightly-used laptop of
| similar vintage, which is low.
| pixl97 wrote:
| Is it a fanless laptop? Fans aren't any different than
| filters in cars. They fail and need replaced.
| joe_mamba wrote:
| _> let us say they ll last 4 more years_
|
| Why not 50 more years if we're just making up numbers? I
| still have an IBM thinkpad from 2006 in my possession
| with everything working. I also see people with Macbooks
| from the era with the light up apple logo in the wild and
| at DJs.
|
| _> A PI would last a lot longer._
|
| Because you say so? OK, sure.
| qsera wrote:
| In your comment you didn't say Apple computers or
| Thinkpads. Those are different. I was talking about plain
| old vanilla business class laptop (because we are talking
| about raspberry alternative).
| joe_mamba wrote:
| You're contradicting yourself
| qsera wrote:
| I was referring to your original comment
|
| >I can buy on the used market a ~2018 laptop with a 15W
| quad core CPU, 8GB RAM, 256 NVME and 1080p IPS display,
| that's orders of magnitude more capable..
| close04 wrote:
| I have computers that are ~20 years or even more and
| still work fine. My main computer which I just replaced
| is ~14 years old (with some components even older than
| that), was used every single day, and is now a perfectly
| functioning server. I have stacks of SFFs and minipcs
| from eBay going back to 2008 but most from 2012-2015,
| which have been running virtually uninterrupted for a
| decade, and still working fine. I have several laptops
| from different OEMs, business and consumer lines, that
| are as old as 2008 and have been used regularly for at
| least 10 years, all still fine.
|
| I understand what you're saying but _saying_ it isn 't
| enough. There's nothing to support your claim.
| shakna wrote:
| 3-5 years for a cheap laptop [0].
|
| 3-5 years of office use for a Pi. [1]
|
| Sure, there's other numbers to find as well, but I'd
| suggest that they're pretty comparable in the way they
| handle environments. If one would fail, so would the other.
|
| [0] https://pcpatching.com/2025/11/extend-your-pcs-life-
| how-long...
|
| [1] https://raspberrypicase.com/how-long-does-a-raspberry-
| pi-las...
| Mashimo wrote:
| I still use Pis in my 3d printers. Laptop would be too big,
| and a ESP could not run the software. "China clone" might
| work, but the nice part of the pi is the images available. It
| just works(tm)
|
| I'm also currently building a small device with 5"
| touchscreen that can control a midi fx padle of mine. It's
| just so easy to find images, code and documentation on how to
| use the GPIO pins.
|
| Might be niche, but that is just what the Pi excels at. It's
| a board for tinkers and it works.
| joe_mamba wrote:
| Yeah sure, for niche use cases it's the best and only
| choice, but that's why it now commands niche prices ;)
| Mashimo wrote:
| Yeah, Pi 5 2gb is ~20% more expensive compared to pi3b on
| release, factoring in inflation (Both in including VAT
| and local prices)
|
| It's 10 bucks more. -\\_(tsu)_/- Still half the price
| that I see intel NUCs for sale. Which of course are way
| more capable. But still, I don't mind the price that
| much.
|
| I could go with a cheaper alternative, but then AFAIK you
| might have to fiddle with images, kernel and
| documentation. For me that is worth 10 bucks.
| joe_mamba wrote:
| _> Yeah, Pi 5 2gb is ~20% more expensive compared to pi3b
| on release, factoring in inflation (Both in including VAT
| and local prices)_
|
| I don't really care how it compares to past models or
| inflation to justify its price tag. I was just comparing
| to to what you can buy on the used market today for the
| same price and it gets absolutely dunked on in the value
| proposition by notebooks since the modern full spec RPi
| is designed to more of a ARM PC than an cheap embedded
| board.
|
| 60 Euros for 2GB and 100 for 8GB models is kind of a
| ripoff if you don't really need it for a specific niche
| use case.
|
| I think an updated Pi-zero with 2GB RAM and better CPU
| stripped of other bells and whistles for 30 Euros max,
| would be amazing value, and more back to the original
| roots of cheap and simple server/embedded board that made
| the first pi sell well.
| Mashimo wrote:
| A used notebook was also better in price to performance
| 10 years ago, no?
| joe_mamba wrote:
| Yeash, but not as good as an alternative to a PI back
| then, since 8 year old notebooks 10 years ago (so 18 year
| old notebooks today) were too bulky and power hungry to
| be a real alternative. Power bricks were all 90W and CPU
| TDW was 35-45W. But notebooks from the 2018 era (intel
| 8th gen) have quite low power chips that make a good PI
| alternatives nowadays.
|
| The mobile and embedded X86 chips have closed the gap a
| lot in power consumption since the PI first launched.
|
| Now you can even get laptops with broken screens for
| free, and just use their motherboard as a home server
| alternative to a PI. Power consumption will be a bit
| higher, but not enough to offset the money you just saved
| anytime soon.
| kalaksi wrote:
| Laptops are still pretty bulky and power hungry in
| comparison if you're looking for very SFF and passive
| cooling.
| kube-system wrote:
| Pop open a cheap Celeron N laptop, and often, the
| motherboard inside is essentially a passively cooled SBC.
|
| https://www.insidemylaptop.com/wp-
| content/uploads/2018/03/Le...
|
| The big pain with using something like this would mostly
| be the IO and odd form factor.
| joe_mamba wrote:
| You can passively cool a laptop CPU too if that's your
| jam. Plus, latest RPI 5 cases now feature fans to not
| thermal throttle :))
| jasomill wrote:
| You can get a 5-year-old laptop with a perfectly working
| screen for free if you're on good terms with the owner of
| a company who has a stack of them sitting in a storage
| closet waiting for disposal. :)
|
| Which is basically just cutting out the middlemen in a
| transaction that might cost $100 on eBay.
|
| Used corporate laptops are particularly cost-effective if
| you're interested in running Windows, as unlike Intel
| NUCs and most SBC products, they typically include
| hardware-locked Windows 10 Pro licenses which can be
| upgraded to Windows 11 Pro for free.
| joe_mamba wrote:
| 5-year old laptops for free aren't really a thing in most
| of Europe unless maybe you're in Norway or some super
| rich country where 100$ is pocket change. In most large
| places I worked in Europe, laptops are leased from a
| service provider, not owned by the company. When they're
| obsolete they get sold in bulk locally or abroad. But
| never for free.
| ssl-3 wrote:
| That comparison was true back in 2012 when the first
| version was released, too.
|
| Things like _used_ PCs and forgotten closet laptops were
| running circles around _brand-new_ Raspberry Pi systems,
| in performance per dollar, for as long as we 've had new
| Raspberry Pis to make that comparison with.
|
| Those first Pis didn't even have wifi, and they were as
| picky about power supplies and stuff back then as a Pi 5
| is today.
|
| The primary aspects that are new are that the featureset
| of new models continues to improve, and the price of a
| bare board has increased by an inflation-adjusted ~$10.
|
| (Meanwhile: A bare Pi 3B still costs $35 right now --
| same as in 2016. When adjusted for inflation, it has
| become cheaper. $35 in 2016 is worth about $48 today.)
| hypeatei wrote:
| > Yeah, Pi 5 2gb is ~20% more expensive compared to pi3b
|
| What prices are you using for the 3b and 5 to get this
| percentage? The _lowest_ percentage I got from available
| data is a 57% increase ($35 - > $55)
| Mashimo wrote:
| I added inflation.
|
| 40 EUR form 2016 is now ~52 EUR.
|
| Compared to 62 EUR for the current model.
| pixl97 wrote:
| Are you adding in the correct 'RAM' inflation being that
| it's costs are up dramatically?
| Mashimo wrote:
| I just looked up the current local prices I can buy a
| unit for.
| jasomill wrote:
| The main things I use Pis for are
|
| 1. Testing images to be deployed on customer Pis.
|
| 2. Testing software on ARM64 Linux. Pis are still cheaper
| than used Apple Silicon Macs, and require less fiddling
| to run Linux. I currently have a free Oracle Cloud
| instance that would work just as well for this, but it
| could go away at any time and it's a PITA to reprovision.
|
| 3. Running Mathematica, because it's free on Pi, I only
| use it a few times a year, and a fully-loaded Pi 5 is
| cheaper than a single-year personal license to run it on
| any other platform.
|
| 4. Silly stuff like one Pi 3 I have set up to emulate a
| vintage IBM mainframe.
| 05 wrote:
| You can run Klipper on any Linux SBC with a USB port, RPi
| works but so does an old router that supports OpenWRT, a
| cheap Android TV box that could be flashed to run Linux, or
| any of the OrangePi/Banana Pi/Alliwinner H3 boards. You
| don't really need hardware UART because most of the printer
| boards you'd be using have either native USB or USB to UART
| converters. For that pedal, would an old Android tablet
| that supports USB OTG work? Because that's got to be much
| cheaper, and with much better SDK.
| Mashimo wrote:
| Correct. But when I looked into it a few years back fir
| OrangePi it was not as easy as downloading raspbian. All
| the images made for the pi would not work, you had to
| download a kernel from another place or something like
| that? Sorry I don't remember the details, but it was not
| as easy as a pi.
|
| How much cheaper then 50 bucks can a tablet get? With the
| pi I can quickly in a hacky way connect rotary encoders
| with female-female dupon cables, use a python GPIO
| library made for raspberry pi.
|
| https://media.discordapp.net/attachments/1461079634354639
| 132...
|
| I can also use it for Zynthian. And if I'm done with it,
| I can build a new printer :P
| cyberax wrote:
| RPi will still have lower power consumption and is far more
| compact. And mechanically reliable.
|
| I'm in the market to replace my aging Intel NUCs, but RPi is
| still cheaper.
| noodlesUK wrote:
| I agree completely - the NUC segment has a gaping hole post
| 2023, and faster raspberry pis can probably fill a lot of
| it especially for small scale commercial stuff.
| nirav72 wrote:
| >the NUC segment has a gaping hole post 2023
|
| There are dozens and dozens of NUC style / form factor
| machines available these days. Especially cheap ones from
| China. Not sure what you mean by gaping hole post 2023.
| I'm running 3 of them with N97 and N150 Cpus. All bought
| within the last 18 months.
| joe_mamba wrote:
| _> And mechanically reliable._
|
| What moving parts do competitors have to be less
| mechanically reliable?
|
| In fact, a NUC or used laptop would be even more reliable
| since you can replace NVME storage and RAM sticks. If your
| RPI ram goes bad you're shit out of luck.
|
| _> RPi will still have lower power consumption and is far
| more compact._
|
| Not that big of on an issue in most home user cases as a
| home server, emulator or PC replacement. For industrial
| users where space, power usage and heat is limited,
| definitely.
|
| _> I'm in the market to replace my aging Intel NUCs, but
| RPi is still cheaper._
|
| Cheaper if you ignore much lower performance and
| versatility vs a X86_X64 NUC as a home server.
| kalaksi wrote:
| It feels like you think that the parent hasn't really
| considered their options and don't know what they really
| want.
|
| > Not that big of on an issue in most home user cases as
| a home server
|
| I don't know what "most home users" want, but I can
| understand wanting something more compact and efficient
| (also easier to keep cool in tighter or closed spaces),
| even at home.
|
| > Cheaper if you ignore much lower performance and
| versatility vs a X86_X64 NUC as a home server.
|
| Or maybe they noticed they don't need all the performance
| and versatility. Been there. It's plenty versatile and
| can run everything I need.
| cyberax wrote:
| I don't want a used laptop. I have my NUC mounted inside
| a small enclosure on a bracket, with PoE for power. It's
| a single-purpose device, only used for HomeAssistant and
| nothing else. It also has to be located centrally in my
| house for better ZigBee/ZWave networking.
|
| Unfortunately, it's close to dying. The heat from the CPU
| disintegrated the plastic of the SATA cable header on the
| motherboard. I fixed it for now with a bit of glue, but
| it's not going to hold indefinitely. And NUCs were pretty
| pricey.
|
| RPi with a SATA/M.2 disk and a PoE hat is not that much
| cheaper than Intel, but it uses much less power. They
| also tend to not have cables that are kept under
| mechanical strain. I have a single-purpose RPi that's
| been running since 2014, and it's doing just fine.
| nl wrote:
| I got a fanless Celeron N4020 4GB RAM 64G Storage new on
| Amazon for under $150 in Jan 2025, and it has been running
| home assistant ever since.
|
| I don't think I could a RPi as cheaply once parts and power
| supply etc are taken into account.
| overfeed wrote:
| > I don't think I could a RPi as cheaply once parts and
| power supply etc are taken into account
|
| The RPi Zero 2W costs $15 and runs HA just fine. One can
| splurge on a pricey case, microSD, and high-amp GAN
| charger, and still be under 50% of your spend. You don't
| _have_ to buy the flagship RPi.
| Dr4kn wrote:
| 500MB is just not enough RAM for a good HA experience. I
| would go with at least 2GB. If you have a few add-ons
| running I would go with 4 or more.
|
| If it's an option I would always go with an SSD for HA.
| It makes a big difference in usability. Writing often and
| a lot to SD cards, like HA does, kills them way too fast
| overfeed wrote:
| > 500MB is just not enough RAM for a good HA experience
|
| Strong disagree: my experience has been great, my HA has
| been running on a Zero 2W for more than 2 years! I have
| several HACS plugins enabled - just not any if the video
| or AI stuff. The same Pi concurrently runs PiHole. For a
| while, it also acted as a git mirror via iSCSI-backed
| Gitea, but I had to migrate Gitea off of it since it was
| memory-hungry. You can do a _lot_ with 500MB in headless
| mode.
| zer00eyz wrote:
| Its not
|
| Go price out a used 1l form factor PC.
|
| After you buy a case, and a real disk, the pi, cost savings
| is gone.
|
| Meanwhile you can pick up a used 8th gen intel 1L form
| factor for about 100 bucks. You can pick up one that will
| take a PICE card for 150ish bucks, with remote management.
|
| The 8th gen or better intel has all sorts of extra features
| that may make it worth while (transcoding/video support).
| cyberax wrote:
| Do you have actual links for that? I looked at small
| form-factor PCs and I have not found anything cheaper
| than RPis.
| u8080 wrote:
| There are also N100/N150 SBC offers which are superior to RPi
| systemtest wrote:
| In The Netherlands the first generation RPi was only sold to
| users with a Chambers of Commerce registration, I figured
| this was always the typical end user for it. Like schools,
| universities, prototyping for companies. Was the RPi in the
| rest of the world targeted towards home users?
|
| * https://tweakers.net/nieuws/80350/verkoop-goedkoop-arm-
| syste...
| firesteelrain wrote:
| Yes, anyone can buy it.
| microtonal wrote:
| What? I ordered the original Pi in May 2012 from
| Farnell/Element14 without a Chamber of Commerce
| registration (KvK nummer). A couple of my colleagues did
| too.
| tirant wrote:
| Same here. Also bought one in NL back then without
| issues.
| systemtest wrote:
| That was only possible with a KvK registration back then.
| They changed it after a while so that end consumers could
| buy one directly.
| systemtest wrote:
| Yes they changed it after a while so people without a
| CoC/KvK could also purchase them. But initially only
| companies and institutions could buy them.
| pipes wrote:
| I'd love to buy a 500+
|
| https://www.raspberrypi.com/products/raspberry-pi-500-plus/
|
| I can't justify it though as I've no use for it.
|
| However I think it is way closer to their original vision
| than anything else, i.e. It is a lot like the 1980s
| computers, the magic they were trying to capture.
| theshrike79 wrote:
| I love the idea, but the price/power ratio is just a teeeny
| bit on the expensive side for me.
|
| For 100EUR that would be something I'd buy for every niece
| and nephew to play with. For 200EUR it's not even for me,
| I'd rather buy something like the uConsole RPI-CM4:
| https://www.clockworkpi.com/product-page/uconsole-kit-rpi-
| cm...
| patjensen wrote:
| Clockwork Pi experience with the CM4 is not good. 10
| months to ship. Horrible Wifi performance, can't hold a
| link, and it only has around 50 minutes of battery life.
| I regret my purchase and it's sitting in my rack next to
| a bunch of old ham radios.
| theshrike79 wrote:
| It's still more up my alley than the pi 500
| M95D wrote:
| Sell it on ebay to one of those people that have to wait
| 10 months, instead of letting it rot on a shelf?
| theshrike79 wrote:
| I had just gotten into Arduinos when the first Raspberry Pi
| came out.
|
| I noticed I can do 90% of the stuff I'd use an Arduino for
| with a RPi, except I had the full power of an internet
| connected Linux machine available. The Arduinos are still
| collecting dust somewhere =)
|
| But now we have the ESP32 filling the same niche along with
| the Pi Zero W, so I don't really understand the purpose of
| RPi 4 and 5. They're not cheap compared to the price nor very
| powerful in any measure.
|
| You don't even need a full laptop, any Chinese miniPC will
| blow the RPi5 out of the water AND some of them have
| expandable storage+RAM, while also having 5-20x more CPU/GPU
| oomph. They do consume a few watts more power, so there
| _might_ be a niche for the Raspberry Pi, but it's not a big
| one.
| theodric wrote:
| You don't buy the Pi for its price: performance as a
| desktop replacement, you buy it for the incredible
| stability of the platform as a target, the support, and the
| addon ecosystem. If you want to screw around with taking a
| motherboard out of a laptop, go right ahead. EUR160 for a
| 16GB Pi5 that I know for sure will be available,
| replaceable, and supported for the next decade is more than
| worth the small investment.
| pibaker wrote:
| > I don't really understand the purpose of RPi 4 and 5.
| They're not cheap compared to the price nor very powerful
| in any measure.
|
| They are good for commercial installations like smart
| displays in stores (think big screens with menus behind
| fast food counters) and information kiosks. The extra HDMI
| port lets you drive two screens with one pi and the extra
| processing power keeps the UI smooth on high resolution.
| They also have hardware acceleration video decoding for
| shops wishing to play hi res promo videos and hobbyists
| building media terminals.
|
| Cost is not a major concern here because the installation
| volume is low and there are far bigger expenses anyway.
| Just take a look at how much commercial displays are. The
| Pi company's future supply guarantee is also nice because
| you know that within a given number of years if something
| breaks or you need another screen, you can just buy another
| identical pi and be done with it. Good luck sourcing a
| Chinese mini pc with compatible footprints, port
| orientations etc five years down the road.
| worldsavior wrote:
| RPi supporting Linux is already a big benefit. Using a laptop
| with a battery isn't very convenient and dangerous (always
| being charged).
| knowitnone3 wrote:
| then remove the battery?
| TazeTSchnitzel wrote:
| The Raspberry Pi probably still has the advantage of an
| actually robust firmware/software ecosystem? The problem with
| SBCs has always been that the software situation is awful. That
| was the Raspberry Pi's real innovation: Raspbian and a
| commitment to openness.
| vachina wrote:
| > The problem with SBCs has always been that the software
| situation is awful
|
| Awful how? A SBC can take advantage of many software written
| from the dawn of x86.
| sam_lowry_ wrote:
| x86 SBCs are niche, and ARM or RISC SBCs are too complex to
| figure out the boot process and the drivers for an average
| tinkerer.
| cbm-vic-20 wrote:
| Fragmentation in the non-x86 world really hurts adoption. RPi
| presents a very well documented configuration that can be
| used as a target for development.
|
| RISC-V is going through this exact same problem right now.
| All of the current implementations have terrible
| documentation, and tailoring Linux for each of these is
| proving to be difficult. All of these vendors include on-
| board devices that have terrible doc and software support.
| bitwize wrote:
| ARM has a mitigation for this called SystemReady. It's
| basically "does your board support UEFI enough to usefully
| boot a battery of generic ARM Linux images". The Raspberry
| Pi can be made SystemReady, and Radxa also makes
| SystemReady-compliant SBCs you can buy.
|
| RISC-V would do well to adopt and promote a similar spec.
| crimsoneer wrote:
| Not everything needs to be for everyone. I think this is super
| cool - I run a local transcription tool on my laptop, and the
| idea of miniaturising it is _super_ cool.
| noodletheworld wrote:
| It is neat, and at 32GB it might be useful.
|
| Almost nothing useful runs in 8.
|
| This is the problem with this gen of "external AI boards"
| floating around. 8, 16, even 24 is not really enough to run
| much useful, and _even then_ (ie. offloading to disk) they
| 're so impractically slow.
|
| Forget running a serious foundation model, or any kind of
| realtime thing.
|
| The blunt reality is fast high memory GPU systems you
| _actually need_ to self host are really really expensive.
|
| These devices are more optics and dreams ("itd be great
| if...") than practical hacker toys.
| ChromaticPanic wrote:
| Vision models are way smaller and run decently on these
| things.
| buran77 wrote:
| > Not everything needs to be for everyone
|
| I wouldn't dare suggest that. The RPi was never for everyone
| yet it turned out it was for many. It was small but powerful
| for the size, it was low power, it was extremely flexible, it
| had great software support, and last but not least, it was
| dirt cheap. There was nothing like that on the market.
|
| They need to target a "minimum viable audience" with a unique
| value proposition otherwise they'll just Rube-Goldberg
| themselves into irrelevance. This hat is a convoluted way to
| change the parameters of an existing compromise and turn it
| into a different but equally difficult compromise. Worse
| performance, better efficiency, adds cost, and it doesn't
| differentiate itself from the competing Hailo-10H-based
| products that work with _any_ system not just RPi (e.g. ASUS
| UGen300 USB AI Accelerator).
|
| > the idea of miniaturising
|
| If you aren't ditching the laptop you aren't miniaturizing,
| just splitting into discrete specialized components.
| dev_l1x_be wrote:
| I love RPI4s for local homelab roles like DNS servers, even for
| NAS with USB attached external storage, VPN gateways (with
| Tailscale).
| baq wrote:
| It's actually not great since Ethernet is over USB on the pi
| 4 (edit this is not true, confused with pi 3). Not that it
| doesn't work, but I'd rather have an N100 minipc.
|
| OTOH with ram prices being where they are and no signs of
| coming back down in the foreseeable future a second hand pi 4
| may be a very wise choice.
| ajb wrote:
| I think you mean the pi3? On the pi4 the ethernet is
| connected to the main SoC.
| vardump wrote:
| > It's actually not great since Ethernet is over USB on the
| pi 4.
|
| Not true, you're thinking about earlier models.
| baq wrote:
| yeah... sorry about that (note I do have an N100 box as a
| home server, though)
| oliwarner wrote:
| Nah, they released products better suited to what people were
| already using Pis for.
|
| The Picos are _great_ for the smaller stuff, new Pis are great
| for bigger stuff, and old Pis and Zeros are still available.
| They 've innovated around their segment.
|
| The AI stuff is just an expression of that. People are doing AI
| on Pi5s and this is just a way to make that better.
| avhception wrote:
| I don't know of any other ARM device that fulfills:
|
| - I can boot it w/o having to learn about custom U-Boot
| implementations
|
| - I, as a consumer or small business, can buy
|
| - Can not only buy today but also still buy in 2 years
|
| - Doesn't cost a small fortune
|
| - Can be tugged away behind TVs and other small niches
| muggesmuds wrote:
| Odroids!
| avhception wrote:
| Do they still need a custom kernel?
| dominicrose wrote:
| I understand wanting to run a local LLM for privacy, but on
| something more powerful. Yes it's costly but what use is an
| LLM on a cheap board?
| avhception wrote:
| I was mostly responding to
|
| > [...] if I want some low power linux PC replacement with
| display output, for the price of the latest RPi 5, I can
| buy on the used market a ~2018 laptop
|
| I guess. I don't care about the AI hat at all.
| horsawlarway wrote:
| Depends entirely on the model you want to run.
|
| There are some absolutely useful things you can do with
| TTS/STT/Diarization/etc on even really minimal specs.
|
| Some of those will run fine on RPis even without this new
| hat.
|
| The extra ram probably opens the door to a large number of
| vision/image models, which typically want a minimum of
| 16Gb, but do better with 24/32.
|
| There are just a HUGE number of case specific models that
| do just fine on hardware at the RPi level, assuming you
| have the ram to load them.
| wpm wrote:
| Why does it need to be ARM?
| theshrike79 wrote:
| Not ARM, but:
|
| https://www.gmktec.com/products/nucbox-g3-plus-enhanced-
| perf...
|
| If ARM is a requirement, then RPi is your only option that I
| know of.
| Mashimo wrote:
| But also seems to cost ~twice as much.
| pibaker wrote:
| Whenever the Pi gets brought up people on HN will tell you
| you are wasting money if you are not buying a Chinese Pi
| clone with beefier specs instead. But at least for me, "Being
| able to boot into a Linux system without having to dig
| through outdated wikis and Chinese language support forums to
| hunt down a google drive link to an OS image from 2021 that
| has since then received zero updates" is definitely worth
| paying ten extra dollars for.
| avhception wrote:
| Exactly this.
| iamrobertismo wrote:
| really good point, but I actually don't think the Pi's
| biggest competitor is Chinese clones, but just regular
| laptops and mini-pcs that have dramatically lowered in
| price over the years. I do still think Pis have a purpose
| though, it's just harder to justify in certain cases.
| pibaker wrote:
| It depends on what you use your Pi for. If you are self
| hosting anything beyond what a Pi Zero can handle then a
| mini PC would most likely be a better choice than a Pi 4
| or 5. If you are building embedded devices or are a
| hobbyist then the Chinese ARM SBCs are definitely in the
| competition.
| 15155 wrote:
| Some of these Chinese 'clones' have substantially more
| mainline Linux support than any of the Pi boards.
| ajsnigrutin wrote:
| You don't need an ARM, and for the same price, before the RAM
| crisis, you could get many tiny PCs for the same amount of
| money, with proper SSDs and power supplies and in tiny form
| factors.
|
| https://teampandory.com/2024/09/24/gmktec-g5-mini-pc-
| review-...
|
| Something like this ($155) cost less than the Pi + case +
| power supply + nvme addon board + ssd, and it also runs
| windows and any other x86 OS.
| avhception wrote:
| The problem here is "something like this". I may buy one of
| these today. Then in 6 months, I have to buy a different
| one. Only to find out that it's firmware has a bug. Or it
| uses a different ethernet chips that is a problem for me
| for some reason. Or something like that.
|
| The RPi I can depend upon to be shitty as well, but in the
| exact same way. So it stays fit for purpose.
| Croak wrote:
| People have for quite some time been using Googles Tensor chip
| to accelerate AI workloads on the Pi. I doubt that anyone runs
| Llms on Pis but stuff like security cameras with object
| detection...
| nirav72 wrote:
| I read somewhere that Google has largely abandoned support
| for their Coral TPUs? I still use a Coral TPU for Frigate
| NVR. But not sure how long they'll be supported.
| Dr4kn wrote:
| The Coral isn't recommended by it anymore for most use
| cases. They're going to support it, but it shows it's age
| by never being updated for years
| mindcrash wrote:
| > I think Raspberry lost the magic of the older Pis, they lost
| that sense of purpose. They basically created a niche with the
| first Pis, now they're just jumping into segments that others
| created and are already filled to the brim with perhaps even
| more qualified competition.
|
| I don't think you will find anything on the market enabling you
| to create your own audiophile quality AMP, DAC, or AMP+DAC for
| a pretty attractive price except a Pi 3/4/5 with a HifiBerry
| (https://www.hifiberry.com/) HAT.
| yoavm wrote:
| There are some nice ESP-based solutions -
| https://sonocotta.com/esp-products/
| mindcrash wrote:
| HifiBerry is basically the only choice when you live in
| Europe and do not want to deal with tariffs.
| yoavm wrote:
| AFAIK what I sent above is made and shipped from Poland.
| NoboruWataya wrote:
| IMO this is a consequence of Raspberry Pi going for-profit and
| IPOing. Now they are incentivised to chase the same hype trains
| as every other public tech company. I can't see them having
| another "Raspberry Pi moment", those are too risky now.
|
| That said, more options at the (relatively speaking) low end of
| the AI hardware market probably isn't a bad thing. I'm not
| particularly an AI enthusiast generally, but if it is going to
| infest everything anyway, then at least I would like a decent
| ecosystem for running local models.
| windexh8er wrote:
| I think this is a miss on what the Pi is: an experiment. Sure,
| it stood on the shoulders of other SFF boards that came before
| it - but it broke into the general computing landscape
| targeting makers and builders. If the AI hat doesn't work out,
| so be it. The use cases for this type of hat may yet to be
| seen. On one hand it may feel shortsighted to bringing hardware
| to market with no explicit use case, but that's part of the Pi
| brand.
|
| As someone else mentioned: if the hat could efficiently be
| leveraged with the YOLO models on Frigate for a low volume
| camera setup that could be a nice niche use case for it.
|
| Either way I hope the RPi org keeps dropping things like this
| and letting the users sort out the use cases with their
| dollars.
| ChromaticPanic wrote:
| On device processing for robots and drones
| Zetaphor wrote:
| Everyone here is missing the fact that this board was made by a
| third party, not the Raspberry Pi organization.
| rocketvole wrote:
| What? Sure, its technically using technology by hailo, but
| it's definitely from the RPI organization
| jasongill wrote:
| It's not "made by a third party" - it's sold as a first-party
| add-on, by Raspberry Pi on their own website, in a Raspberry
| Pi branded package.
|
| https://www.raspberrypi.com/products/ai-hat-plus-2/
|
| It's no more "made by a third party" than any other
| electronics device made by a contract manufacturer.
| Zetaphor wrote:
| Oh, I did not realize it was getting that first class
| treatment, I thought (from only reading the article) that
| this was just a hat made by a third party and sold for the
| ecosystem.
| lenerdenator wrote:
| Gotta get that investor cash.
|
| In regard to their niche, their niche is a ridiculously well-
| documented ecosystem for SBCs. Want to do something with your
| RPi? You can find it on Google, and the LLM of your choice is
| probably trained to give you the answer on how to do it. If
| you're just tinkering or getting a POC ready, that's a big
| help.
|
| Of course, if you're in the business of hardware prototyping,
| and have a set of libraries and programs you know you're going
| to work with, you don't need to care as much.
| andix wrote:
| My biggest issue is the lack of really good cases. There are
| all those fancy peripherals you can buy, but it's really hard
| to find simple case that works without overheating and no
| cables sticking out on all four sides.
| nkko wrote:
| That magic now moved to ESP32.
| rpcope1 wrote:
| This is why I keep buying 3B+ and Zero 2 W and not any of the
| newer versions...it's much more in keeping with the relatively
| low cost board with GPIO and reasonable compute. It's kind of
| the last one they made that does what I kind of expected out of
| a Raspberry Pi at a reasonable price point. If I needed more
| compute I would have skipped the travesty that is ARM and just
| bought an x86 system.
| iamrobertismo wrote:
| I feel like if RPI doubled down heavily into education, they
| would be in a much better spot. They really could never win on
| price in the long run. But having a bit of K-12 and university
| budgets going to RPIs every year, especially during the "teach
| the kids programming" era, would I think make them a much
| healthier business.
|
| Chromebooks did what RPI should have done.
| alnwlsn wrote:
| I still remember the email I got telling me that they were
| going to upgrade the RAM of the 256Mb Model B I ordered and
| that I would receive a brand new model with 512Mb for no extra
| cost. Hard to believe that was nearly 14 years ago.
|
| Was very helpful in me learning Linux. The only alternative I
| had at the time was a few old Pentium 4 machines, which were
| very noisy and my parents didn't like me leaving turned on for
| a long time.
| 0xbadcafebee wrote:
| The original Pis are still for sale, are cheap, and still do
| everything you need. This doesn't conflict with an expanded
| product line. The whole reason for Pi is still GPIO plus
| general purpose computing. AI is now a part of general purpose
| computing, so it only makes sense to adopt it too.
|
| The things you can do locally with AI now are amazing. For
| several years there's been multiple open source products that
| can do both audio and visual processing locally using AI
| models. Local-only Home Assistant is almost equivalent to Siri.
| The more things you throw at it, the more computing power it
| needs (especially for low latency), and that's where the
| dedicated GPUs/NPUs (previously ASICs) are needed. And consider
| the expanded use cases; drones and robots can now navigate the
| world autonomously using a $150 SoC and some software.
| atmosx wrote:
| Do sales backup these claims that come up very often or not?
| Does anyone has any data?
|
| Although the op is not wrong, maybe their decisions are data
| driven and pay off?
| turtlebits wrote:
| May have lost the magic with regards to price, but they're
| commodity hardware now.
|
| My rpi3 (that's been running since 2019) died last year and I
| was able to buy another and just plug in the SD card.
| moffkalast wrote:
| > The Pi's built-in CPU trounces the Hailo 10H.
|
| Case closed. And that's extremely slow to begin with, the Pi 5
| only gets what, a 32 bit bus? Laughable performance for a purpose
| built ASIC that costs more than the Pi itself.
|
| > In my testing, Hailo's hailo-rpi5-examples were not yet updated
| for this new HAT, and even if I specified the Hailo 10H manually,
| model files would not load
|
| Laughable levels of support too.
|
| As another datapoint, I've recently managed to get the 8L working
| natively on Ubuntu 24 with ROS, but only after significant
| shenanigans involving recompiling the kernel module and building
| their library for python 3.12 that Hailo for some reason does not
| provide outside 3.11. They only support the Pi OS (like anyone
| would use that in prod) and even that is very spotty. Like, why
| would you not target the most popular robotics distro for an AI
| accelerator? Who else is gonna buy these things exactly?
| renewiltord wrote:
| What's the current state of the art in low power wake word and
| speech to text? Has anyone written a blog post on this?
|
| I was able to run a speech to text on my old Pixel 4 but it's a
| bit flaky (the background process loses the audio device
| occasionally). I just want to take some wake word and then send
| everything to remote LLM and then get back text that I do TTS on.
| geerlingguy wrote:
| Maybe not SOTA but the HA Voice Preview Edition [1] in tandem
| with a Pi 5 or some similar low-power host for the Piper /
| Whisper pipeline is pretty good. I don't use it but was able to
| get an Alexa/Google Home-like experience going with minimal
| effort.
|
| I was only using it for local Home Assistant tasks, didn't try
| anything further like retrieving sports scores, managing TODO
| lists, or anything like that.
|
| [1] https://www.home-assistant.io/voice-pe/
| monocasa wrote:
| Wake word can be tiny. Like 10k weights and can run on an esp32
| or similar with plenty of compute to spare.
|
| TinyML is a book that goes through the process of building a
| wake word model for such constrained environments.
| folmar wrote:
| Wake word is not expensive, you can do it on esp32
| https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3...
| (and then send audio to something more beefy as TTS will be
| marginal at best).
| huntercaron wrote:
| Glad Jeff was critical here they need a bit of a wake up call it
| seems.
| speedgoose wrote:
| Is there any usefulness with the small large language models,
| outside perhaps embeddings and learning?
|
| I fail to see the use-case on a Pi. For learning you can have
| access to much better hardware for cheaper. Perhaps you can use
| it as a slow and expensive embedding machine, but why?
| kouteiheika wrote:
| A natural language based smart home interface, perhaps?
|
| Tiny LLMs are pretty much useless as general purpose
| workhorses, but where they shine is when you finetune them for
| a very specific application.
|
| (In general this is applicable across the board, where if you
| have a single, specific usecase and can prepare appropriate
| training data, then you can often fine-tune a smaller model to
| match the performance of a general purpose model that is 10x
| its size.)
| michaelmior wrote:
| I think there's a lot of room to push this further. Of course
| there are LLMs being used for this case and I guess it's nice
| to be able to ask your house who the candidates were in the
| Venezuelan presidential election of 1936, but I'd be happy if
| I could just consistently control devices locally and a small
| language model definitely makes that easier.
| cmpxchg8b wrote:
| 8GB? What is this, an LLM for ants?
| matja wrote:
| It's more about demonstrating what's possible on a Pi than
| expecting GPT-4 level performance. It's designed for LLMs that
| specialize in tiny, incredibly specific tasks. Like, "What's
| the weather in my ant farm?" ;)
|
| The vision processing boost is notable, but not enough to
| justify the price over existing HATs. The lack of reliable
| mixed-mode functionality and sparse software support are
| significant red flags.
|
| (This reply generated by an LLM smaller than 8GB, for ants,
| using the article and comment as context).
| kirurik wrote:
| You can run some models pretty decently using CPU inference
| only, things like Gemma 3 that are built for exactly that use
| case or some tiny speech to text models via llama.cpp that I
| have tested out (not so good). Although not the best for
| "heavy" tasks, if you just need a decent text generator that
| can produce more or less sensible, generic output you are good
| to go.
| kotaKat wrote:
| "For example, the Hailo 10H is advertised as being used for a
| Fujitsu demo of automatic shrink detection for a self-checkout."
|
| ... _why though_? CV in software is good enough for this
| application and we 've already been doing it forever (see also:
| Everseen). Now we're just wasting silicon.
| Barathkanna wrote:
| As an edge computing enthusiast, this feels like a meaningful
| leap for the Raspberry Pi ecosystem. Having a low-power inference
| accelerator baked into the platform opens up a lot of practical
| local AI use cases without dragging in the cloud. It's still
| early, but this is the right direction for real edge workloads.
| saagarjha wrote:
| Which ones?
| Croak wrote:
| For example camera based object detection. People have been
| using usb Dongel Google Tensors for that previously
| Croak wrote:
| For example camera based object detection. People have been
| using usb Dongel Google Tensors for that previously.
| lopis wrote:
| A smarter home assistant would be one
| teekert wrote:
| At this moment my two questions for these things are:
|
| 1. Can I run a local LLM that allows me to control Home Assistant
| with natural language? Some basic stuff like timers, to
| do/shopping lists etc would be nice etc.
|
| 2. Can I run object/person detection on local video streams?
|
| I want some AI stuff, but I want it local.
|
| Looks like the answer for this one is: Meh. It can do point 2,
| but it's not the best option.
| worksonmine wrote:
| 1. Probably, but not efficiently. But I'm just guessing I
| haven't tried local LLMs yet.
|
| 2. Has been possible in realtime since the first camera was
| released and has most likely improved since. I did this years
| ago on the pi zero and it was surprisingly good.
| noodletheworld wrote:
| > Can I run a local LLM that allows me to control Home
| Assistant with natural language? Some basic stuff like timers,
| to do/shopping lists etc would be nice etc.
|
| No. Get the larger PI recommended by the article.
|
| Quote from the article:
|
| > So power holds it back, but the 8 gigs of RAM holds back the
| LLM use case (vs just running on the Pi's CPU) the most. The Pi
| 5 can be bought in up to a 16 GB configuration. That's as much
| as you get in decent consumer graphics cards1.
|
| > Because of that, many quantized medium-size models target
| 10-12 GB of RAM usage (leaving space for context, which eats up
| another 2+ GB of RAM).
|
| ...
|
| > 8 GB of RAM is useful, but it's not quite enough to give this
| HAT an advantage over just paying for the bigger 16GB Pi with
| more RAM, which will be more flexible and run models faster.
|
| The model specs shown for _this device_ in the article are
| small, and not fit for purpose even for the relatively trivial
| use case you mentioned.
|
| I mean, look, lots of people have lots of opinions about this
| (many of them wrong); it's cheap, you can buy one and try...
| but, look. The OP really gave it a shot, and results were kind
| of shit. The article is pretty clear.
|
| Don't bother.
|
| You want a device with more memory to mess around with for what
| you want to do.
| rballpug wrote:
| Catalog TG 211, 1000 Hz.
| endymion-light wrote:
| can't wait to not be able to buy it, and also for it to be more
| expensive than a mini-computer
|
| I buy a raspberry pi because I need a small workhorse - I
| understand adding RAM for local LLMs, but it would be like a
| raspberry pi with a GPU, why do i need it when a normal mini
| machine will have more ram, more compute capacity and better
| specs for cheaper?
| rocketvole wrote:
| a lot of people buy rpis because they are the only reasonable
| option for connectivity with power. I'm not sure what other
| devices you can get that have gpio and mipi connectivity with
| the ability to (potentially) run vlms and llms on them.
|
| I daresay they could charge more than a comparably specced
| computer (if they don't already) and they would still be a
| viable purchase.
| endymion-light wrote:
| Surely with this hat you don't have any access to any GPIO?
|
| Unless i'm missing something - which is where i'm like why
| not just buy a NUC with similiar RAM for far less.
| Rohansi wrote:
| You do still have access to the GPIO. This HAT [1] stacks
| on top of the GPIO connector but passes through all the
| pins so you can still use them. This one is connected
| through PCIe so it shouldn't be blocking off any pins from
| use, unless you wanted an NVMe SSD hooked up!
|
| [1] https://www.raspberrypi.com/news/introducing-raspberry-
| pi-ha...
| t43562 wrote:
| In the UK I've never seen the hailo hats (which are quite old
| BTW) advertised for LLMs. The presented usecase has been object
| detection from lots of video cameras in realtime.
|
| They seem very fast and I certainly want to use that kind of
| thing in my house and garden - spotting when foxes and cats
| arrive and dig up my compost pit, or if people come over when I'm
| away to water the plants etc.
|
| [edit: I've just seen the updated version in Pimonori and it does
| claim usefulness for LLMs but also for VLMs and I suspect this is
| the best way to use it].
| esskay wrote:
| What a pointless product to waste time making.
| joelthelion wrote:
| 8GB is really low.
|
| That said, perhaps there is a niche for slow LLM inference for
| non-interactive use.
|
| For example, if you use LLMs to triage your emails in the
| background, you don't care about latency. You just need the
| throughput to be high enough to handle the load.
| wyldfire wrote:
| I wonder -- how does this thing compare against the Rubik Pi [1]?
|
| [1] https://rubikpi.ai/
| philipallstar wrote:
| Is that affiliated with raspberry pis in some way, or are they
| just freeloading on the "pi" suffix to confuse people?
| Elfener wrote:
| > are they just freeloading on the "pi" suffix to confuse
| people?
|
| Yes, but that is normal I guess:
|
| - https://banana-pi.org/
|
| - http://www.orangepi.org/index.html
|
| - https://radxa.com/products/rockpi
| wyldfire wrote:
| They are "freeloading" indeed but in this case it is a
| valuable indicator that it should be physically compatible
| with many add-ons/attachments that are compatible with the
| Raspberry Pi.
| imtringued wrote:
| This looks pretty nice for what it is. However, the RAM is a bit
| oversized for the vast majority of applications that will run on
| this, which is giving a misleading impression of what it is
| useful for.
|
| I once tried to run a segmentation model based on a vision
| transformer on a PC and that model used somewhere around 1 GB for
| the parameters and several gigabytes for the KV cache and it was
| almost entirely compute bound. You couldn't run that type of
| model on previous AI accelerators because they only supported
| model sizes in the megabytes range.
| Lio wrote:
| I've seen the AI-8850 LLM Acceleration M.2 Module advertised as
| an alternative RPi accellorator (you need an M.2 hat for it).
|
| That's also limited to 8Gb RAM so again you might be better off
| with a larger 16Gb Pi and using the CPU but at least the space is
| heating up.
|
| With a lot of this stuff it seems to come down to how good the
| software support is. Raspberry Pis generally beat everything else
| for that.
| Havoc wrote:
| That seems completely and utterly pointless.
|
| A NPU that adds to price but underperforms a rasp cpu?
|
| You get SBC with 32gb ram...
|
| Nevermind the whole minipc ecosystem which will crush this
| nottorp wrote:
| Hmm. Can this "AI" hardware - or any other "AI" hardware that
| isn't a GPU - be used for anything other than LLMs?
|
| YOLO for example.
| dismalpedigree wrote:
| Yes. The Hailo chips are mainly for AI vision models. This is
| the first time I have seen them pushed for LLM. They are very
| finicky and difficult to setup outside of the examples.
| Documentation is inconsistent and the models have to be
| converted to a different format to run. It is possible to run a
| custom yolo8 model, but is challenging.
| jsight wrote:
| I'd expect that kind of thing to be the primary use case, tbh.
| Maybe even running whisper models?
|
| If it could run whisper, it'd be a solid addition to a pi based
| home assistant setup.
| vander_elst wrote:
| I had a couple of Pis that I wanted to use as a Media center, I
| always had some small issues that created a suboptimal
| experience. Went for a regular 2nd hand amd64 with a small form
| factor and never looked back, much better userspace support and
| for my use case a much smoother experience, no lags no memory
| swapping and if needed I can just buy a different memory bank or
| a different component. I have no plans to use a raspberry pi any
| time soon. I am not sure these days if they really still have a
| niche to fill and if yes how large this niche is.
| JustFinishedBSG wrote:
| It's useless for LLMs and it's actually slower than Hailo 8H for
| standard vision tasks, so, why ?
| myrmidon wrote:
| Are there significant usecases for the really small LLMs right
| now (<10b distills and such)?
|
| My impression so far was that the resulting models are unusably
| stupid, but maybe there are some specific tasks where they still
| perform acceptably?
| arkmm wrote:
| They're still very good for finetuned classification, often
| 10-100x cheaper to run at similar or higher accuracy as a large
| model - but I think most people just prompt the large model
| unless they have high volume needs or need to self host.
| syntaxing wrote:
| Interesting idea. I think the Jetson Orin Nano is a better
| purchase for this application. The main downside is the RAM is
| shared so you lose about 1G from the OS overhead.
| giantg2 wrote:
| What not use a USB Coral TPU? Seems to do mostly the same stuff
| and is half the price.
| geerlingguy wrote:
| Coral is many times slower at this point (2 TOPS IIRC), but if
| it meets your needs, it's okay.
| incomingpain wrote:
| I wonder if this is the magic hardware for
| LiquidAI/LFM2.5-Audio-1.5B
|
| Dont need more than 8gb. It'll be enough power. IT can do audio
| to audio.
| djhworld wrote:
| A few years ago this product would have just been called an ML
| Accelerator and marketed as helping accelerate ML workloads like
| object detection in images
|
| Hitching their wagon to the AI train comes with different
| expectations, leading to a mixed bag of reviews like this.
| yawniek wrote:
| i wonder how the Hailo 10H compares to Axera AX8850. add on
| boards seem to be cheaper and its a full SoC that can also draw
| much more power.
| yjftsjthsd-h wrote:
| Any chance you can split layers so that some run on the CPU and
| some run on this board to let you run bigger models and/or get
| better performance?
| xp84 wrote:
| Gigabytes?? In THIS economy?
| 1970-01-01 wrote:
| It does seem like this is a marketing gimmick. I doubt it will
| have worked in terms of sales numbers. "What else does it do?"
| venturecruelty wrote:
| Are computers even for anything else anymore? Is it just AI until
| the bubble pops? This is exhausting.
| janalsncm wrote:
| Excited to see more hardware competition at this level. Models
| that can run on this amount of RAM are right in the sweet spot of
| small enough to train on consumer-grade GPUs (e.g. 4090) but big
| enough to do something interesting (simple audio/video/text
| processing).
|
| The price point is still a little high for most tasks but I'm
| sure that will come down.
| rootusrootus wrote:
| Within minutes of each other I can read a bunch of comments about
| how an 8GB AI hat for RPi would be grand for LLMs, while another
| contingent points out that my M2 MAX 96GB MacBook is completely
| useless for LLMs. All I can say is at least the latter is also a
| great laptop.
| sxzygz wrote:
| My primary concern with this offering is vendor of the enabling
| silicon. I think it's important to consider who you do business
| with. The industrial uptake of RPi boards has poisoned their
| mandate and made them forget making a world where _all_ children
| can discover the magic of computing, not just the special chosen
| ones.
| tonymet wrote:
| Can someone share a non-theoretical LLM application that demands
| local inference and pocketable hardware? The internet hype for
| these is sky high, but the actual results are disappointing.
| 5ersi wrote:
| This AI 8GB HAT is $150, while better performing Pi5 8GB is $100.
| Makes no sense to buy the AI HAT.
___________________________________________________________________
(page generated 2026-01-15 23:01 UTC)