[HN Gopher] Internet in a Box
___________________________________________________________________
Internet in a Box
Author : doodlesdev
Score : 294 points
Date : 2023-04-29 04:57 UTC (18 hours ago)
(HTM) web link (internet-in-a-box.org)
(TXT) w3m dump (internet-in-a-box.org)
| asicsp wrote:
| Previous discussion:
| https://news.ycombinator.com/item?id=27568332 _(620 points | June
| 20, 2021 | 139 comments)_
| antman wrote:
| I was wondering, if there was enough cpu power and no
| hallucinations, a gpt kind of llm would compress better?
|
| Will knowledge be stored in an llm in the future?
| bobsmooth wrote:
| You might find this interesting.
|
| Text Compression as a Test for Artificial Intelligence
|
| https://news.ycombinator.com/item?id=33002312
| rozab wrote:
| The current size of the full text of Wikipedia is about 21GB.
| Text compression is not a field which needs disrupting
| shaburu wrote:
| i checked their GitHub repo and looks like the project is dead,
| any update from anyone working on it?
| doodlesdev wrote:
| Which repo did you look at? The one at [0] seems pretty active,
| specially for an application that probably doesn't need updates
| all that often anyways.
|
| [0]: https://github.com/iiab/iiab
| nynx wrote:
| Eventually, when local compute gets good enough, running LLMs on
| this kind of system will be far more useful than most internet
| resources could be directly.
| dynjo wrote:
| Or the IT Crowd version https://youtu.be/iDbyYGrswtg
| jaxr wrote:
| Thought the exact same thing when I saw the title :)
| prmoustache wrote:
| First thing I looked is if someone would mention it in the
| comments ;-)
| krm01 wrote:
| I wonder if something like a locally run LLM would be a 10X
| better (and viable) solution? You could frankly use a chatGPT
| like UI to have even the illiterate part of the community
| converse and learn about any topic with tts and stt.
| aflag wrote:
| LLMs get a lot of facts wrong. Computer literate people can
| double check. But completely illiterate people can't even look
| it up in a book. That sounds like a way to fabricate myths and
| straight up lies that are believed by the most vulnerable in
| the population.
|
| Using llms can be useful, but it's very advanced tech that
| should be done by people who understand what it is and what its
| limitations are. And even then it has its dangers. Letting it
| loose among uneducated people has the potential to be
| disastrous
| mftrhu wrote:
| No, a locally-run LLM would not be a 10x better solution than
| having a copy of Wikipedia - plus StackOverflow, the other
| StackExchange websites, and a lot of other wikis - thanks to
| Kiwix, a copy of Khan Academy's educational content, of the
| relevant OpenStreetMap data, of Project Gutenberg, and of
| dozens more GB in textbooks, reference material, and
| entertainment.
|
| The illiterate part of the community would be better served by
| interacting with _the community_ - by fostering connections
| between people, improving literacy, and giving them tools -
| rather than with _any sort of LLM_ , let alone one that has
| been cut down enough to run on cheap, under- (and often
| intermittently-)powered commodity hardware.
| butz wrote:
| It would be great if it had some "snail mail" like features:
| collect "sent" emails and "posted" posts, and synchronize them
| with real internet on regular intervals, probably on monthly
| basis.
| dimisdas wrote:
| It does, it has https://github.com/ascoderu/lokole
| comprev wrote:
| aka "Sneakernet"
|
| https://en.m.wikipedia.org/wiki/Sneakernet
| dan_linder wrote:
| Lol, you've just rediscovered "uucp" and the other related
| protocols!
| drummojg wrote:
| This comment is bang on.
| kervantas wrote:
| I wonder if the internet will be handled in a similar mater
| once humanity spreads to other plants.
| troupe wrote:
| So basically a BBS that you access over wifi instead of dial
| up? :). Long live FIDO net!
| hoppla wrote:
| Related; Othernet (previously named outernet)
| https://othernet.is/ is also worth looking at
| myself248 wrote:
| Natural allies, it's weird that they're not already grafted
| together somehow.
|
| IMHO, the ultimate feature-set would be:
|
| * IIAB content library support, especially the new map features
| which rock
|
| * wrolpi-like tagging and automatic updating when connected to
| the internet
|
| * Othernet updating of content when satellite signal is
| available
|
| * Toosheh extractor for unpacking files saved in an mpeg-ts on
| a USB stick
|
| * Local Fdroid app repo
|
| * Piratebox-like file drop area
|
| * BBS or local forum message-board
|
| * Mesh with other nodes, perhaps using NNCP or Scuttlebutt
| protocols to exchange messages over spotty connectivity or
| traveling relay nodes
|
| If I had any software clue whatsoever, I'd be actively trying
| to graft all these things together.
| owenpalmer wrote:
| Honestly this is something I would love to setup for my future
| family some day, even when full internet is an option.
| fabiensanglard wrote:
| There is a mention of "Internet-in-a-Box searchability" but I am
| unable to figure out how it works now. Wikipedia and
| Stackoverflow are way better if they can be searched.
| throw2022110401 wrote:
| You arrive at the village. You've been separated from your family
| and you are desperate to get some kind of Internet access so that
| you can try to get in touch with them. Your phone still has 20%
| battery but no reception. You finally find a Wi-Fi hotspot! You
| connect... and it turns out to be one of these things instead of
| the real deal.
| freetanga wrote:
| Either use gsm or ask the locals to relay a message via radio?
| 2b3a51 wrote:
| Just wondering aloud: once a region has a number of villages
| with these services going well, perhaps a low bandwidth mesh
| network between them would allow a slow messaging service like
| email to exist?
| thomasjb wrote:
| Low bandwidth mesh for email ought to be viable, I'm sure
| that used telecoms equipment could enable a 'good enough'
| network, although perhaps using QRP APRS equipment would
| reduce the cost when starting from an energy demands point of
| view, and better account for the fact that packets will
| occasionally have to wait a few hours until the batteries in
| other nodes are charged.
| akho wrote:
| Text does not consume meaningful bandwidth, so a mesh network
| should allow IM. (You'd need to limit torrenting, video, and
| hi-res audio)
|
| Email and ssb can be supported by having a bicycle and a
| thumb drive.
| nunobrito wrote:
| The solution between villages could be LoRa since it is
| cheap, less than 50 euro per device albeit too high-tech.
|
| A lower tech option is plain old radio, for example:
| https://frn.dc4fs.de/
| chinaman425 wrote:
| [dead]
| xg15 wrote:
| It's a really great project and I hope there will be more like
| that in the future!
|
| I wonder, do you have any plans do enable https support? With
| browsers increasing becoming hostile to http and disabling modern
| features, it might be a good idea to enable access via https.
|
| I think a streamlined way how a teacher could quickly generate a
| custom root CA and install it on student laptops could be useful
| here.
|
| (And yes, I still think it sucks that this is the only option for
| using https without internet)
| otabdeveloper4 wrote:
| > install root CA
|
| Lol. Yeah, we didn't spend all this effort to centralize the
| Internet just to let you have such a powerful and dangerous
| tool as a private key, buddy.
| throwawaaarrgh wrote:
| one day someone is going to die, and the root cause will be
| found to be some nonsense like an expired certificate,
| invalid [old] ciphers, ocpp, lack of publicly signed cert, or
| some other ridiculous security feature that you should be
| able to turn off or ignore in an emergency
| mytheory wrote:
| This is a fantastic project, and we need more of those. Let's
| keep in mind that 50% of the world's population did not connect
| to the internet in the last three months (last un UN starts).
|
| On my end, I worked on an Android version since getting &
| maintaining a Raspberry Pi or a Linux machine is difficult in
| remote communities - where finding reliable access to a network
| or electricity can be tricky. Still, you will see those
| inexpensive Android phones everywhere. If you are curious: grey-
| box.ca/uni Early stuff, but we managed to deploy and test in 12
| countries, and we are looking for volunteers for some of our
| digital divide open-source projects.
| moffkalast wrote:
| Yeah a Pi is a completely terrible solution for something like
| this because it doesn't really play to any of its strengths.
| Android phones are cheaper, more reliable, have a built in
| battery, touchscreen, GPS and compass (so you can navigate with
| one too), and have you know... actual power management so you
| don't run out of said battery. Sometimes they're even dust and
| waterproof which is probably good in a tropical rainforest or a
| sandy desert.
| kangalioo wrote:
| Where do you get Android phones for cheaper than a Raspberry
| Pi (e.g. $35 for a Pi 4)?
| moffkalast wrote:
| > $35 for a Pi 4
|
| Lol, if you have them at that price I'll take 5. More like
| $80-120 at current market rates, and then you still need to
| buy everything else that's already included in a phone.
| asynchronous wrote:
| I'm so tired of people citing $35 for a Pi. Maybe for a
| Pi Zero (non W). But $120 is standard at this point.
| ChuckNorris89 wrote:
| Second hand market? Samsung, OnePlus, HTC, LG, Nexus,
| Nokia, Pixels, with quad core CPUs and RAM >2GB can be had
| dirt cheap depending where you live.
|
| Choose a popular, old flagship that's well supported by the
| community, and can be flashed with a de-Googled ROM and
| you're gold.
|
| Old OnePlus devices are still supported today by the
| community.
| myself248 wrote:
| Ooh ooh ooh!
|
| I've been trying to get Termux/Alpine going on my old Android
| so I can install IIAB in it, for precisely all these reasons.
| But I'm linux-clueless and haven't had the best of luck.
|
| UNI doesn't look like it's actually IIAB-on-Android, more like
| another similar project, yes?
| ChatGTP wrote:
| I know this will get absolutely smashed but is this such a bad
| thing anymore ? If you can't connect to the internet you're
| immediately protected from a lot of garbage information,
| attention hijacking, scams and hacking, child grooming, IP
| theft, you're job is probably a lot more secure etc.
|
| I know there is access to information which is beneficial but
| at some stage I'm not convinced having internet access will
| necessarily be considered a luxury in the best future.
|
| I know it's an outrageous sounds claim but I still feel it
| could go either way.
| bahmboo wrote:
| This does not connect to the internet.
| nibbleshifter wrote:
| This is an outrageous claim lol.
| ChatGTP wrote:
| I don't post to make friends.
|
| Edit: Follow up thought, I'm not sure who the internet
| benefits anymore. It used to be about users. Now I don't
| feel the same way.
| nibbleshifter wrote:
| I mean, you basically out of hand disregard the infinite
| potential network connectivity grants you to learn nearly
| _anything_ to a good level.
|
| Languages? State department language courses are
| available online, as are many other resources, language
| classes, language learning communities, etc.
|
| Math? Physics? Chemistry? Again, infinite repository of
| info.
|
| And so on...
| beebeepka wrote:
| Oh boi. Don't ever go outside your home because you may get
| robbed, stabbed, spat on, laughed at, run over, kidnapped,
| etc. And even worse things await unsupervised children! No
| fun can be had outside and all knowledge must be carefully
| curated.
|
| Ugh.
| cogitoergofutuo wrote:
| The internet may be full of scams and IP theft, but it also
| allows people to fulfill their impulse to post, so who is to
| say that it's bad?
| xg15 wrote:
| Also starting to believe that the real status symbol will be
| the ability to _not_ have an internet connection or only
| connect at your own discretion.
|
| I also see a lot of temptation especially in the IoT space to
| use the internet to control the devices against the users
| wishes and spy on the user.
|
| You also see tge first attempts of devicemakers to actively
| ensure an internet connection themselves, independently of
| what the user set up. (I.e. Apple's Find My network, Amazon's
| Sidewalk, etc).
|
| I don't think this will become less in the future.
| stareatgoats wrote:
| Thanks, I think we need to discuss this too. I don't believe
| we can wish the internet away, but we're still in internet
| infancy, the "wild west" of plugged in, always online, life.
| Regulators need to catch up, but at the moment they seem to
| miss the point, mainly trying to cash in and serve their own
| narrow interests.
|
| We likely need a movement that promotes a healthier use of
| the tech, which could include things like making it socially
| unacceptable to be constantly hooked up to your phone, and
| hefty fines for the manufacturers whenever their
| hardware/software made clandestine surveillance possible.
|
| If things like these are not technically or socially
| possible, then I'm guessing we'll see an "anti-internet"
| movement eventually, too.
| stareatgoats wrote:
| Actually, such local "internet in the box" as per the
| linked article might prove to be a basis for one such
| "anti-internet" solution: detach from the internet as a
| whole and enable local communication, resources and traffic
| only.
|
| Every prepper should really have one!
| jasonjayr wrote:
| What's old is new again: if everyone has their own
| isolated network, then if you _need_ message exchange,
| you can light up FIDO or UUCP!
| qsort wrote:
| Access to high-quality internet resources is absolutely a
| luxury. You can pick and choose what resources you want, it's
| not TV.
|
| It's like saying that access to a well-stocked university
| library is not a privilege because most pop books that get
| published are crap. It's not even the same category of
| "book".
|
| Negatives of the internet do exist but you're overstating
| your case. This is about getting basic resources to people
| who couldn't otherwise access them, I don't see how it can
| possibly be net negative.
| ifyoubuildit wrote:
| I have a similar feeling. I think there is a definite
| lifecycle to consumer technology (on the internet you have
| things like wikipedia, youtube, social media, google search,
| but this also applies to things like smart phones, operating
| systems, tvs even).
|
| These things all seem to follow a curve. They are released,
| adopted and improved, they become amazing, and then at some
| point they start to decline for whatever reason (even as they
| may be improving in some aspects - like smart tvs continue to
| get a better picture but only if you can stomach the extra
| ads, surveillance, and otherwise creepy anti features).
|
| I wish I could just have windows 7 with security updates, a
| dumb tv with an awesome screen, almost no subscriptions or
| cloud based doodads.
|
| Id love a frozen version of the internet that nobody can come
| along and ruin for whatever reason. Internet in a box seems
| like a good idea to me.
| zamnos wrote:
| The privilege is in being able to _not_ have Internet access
| for an extended period and just enjoy life from a tropical
| beach, sipping on a mai tai, with no work to check-in on to
| make sure nothing 's blown up. From the first world, only the
| middle class and higher can afford such a luxury. But the
| Internet is just so damned _useful_ that, sure, it enables
| doomscrolling, but it enables _so_ much more as a
| communication medium (the MVP for many businesses is a email
| address or a whatsapp number) that the assertion is just
| ignorant of life before the Internet.
| hammyhavoc wrote:
| Well, some content available through the internet is
| useful. That content was available on CD-ROMs in the '90s
| as encyclopedias.
|
| The function provided with this project is some content
| available offline just like those CD-ROMs. You aren't going
| to be communicating with folks on the other side of the
| world via it. Ergo, it is not "the internet in a box".
| drra wrote:
| Look how few projects containing valuable knowledge are
| available offline. There's less and less stuff that can work
| offline.
| rakoo wrote:
| Thanks for what you're doing. Couldn't that be achieved with
| kiwix as well ? It can serve content over WiFi as well
| lxgr wrote:
| The project seems to be using Kiwix, but not only that. For
| example, you can't serve OpenStreetMaps cards using Kiwix (at
| least to my knowledge).
| asynchronous wrote:
| It seems to be Kiwix+OSM+Archive.org downloads which is
| really nice. I wonder how the interface works
| raybb wrote:
| Very cool idea!
|
| How this project funded? Do you plan to make enough money
| selling boxes to support development or rely on donations?
|
| Kinda wondering if you'd consider the org a social enterprise.
|
| Btw your site doesn't redirect to https, not sure if that's
| intentional.
| zikduruqe wrote:
| Years ago I worked with a project that used eGranary and FabFi
| mesh networking:
|
| https://widernet-egranary.org/?page_id=5
|
| https://en.wikipedia.org/wiki/FabFi
|
| We were trying to bring Wifi and digital content to some
| struggling communities.
| aaron695 wrote:
| This is stupid for schools as always, but does seem to be working
| in medical settings - https://www.youtube.com/watch?v=JHlfs__nBkE
|
| This seems to be the content - http://med.iiab.me/
|
| The "The Global Emergency Medicine Wiki" is pretty cool, external
| link where they get it from - https://wikem.org/
|
| Preppers seem interested too which is cool -
| https://www.tiktok.com/@deweylikedonuts/video/71236508943983...
| doodlesdev wrote:
| You were flagged, I imagine because of the tone of your
| comment, but I wonder why you think this is stupid for schools?
|
| Living in a third world country I can tell you this has huge
| potential to improve schooling in areas with limited or no
| internet accessibility.
| hammyhavoc wrote:
| I find the name to be misleading. By the same logic, could you
| describe '90s encyclopedia CD-ROMs as "the internet in a box"?
|
| It's content that was hosted on the internet. The actual internet
| is infrastructural, not content.
|
| Is a DVD box set thus "Netflix in a box"?
| embedded_hiker wrote:
| There was a product in the 90s called "Internet in a Box". I
| remember seeing it at a store. It appears to be a way of
| selling Compuserve.
| https://www.computerhistory.org/revolution/the-web/20/408/20...
| sph wrote:
| Is there something like this, for "guerrilla" situations? An open
| source DIY project, a Raspberry PI image, and some kind of WiMAX
| antenna/WiFi AP combo to give internet access to a wide area in
| emergency situations.
|
| I have no experience with that field but I think that would be
| really neat, for people in unstable countries where internet is
| blocked, or inaccessible. A plug and play, ghetto way of
| restoring internet access on the go, when all you have is a
| crappy, flaky and possibly filtered internet connection.
| 29athrowaway wrote:
| This is how people consume content in Cuba
|
| https://www.youtube.com/watch?v=fTTno8D-b2E
|
| If you like this story, I recommend this documentary:
|
| https://www.imdb.com/title/tt2442080/ (Chuck Norris vs Communism)
| RecycledEle wrote:
| This is great. Thank You for the link!
|
| I have spent a little time each year over decades planning a
| library. The first draft planned a shipping container full of
| rack-mounted equipment, and several more shipping containers full
| of the equipment to power the library. That version never got
| built.
|
| My current iteration uses a bunch of 18 TB HDDs in a HP Z-series
| workstation.
|
| If I had the money to actually deploy prototypes outside my home,
| I would switch to much more power efficient SATA SSDs, a
| Raspberry Pi, some kind of hat that connects many SATA drives,
| and a low-power Wi-Fi router that connects many users at once.
|
| Over the years I learned that: * The US Library of Congress (LOC)
| cataloging system is the best way to index information, but... *
| Databases are temporary. Files on disk are much more permanent.
| You will change databases several times while leaving the same
| files on disk.
|
| LLMs will change my design, as will StarLink and other satellite-
| based Internet Service Providers (ISPs)
| nunobrito wrote:
| I've built http://arquiva.me exactly with that concern in mind.
|
| Databases are temporary and difficult to move around when
| getting too big. On the other hand, files are easy to transport
| in portions and to continue working if a part has gone missing
| for some reason.
|
| The backend on that platform was written from scratch and is
| file-based. Reduces duplication of images to permit archiving
| parts of the digital history for my country (Portugal).
|
| Works fully offline for benefit of archivers in future
| generations.
| Infernal wrote:
| Are you comfortable sharing the current version of your draft?
| frickinadorable wrote:
| This seems very cool. I'm trying to figure out if anyone would
| take a donation to build say 1000 of these and give them away,
| but I can't find any kind of contact info for IIAB. They link to
| Wikimedia Medicine which I also can't find any contact info for,
| apparently communication should happen over "meta" which must be
| some wiki thing?
|
| Anyway, if you have or know about a 501c3 that can build and give
| away these devices, please share contact info here.
| Uptrenda wrote:
| I really don't get it. Many of these countries don't even have
| the resources for clean water, sewage, and reliable agriculture.
| The idea that they're going to just think their way out of
| poverty is a very western idea that comes from quite a privileged
| background. You're not going to be able to do any of that
| luxurious learning when your stomach is empty and you're having
| to sleep each night in some war-torn slum. These countries need a
| lot of basic things before getting them online will have any real
| impact... Some of the poorest countries don't even have stable
| governments.
|
| I think most idealist people here would be utterly crushed by how
| bad the situation is in some countries. Going in with ideas of
| how tech will be used to raise everyone up... Then having to live
| in camps with armed guards outside because there's frequent
| raids... No way to even get supplies in because corrupt
| beaurocrats keep asking for bribes. The same beaurocrats who are
| supposed to be on the side of their country are also massively
| part of the problem. Then you end up realizing that there's so
| many problems and inter-dependencies that it's not clear if any
| single person can do anything positive.
|
| All aboard the FEEL TRAIN. Yeah, its hard, I wish I knew the
| answer.
| kilroy123 wrote:
| Very true. It reminds me of the time I visited a remote
| Ethiopian village. Very poor area, but they seemed fairly self-
| sufficient. There was a large solar oven given to the village
| by some NGO. Looks like:
|
| https://www.voanews.com/a/south-african-firm-aims-to-supply-...
|
| You can see it was never used. I asked the guide, do they even
| use this thing? He laughed and said no. It was basically a
| decoration piece.
|
| What did they use to cook? The wood they go out to collect.
| MandieD wrote:
| There are billions of people living in situations between the
| poorest parts of the EU (yet still in better circumstances than
| the majority of the world) and war-torn starvation in Sudan,
| and things like this are for them. Plenty of people who can't
| afford satellite internet can read just fine.
|
| And I, a well-off tech worker in urban Germany, use the project
| as part of my own emergency communications prep. Internet
| outages are possible even for rich Europeans and Americans, and
| we're awfully dependent on being able to look stuff up.
| zamnos wrote:
| For their businesses owners and upper class, I'm pretty sure
| that sending money abroad to Facebook for advertising or Uber
| for taxi service doesn't help the situation. Local apps like
| FoodPanda or Grab at least let the money stay in-country or in-
| region, instead of off to California.
|
| But the level of poverty is unreal. I had multiple hotels
| charge me a deposit for key card so I'd return it, and breaking
| large bills (the equivalent to US $20) wasn't something that
| most shops actually had change for.
| brazzy wrote:
| You're just being contrarian, not more realistic. Which is also
| a very privileged position. I also suspect that you haven't
| actually visited many poor countries.
|
| Even for extremely poor people, free and easy access to
| information and communication can have very substantial,
| practical benefits.
|
| A layman's medical encyclopedia that lists smyptoms and
| treatments for common illnesses may not be an adequate
| replacement for a trained doctor, but it's a hell of a lot
| better than nothing.
|
| One of _the_ most important factors to enable the "reliable
| agriculture" you mentioned is access to a good weather
| forecast. Even one that's only good for the next two days can
| be a game-changer when it allows you to harvest a crop that
| might otherwise get spoiled.
|
| Another example I've read: access to cell phones _massively_
| improved the lives of fishermen in Northern Africa by allowing
| them to find out in which nearby port the demand for fish is
| highest. Previously, they 'd have to guess and often there
| would be an oversupply (and thus low prices and fish that goes
| to waste) in one town, while another twon 20km away had people
| go hungry because there was too little fish and the prices too
| high.
| bobsmooth wrote:
| It's such a privileged position to be even able to
| underestimate the importance of knowledge.
| brianvli wrote:
| Is it really so binary that either they're rich westerners or
| they're so destitute they can't benefit from this project?
|
| Also, this project ends up being a compact way to deliver
| educational material. Imagine sending them the equivalent in
| physical books and manuals
| kevviiinn wrote:
| it's a very privileged western view that all "third world"
| countries are destitute and starving
| wil421 wrote:
| > You're not going to be able to do any of that luxurious
| learning when your stomach is empty and you're having to
| sleep each night in some war-torn slum.
|
| Not sure why your calling out this poster when the OP said
| the above.
| NikolaNovak wrote:
| I feel that's exactly what the op was saying in response to
| gp?
|
| There's a spectrum, and while we should have increased
| empathy and awareness of a LOT of places in the world where
| life is spectacularly rough and wildly different than ours
| (2 years of my childhood were in a civil war and my
| Canadian wife just cannot fathom any of those experiences),
| there are indeed a lot of places in the world where
| knowledge and access and technology are helpful.
| kevviiinn wrote:
| I was rephrasing the comment I'm replying to in agreement
| wffurr wrote:
| Did you see what this project actually is? It's not a live
| internet connection. It's a content pack of extremely useful
| reference materials and such for solving exactly a lot of those
| problems, with solar power and local wifi.
|
| It's a end run around those corrupt bureaucrats.
|
| Is it perfect or going to solve everything? Obviously not, but
| it's cheap, simple, and robust.
| rolobio wrote:
| I thought about creating one of these in the past, but it didn't
| fit my needs. I decided to start my own project, called WROLPi.
| It also uses a Raspbery pi as its base, but also has a Debian
| installer. I have RPi images available at https://wrolpi.org as
| well as a link to a demo (the demo needs an update, lots of new
| features since it's creation).
|
| The basic premise of WROLPi is creating your own off grid digital
| library: videos, web archives, PDFs, ebooks, etc. I have full
| text search for all of these. Offline maps as well via Open
| Street Map. Wi-Fi hotspot. Automatic downloads of videos (entire
| channels), RSS feeds, etc.
| asjfka wrote:
| Is this a consumption-only thing or does it allows some kind of
| interractions with the outside world, like emails sent to the box
| ultimately get delivered (when someone brings the box to a true
| internet connection to update it), edits to wikipedia get merged,
| ect. ?
| praveen9920 wrote:
| Imagine an LLM that is trained on public data like the internet
| archive and it is made available on these boxes without the
| internet, now we can have almost the entirety of human knowledge
| accessible on a single box.
|
| I know that LLMs sometime hallucinate and practical limits of
| computation power availability etc. but imagine how quickly we
| can spread the knowledge. It is almost like the printing press
| during the renaissance.
| CrampusDestrus wrote:
| LLMs are not magic.
|
| If wikipedia takes up 100GB of space and your model trained on
| it is only 5GB big, there is _NO WAY_ it will hold all the
| original information.
| grumbel wrote:
| It doesn't need to be lossless to be useful. A lossy LLM can
| just throw all the redundant stuff away and focus on the
| actual information in the text, instead of the text word by
| word. It wouldn't need to keep dozens of languages around for
| if it could translate on the fly. And it wouldn't need to
| keep a full text when it can just have the raw facts and
| generate the text on the fly.
|
| Not sure what the state of the art is with LLM text
| compression, but with StableDiffusion compression ratios of
| up to 155:1 have been achieved[1] compared to 10:1 for JPEG.
|
| [1] https://pub.towardsai.net/stable-diffusion-based-image-
| compr...
| SXX wrote:
| Let's be honest disk space is cheap enough for people use
| actual data instead of LLM-generated.
|
| What people in poor countries need the least is learn from
| source that have tons of random facts of knowledge censored
| out since some western corporation decided they're unetical
| or disturbing.
| OscarTheGrinch wrote:
| Furthermore, LLMs are good at condensing information, but
| terrible at knowing which information will actually be
| important or true. This is an obvious problem in a life or
| death medical situations.
| Robotbeat wrote:
| ...which is also unsolved by any other system or by humans.
| It's a fundamental constraint of reality.
| marginalia_nu wrote:
| I'm not sure this is true.
|
| Lossless compression algorithms exist, and HTML compresses
| almost absurdly well. A 20x compression rate would be quite
| impressive, but I don't think it's impossible, since 10x
| compression is fairly common in this domain, and there are
| reasons to expect wikipedia to compress even better given the
| significant amount of markup on each page.
|
| I've worked with cleaning up rendered wikipedia texts, and if
| you strip the mediawiki class names and cruft, the
| uncompressed HTML shrinks a lot, often from hundreds of
| kilobytes to tens of kilobytes. You can then probably
| compress that quite a lot as well since plain text compresses
| very well on its own.
|
| Basically, mediawiki uses this much markup to say <li><a
| href="#Assessment">Assessment</a></li>: <li
| id="toc-Assessment" class="vector-toc-list-item vector-
| toc-level-1 vector-toc-list-item-expanded"> <a
| class="vector-toc-link" href="#Assessment"> <div
| class="vector-toc-text"> <span class="vector-toc-
| numb">6</span>Assessment</div> </a> <ul
| id="toc-Assessment-sublist" class="vector-toc-list">
| </ul> </li>
|
| So yeah. 20x isn't too far fetched.
| CrampusDestrus wrote:
| That is not the point.
|
| There is a physical limit of how much you can compress
| stuff. If your LLM is significantly smaller than even the
| most optimized compression we can achieve, then it simply
| doesn't actually preserve all the information
| marginalia_nu wrote:
| Be that as it may, 5Gb just isn't realistically
| significantly smaller than you could squeeze the text of
| wikipedia.
| bmacho wrote:
| Indeed, if the theoretical limit to compress wikipedia is
| n GB, then you will need n GB at least to create the
| exact same copy of wikipedia. However you may create a
| jpg of it. (And you can then remove the artifacts, and
| even enhance it with AI, and you will get something
| better than the original wikipedia, to keep the image
| analogy.)
| T-A wrote:
| _As of 21 September 2022, the size of the current version of
| all articles compressed is about 21.23 GB without media._
|
| https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia
| doodlesdev wrote:
| By the way, note that is the Wikipedia in English which
| AFAIK is the largest one. For instance, Wikipedia in
| Portuguese weighs about 13gb with a few pictures in my
| Kiwix install.
| fbdab103 wrote:
| Has Kiwix gotten any better? I tried to use it a few
| years ago, but the user experience was super rough.
| Fantastic idea, but there were lots of UI stalls and
| glitches that soured my trial.
| doodlesdev wrote:
| Honestly? The UX is dogshit lol, at least on the Android
| app, when you go and download something to find out if it
| has downloaded or not is a chore and there is no progress
| notification or indication it's doing anything at all.
|
| I'm considering sending some contributions whenever I
| have time again for OSS contributions.
|
| However, I find it usable enough that I've been using it
| quite a lot recently when I don't have internet access to
| read Wikipedia, StackOverflow and a few other things I
| downloaded.
| unnah wrote:
| NNCP, a state-of-the-art lossless text compressor based on a
| transformer neural network (i.e. containing similar
| structures to a LLM), can reach a compression ratio of 10.8%
| (0.867 bits per byte) on Wikipedia text:
| https://bellard.org/nncp/
|
| Compared to NNCP, a hypothetical LLM that "knows" every fact
| in the English Wikipedia, would not need to keep a lossless
| record of the entire text of Wikipedia, since many Wikipedia
| pages repeat facts that also occur on other pages, and
| instead of reproducing every word it would be enough to
| paraphrase the ideas. Thus the size of such a LLM could be
| much smaller than 10% of the size of Wikipedia. How much,
| it's hard to say...
| moffkalast wrote:
| Yeah neural nets are amazing compression algorithms since
| the best way to get the gist of something is to actually
| understand it, then algorithmically generating the data.
| Like a calculator having a near infinite compression ratio
| because it can do math instead of keeping a table of all
| possible inputs and outputs.
|
| Same goes for human brains I suppose, we can only remember
| as much as we do because of the effective compression our
| understanding allows for.
| nibbleshifter wrote:
| In my experience LLM's hallucinate a _lot_.
|
| Often giving "plausible but entirely wrong" output that is hard
| to discern from factual, real output.
| eru wrote:
| Yes.
|
| At the moment they are great for entertainment.
|
| But for factual information, you are better off just grabbing
| a copy of Wikipedia.
| zamnos wrote:
| Wikipedia's information often isn't collated how you want.
| Eg List the states where the biggest city is larger in
| population than the state capital? There isn't specifically
| an article for that. Sure you could take the _List of
| United States cities by population_ article and assemble
| the list yourself, or you could just have ChatGPT do it for
| you.
| quechimba wrote:
| Cool! I'm really happy Iquitos got a fiber cable now. Just
| measured 23Mbps on 4G and I'm a few kilometers down the river. A
| few years ago I got about 5-10Kbps so it's a huge difference.
| amelius wrote:
| The box should also contain ChatGPT, so the internet-in-a-box can
| keep growing and you could have conversations on forums like HN.
| moffkalast wrote:
| It would need to be a significantly larger box for that. And
| require a field of solar to run. It would also cost a few
| orders of magnitude more than just a starlink terminal would.
| marcodiego wrote:
| Nice educational project. In terms of entertainment... I wonder
| how cool would be a ChatGPT in a box. I think... how interesting
| would it be for isolated people...
| enos_feedler wrote:
| We've gone full circle. I remember the day this little box came
| in the mail and my world changed:
|
| https://www.computerhistory.org/revolution/the-web/20/408/20...
|
| Hopefully this project does something similar for others.
| shortrounddev wrote:
| Why are half the comments about chatbots
| RajT88 wrote:
| Half of all comments are now about chatbots.
|
| I'm a little afraid of the implications.
| throwawaaarrgh wrote:
| half of the internet is now just chatbots taking to chatbots.
| it's why the internet has been slow lately
| rektide wrote:
| How long has this project been an Ansible script?
|
| Iirc the Internet-in-a-Box project pre-dates Ansible, which was
| started sometime in 2012.
| Epa095 wrote:
| Nice to see the OLPC xo laptop again "in the wild" (the green
| laptop in the bottom picture) Hopefully it's still useful for
| someone out there:-)
___________________________________________________________________
(page generated 2023-04-29 23:02 UTC)