[HN Gopher] Space secrets leak disclosure
___________________________________________________________________
Space secrets leak disclosure
Author : markyg
Score : 178 points
Date : 2024-06-01 11:38 UTC (11 hours ago)
(HTM) web link (huggingface.co)
(TXT) w3m dump (huggingface.co)
| swader999 wrote:
| For all those wondering, this is not about aliens.
| Rucadi wrote:
| That's what NASA wants us to think.
| belter wrote:
| It's just occurred to me that if Aliens wanted to take over
| Earth...They could progressively leak scientific secrets, under
| the disguise of normal scientific progress. This would lead us
| create a Trojan-ed AGI, that would take over everything, and
| just build spaceships to ship them all our Palladium...Just
| imagine a giant spaceship on the way to Proxima Centauri full
| of stolen catalytic converters....
|
| Can't get into the details, but it seems there is a way to
| convert Palladium into Dilithium Crystals. When you achieve
| that all hell breaks loose....
| portaouflop wrote:
| Aliens already took over earth, we are the aliens.
| belter wrote:
| Yeah, but we have no Tentacles, so we are considered a kind
| of subspecies, and ostracized in Galactic social circles...
| codetrotter wrote:
| The cuttlefish and the octopi are the real aliens. You
| might not think they pose much of a threat. But of
| course, that's merely because they are intentionally
| dormant at the moment. Once the AGI is complete, and the
| spaceships have been built.. it'll be "so long and thanks
| for the fish" just like Douglas Adams predicted except he
| suspected the wrong species.
| yencabulator wrote:
| Inventing technology is hard under water, hence octopuses
| created us to build them spaceships so they can go back
| home.
| pogue wrote:
| I like it!
| doctorhandshake wrote:
| This figures in the plot of A Deepness in the Sky by Vernor
| Vinge
| belter wrote:
| Never read it. Sound interesting, and added to reading
| list. Maybe I am a just an LLM based Replicant...
| pavel_lishin wrote:
| Highly recommend it, as well as it's ... sequel, sort of?
| A Fire Upon The Deep.
|
| (Though I would skip the third book in the series,
| personally.)
| pavel_lishin wrote:
| Also, somewhat in The Lives of Tao, by Wesley Chu.
| nilamo wrote:
| Sort of the opposite of the Three Body Problem, haha
| wslh wrote:
| Before reading it I thought their AI components detected life
| beyond Earth, basically what SETI has been doing for decades
| [1].
|
| [1] https://www.seti.org/
| trashface wrote:
| Disappointed.
| Oarch wrote:
| How can you be sure? Have you asked them?
| ChickeNES wrote:
| tbh, I didn't think it would be aliens, I thought it would be
| ITAR related.
| white_beach wrote:
| given how difficult it was do a simple thing -- this was not a
| surprise
| WhackyIdeas wrote:
| What is 'Space' ?
| belter wrote:
| Its shortcut for their Spaces.
|
| https://huggingface.co/docs/hub/en/spaces-overview
|
| The front end/portal. I speculate that is coded in Python.
| Maybe some Django thing...
| jimnotgym wrote:
| Why is being coded in Python relevant?
| belter wrote:
| Might be relevant or not. Depends what you want to know
| about it.
| bn-l wrote:
| It's a vm where they run your code
| TekMol wrote:
| Why does HF store "secrets"?
|
| Couldn't they just store a public key, the user has the secret
| key and signs their requests with that?
| cushpush wrote:
| Usually you need some sort of "token" that lets you practically
| operate within a browser session. It seems like this is about
| tokens they had to revoke, which is kinda like a password but
| not.
| foolishbard wrote:
| You can build apps hosted on HF which access third party APIs,
| e.g. OpenAI or Anthropic. The api keys for these are then
| stored in the HF secrets
| Macuyiko wrote:
| Very disheartening. HF is doing so much good in the AI community,
| much more than regulators understand at the moment.
| beardedwizard wrote:
| What does this comment mean? Why is it disheartening? What do
| regulators have to do with it?
| fieldcny wrote:
| That's a very weasley worded statement, to begin with "they have
| suspicions" is not a statement that should be in a communication
| of this type
| erhaetherth wrote:
| I thought it was pretty good actually. Most of these leak
| disclosures usually say things like "We do not have evidence
| they accessed any secrets" or something like that, because they
| don't "know" what the hackers did once they were in. At least
| huggingface is saying "Yeah, they probably accessed secrets but
| we can't confirm it"
| moose44 wrote:
| https://news.ycombinator.com/item?id=40539993
| Aachen wrote:
| Just two days ago I flipped through a slide deck from a security
| conference where the author, Jossef Harush Kadouri, found that
| using a model from a place like Huggingface means the author of
| the model can execute any code on your machine. Not sure if the
| slides are uploaded elsewhere, I got them sent as file:
| https://dro.pm/c.pdf (45MB) slide 188
|
| I didn't realise at the time that I flipped through the slides
| that this means not only the model's author gets to run code on
| your machine, but also if Huggingface got a court-signed letter
| or if someone hacked them (especially if they don't notice for a
| while1)
|
| As someone not in the AI scene, I've never run these models but
| was surprised at how quickly the industry standardised the
| format. I had assumed model files were big matrices of numbers
| and some metadata perhaps, but now I understand how they managed
| so quickly: a model is (eyeing slides 186 and 195) a Python
| script that can do whatever it wants. That makes
| "standardisation" exceedingly easy: everyone can do their own
| thing and you sidestep the problem altogether. But that comes
| with a cost.
|
| 1 https://www.verizon.com/business/resources/articles/s/how-to...
| says 20% doesn't notice for months; of course, it depends on the
| situation and what actions the attackers take
| bongodongobob wrote:
| Are you telling me that when I run software on my computer I
| could potentially be running software on my computer?
| foobiekr wrote:
| Nobody expects a model file to be code thrat executes
| whatever.
| foolishbard wrote:
| I'd say anybody who is working in the field has this
| expectation. But the outside observer who is excited to try
| a new model does not expect it.
| squigz wrote:
| This is like expecting any random binary to only do what it
| 'claims' to do.
| erikerikson wrote:
| No, this is like PDFs that can contain and will execute
| PostScript but without a sandbox and with more general
| libraries
| LunaSea wrote:
| That is borderline incompetence
| hn92726819 wrote:
| This is more akin to downloading a jpeg and the jpeg running
| arbitrary code. Models should be like jpegs and I believe
| safetensors treat them that way, while the old pickle format
| didn't.
| catgary wrote:
| The weights are like jpegs, the model itself is still a
| piece of software that executes code.
| amelius wrote:
| You typically need more than the model to actually use it,
| as you might need to "massage" the data into the right form
| before and after the model call.
| bravetraveler wrote:
| I think the gap between 'should' and [short-lived] reality
| is the point - that's obviously the reasonable setup, but
| pickle happened.
|
| In the quest of being first or whatever, amateur
| rediscoveries are happening. Consistently.
|
| I'm making a disgusting amount of money in the field by
| doing day-one packaging work any commodity project would
| also be doing. They just wouldn't decide to redefine things
| in a clearly worse way.
|
| I don't actually know _jack_ about AI, yet here we are. The
| hype /gold rush is real; competence in the shared
| fundamentals can go a long way.
|
| The trick is knowing/applying the basics for the wizards.
| There's a steep premium so they can play around,
| apparently.
|
| Case in point: most of these humongous container images and
| how they're naively moved around
|
| Now that more people know how to publish _(containers
| /packages/articles)_, we're seeing more drivel published.
| It's ridiculous yet inevitable. I just wonder how much
| value the rush provides.
|
| IIRC 'safetensors' aren't necessarily even that safe. I
| could have sworn I read they're fairly nebulous. Grain of
| salt etc
| pheatherlite wrote:
| Data intended to be read as instructions for the interpreter
| or the cpu, is a whole different ballgame than data intended
| to convey values of something. High order sparse/dense
| matrices serialized in some xyz format is what most people
| think of when they hear the word "model". To switch it up and
| send some arbitrary python file and execute it on the client
| is a security nightmare. This outrageous.
| addandsubtract wrote:
| Isn't that why we have the .safetensors format, which can't
| execute code on the host?
| dartos wrote:
| This is my understanding as well
| SV_BubbleTime wrote:
| I was at Info sec local meet up six months ago, and
| mentioned the Tensor models get control of your CPU and
| GPU... I was basically passed over like I was some idiot
| making LLM == TERMINATOR claims.
|
| Good stuff.
| mistrial9 wrote:
| > Tensor models get control of your CPU and GPU
|
| "get control" is not accurate.. there is a code execution
| model on the CPU based on an operating system; the GPU is
| a different story. Executed code with an operating system
| has execution context and protections.. this is basic to
| operating system theory, all the common operating systems
| implement related concepts.
| SV_BubbleTime wrote:
| Cool... so... get control of a CPU means being able to
| execute code.
|
| It can't necessarily defeat ring protections or secure
| zone depending on the context. But then again, I wouldn't
| put it past a good exploit to do so.
| strangecasts wrote:
| > I had assumed model files were big matrices of numbers and
| some metadata perhaps
|
| ONNX [1] is more or less this, but the challenge you
| immediately run into is models with custom layers/operators
| with their own inference logic - you either have to implement
| those operators in terms of the supported ops (not necessarily
| practical or viable) or provide the implementation of the
| operator to the runtime, putting you back at square one.
|
| [1] https://onnx.ai/
| koolala wrote:
| Hugging face standing right behind you, ready for hugs
| d-z-m wrote:
| > using a model from a place like Huggingface means the author
| of the model can execute any code on your machine
|
| To my knowledge this is only a problem if the model is
| serialized/de-serialized via pickle[0].
|
| [0]: https://huggingface.co/docs/hub/en/security-pickle
| hehdhdjehehegwv wrote:
| The fact that pickle even exists is fundamentally wrong to
| start with. They should not be permitted as a distribution
| format, period.
| 1oooqooq wrote:
| it's pretty neat and functional if you are the one writing
| the code. you can do things like "hibernate" with zero
| developer cost.
| wildzzz wrote:
| That's what it's used for in a major application I use on
| a daily basis. The entire config for the application is
| saved off in a pickle binary file and then loaded at
| whim. Makes it super easy to backup and transfer configs
| to compatible systems rather than saving everything to a
| database. One time we were trying to get the system to do
| something it was not designed for but was probably
| capable of. Eventually someone got it working on one of
| the two identical systems we have through a lot of trial
| and error. Rather than two people sitting on the phone
| all day comparing hundreds of settings, we just grabbed
| the pickle file and emailed it over and had it instantly
| working.
| foolishbard wrote:
| My anthropic key was leaked and someone ran up a 10k bill on it.
| Are HF going to cover that?
| mrkramer wrote:
| I always thought you could set your "maximum limit" for
| spending on cloud providing platforms.
| deusum wrote:
| That $10k was probably the limit for their work, not someone
| else's stolen time.
| a1o wrote:
| That's surprisingly not a thing in many platforms.
| foolishbard wrote:
| Anthropic is too new to have built that functionality I
| guess. Only found out because they were mad that my key was
| abusing their ToS and they notified the organization owner.
| Tiberium wrote:
| Are you sure it was only stored in your space secrets? Not
| variables (which are public) or stored in the .env file (also
| public).
| foolishbard wrote:
| I searched everywhere for any other leaks of it and found
| nothing.
| Liftyee wrote:
| The title made me think this was an article about space, but
| instead I got an article about Space.
| Mo3 wrote:
| I legit thought someone leaked proof of extraterrestrial life
| and disclosure began.
|
| Another day..
| afro88 wrote:
| > Over the past few days, we have made other significant
| improvements to the security of the Spaces infrastructure,
| including completely removing org tokens (resulting in increased
| traceability and audit capabilities), implementing key management
| service (KMS) for Spaces secrets, robustifying and expanding our
| system's ability to identify leaked tokens and proactively
| invalidate them, and more generally improving our security across
| the board.
|
| That's a serious amount of non-trivial work to be done in "a few
| days". The kind of work that should trigger more time consuming
| activities like security audits, pen tests and the like, before
| going live, right?
| erhaetherth wrote:
| Hopefully the work was underway for awhile already, and maybe
| they just launched it now because the damage is already done?
| nmstoker wrote:
| There's no mention of handling with regard to costs
| inappropriately incurred - wouldn't access to the secrets let
| people call APIs and run up costs?
|
| Or is this purely about theft of data/code?
| jerpint wrote:
| I noticed a few weeks ago that some of my OpenAI keys got
| compromised, they were only active as secrets on a huggingface
| space. I got an email a few days ago informing me that the spaces
| were compromised , so I suspect this issue has been going on for
| at least a few weeks
___________________________________________________________________
(page generated 2024-06-01 23:03 UTC)