[HN Gopher] Ask HN: For those using Stable Diffusion locally, ho...
___________________________________________________________________
Ask HN: For those using Stable Diffusion locally, how do you filter
fishy repos?
I have been using the official release so far, and I see many new
tools popping up every day, mostly GUIs. A substantial portion of
them are closed-source, sometimes even simply offering an
executable that you are supposed to blindly trust. Not to go full
Richard Stallman here, but is anybody else bothered by that? How do
you deal with this situation, do you use a virtual machine, or is
there any other ideas I am missing here?
Author : epups
Score : 86 points
Date : 2022-08-31 11:48 UTC (11 hours ago)
| gigel82 wrote:
| I use the Docker one, fully insulates it from my actual machine;
| it's OSS, but there's a lot of code and downloading a bunch of
| models, packages and tools at build / run time so better safe
| than sorry.
| f38zf5vdt wrote:
| The official repos work fine.
|
| https://github.com/huggingface/diffusers
|
| https://github.com/CompVis/stable-diffusion
| capableweb wrote:
| I simply don't use the closed source ones? Easy to filter (can I
| see the source?), and helps if I want to contribute as well.
|
| Currently using https://github.com/hlky/stable-diffusion +
| https://github.com/hlky/stable-diffusion-webui which are both
| FOSS and works well.
| rcoveson wrote:
| Whoa there, that first link looks super sketchy. It seems like
| they just forked the original `CompVis/stable-diffusion` and
| replaced the license file in this commit:
|
| https://github.com/hlky/stable-diffusion/commit/b4c61769dfa1...
| knaik94 wrote:
| The have added a lot more to it than just change the license,
| and the replacement is to the GNU AGPL. The original wording
| in the license puts arbitrary and unenforceable limits on
| what the end user can do. I would argue the original repo
| license applies to the weights only and not the code wrapping
| it. This is completely valid.
|
| They have also made a way to use real-esrgan/gfpgan and so a
| license change is practicality required.
| rcoveson wrote:
| > The have added a lot more to it than just change the
| license
|
| I see that, but that doesn't mean they can ignore the
| original license.
|
| > The original wording in the license puts arbitrary and
| unenforceable limits on what the end user can do.
|
| I'm totally with you here. The original license is absurd.
| Still doesn't mean we can fork the repo and replace the
| license without (probably) breaking the law. Which, by all
| means, do. In practice maybe nobody will care. I'm just
| pointing it out because it's sketchy.
|
| > I would argue the original repo license applies to the
| weights only and not the code wrapping it.
|
| Most of the clauses only apply to the weights, yes. The
| first clause in the license applies to the whole repo,
| though: "All rights reserved by the authors."
|
| > They have also made a way to use real-esrgan/gfpgan and
| so a license change is practicality required.
|
| I don't see how this is relevant. Even if there is a
| license conflict, the authors retain control over their
| source. A license conflict might lead to a damages
| settlement, or an order to halt distribution. It doesn't
| magically switch the license by implication.
| knaik94 wrote:
| This repo doesn't distribute the model or weights
| directly. You are agreeing to the more limited upstream
| license of the original repo when you download the pre-
| trained model. If the repo came bundled with the pre-
| trained model then your concern is valid. In this case it
| is not.
| fjfbsufhdvfy wrote:
| It's now enough to use the main repo only, because he's
| regularly copying over any changes. So don't need to manually
| copy those files across anymore.
| capableweb wrote:
| Ah, I didn't know that, already have symlinks + .gitignore
| setup so I can update both easily with `git pull`, but good
| to hear it's no longer needed.
|
| Any reason why the webui repository is not archived with a
| warning up top if it's been deprecated like you say?
| fjfbsufhdvfy wrote:
| It's not deprecated. He uses it for development and then
| copies the files across to the main repo. I don't really
| understand what benefits this setup has over making a dev
| branch in the main repo, but to each their own!
| fjfbsufhdvfy wrote:
| Currently there is so much activity that for every closed source
| tool chances are there is an open source one that does the same.
| I simply use those instead, after skimming over the code for any
| obviously malicious activity.
| cpdean wrote:
| What malicious activity in these repos have you come across?
|
| Do you know when you've missed something?
| fjfbsufhdvfy wrote:
| So far actually, none in the ones I used. It seems everyone
| is just excited about the tech :)
|
| For now, most of these tools are rather small wrappers around
| the original stable diffusion repo which is considered
| trustworthy, so there isn't that much to review.
|
| Things I generally look out for are setup scripts that
| install unusual packages, any file or network io activity,
| code that's been obfuscated, instructions that have you
| download checkpoints from unofficial sources, etc.
|
| Of course I can't know if I missed something...
| cube2222 wrote:
| I've used [0], [1] and [2] so far. I only use open-source ones
| and quickly skim the source code for anything suspicious. I also
| only use ones with some degree of popularity, meaning that others
| have probably taken a look at the code as well.
|
| [0]: https://github.com/lstein/stable-diffusion
|
| [1]: https://github.com/hlky/stable-diffusion
|
| [2]: https://github.com/basujindal/stable-diffusion
| TekMol wrote:
| How easy / hard is it to run those repos in Google Colab?
|
| That would be my preferred way to shield myself from the repo.
| 7373737373 wrote:
| One big problem are also model weights serialized with (the likes
| of) pickle - which allows arbitrary code execution
|
| A lot of trust just to get some numbers
| zorgmonkey wrote:
| Everything I've seen is still using the model from the original
| researchers, so as far as that threat goes you only need to
| trust the original model author and the author of the
| particular script you are using. If you want to inspect the
| serialized model file fickling looks like a very promising tool
| https://blog.trailofbits.com/2021/03/15/never-a-dill-moment-...
| 7373737373 wrote:
| The model is hosted on many sites by now, so it is important
| to compare the checksums
| Jach wrote:
| VM is a good idea. I barely even trust a lot of the open source
| stuff, there's deep stacks of magic (and not so magic) shit in
| modern machine learning, and too much "we depend on pip
| installing this particular git repo, sometimes a particular
| commit that you'll need to figure out lol". Some of the stuff
| people are building has looked interesting, but I'm going to let
| the dust settle for a while before I look into them more, and I'm
| particularly trying not to gum up my new machine with npm BS. For
| now, I've had enough fun with just the original repo (regrettably
| finally mostly figuring out how to use conda) which I setup a bit
| before the weights were released and haven't updated. (So I
| didn't need to bother removing the last-minute added
| watermark/filtering stuff.) I also sometimes test things with the
| network down to see if there's blatant surprising network
| connections, interestingly the default repo will ping a site for
| a resource it needs to download, but continues to do so even
| after you have it. Add the "local_files_only=True" param to the
| from_pretrained() method calls in ldm/modules/encoders/modules.py
| to stop it. (Oh thank the gods that I can just edit the py files
| to make local changes and they haven't tried to do some weird
| hybrid binary thing like other projects (ActivityWatch).) I also
| setup Real-ESRGAN to upscale some favorites, its results are
| pretty interesting.
|
| The executable binary blobs I've a natural inclination to
| distrust, but I can also see where they're coming from,
| culturally, and Most Of The Time it's not a problem, like random
| indie games you might download to try. (Besides, there are other
| binary blobs I depend on, like the nvidia driver...) Culturally
| it seems somewhat comparable to the gaming world or even demo
| scene or modding scene, where traditionally you don't often find
| much open source.
| foxhop wrote:
| I tried to run locally but all I got is a black square for
| output.
|
| I used the scripts from this Repo
|
| https://github.com/basujindal/stable-diffusion
|
| which didn't give me a GPU memory error which the original link
| does.
| paskozdilar wrote:
| Please go full Richard Stallman.
|
| Control of computing is an all-or-nothing business - even a
| single compromised component can lead to compromise of the
| complete system.
|
| Don't trust opaque binaries.
|
| > How do you deal with this situation, do you use a virtual
| machine, or is there any other ideas I am missing here?
|
| If you _really_ want to run that opaque binary, a virtual machine
| will give you a decent amount of security. With GPU passthrough,
| you can even get near-native speed, too.
| bhedgeoser wrote:
| If you've ever run an `npm install` you've executed 100s of
| opaque binaries on your machine.
| paskozdilar wrote:
| Would you care to elaborate?
|
| I always thought npm was open-source-centric. If npm somehow
| ran opaque binaries, I'd really like to know about that.
| Something1234 wrote:
| Do you really think your average javascript developer is
| going to read and understand all of their dependencies?
| leppr wrote:
| There is no open-source requirement, like there would be on
| Gentoo packages for instance. NPM packages frequently pull
| arbitrary binaries in their install scripts.
| sp332 wrote:
| You could use the Windows Sandbox to prevent them from accessing
| anything sensitive on your computer.
| https://docs.microsoft.com/en-us/windows/security/threat-pro...
| dncornholio wrote:
| Why would I be bothered about what exactly? People should be free
| on what and how they release their stuff.
| fjfbsufhdvfy wrote:
| Someone taking advantage of all this excitement to get people
| to download malware sounds like a very reasonable fear,
| especially if the software is not coming from a known and
| reputable person/business.
|
| No one's saying you can't release closed source software - but
| of course other people are free to be afraid of running it as
| well.
| avg_dev wrote:
| What's more, I believe it is possible to release code with a
| closed license, permitting analysis without allowing
| derivative work.
|
| Edit: IANAL but here's a resource:
|
| > You're under no obligation to choose a license. However,
| without a license, the default copyright laws apply, meaning
| that you retain all rights to your source code and no one may
| reproduce, distribute, or create derivative works from your
| work. If you're creating an open source project, we strongly
| encourage you to include an open source license. The Open
| Source Guide provides additional guidance on choosing the
| correct license for your project.
|
| From https://docs.github.com/en/repositories/managing-your-
| reposi...
| dncornholio wrote:
| Ok, but again, why should I, or even Richard Stallman, be
| bothered about this? I couldn't care less, as does Richard,
| I assume..
|
| I have the feeling OP feels left out because of closed
| binaries and thinks he should be entitled for everything
| based off Stable Diffusion __should__ be open source.
|
| OP asks how we should deal with this, but there's nothing
| to deal with actually.
| epups wrote:
| I believe any developer should be free to release things
| in the way they please. In fact some people might prefer
| an easy executable file. However, I do reserve the right
| to be suspicious and consider any such file as
| potentially malicious. There are some ways to deal with
| it, including using a virtual machine. I was just
| wondering if anybody else had a better idea.
___________________________________________________________________
(page generated 2022-08-31 23:02 UTC)