[HN Gopher] X0.at: upload files from cURL
___________________________________________________________________
X0.at: upload files from cURL
Author : sirffuzzylogik
Score : 121 points
Date : 2021-03-15 15:15 UTC (7 hours ago)
(HTM) web link (x0.at)
(TXT) w3m dump (x0.at)
| umvi wrote:
| These are always nice little sites to have around, but they can't
| really grow much in popularity before users start abusing them to
| distribute illegal things at which point the site has to start
| doing more and more content moderation or be shut down.
| geek_at wrote:
| Can confirm. I had a public demo of my open source image
| hosting solution [1] (where you can resize images and videos by
| just entering a different URL) up for years without problems,
| until idiots started uploading CSAM (Children sexual abuse
| material).
|
| Luckily I found out before law enforcement did [2] so I
| proactively talked to my federal bureau for months generating
| Excel sheets of IPs and access times and devices and countries.
| I didn't see many of the images myself, basically just looked
| at one upload per IP which was like three in total and
| forwarded all uploads of that IP to the police but man.. what
| the hell is wrong with people. 4 digit number of uploads of
| CSAM.
|
| [1] https://github.com/HaschekSolutions/pictshare [2]
| https://blog.haschek.at/2018/fight-child-pornography-with-ra...
| ivan888 wrote:
| The process of properly reporting and working with
| authorities seems daunting. (Anecdotally,) It sounds too easy
| to implicate yourself for a technical violation of the law by
| (even unknowingly) hosting this content, or accidentally
| transferring it to one of your personal devices. Much worse
| following the advice of your local police to print out images
| which would be completely illegal! On the other hand, if the
| process was too lenient on reporters, hosting a file sharing
| service that "gets abused" with illegal content might turn
| into the ultimate scapegoat for illegal content
| users/creators/brokers
|
| Nice job going through the reporting process and I'm glad you
| blogged about it to share with others
| anderspitman wrote:
| Sad but true. One reason why easy self hosting is important for
| these types of projects.
| tobylane wrote:
| Agreed. I was using it* for TravisCI, and having moved to
| Github Actions I'm glad they have uploads stored per run.
|
| *this one and another few before it.
| jtokoph wrote:
| Came to say this. OP: If this is your site it will be used for
| piracy, underage porn and phishing within hours.
| faeyanpiraat wrote:
| It might be a honeypot for exactly that purpose aswell
| anderspitman wrote:
| What would it lead them to other than the user's VPN ip/Tor
| exit node?
| ivan888 wrote:
| It might lead to: someone who wasn't careful enough to
| use one of those options, identifying metadata in the
| file, etc
| philshem wrote:
| which makes the p2p file transfer websites so special
| https://file.pizza/ and https://webwormhole.io/
|
| (*based on https://github.com/magic-wormhole/magic-wormhole)
| [deleted]
| bityard wrote:
| Not OP, but this site has been around for months at least,
| maybe a year, it would be sad if it had to taken down right
| after landing on the front page of HN.
| [deleted]
| 40four wrote:
| I like the simplicity of it. One PHP file, throw it on a server
| with Apache and rock and roll.
|
| Other comments are right to point out that this site is setting
| itself up to be abused. My feeling is that this is intended to be
| a demo. I doubt the creator is trying to provide a real service
| here. And they might be in for a rude awakening if it gains
| traction.
|
| But, it looks like they intend this to be open source. Anyone can
| clone the repo and run this on their own server! Unfortunately,
| the repo does not have a license file, which makes me a little
| uneasy.
|
| Edit: I didn't say that very well. With no license file,
| technically we cannot actually use this code since it defaults to
| 'All rights reserved'. I think the author might not realize that
| though. It seems they intend it to be 'open' based on line 334.
|
| Also, it is not particularly _good_ PHP code, a little rough
| around the edges. But hey, it 's a cool demonstration on a very
| straight forward way to upload & share files! Could be a good
| starting point to develop further.
| s_dev wrote:
| >the repo does not have a license file, which makes me a little
| uneasy.
|
| Surely the author is bearing the liability of getting burned by
| not specifying a licence.
| cyberbanjo wrote:
| Why? I thought this type of situation would default non-
| permissive licensing in-lieu of an explicitly permissive one.
| frabert wrote:
| I don't think you have much responsibility in not specifying
| a license, since it defaults to "all rights reserved", thus
| preventing anyone else from using that code.
| gugagore wrote:
| No. If you find some code online, or on a thumb drive on the
| sidewalk, and it is unlicensed, it's incorrect to assume that
| it's equivalent to being permissively licensed for you to do
| whatever you want with it.
| detaro wrote:
| How do they "get burned" by that? Not having a license means
| you don't get to use it and are violating their copyright if
| you do so (possibly except the things specified in Github
| ToS: look at the code on github)
| s_dev wrote:
| So not having a license means a default one was used? Which
| licence is that?
| detaro wrote:
| The default is that you don't have rights to code unless
| they are explicitly given to you. The Github ToS do
| specify some rights that are granted through uploading to
| GH, but they don't constitute any usual license, nor one
| that makes any of it usable to you.
| blueflow wrote:
| "Not licensed" is the default. Its similar to what the IP
| industry calls "stealing".
| geocrasher wrote:
| With the CurlyTP protocol, any server can be uploaded to using
| curl or wget. Access might be a problem though.
|
| https://miscdotgeek.com/curlytp-every-web-server-is-a-dead-d...
| koeng wrote:
| This is kinda similar to ix.io. Cool!
| apayan wrote:
| If you like the convenience of transferring a file temporarily
| into the cloud to download it elsewhere (great for getting stuff
| out of a rancher environment), check out patchbay[0]. It uses
| what it calls 'HTTP channels' so if you start a POST request to a
| patchbay URL, it will block until a corresponding GET is made to
| the same endpoint which will receive the data from your POST. The
| operation can be done in reverse as well, with the GET blocking
| until the POST begins.
|
| [0] https://patchbay.pub/
| rogual wrote:
| This is brilliant, thank you!
| orf wrote:
| Gotta love PHP
| https://github.com/Rouji/single_php_filehost/blob/master/ind...
| calvinmorrison wrote:
| The longest running one that I've used for a long time is
|
| ix.io
| qrv3w wrote:
| I'll tack on mine as well, which you can self-host:
| https://github.com/schollz/share
|
| Also there is a cute alias you can do to easily 'share' files:
|
| alias share='f() { curl --progress-bar --upload-file "$1"
| https://share.schollz.com | tee /dev/null; echo };f'
| sparkling wrote:
| Another self-hosted script (Python) that aims for easy
| integration with curl
|
| https://github.com/kennell/curldrop
| hprotagonist wrote:
| i've been using 0x0.st pretty happily.
| _joel wrote:
| Alternatively https://transfer.sh/
| jonathantf2 wrote:
| Link is dead for me.
| _joel wrote:
| Which one? https://downforeveryoneorjustme.com/transfer.sh
| says it's up and fine for me. If you're curling don't include
| the last character that the URL has in response. That's
| effectively a carriage return/end of data
| poyu wrote:
| This is what my dig shows
|
| `transfer.sh. 2 IN A 0.0.0.0`
| _joel wrote:
| Probably your DNS blocking it then
| poyu wrote:
| Seems like my Pi-hole blocked it.
| smartbit wrote:
| It's back! I thought it stopped nov 30, 2018? +1
| _joel wrote:
| Indeed, they had a bit of a hiatus but the service was
| brought back.
| timvisee wrote:
| Shameless plug for `ffsend`, a CLI tool for Send. Though it
| doesn't use curl, it features E2E encryption. Send was originally
| created by Mozilla, and is self hostable.
|
| https://github.com/timvisee/ffsend
|
| https://github.com/timvisee/send
| the_arun wrote:
| Uploading files without auth layer - is asking for trouble IMHO.
| Change without audit trail will encourage wrong doers. But I get
| the idea, this is an example for a file upload in a simple way
| using Curl or other tools.
| ju-st wrote:
| The website is already blocked because of "Malicious
| Sources/Malnets" in the firewall of the company I work at.
| raverbashing wrote:
| "This is why we can't have nice things..." _sigh_
| southerntofu wrote:
| > Uploading files without auth layer - is asking for trouble
| IMHO.
|
| If you make it super user-friendly and advertise it as the next
| Megaupload, sure. But if you keep a small audience of good-
| faith users it's not asking for problems.
|
| If you can teach me to make my file upload as hacker-friendly
| as this service while implementing auth, i'd be glad. Here the
| entire point is you don't need further
| configuration/credentials for example to upload log/config from
| a server.
| codegeek wrote:
| But it is not a matter of "if". It is a matter of "when". Bad
| actor(s) will find out at some point if it is even remotely
| popular and then it's game over.
| derefr wrote:
| This is incompatible with using curl as your client, but one
| "hacker-friendly way to do auth" is to use Github's public
| SSH keys API.
|
| You can stand up an (SCP/SFTP-subprotocol-only) SSH server,
| and tell the user to log in with their GitHub username +
| GitHub SSH key. Then configure your SSH server to call[1] a
| check on GitHub's API to map the provided username to the
| GitHub user's set of public SSH keys. From there, the server
| treats that list exactly as if it were the user's
| ~/.ssh/authorized_keys file.
|
| [1] As it happens, I wrote an OpenSSHD plugin for exactly
| this: https://github.com/tsutsu/github-auth3
|
| Following that, you can configure PAM to continue the auth
| process however you like, policy-wise: let any GitHub user
| in; only let GitHub users in from a specific GitHub org; keep
| an LDAP directory of GitHub usernames such that you can
| attach metadata to them like "is banned" or "has used up
| their upload credits for the day" or "is on plan tier X";
| etc.
|
| Then, to actually handle the uploads, you can 1. set up
| automatic local user instantiation per remote user; 2.
| populate /etc/skel with just the right set of limited files
| to allow the user to upload into one "spool" directory; 3.
| have an inotify-like daemon that watches for files to be
| closed in that directory and handles them from there (e.g.
| uploading them to an S3 bucket, etc.)
|
| ----------
|
| Or, alternately, you can avoid building this on top of
| OpenSSH, since you're really fighting against the current by
| trying to virtualize everything, when OpenSSH expects to be
| relying on, and providing access to, a traditional POSIX
| environment.
|
| Instead, you can have your own SSH server daemon that
| provides access to a _pretend_ environment inside the SSH-
| server process, and handles SCP /SFTP upload streams through
| a custom in-process handler, the same way a web framework
| handles PUT requests.
|
| I don't know how common this is in other runtimes, but Erlang
| has an SSH server framework that you can use to implement
| exactly this. (As it happens, I've also written a high-level
| service that uses this SSH server framework to implement an
| alternative carrier for Erlang remote shell, where you can
| just SSH into the Erlang node to get a shell on it:
| https://github.com/tsutsu/exuvia. This app is also, AFAIK,
| the _only_ public /FOSS demonstration of how to use Erlang's
| SSH server library--which is kind of sad. People should play
| with things like this more! Make MUDs and such!)
| mhitza wrote:
| That's a neat idea, but I would try to simplify it a bit.
| Now, the details are fuzzy as I haven't set up anything
| like this but I know the theory behind it.
|
| You could setup a website which accepts file uploads at
| username:file_signature@example.com , when a request is
| accepted you retrieve the public ssh key for "username" and
| validate that the "file_signature" is valid for the file
| uploaded.
|
| It's a bit unintuitive to sign files with your SSH key but
| possible https://superuser.com/a/1616656/51984
| derefr wrote:
| The point of including GitHub in the chain is that it's
| acting as an SSO provider, thus:
|
| 1. letting you ban bad actors from your system by
| creating a blacklist of GitHub usernames; and
|
| 2. putting a stumbling block in the way of getting around
| #1 by signing up for 1000 accounts (signing up for a
| GitHub account takes a while; adding a _unique new_ SSH
| key[1] to that GitHub account means generating a new SSH
| keypair, which takes another while; etc. Having a GitHub
| account with an associated SSH key constitutes a very
| small proof-of-work, similar to an e-stamp.)
|
| [1] IIRC, a given SSH public key cannot be associated
| with more than one GitHub account at a time.
|
| Any SSO provider would provide the same guarantees; but
| GitHub is the only SSO provider that has people's SSH
| pubkeys on file and accessible through a public API, and
| so GitHub is the only SSO provider you can use as an SSO
| provider _through_ SSH.
|
| Also, as a separate "benefit" (though some would argue
| it's the opposite), GitHub accounts -- SSO accounts
| generally -- are more-often-than-not associated with
| people's personal identities. SSO identity-provider sign-
| up processes usually collect quite a bit of PII, and
| often do a bit of KYC with phone verification, etc. This
| means that requiring sign-in through SSO is an almost-
| perfect ward against criminal use of your service --
| since any criminal stupid enough to SSO into your site
| and then do something illegal, can be quickly-and-easily
| identified by the police by sending a warrant to the SSO
| provider.
|
| -----
|
| Besides all that, the point here is to make the process
| of uploading a file as simple as possible _for the end-
| user_ (presuming the end-user is a developer familiar
| with common CLI tools.) The backend complexity isn 't
| really a concern, if it presents a simple and
| straightforward API.
|
| With GitHub-identity-supported SSH-pubkey-auth, anyone
| who has a default SSH keypair and a GitHub account (and
| who has registered their default SSH keypair _against_
| their GitHub account) can directly jump to typing `scp .
| /file mygithubuser@uploader.example.com:` and it'll "just
| work" -- _without_ having to ever even visit
| uploader.example.com to register an account! Most
| developers have a GitHub; and most developers have used
| SCP before at some point. It 's a very familiar process.
| Less confusing, even, than GitHub's own requirement of
| needing to SSH into github.com as the "git" user.
|
| With your scheme, meanwhile, the user has to figure out
| how to sign a file using SSH -- something I don't think
| more than a handful of people in the world will have ever
| had previous experience with. That's not quite "easy" on
| the level of "just use cURL" or "just use SCP" :)
| ericbarrett wrote:
| This is a one-way road, though. The minute a bad actor finds
| out about your ungated image-hosting service, it's over, and
| you'll have a hell of a mess to clean up. If you're lucky
| it'll just be somebody trying to sell penis pills. If you're
| not, you'll have federal investigators knocking.
| pokoleo wrote:
| Try using basic auth:
|
| curl -u username:password https://
| opan wrote:
| chunk.io is a similar thing, but with auth.
|
| https://chunk.io/
| the_arun wrote:
| 1. How are you blocking someone from uploading millions of
| files programmatically?
|
| 2. How do you know someone is from small audience of good-
| faith?
|
| 3. What if a file has virus and corrupt all the files on your
| end?
|
| If you don't need auth, there are few measures you can take
| on your end:
|
| 1. TTL - Make these files temporary - They will be erased
| after x hours. Eg. x=1
|
| 2. Throttle - Limit number of uploads from a given IP/machine
| or control by uploads per sec
|
| 3. May be adding a malware scanner?
| xvector wrote:
| 4. Block IPs known for abuse. This includes VPNs, Tor exit
| nodes, and more.
| banana_giraffe wrote:
| Agreed.
|
| I use a little python script that creates a curl command to
| upload to S3 for cases where I don't have the AWS toolchain on
| a remote box.
|
| Not as easy as a single command, but at least I'm less likely
| to be sending files off to some random site for everyone to
| see.
| mratzloff wrote:
| For simple peer-to-peer file sending between technically-inclined
| people, I use Magic Wormhole.
|
| https://github.com/magic-wormhole/magic-wormhole
| naturalpb wrote:
| One can upload a file to their Dropbox via a cURL post, provided
| they have created an app and have an access token, which just
| takes a few minutes to set up.
|
| curl -X POST https://content.dropboxapi.com/2/files/upload
| --header "Authorization: Bearer ACCESSTOKEN" --header "Dropbox-
| API-Arg: {\"path\": \"/DROPBOXFILEPATH/DROPBOXFILENAME\"}"
| --header "Content-Type: application/octet-stream" --data-binary
| @/LOCALFILEPATH/LOCALFILENAME
| acco102 wrote:
| Sia https://siasky.net/ has also upload from curl.
| laurensr wrote:
| Alternatively https://bashupload.com/
| uploaderwin wrote:
| Yeah this is asking for trouble. We only had a small demo on our
| homepage where users could upload media files and they were
| deleted after 24 hours and still some people managed to abuse it
| and nearly got our site killed, domain blacklisted in Google with
| a big red screen of death.
|
| I don't want to spam any links here but if you are interested
| please do look at my last post about the dangers of doing this
| and lessons I learned from my mistake.
|
| Please do not keep the files for 10 days. Even 24 hours is a
| deal-breaker. From what I've learned, anything more than 30
| minutes can get you into trouble.
| dheera wrote:
| I once had a location-based file sharing service that also got
| blacklisted by Google with no recourse. I hate Google trying to
| police the internet with no timely appeals process.
|
| I wonder though if you could simply just block the Google
| crawler and bypass it. Or use a JavaScript to auto-POST
| something before the file gets sent for download. The Google
| crawler doesn't issue POST requests as far as I know.
| gowld wrote:
| By "police", do you mean "warn people about dangers" ?
| unglaublich wrote:
| More like taking suspects into custody.
| cjohansson wrote:
| and without responding to questions or providing evidence
| dheera wrote:
| There wasn't any danger. Nothing more than Google Drive or
| Dropbox. And they didn't have any way to contact them and
| explain. Way to heavy-handedly shut down a potential
| business idea.
| xyzzy_plugh wrote:
| Drive and Dropbox are at least moderated.
| DyslexicAtheist wrote:
| I'm surprised to hear that because their response in
| dealing with actual child porn is absolute atrocious. A
| sick and sad story:
|
| Couple years ago got a DM from an ex-colleague and
| security researcher who discovered child porn that was
| publicly accessible. he contacted Dropbox several times
| over the course of 3 weeks. Weeks later the links were
| still up. I reached out to somebody I knew at Dropbox who
| said they were reluctant to do me this favor and deal
| with that matter and would prefer if I continue
| contacting their security. I continued trying on LinkedIn
| and contacted several people in Germany and the UK. No
| response other than "thank you for your email". The head
| of security in Germany even blocked me for saying _"
| there is child porn on the site please help me get hold
| of somebody in charge"_. Getting really fed up by then, I
| contacted sales from my company email (a fortune 500
| company) and asked them to give me a quote for what
| looked to them like a multi million $ client. Within 2
| hours I got a call from the VP of sales to talk about my
| _" storage needs"_. I told them about the child porn and
| that they are helping to actively distribute it now since
| several weeks. One of the videos was a girl not older
| than 7 getting raped and tortured. It took another 3 days
| to take down the material.
| planetafro wrote:
| Source? ...moderated via automation or human?
| CobrastanJorji wrote:
| https://support.google.com/a/answer/172541?hl=en
|
| > Google Drive scans a file for viruses before the file
| is downloaded or shared. If a virus is detected, users
| cannot convert the infected file to a Google Doc, Sheet,
| or Slide, and they'll receive a warning if they attempt
| these operations.
|
| So at least some degree of automated moderation is going
| on. Frankly, I'd be astounded if some amount of scanning
| isn't being done for illegal content and/or phishing
| stuff.
| PeterisP wrote:
| Automation; the bare minimum would be to scan for known
| child sexual abuse material hashes - if you're not doing
| that, then opening up anonymous uploads is very risky, as
| for CSAM (unlike most other things) you may be personally
| liable even if it's distributed there without your
| knowledge. Cloudfare's CSAM scanning tool is one option
| that may help, there are other options.
|
| You can't rely on the good faith of users, if your
| service is easily usable for crime, it will be used for
| it.
| SeriousM wrote:
| Because this is an austrian domain you need to put an Impressum
| on it with details about the hosters name and address. Sorry to
| spoil the fun :(
| southerntofu wrote:
| Another alternative: the famous nullpointer
| https://github.com/mia-0/0x0
|
| A small script i use very regularly:
| #!/usr/bin/env bash if [ ! -f $1 ]; then echo "MISSING:
| $1"; exit 1; fi torify curl -F"file=@$1"
| https://YOURSERVER || echo "UPLOAD FAILED (code: $?)"
___________________________________________________________________
(page generated 2021-03-15 23:00 UTC)