[HN Gopher] Ask HN: Suggestions to host 10TB data with a monthly...
___________________________________________________________________
Ask HN: Suggestions to host 10TB data with a monthly +100TB
bandwidth
I'm looking for suggestions to possibly replace dedicated servers
that would be cost-effective considering the bandwidth.
Author : magikstm
Score : 124 points
Date : 2023-05-27 14:52 UTC (8 hours ago)
| justinclift wrote:
| Any idea on how many files/objects, and how often they change?
|
| Also, any idea on the number of users (both average, and peak)
| you'd expect to be downloading at once?
|
| Does latency of their downloads matter? eg do downloads need to
| start quickly like a CDN, or as long as they work is good enough?
| andai wrote:
| If it's for internal use, I have had good results with Resilio
| Sync (formerly BitTorrent Sync).
|
| It's like Dropbox except peer to peer. So it's free, limited only
| by your client side storage.
|
| The catch is it's _only_ peer to peer (unless they added a
| managed option), so at least one other peer must be online for
| sync to take place.
| danieldk wrote:
| They don't really maintain the regular Sync client anymore,
| only the expensive enterprise Connect option. My wife and I
| used Resilio Sync for years, but had to migrate away, since it
| had bugs and issues with newer OS versions, but they didn't
| care to fix them. Let alone develop new features.
| j45 wrote:
| The OP request would benefit from details, but the solution
| depends on what format the data is and how to be shared.
|
| Assuming the simplest need is making files available :
|
| 1) Sync.com provides unlimited hosting and file sharing from it.
|
| Sync is a decent Dropbox replacement with a few more bells and
| whistles.
|
| 2) BackBlaze business let's you deliver files for free via their
| CDN. $5/TB per month storage plus free egress via their CDN.
|
| https://www.backblaze.com/b2/solutions/developers.html
|
| Backblaze seems to be 70-80% cheaper than S3 as it claims.
|
| Traditional best practice cloud paths are optimized to be a best
| practice to generate profit for the cloud provider.
|
| Luckily it's nice to rarely be alone or the first to have a need.
| subhro wrote:
| tarsnap
| aborsy wrote:
| S3 is probably the highest quality. It's enterprise grade : fast,
| secure with a lot of tiers and controls.
|
| If you recover only small data, it's also not expensive. The only
| problem is if you recover large data. That would be a major
| problem.
| capableweb wrote:
| 10TB storage + 100TB bandwidth and S3 will easily be +1000 USD
| per month, while there are solutions out there that are fast
| and secure with unrestricted bandwidth for less than 100 USD
| per month. Magnitude cheaper with same grade in "enterprisey".
| aborsy wrote:
| Well, I said, if you store small data. For large data, sure,
| prohibitively expensive!
|
| I don't think many other solutions are equally fast and
| secure.
|
| AWS operation is pretty transparent, documented, audited and
| used by governments. You can lock it down heavily with IAM
| and a CMK KMS key, and audit the repository. The physical
| security is also pretty tight, and there is location
| redundancy.
|
| Even hetzner doesn't have proper redundancy in place. Other
| major providers in France burned down (apparently with with
| data loss), or had security problems with hard drives stolen
| in transport.
|
| I don't work for AWS, don't have much data in there, just
| saying. GCP and Azure are probably also good.
| KomoD wrote:
| OPs post is literally about 10TB storage and 100+TB/mo
| charcircuit wrote:
| >Well, I said, if you store small data.
|
| Well, the OP said he would be using >100 TB a month.
|
| >GCP and Azure are probably also good.
|
| They similarly charge 100x for bandwidth. No they are not a
| good option either.
| sacnoradhq wrote:
| Too expensive. RRS would be the only consideration, and it's
| not a for-sure thing compared to other options.
| influx wrote:
| I would do it in S3 + Cloudfront if your budget supported it.
| Mostly because I don't want to maintain a server somewhere.
| indigodaddy wrote:
| BuyVM has been around a long time and have a good reputation.
| I've used them on and off for quite a while.
|
| They have very reasonably priced KVM instances with unmetered 1G
| (10G for long-standing customers) bandwidth that you can attach
| "storage slabs" up to 10TB ($5 per TB/mo). Doubt you will find
| better value than this for block storage.
|
| https://buyvm.net/block-storage-slabs/
| j45 wrote:
| Nice. How have you found their uptime and reliability?
| indigodaddy wrote:
| Honestly haven't hosted anything important enough for me to
| track that. There is an unofficial site that tracks their
| uptime apparently: https://www.buyvmstatus.com/
| j45 wrote:
| Great, I'll try it out.
| sacnoradhq wrote:
| Backblaze, MEGA, or S3 RRS.
| winrid wrote:
| You could do this for about $1k/mo with Linode and Wasabi.
|
| For FastComments we store assets in Wasabi and have services in
| Linode that act as an in-memory+on disk LRU cache.
|
| We have terabytes of data but only pay $6/mo for Wasabi, because
| the cache hit ratio is high and Wasabi doesn't charge for egress
| until your egress is more than your storage or something like
| that.
|
| The rest of the cost is egress on Linode.
|
| The nice thing about this is we gets lots of storage and
| downloads are fairly fast - most assets are served from memory in
| userspace.
|
| Following thread to look for even cheaper options without using
| cloudflare lol
| qeternity wrote:
| > You could do this for about $1k/mo with Linode and Wasabi.
|
| This is still crazy expensive. Cloud providers have really
| warped people's expectations.
| winrid wrote:
| You could do it through interserver for $495/mo (5 20gb sata
| disks, 150tb free bandwidth). 10gbps link. 128gb ram for page
| cache.
|
| Backups probably wouldn't be much more.
| winrid wrote:
| Well, for us it's actually really cheap because we really
| just want the compute. The bandwidth is just a bonus.
|
| Actually, since the Akami acquisition it would be even
| cheaper.
|
| $800/mo to serve 100TB with fairly high bandwidth and low
| latency from cold storage is a good deal IMO. I know
| companies paying millions a year to serve less than a third
| of that through AWS when you include compute, DB, and
| storage.
| qeternity wrote:
| Fine, but now you're changing the comparison. Spending
| millions on compute with low bandwidth requirements doesn't
| make it stupid. It probably still is, but that's a
| different conversation.
| winrid wrote:
| lol no, it sure is. But OP didn't give a price range or
| durability requirements...
| winrid wrote:
| Actually, with Interserver and Wasabi you could probably get it
| under $100/mo.
|
| You could just run Varnish with the S3 backend. Popular files
| will be cached locally on the server, and you'll pay a lot less
| for egress from Wasabi.
| winrid wrote:
| EDIT it would be ~$500 through linode + wasabi. I was thinking
| of included bandwidth, but if you just pay overages it's
| cheaper.
| rapjr9 wrote:
| I helped run a wireless research data archive for a while. We
| made smaller data sets available via internet download but for
| the larger data sets we asked people to send us a hard drive to
| get a copy. Sneakernet can be faster and cheaper than using the
| internet. Even if you wanted to distribute 10TB of _new_ data
| every month, mailing hard drives would probably be faster and
| cheaper, unless all your customers are on Internet2 or unlimited
| fiber.
| radar1310 wrote:
| Nothing beats the bandwidth of a station wagon full of hard
| drives:-)
| mindcrash wrote:
| Several huge infrastructure providers offer decent VPS servers
| and bare metal with free bandwidth for pretty reasonable prices
| nowadays.
|
| You might want to check out OVH or - like mentioned before -
| Hetzner.
| scottmas wrote:
| Surprised no one has said Cloudflare Pages. Might not work though
| depending on your requirements since there's a max of 20,000
| files of no more than 25 mb per project. But if you can fit under
| that, it's basically free. If your requirements let you break it
| up by domain, you can split your data across multiple projects
| too. Latency is amazing too since all the data is on their CDN.
| risyachka wrote:
| Find a bare metal server with 1GBit connection and you are all
| set.
| dboreham wrote:
| No, you'd need to find peering that supports 100T/mo at a
| reasonable cost.
| risyachka wrote:
| why do I need it?
| jedberg wrote:
| It's impossible to answer this question without more information.
| What is the use profile of your system? How many clients, how
| often, what's the burst rate, what kind of reliability do you
| need? These all change the answer.
| aledalgrande wrote:
| And what kind of latency is needed, what geo areas are
| involved? Budget? Engineers available?
| noja wrote:
| To host it for what? A backup? Downloading to a single client?
| Millions of globally distributed clients uploading and
| downloading traffic? Bittorrent?
| ericlewis wrote:
| Cloudflare R2, egress is free. Storing 10TB would be about $150 a
| month.
| activiation wrote:
| Holy shit that's expensive
| wwwtyro wrote:
| If it fits your model, WebTorrent[0] can offload a lot of
| bandwidth to peers.
|
| [0] https://github.com/webtorrent/webtorrent
| giantrobot wrote:
| At some point you still need a seed for that 10TB of data with
| some level of reliability. WebTorrent only solves the monthly
| bandwidth iff you've got some high capacity seeds (your servers
| or long-term peers).
| sacnoradhq wrote:
| And, at that point, you might as well be mirroring it over
| IPFS too.
|
| One multi-vendor, zero-knowledge, HA solution is
| https://tahoe-lafs.org/trac/tahoe-lafs
| callamdelaney wrote:
| Wasabi, unlimited bandwidth. $5.99/tb/month. Though it is object
| storage.
|
| See: https://wasabi.com/cloud-storage-pricing/#cost-estimates
|
| They could really do with making the bandwidth option on this
| calculator better.
| mkroman wrote:
| Wasabi does not have any sort of unlimited bandwidth. In fact,
| they explicitly say that if your download exceeds 100TB/month
| your use case is not a good fit. https://wasabi.com/paygo-
| pricing-faq/#free-egress-policy
| spiffytech wrote:
| Wasabi also bills you for a minimum of 90 days of storage, so
| it's only appropriate if your data has low turnover.
|
| https://wasabi.com/paygo-pricing-faq/#minimum-storage-
| durati...
| mdasen wrote:
| The 100TB was just an example. They don't want you using more
| bandwidth than your storage. If you're storing 500GB, then
| your bandwidth usage should be less than 500GB.
|
| Wasabi isn't meant for scenarios where you're going to be
| transferring more than you're storing.
| duskwuff wrote:
| > Wasabi isn't meant for scenarios where you're going to be
| transferring more than you're storing.
|
| Which is basically a roundabout way of saying, they're
| offering storage for backups, not for content distribution.
| sacnoradhq wrote:
| Personally, at home, I have ~600 TiB and 2 Gbps without a data
| cap.
|
| I can't justify colo unless I can get 10U for $300/month with 2kW
| of PDU, 1500 kWh, and 1 GbE uncapped.
| amluto wrote:
| This is pretty close:
|
| http://he.net/colocation.html
| sacnoradhq wrote:
| While Hurricane is nice, they're 1800 miles from me. :)
|
| I'll take the risk of colo within the same region. If this
| region were gone, my data would be meaningless.
| jayonsoftware wrote:
| Spend some time on https://www.webhostingtalk.com/ and you will
| find a lot of info. For example https://www.fdcservers.net/ can
| give you 10TB storage and 100GB bw for around $300....but keep in
| mind the lower the price you pay, the lower the quality...just
| like any other products.
| roetlich wrote:
| As a previous employee there I'm very biased, but I think
| bunny.net has pretty good pricing :)
| pierat wrote:
| Sounds like you could find someone with a 1Gbps symmetric fiber
| net connection, and pay them for it and colo. I have 1Gbps and
| push that bandwidth every month. You know, for yar har har.
|
| And that's only 309Mbits/s (or 39MB/s).
|
| And a used refurbished server you can easily get loads or ram,
| cores out the wazoo, and dozens of TB's for under $1000. You'll
| need a rack, router, switch, and batt backup. Shouldn't cost much
| more than $2000 for this.
| k8sToGo wrote:
| People here suggest Hetzner. Just be aware that their routing is
| maybe not as good as you get for more expensive bandwidth.
| GC_tris wrote:
| Can you be more specific?
|
| Hetzner has excellent connectivity:
| https://www.hetzner.com/unternehmen/rechenzentrum/ They are
| always working to increase their connectivity. I'd even go so
| far to claim that in many parts of the world they outperform
| certain hyperscalers.
| k8sToGo wrote:
| I used to have a dedicated server there and what happened to
| me is that my uploads were fast, but my downloads were slow.
| Looking at an MTR route, it was clear that the route back to
| me was different (perhaps cheaper?). With google drive for
| example I could always max out my gbit connection. Same with
| rsync.net
|
| Also I know that some cheaper Home ISPs also cheap out on
| peering.
|
| Now, this was some time ago, so things might have changed,
| just as you suggested.
| KaiserPro wrote:
| whats your budget?
|
| who are you serving it to?
|
| how often does the data change?
|
| is it read only?
|
| What are you optimising for, speed, cost or availability? (pick
| two)
| delduca wrote:
| I would go with BunnyCDN at front of some storage.
| johnklos wrote:
| If price is a consideration, you might consider two 10 TB hard
| drives on machines on two home gbps Internet connections. It's
| highly unlikely that both would go down at the same time, unless
| they were in the same area, on the same ISP.
| sireat wrote:
| How do you set up load balancing for those two connections?
|
| That is yourdomain.com -> IP_ISP1, IP_ISP2
|
| Going the other way from yourserver -> outside would indicate
| some sort of bonding setup.
|
| It is not trivial for a home lab.
|
| I use 3 ISPs at home and just keep each network separate
| (different hardware on each) even though in theory the
| redundancy would be nice.
| johnklos wrote:
| Just use two A records for the one DNS name, and let the
| clients choose.
|
| The other way is to have two names, like dl1 and dl2, and
| have your download web page offer alternating links,
| depending on how the downloads are handled.
|
| You very rarely can do multi-ISP bonding, often not even with
| multiple lines from the same ISP, unfortunately.
| bombcar wrote:
| That's not even saturating a Gb/s line. Many places offer
| dedicated with that kind of bandwidth.
| jstx1 wrote:
| Is there a reason people aren't suggesting some cloud object
| storage service like S3, GCS or Azure storage?
| abatilo wrote:
| Those bandwidth costs are going to be so expensive
| cheeselip420 wrote:
| Use R2
| Hamuko wrote:
| I once had a Hetzner dedicated server that held about 1 TB of
| content and did some terabytes of traffic per month (record
| being 1 TB/24 hours). Hetzner charged me 25EUR/month for that
| server and S3 would've been like $90/day at peak traffic.
| hdjjhhvvhga wrote:
| Because they are an order of magnitude more expensive.
| bakugo wrote:
| The answer to this question depends entirely on the details of
| the use case. For example, if we're talking about an HTTP server
| where a small number of files are more popular and are accessed
| significantly more frequently than most others, you can get a
| bunch of cheap VPS with low storage/specs but a lot of cheap
| bandwidth to use as cache servers to significantly reduce the
| bandwidth usage on your backend.
| wongarsu wrote:
| Depends on what exactly you want to do with it. Hetzner has very
| cheap Storage boxes (10TB for $20/month with unlimited traffic)
| but those are closer to FTP boxes with a 10 connection limit.
| They are also down semi-regularly for maintenance.
|
| For rock-solid public hosting Cloudflare is probably a much
| better bet, but you're also paying 7 times the price. More than a
| dedicated server to host the files, but you get more on other
| metrics.
| novok wrote:
| Hetzner is also fairly slow network bandwidth wise unless
| you're in Europe.
| codersfocus wrote:
| I wouldn't call Cloudflare rocksolid, assuming you mean their
| R2 offering. It goes down pretty regularly.
| KomoD wrote:
| > Hetzner has very cheap Storage boxes (10TB for $20/month with
| unlimited traffic)
|
| * based on fair use
|
| at 250 TB/mo:
|
| > In order to continue hosting your servers with us, the
| traffic use will need to be drastically reduced. Please check
| your servers and confirm what is using so much traffic, making
| sure it is nothing abusive, and then find ways of reducing it.
| coverband wrote:
| Thanks. It's important to be very much aware of this when
| being enticed by the promise of unlimited bandwidth.
| j45 wrote:
| Thanks for finding this.
|
| Unlimited rarely is.
|
| Looks like backblaze (other post below) has a free bandwidth
| and cheap storage solution
| yakubin wrote:
| Wow! It's the first time I'm hearing of this Hetzner offering.
| It's ideal for my offsite backup needs. Thanks!
| kens wrote:
| That reminds me of the entertaining "I just want to serve 5
| terabytes. Why is this so difficult?" video that someone made
| inside Google. It satirizes the difficulty of getting things done
| at production scale.
|
| https://www.youtube.com/watch?v=3t6L-FlfeaI
| trhr wrote:
| Nothing in that video is about scale. Or the difficulty of
| serving 5TB. It's about the difficulty of implementing n+1
| redundancy with graceful failover inside cloud providers.
|
| User: "I want to serve 5TB."
|
| Guru: "Throw it in a GKE PV and put nginx in front of it."
|
| Congratulations, you are already serving 5TB at production
| scale.
| sacnoradhq wrote:
| The interesting thing is there also paradoxes of large scale:
| things that get more difficult with increasing size.
|
| Medium- and smaller-scale can often be more flexible because
| they don't have to incur the pain of nonuniformity as scale
| increases. While they may not be able to afford optimizations
| or discounts with larger, standardized purchases, they can
| provide more personalized services large scale cannot hope to
| provide.
| ergocoder wrote:
| I thought that was making fun of how difficult to implement
| anything at Google.
| psychphysic wrote:
| I'd suggest looking into "seedboxes" which are intended for
| torrenting.
|
| I suspect the storage will be a bigger concern.
|
| Seedhost.eu has dedicated boxes with 8TB storage and 100TB
| bandwidth for EUR30/month. Perhaps you could have that and a
| lower spec one to make up the space.
|
| Prices are negotiable so you can always see if they can meet your
| needs for cheaper than two separate boxes.
| GOATS- wrote:
| Yep, resellers of dedicated machines rent servers in bulk so
| you can often get boxes for way cheaper than you would directly
| from the host. Take a look at https://hostingby.design as an
| example.
| dspillett wrote:
| _> I 'd suggest looking into "seedboxes" which are intended for
| torrenting._
|
| Though be aware that many (most?) seedbox arrangements have no
| redundancy, in fact some are running off RAID0 arrays or
| similar. Host has a problem like a dead drive: bang goes your
| data. Some are very open about this, afterall for the main use
| case cheap space is worth the risk, some far less so...
|
| Of course if the data is well backed up elsewhere or otherwise
| easy to reproduce or reobtain this may not be a massive issue
| and you've just got restore time to worry about (unless one of
| your backups can be quickly made primary so restore time is as
| little as a bit of DNS & other configuration work).
| KomoD wrote:
| Ultra.cc is pretty great too.
| bluedino wrote:
| What are you using now and what does it cost?
| snihalani wrote:
| Cloudflare R2 or Oracle Cloud Infra or Hetzner
| rozenmd wrote:
| Cloudflare R2?
| qeternity wrote:
| Hetzner auctions have a 4x 6TB server for EUR39.70/mo.
|
| Throw that in RAID10 and you'll have 12TB usable space with >
| 300TB bandwidth.
| princevegeta89 wrote:
| Isn't that enterprise HDD going to be slower for a cloud
| instance though?
| Hamuko wrote:
| SSDs are definitely not "cost-effective" for 10 TB if that's
| what you're suggesting.
| dehrmann wrote:
| It looks like HDD prices are at ~40% of SSD prices (new
| drives, not cloud-hosted). SSDs are starting to make sense
| for more things, now.
| ComputerGuru wrote:
| That comparison only holds for the smaller sizes.
| dehrmann wrote:
| For an 8 TB SSD and 18 TB HDD, it's more like 3x.
| dinvlad wrote:
| Storage Boxes are even cheaper, 20 EUR (or less if you're
| outside Europe) for 10TB + unlimited bandwidth.
| midasuni wrote:
| 1gbit is 300T a month, 10g is 3000T a month.
|
| There's always a limit, that might be measured in TB, PB or
| EB, and may be what you determine practical or not, but it's
| there
| j45 wrote:
| Really good way of putting it.
| wongarsu wrote:
| If we are talking about serving files publicly I'd go with
| the EUR40 server for flexibility (the storage boxes are kind
| of limited), but still get a EUR20 Storage Box to have a
| backup of the data. Then add more servers as bandwidth and
| redundancy requires.
|
| But if splitting your traffic across multiple servers is
| possible you can also get the EUR20 storage box and put a
| couple Hetzner Cloud servers with a caching reverse proxy in
| front (that's like 10 lines of Nginx config). The cheapest
| Hetzner Cloud option is the CAX11 with 4GB RAM, 40GB SSD and
| 20TB traffic for EUR3,79. Six of those plus the Storage Box
| gives you the traffic you need, lots of bandwidth for usage
| peaks, SSD cache for frequently requested files, and easily
| upgradable storage in the Storage Box, all for EUR42. Also
| scales well at $3,79 for every additional 20TB traffic, or
| $1/TB if you forget and pay fees for the excess traffic
| instead.
|
| You will be babysitting this more than the $150/month
| cloudflare solution, but even if you factor in the cost of
| your time you should come out ahead.
| aledalgrande wrote:
| > even if you factor in the cost of your time you should
| come out ahead
|
| There is always the hidden cost of not spending time on
| activities that are core to your business (if this is
| indeed for a business) that would make multiples of the
| money CF costs you.
| midasuni wrote:
| Yup. Managing the CF account is a hidden cost you need to
| account for.
| j45 wrote:
| There is more and more that can be done like proxmox to
| help make self hosted as much as an appliance as anything
| else.
| wahnfrieden wrote:
| bittorrent and some home servers
| bosch_mind wrote:
| Cloudflare R2 has free egress, cheap storage
| ttul wrote:
| And they just added TCP client sockets in Workers. We are just
| one step step away from being able to serve literally anything
| on their amazing platform (listener sockets).
| itake wrote:
| Does this mean you can self host ngrok?
| ttul wrote:
| Only client sockets are available. So what you can do is
| build a worker that receives HTTP requests and then uses
| TCP sockets to fetch data from wherever, returning it over
| HTTP somehow.
| itake wrote:
| oh darn. You can't receive tcp connections. lame.
| gok wrote:
| Looks like it would be $150/month for OP's needs?
| slashdev wrote:
| Cloudflare has free bandwidth up to a point and then they will
| charge you. That's not really that surprising though.
| ozr wrote:
| I think that's the case with their free CDN/DNS/proxy
| offering, not R2.
| FBISurveillance wrote:
| PSA: The informal point at which they'll reach out used to be
| about 300TB/month about 2 years ago.
| slashdev wrote:
| Thanks, I've heard about this, but never actually with
| numbers. It's good to know.
| Atlas22 wrote:
| It may depend on the makeup of data or something. They
| "requested" one of my prior projects go on the enterprise
| plan after about 50TB, granted the overwhelming majority of
| transfer was for distributing binary executables so I was
| in pretty blatant violation of their policy. This was
| 2015ish, so the limit could also have gone up over time as
| bandwidth gets cheaper too.
| dark-star wrote:
| you can definitely do this at home on the cheap. As long as you
| have a decent internet connection, that is ;) 10TB+ harddisks are
| not expensive, you can put them in an old enclosure together with
| a small industrial or NUC PC in your basement
| sacnoradhq wrote:
| I current have 45 WUH721414ALE6L4 drives in a Supermicro JBOD
| SC847E26 (SAS2 is way cheaper than SAS3) connected to an LSI
| 9206-16e controller (HCL reasons) via hybrid Mini SAS2 to Mini
| SAS3 cables. The SAS expanders in the JBOD are also LSI and
| qualified for the card. The hard drives are also qualified for
| the SAS expanders.
|
| I tried this using Pine ROCKPro64 to possibly install Ceph
| across 2-5 RAID1 NAS enclosures. The problem is I can't get any
| of their dusty Linux forks to recognize the storage controller,
| so they're $200 paperweights.
|
| I wrote a SATA HDD "top" utility that brings in data from
| SMART, mdadm, lvm, xfs, and the Linux SCSI layer. I set
| monitoring to look for elevated temperature, seek errors, scan
| errors, reallocation counts, offline reallocation, and
| probational count.
| InvaderFizz wrote:
| What's your redundancy setup in the 45 drive configuration? I
| would guess 20-22 mirrors with 1-5 hotspares, but it's not
| clear.
| jszymborski wrote:
| Wasabi.com is usually a good bet when your primary cost is
| bandwidth.
| slashdev wrote:
| > If your monthly egress data transfer is less than or equal to
| your active storage volume, then your storage use case is a
| good fit for Wasabi's free egress policy
|
| > If your monthly egress data transfer is greater than your
| active storage volume, then your storage use case is not a good
| fit for Wasabi's free egress policy.
|
| https://wasabi.com/paygo-pricing-faq/
| renewiltord wrote:
| Their pricing was good enough that I just stored a spacer
| file with them to qualify for higher bandwidth transfer.
| jszymborski wrote:
| wow, that's news to me. thanks for the info.
| ez_mmk wrote:
| Had 300tb of traffic on a Hetzner server up and down no problem
| with much storage
| walthamstow wrote:
| I would also like to ask everyone about suggestions for deep
| storage of personal data, media etc. 10TB with no need for access
| unless in case of emergency data loss. I'm currently using S3
| intelligent tiering.
| msh wrote:
| Hertzner storagebox or back blaze b2 is the cheapest options.
| Hamuko wrote:
| I do my archiving to S3 Glacier Deep Archive but my data volume
| is still so low that Amazon doesn't bother charging my card.
| hossbeast wrote:
| Does it accumulate month to month and eventually they charge
| you once a threshold is reached?
| Hamuko wrote:
| I think they'll charge me only when my current monthly
| statement is enough to charge. Pretty sure I've never been
| charged so far with my monthly statement being like
| 0.02EUR.
| kuratkull wrote:
| I have the same situation with backblaze B2, cents a
| month bill, they roll that into one ~ yearly charge.
| m-p-3 wrote:
| how much data are we talking about?
| Hamuko wrote:
| Some tens of gigabytes at this point? It's definitely not a
| lot. Mostly just some stuff that doesn't make sense to keep
| locally but I still want to have a copy in case a disaster
| strikes.
| xrisk wrote:
| How do you achieve deduplication with S3?
| oefrha wrote:
| <s>GSuite</s> Google Workplace business plan is still
| essentially unlimited, for 10TB anyway.
| harrymit907 wrote:
| Wasabi is the best option for you. 10TB would be around
| 60$/month and they offer free egress as much as your storage.
| So you can download upto 10TB per month.
| KomoD wrote:
| $60/mo for NA and EU, otherwise $70/mo
|
| Also read their free egress policy first:
| https://wasabi.com/paygo-pricing-faq/#free-egress-policy
| NotYourLawyer wrote:
| Tarsnap.
| sacnoradhq wrote:
| Too expensive for all but critical use-cases. MEGA and
| Backblaze are way, way cheaper.
| jedberg wrote:
| AWS Glacier. That's where I keep (one copy of) my wedding
| video, all the important family photos, etc.
| Atlas22 wrote:
| I like to use rsync.net for backups. You can use something like
| borg, rsync, or just sftp/sshfs mount. Its not as cheap as
| something like S3 deep (in terms of storage) but it is pretty
| convient. The owner is a absolute machine and frequently visits
| HN too.
| ericpauley wrote:
| S3 is tough to beat on storage price. Another plus is that the
| business model is transparent, i.e., you don't need to worry
| about the pricing being a teaser rate or something.
|
| Of course the downside is that, if you need to download that
| 10TB, you'll be out $900! If you're worried about recovering
| specific files only this isn't as big an issue.
| seized wrote:
| Glacier Deep Archive is exactly what you want for this, that
| would be something like $11/month ongoing, then about $90/TB in
| the event of retrieval download. Works well except for tiny
| (<150KB) files.
|
| Note that there is Glacier and Glacier Deep Archive. The latter
| is cheaper but longer minimum storage periods. You can use it
| as a life cycle rule.
| NilsIRL wrote:
| OVH Cloud Archive seems to have very attractive prices if
| you're not accessing the data often
| https://www.ovhcloud.com/en-gb/public-cloud/prices/#473
| bicijay wrote:
| Backblaze B2 + CDN on top of it
| gravitronic wrote:
| Cloudflare on top would be free bandwidth (bandwidth alliance)
| Atlas22 wrote:
| Unless its 100TB/mo of pure HTML/CSS/JS (lol) cloudflare will
| demand you be on enterprise plan long before 100TB/mo. The
| fine print makes it near useless for any significant volume.
| FBISurveillance wrote:
| Hetzner would work too.
| pollux1997 wrote:
| Copy the data to disk and ship them to the place they need to be
| used.
| cheeseprocedure wrote:
| Datapacket has a number of dedicated server configurations in
| various locations and offers unmetered connections:
|
| https://www.datapacket.com/pricing
___________________________________________________________________
(page generated 2023-05-27 23:01 UTC)