[HN Gopher] Backblaze is now a Terraform provider
___________________________________________________________________
Backblaze is now a Terraform provider
Author : caution
Score : 206 points
Date : 2021-03-11 17:20 UTC (5 hours ago)
(HTM) web link (www.backblaze.com)
(TXT) w3m dump (www.backblaze.com)
| OJFord wrote:
| But unfortunately plugins cannot provide remote backends, and
| it's a little clunky to use the S3 one:
|
| https://github.com/hashicorp/terraform/issues/27304
| javajosh wrote:
| It's interesting because it feels like a small turf war. The
| "Amazon Provider Team" wants to own the s3 provider backend,
| and you want to make it pluggable. Is this to avoid an explicit
| "b2" provider? I can see how it might be confusing to use
| terraform and see an s3 provider and then have it...not use s3.
| :)
| renewiltord wrote:
| Very cool! Great job, guys.
|
| Do you use a similar technique to
| https://poweruser.blog/embedding-python-in-go-338c0399f3d5 to
| embed the Python SDK?
|
| Also, was there a reason it's not against the B2 API? Not a
| judgment, just curious about the design tradeoffs in a
| professional-talking-to-professional sense.
| AYBABTME wrote:
| That design choice prevents their provider from working with
| official Terraform container images. They should make the API
| calls directly in Go, it's just weird to do it via Python...
| They have very few resources exposed on their API so it's not
| like writing a Go client would be all that hard.
| [deleted]
| nikisweeting wrote:
| I love B2's free ingress and egress when you use CloudFlare, they
| utterly destroy the competition on price.
|
| But my true dream would be for backblaze to someday offer ZFS as
| a service.
|
| I want to `zfs send -i my_pool@2020-03-11 | b2zfs recv
| some_bucket_id`, then be able to view my snapshots and files
| within in the backblaze web UI, and restore with `b2zfs send -i
| some_bucket_id/my_pool@2020-03-11 | zfs recv my_pool`.
|
| You can already mount B2 as a FUSE filesystem with something like
| ExpanDrive, then write ZFS raw file vdevs to the B2 FUSE mount,
| but it's horrifically slow and probably too janky for any real
| use.
| ptomato wrote:
| rsync.net's zfs as a service is really great for that, though a
| bit more expensive than b2 is.
| bpye wrote:
| Rsync.net looks great but the storage cost is 5x B2
| unfortunately
|
| EDIT: As mentioned below this is for the most expensive
| (lowest capacity) tier. I'd been comparing for my own home
| use and so I would be unlikely to exceed 10TB but if you're
| looking at higher capacity then maybe the calculus is
| different.
| rsync wrote:
| ... for small quantities, yes.
|
| For large quantities, it is 3x, and actually less since
| there are no charges for ingress/egress.
|
| Further questions/comments over email, please, since this
| is BBs HN thread and I don't want to butt in.
| kgog wrote:
| maybe you need borg with borgbase.
| snicker7 wrote:
| Terraform is great, but I wish more cloud providers had Guix/Nix
| integration.
| emptysongglass wrote:
| You can deploy and manage NixOS on Big Cloud using Terraform:
| https://nix.dev/tutorials/deploying-nixos-using-terraform.ht...
| CameronNemo wrote:
| Are there any providers that to have such integration? I have
| only lightly used nix. Having a hard time understanding what a
| cloud provider integration would look like.
| ethnt wrote:
| You could use NixOps[0] for Nix but I'm not sure you can
| directly compare Terraform and Guix/Nix? My set up involves
| Terraform for infrastructure and Nix for provisioning, and it's
| working for me so far.
|
| [0] https://github.com/NixOS/nixops
| [deleted]
| majormjr wrote:
| I was just looking at this new Terraform provider yesterday how
| timely. Nice to see the quickstart guide this will be helpful for
| managing the buckets and application keys.
| atYevP wrote:
| Yev from Backblaze here -> That's great! Glad the quick-start
| makes it simple.
| rattray wrote:
| I sure wish this post had included a few code snippets, just to
| make it extra-clear what using Backblaze through Terraform might
| look like.
| vbsteven wrote:
| Inside the article is a section "How to get started using
| Backblaze B2 in Terraform" with a link [1] to the getting
| started guide which should have everything you need.
|
| I can see why they put it on a separate page to not clutter up
| the article.
|
| [1] https://help.backblaze.com/hc/en-us/articles/1260803375989
| rattray wrote:
| Nice, thanks!
|
| I was less interested in the code sample as a "how-to" and
| more for skimmability - rather than reading a bunch of words
| I just wanted to see what it would look like.
|
| Eg, from that link: terraform {
| required_version = ">= 0.13" required_providers {
| b2 = { source = "Backblaze/b2"
| version = "~> 0.2" } } }
| provider "b2" { } resource
| "b2_application_key" "example" { key_name =
| "test-b2-tfp-0000000000000000000" capabilities =
| ["readFiles"] } data
| "b2_application_key" "example" { key_name =
| b2_application_key.example.key_name }
| output "application_key" { value =
| data.b2_application_key.example }
| mattjaynes wrote:
| If you'll be using B2 for live production data, a couple points:
|
| First, B2's pricing is pretty amazing, especially compared to S3
| and similar competitors: https://www.backblaze.com/b2/cloud-
| storage-pricing.html
|
| Second, be aware those savings come with some downsides. The
| major one for us has been their maintenance window every Thursday
| from 2:00-3:00pm Pacific Time. Usually there's no outage, but
| sometimes there is. There's no warning - it's just down sometimes
| during that window. So, if uptime is important for your data,
| consider the cost of also implementing a fallback solution to
| cover your production use during those maintenance windows.
| https://www.backblaze.com/scheduled-maintenance.html
| vishnumohandas wrote:
| B2 claims to offer 99.9% uptime[1], and seems to have SLAs[2]
| similar to that of AWS S3[3].
|
| [1]: https://www.backblaze.com/blog/cloud-storage-durability-
| vs-a...
|
| [2]: https://www.backblaze.com/company/sla.html
|
| [3]: https://aws.amazon.com/s3/sla/
| tedmiston wrote:
| What a strange time for a company with lots of US customers to
| schedule a maintenance window.
| Twisol wrote:
| I'm not so sure. It's better than a late night window in that
| it forces their customers to actually deal with the
| engineering window, rather than just cross their fingers.
| aequitas wrote:
| Also, your office is full of awake engineers at that time.
| Which is better than a handful of on-call sleepy engineers
| and crossing your fingers the rest of the team wakes up
| when you call them if something goes completely haywire.
| bretpiatt wrote:
| With the B2 target use case of backup and archive data I
| suspect it's actually a good time to do it for their customer
| base (and it also then happens to be good for their
| engineering team too, awake and alert!).
| johnjboren wrote:
| Don't push to production on a Friday, but Thursday at the
| latest, sure...
| ericbarrett wrote:
| Thursday is the best maintenance day. If it goes haywire,
| you have a full business day to fix before losing a
| weekend. And if you can't, well, you've given your
| coworkers a "free Friday," which is far less likely to
| result in complaints than screwing up M-Th.
| truetraveller wrote:
| Ouch. Does B2 Cloud actually go down during this time. That's a
| major show-stopper. Was actually counting on B2. Not sure
| anymore.
| mattjaynes wrote:
| Not often, I only remember a few times in the last couple of
| years. If you're just hosting backups, you're unlikely to
| even notice. But if you're serving live production data from
| B2, it can bring down your whole service, which is quite
| painful especially if you have a large customer base.
| atYevP wrote:
| Yev from Backblaze here -> typically not - we've built most
| of our systems to be keep data up and flowing during those
| maintenance windows - if we anticipate longer windows or are
| doing things that can impact performance we typically
| announce it on our blog and twitter!
| ericvanular wrote:
| Thanks for building Backblaze. If you're able to share some
| feedback with the team - this is an extremely important
| factor. If you can ensure downtime is avoided during
| maintenance windows, it will make your service much more
| viable for production systems
| CameronNemo wrote:
| Can anyone be 100% sure that downtime is avoided during
| maintenance windows?
| xchaotic wrote:
| If you have some umm other useful tips for that 100%
| uptime let the rest of the world know. I am more happy
| with realistic and upfront statements from B2 than some
| wishful thinking from potential users.
| barkingcat wrote:
| This is an unreasonable expectation. The whole point of a
| maintenance window is to allot an expected time when
| there might be downtime.
|
| Otherwise, the maintenance window becomes 24hx365, since
| "ensuring downtime is avoided during maintenance window"
| means literally - make a maintenance window have the same
| uptime as non-maintenance window.
| shock-value wrote:
| It's not necessarily unreasonable, it just depends on
| what kind of product they want to offer. S3, Google
| Storage, etc. do not have a maintenance window that I'm
| aware of. That's not to say they would never go down, but
| if they do you would expect an alert and an apology, at
| least. Many application require this kind of expected
| uptime, but of course there are others (backup, etc) that
| would not.
| xoa wrote:
| > _if we anticipate longer windows or are doing things that
| can impact performance we typically announce it on our blog
| and twitter!_
|
| As a customer is there any way to opt-into a more proactive
| notification of an anticipated delay, like an email? I
| understand such things are necessary sometimes, but "always
| pay attention to some blog or twitter for a rare
| occurrence" doesn't seem particularly busy-stressed-admin
| friendly :).
| CameronNemo wrote:
| RSS feed for maintenances would be nice I think.
| truetraveller wrote:
| Side question: Apart from this maintenance window, is B2 Cloud
| reliable? I've heard of problems with the S3 API. Is the
| "native" API more stable? Would love to know your insight, it
| will potentially save me a lot of time!
| tedivm wrote:
| Several years ago Backblaze lost all of my wife's data. Their
| dashboard said it was all there, and we trusted their systems
| to be accurate. When attempting to download the data it
| turned out that none of it was there. When my wife contacted
| support they tried to blame her.
|
| Obviously this was a few years ago, but a backup provider
| failing at their one job and then blaming the customer left a
| really bad impression that keeps me from using them.
| jiofih wrote:
| You were probably a victim of their 30 day deletion policy.
| If for any reason (firewall, etc) you did not connect to
| the backup servers your data would just be purged without a
| grace period. For that reason I built my own backup sync
| using B2 directly instead of their backup service (and it's
| a lot cheaper).
| tedivm wrote:
| We actually checked that- the day we went to get the
| laptop repaired we confirmed that it was active and
| backed up, and a week later the restore failed.
|
| Backblaze eventually admitted that their dashboards
| aren't realtime, and they had a bug which was showing us
| (and their client) files that didn't exist.
| Spivak wrote:
| It is? I avoided it on my Linux laptop for ages since I
| assumed their flat pricing was a lot cheaper.
| mattjaynes wrote:
| Have had live production data on there for a couple of years
| and it's been very solid outside the maintenance windows.
|
| (The exception being the recent outages GoDaddy caused for
| them, but since they've moved to using Cloudflare as their
| registrar, I don't anticipate further issues there:
| https://news.ycombinator.com/item?id=26119619 )
| truetraveller wrote:
| Okay, that's great to hear.
| Dylan16807 wrote:
| I haven't used a it a lot but it's been a solid endpoint for
| backups.
|
| I did some medium-intensity benchmarking a while back and
| decided not to put certain server data on it because I was
| getting a few 20+ second timeouts per thousand read requests.
| I can handle server errors, and I have retry logic, but this
| was something where I needed to be able to access the data
| within a second or two. Maybe it would have worked better if
| I set a very aggressive timeout, I'm not sure. Deeper testing
| is something I'll worry about some other time if the data
| actually grows past a couple hundred gigabytes.
|
| This was mostly with the S3 API, I don't remember if I ever
| succeeded in getting the program to use the native one.
| preommr wrote:
| Digital Ocean has spaces with unlimited uploads, 250gB +
| (0.02/gb), 1TB of outbound transfer all at $5/mo.
|
| That seems way cheaper than this.
|
| For backups and large, long term storage, AWS has Glacier,
| that's really really cheap.
| dr-smooth wrote:
| Wait until you see how much it costs to retrieve your entire
| backup from Glacier.
| jjeaff wrote:
| $0.02/gb compared to $0.005 with b2. So not really in the
| same ballpark as far as price.
| preommr wrote:
| That's just the storage cost per GB, if you have any kind
| of serious throughput, then bandwith costs make it more
| expensive.
| brightball wrote:
| Aren't they in that free bandwidth alliance with
| Cloudflare?
| divbzero wrote:
| Yes, it looks like Backblaze is part of that free
| bandwidth alliance. [1]
|
| Thank you, I was not aware of this policy.
|
| [1]: https://www.cloudflare.com/bandwidth-
| alliance/backblaze/
| preommr wrote:
| Yea, that changes things, that's a pretty good deal then.
|
| Especially because Cloudflare's pricing is "smoother" and
| detached from any one service.
| atYevP wrote:
| Yev from Backblaze here -> yes we are!
| ddorian43 wrote:
| If you ever need those backups, your company will fail from
| the receipt to receive them.
|
| For Digital Ocean, please look that their pricing is higher
| in both bandwidth & storage.
| qw3rty01 wrote:
| Also keep in mind all the data is in a single datacenter, so if
| something like OVH happens to backblaze, you'll probably lose
| the data
|
| I mainly only use them for backups instead of production data
|
| EDIT: As of 2019 there are multiple datacenters, but it doesn't
| seem like the data is stored redundantly across them
| jiofih wrote:
| They have four datacenters, three in the US and one in the
| EU. Details are not given regarding how the 20 shards that
| comprise your data are distributed geographically, but they
| state eight 9s of reliability.
| spondyl wrote:
| Just to clarify, Backblaze states eight 9s of durability,
| not reliability.
|
| Durability refers to the idea that your data will still be
| retrievable (ie no corruption) similar to S3's claim of
| eleven 9s of durability.
|
| Reliability however would be say; the actual availability
| of the Web UI or API server that you download your data
| through. If it were down, that wouldn't impact the actual
| integrity of the data itself.
|
| For anyone interested, any more than about five 9s of
| reliability is basically impossible anyway when it comes to
| human intervention. As an example, 6 nines of reliability
| would allow you 3 seconds of unavailability a year so 250
| milliseconds of unavailability a month.
|
| From a users point of view, being "unavailable" includes
| everything from going through a tunnel and having your
| mobile connection drop out to a shark biting one of the
| undersea cables in the middle of the ocean.
|
| As you might imagine, with a human involved, they couldn't
| even get acknowledge an alert fast enough to meet that
| deadline let alone actually going about doing any repairs
| and diagnosis :)
|
| It could be spread over multiple instances and redundant
| hardware as well but as with any system being touched by
| humans, it's near guaranteed that something will go wrong
| eventually.
| andrewaylett wrote:
| At that scale, a complete outage is unlikely. I have
| services which haven't gone down _at all_ for longer than
| a year. But we lose requests every now and then -- during
| a deploy, or due to a bug. So we've moved from a time-
| based view of outage to a request-based view.
|
| This helps, too, as it lets us build out services to be
| more reliable in combination, rather than less reliable.
| With retries and fail-over, an outage in an entire region
| may not necessarily result in any user requests failing.
|
| For scale, pre-pandemic our published figures claimed
| >100M MAU.
|
| I find https://andrewaylett.github.io/multi-burn-rate-
| calculator/ helpful for visualising error rates --
| largely cribbed from the project it's forked from :) but
| with the tweakables switched around and the time between
| alert and error budget exhaustion in the tooltip.
|
| It's worth noting that we only evaluate our alerts at
| most once a minute.
| Dylan16807 wrote:
| Given that the parity can only lose 3 shards it's easy to
| show that a datacenter loss will always result in data
| loss, so there's no reason to distribute and we should
| assume that all 20 shards are always going to be in the
| same place.
| qw3rty01 wrote:
| Oh how new is that? Last time I was looking through they
| just said their datacenter was in a bunker so nothing
| outside of a major natural disaster would affect it.
| jiofih wrote:
| Since 2019 it seems: https://help.backblaze.com/hc/en-
| us/articles/217667468-B2-Se...
| [deleted]
| 1vuio0pswjnm7 wrote:
| For large downloads, does BB support the Range header. If the
| user is on a connection that that is not suitable for long
| downloads, could the Range header be used to download a large
| file in several parts.
| 0xbkt wrote:
| They support byte serving.
| vbsteven wrote:
| Terraform is an amazing tool and it's always great to see more
| infrastructure providers jumping on the bandwagon.
| atYevP wrote:
| Yev from Backblaze here -> Yea, it's pretty great! They're a
| great tool and we're glad to be involved!
| [deleted]
___________________________________________________________________
(page generated 2021-03-11 23:00 UTC)