[HN Gopher] The Cloud Computer
___________________________________________________________________
The Cloud Computer
Author : CathalMullan
Score : 1700 points
Date : 2023-10-26 10:43 UTC (1 days ago)
(HTM) web link (oxide.computer)
(TXT) w3m dump (oxide.computer)
| XorNot wrote:
| Is this a big deal? I clicked on this skeptically, but...it
| actually seems like kind of a big deal? It is at least
| technically impressive, since it seems like they're eliminated a
| ton of datacenter pain points with the mechanical design.
| hotpotamus wrote:
| It sort of reminds me of a Dell blade chassis that I pulled out
| of a DC last year - it was pretty impressive tech that I hadn't
| dealt with. 6 power supplies that I think could take 110v or
| 220v, but you needed 220v for full capacity. All of them were
| hot-swappable and as long as you had 3 powered it would keep
| running. The compute sleds were modular and it also had
| integrated networking as mentioned in this article. All
| obviously crafted to a very high standard; even the plastic fan
| cages felt premium (and I'm told it cost accordingly back in
| 2010 or so).
|
| Also a former employer who was a bit below the hyperscaler
| level also had what they called "roll-in racks", though I
| believe they were just taking standard servers and networking
| gear and integrating them somewhere (presumably with cheap
| labor) and then bringing them into the DC as needed.
|
| So I don't think it's a particularly new concept, but I agree
| it looks like it has some potential as a product.
| roland35 wrote:
| Oxide is a bit of a darling child on HN - I think a lot of
| people (myself included) find it interesting that they are
| trying to re-engineer something somewhat commoditized and
| boring (servers) into something cool
| deadfece wrote:
| It seems like they have managed to fit 32 1P servers in a 42U
| rack. That's pretty impressive, I have to admit. I don't think
| I've seen that level of density before.
| steve1977 wrote:
| 32 1P servers in a 42U rack is impressive? If you use 1U
| servers, they would only use up 32 units, leaving with 10
| units for storage and networking.
|
| Also, I remember for example HP BladeCenter BL20p systems.
| There you would have up to 8 or 16 servers per 6 rack units
| IIRC. So about 56 to 112 servers in 42U.
|
| https://en.wikipedia.org/wiki/HPE_BladeSystem
| ludjer wrote:
| Ever seen blades you can really pack in density there.
| cpursley wrote:
| Can somebody explain in a sentence or two what this actually is
| and what the benefits are?
| gmaster1440 wrote:
| Modern tech stack for managing data centers using tightly
| integrated hardware and software.
| mickeyp wrote:
| Like Dell and Bladelogic used to be? Or VMWare ESXi and
| friends? Or...
| EvanAnderson wrote:
| I'd say more like Sun, or DEC, at least in terms of the
| hardware. The software is Free, which wasn't the case with
| those older companies.
| junon wrote:
| It appears to be an all in one, preconfigured/partially-or-
| completely assembled rack server for on-prem hosting, complete
| with pre-configured software.
|
| Just my perception given the front page. I agree it's a bit
| vague.
| 7thaccount wrote:
| They have been on HN numerous times over the years iirc. But
| yeah, I had completely forgotten what they did and had to
| read through all of it.
| nithril wrote:
| If it is on-prem, calling it "cloud" computer is imho
| misleading / marketing term.
| steve1977 wrote:
| Absolutely. The point of cloud computing is that I don't
| have to care where something is running and that I
| (theoretically) have infinite elasticity and can scale in
| and out as fast and much as I need to.
|
| None of that applies here.
| dist1ll wrote:
| This server is for cloud _operators_ - i.e. for
| organizations that are _building_ their own cloud
| infrastructure. That 's why Oxide is drawing comparisons
| to hyperscalers.
| steve1977 wrote:
| If you are building your own cloud infrastructure, you're
| not doing cloud computing.
|
| Unless you rent it out and are a cloud _provider_ , but
| the website at least does not seem to target those.
| growse wrote:
| Nonsense. The operative bit of "cloud" is "provision and
| de-provision instantly via an API, without much concern
| for what's going on inderneath", not "lives in someone
| else's datacenter".
| steve1977 wrote:
| It's called cloud _because_ it's not in your own data
| center. Usually cloud symbols were used in network
| diagrams to depict systems /networks outside of our
| concern.
|
| Also you could don't really get elasticity with a system
| like this. If anything, that would be the operative bit
| for me.
| vidarh wrote:
| That _was_ the case ~15+ years ago. Private cloud has
| been a term for the majority of that time to evoke the
| elasticity and virtualisation without the "not in your
| own data center" bit, because to most _users_ of a cloud
| the operative bit is that _they_ don 't have to worry
| about where the computer is or talk to someone to
| provision one, not whether it sits in the corporate data
| centre or off at Amazon.
|
| Hybrid clouds even means devs might not know whether it
| sits in the corporate data centre or a public cloud,
| because it could be either/or depending on current
| capacity.
|
| > Also you could don't really get elasticity with a
| system like this. If anything, that would be the
| operative bit for me.
|
| "You" as in "the organisation as a whole" don't get
| elasticity. "You" as in "your department" or "you as an
| individual" do get elasticity.
| ninkendo wrote:
| > "You" as in "the organisation as a whole" don't get
| elasticity. "You" as in "your department" or "you as an
| individual" do get elasticity.
|
| Right, but to the degree that you get elasticity, it
| starts to look more and more like "someone else's
| computer", no? If multiple people/departments/etc are
| provisioning virtual instances on one shared cloud infra,
| with nobody who's using the provisioning API caring about
| the underlying capacity (and capacity is planned
| indirectly by forecasting, etc), then it really starts to
| sound like "someone else's computer" to me. That "someone
| else" just happens to be another org within the same
| company.
| vidarh wrote:
| Yes, that is why we talk about it as a private cloud --
| it looks almost exactly like a public cloud to the people
| actually using it.
| ninkendo wrote:
| So in other words, "Someone else's datacenter" continues
| to be a perfect description of what Cloud is.
| vidarh wrote:
| As long as everyone has a shared understanding of what
| "someone else's" means, which this thread shows people
| don't.
| brucepink wrote:
| With apologies to everyone who has a "The cloud is
| someone else's computer" T-shirt, things have changed,
| and the language as evolved as it is wont to do.
|
| I've spent the last decade building on-premises systems
| very like what Oxide is doing, but I've had to build them
| out of stacks of servers, switches, storage appliances
| and VMWare licenses. And the network cabling, and fan
| noise, and the number of power cables, and.. oh man, I
| can't wait to install one of these things myself. Having
| a single point of responsibility for the whole thing
| shouldn't be underestimated either - I've spent far too
| long trying to resolve problems with vendors on both
| sides blaming each other.
|
| It's worth mentioning too that building something
| equivalent to this would be across more than one rack,
| and easily cost in excess of $1M.
| growse wrote:
| > It's called cloud because it's not in your own data
| center. Usually cloud symbols were used in network
| diagrams to depict systems/networks outside of our
| concern.
|
| That might be true historically, because the only way you
| could get resources provisioned on-demand via an API from
| _someone else who 'd built it_. You had to run in someone
| else's datacenter to get the capability which you
| actually wanted.
|
| Times have changed. Now, businesses think about "Cloud
| compute" as being synonymous with "on-demand", "elastic"
| etc. Where the actual silicon lives is merely an after-
| thought.
|
| > Also you could don't really get elasticity with a
| system like this. If anything, that would be the
| operative bit for me.
|
| Buy enough of them and you will :)
| steve1977 wrote:
| > Where the actual silicon lives is merely an after-
| thought.
|
| If you have to buy the silicon and plan capacity for it
| (as in the case with Oxide for example), then it cannot
| be an afterthought. Which is exactly why I would not
| consider it cloud computing.
| growse wrote:
| The point is the application team doesn't have to do any
| of that.
|
| Someone's always got to buy the tin and manage that. Some
| people are big enough that they might get a benefit from
| doing that themselves, rather than having Jeff Bezos do
| it for them.
|
| From the application team's perspective, call API then
| container go whirr.
| cpuguy83 wrote:
| I work on Azure. Is it not a cloud for me because of
| where I work?
| c0pium wrote:
| > you could don't really get elasticity with a system
| like this
|
| Of course you do, right up until the point where you've
| used all available capacity. Just like with public clouds
| (ask anyone using meaningful amounts of {G,T}PUs).
| Elasticity doesn't imply infinitely elastic, that would
| be ridiculous.
| ninkendo wrote:
| > provision and de-provision instantly via an API
|
| But that is literally not possible with hardware you
| purchase yourself.
|
| Sure, you can buy X amount of hardware, and provision up
| to X amount of virtual hardware via an API, but then
| what? You can't provision any more until you go and buy
| more hardware. This is why "cloud, but local" is a
| contradiction IMO. You can only be "cloud-like" if you're
| under-provisioning. The moment you want to actually use
| _all_ of the capacity you already paid for, you 're not a
| cloud any more, because you've provisioned all of it.
|
| No elasticity, no cloud.
| growse wrote:
| > But that is literally not possible with hardware you
| purchase yourself.
|
| Sure it is. I think TFA is talking about a company
| selling you exactly that capability.
|
| > You can't provision any more until you go and buy more
| hardware.
|
| But this is also true of AWS etc. When their estate gets
| full, they need to go buy more hardware. Regardless of
| who owns the tin, someone's doing a capacity plan and
| buying hardware to meet demand.
|
| The point of 'cloud' is that you move that function out
| of the domain who are actually using resources to solve
| business problems, which is where it traditionally sat.
| Historically, if you wanted to run a service, you had to
| go buy some hardware and hire someone to manage it for
| you.
|
| A cloud-like model means that the application engineers
| no longer care about servers, disks and switches.
| Instead, they just use some APIs to request some
| resources and then deploy a workload onto them. The
| details of what hardware, where and how is fuzzy and
| abstracted. Or cloud-like.
|
| > You can only be "cloud-like" if you're under-
| provisioning
|
| Everyone under-provisions. Nobody runs at 100%
| utilisation.
| thom wrote:
| There are many "you"s in most enterprises. If your
| platform engineering team builds their own cloud, and
| offers an experience similar to other cloud providers (or
| even better and more targeted), this could be a clear
| win.
| steve1977 wrote:
| You = the enterprise
|
| The definition of cloud is "out there in the sky - not
| here"
| growse wrote:
| Actually, the definition of cloud is "a visible mass of
| particles of condensed vapor (such as water or ice)
| suspended in the atmosphere of a planet (such as the
| earth) or moon".
|
| https://www.merriam-webster.com/dictionary/cloud
| steve1977 wrote:
| So basically hot air. In which case I guess Oxide has a
| point ;)
| thom wrote:
| I understand that's your definition and I'm saying that
| there are many companies where the cloud experience (of
| not worrying about physical infrastructure but having
| flexibility and elasticity) is offered to product teams
| by an internal platform team. That gives you an
| articulation point where you could migrate from AWS to
| Oxide racks, and yes, lose some functionality and some
| guarantees, but also gain more control and potentially
| make huge savings.
| count wrote:
| At least in the United States we have an official
| definition of cloud computing:
| https://csrc.nist.gov/pubs/sp/800/145/final
|
| and that isn't it...
| vidarh wrote:
| "Private cloud" has been a term for at least a decade to
| refer to situations where people are building their own
| cloud infrastructure to provide a "cloud feature set"
| even though the servers are self hosted.
|
| You may not like it, but it's been a _long_ time since
| "cloud" has exclusively referred to "someone in another
| organisation entirely runs the computers" as opposed to
| virtualised allocation of resources.
| mst wrote:
| "Private cloud" as a term still kinda makes me itch, but
| I don't recall encountering an alternative that was more
| obvious and it's clearly the usual term of art at this
| point.
|
| I suspect that while I do appreciate how some posters
| upthread find the website a tad on the vague side, the
| target customers-in-potentia will understand it fine.
| throw0101a wrote:
| > _Absolutely. The point of cloud computing is that I don
| 't have to care where something is running and that I
| (theoretically) have infinite elasticity and can scale in
| and out as fast and much as I need to._
|
| Define "I"?
|
| I-as-developer can call an VMware/OpenStack API with an
| on-prem/private cloud and get a new instance just as
| easily as calling an AWS API. I-as-developer does not
| have to worry about elasticity if the IT hardware folks
| have the capacity.
| steve1977 wrote:
| Private cloud is a marketing misnomer.
| throw0101a wrote:
| I'm sure all the folks running OpenStack in-house,
| including NASA and CERN, would disagree.
|
| From a developer's perspective an API call is an API
| call, and to them it's just another instance.
| zeckalpha wrote:
| VPC doubly so then.
| spamizbad wrote:
| How much you care depends on your role.
|
| As an engineer, an Oxide system works like any other
| cloud provider. You're just interacting with its API and
| tooling like you would with Google Cloud or AWS.
|
| To someone on the IT/Operations side, obviously there are
| differences but theres SIGNIFICANTLY less labor required
| to build-out and operate an Oxide system vs a rack full
| of servers. The biggest difference for these people is
| that there's actual hardware vs a Cloud Provider, but
| also costs are fixed so there's likely no monthly or
| quarterly meetings with finance arguing over the cloud
| bill, tying people up to try and shave a few thousand off
| the bill every month.
|
| In finance/accounting, Oxide is probably the most
| different: now compute is CapEx rather than OpEx.
| Depending on your company's stage that can be a wonderful
| thing for the bean counters.
| steve1977 wrote:
| > To someone on the IT/Operations side, obviously there
| are differences but theres SIGNIFICANTLY less labor
| required to build-out and operate an Oxide system vs a
| rack full of servers.
|
| But it's also gonna be much more restricted. So I guess
| one could see as kind of "Apple for data centers"? Have a
| nice appliance and be happy as long as it runs as it
| should (but hope it never stops working as it should).
| vidarh wrote:
| Every "cloud" computer is on-prem for someone, and "private
| cloud" has been a term for at least a decade to refer to
| "API-based provisioning of resources like with a cloud but
| in your own data centre/office" that may not be
| "technically" a cloud but carries the meaning clearly for
| anyone in the business.
|
| What they're selling is building blocks for
| private/hybrid/public clusters that may or may not fit your
| definition of cloud depending on where they happen to be
| located but where what the term signifies is that it is
| built to be a building block of a cloud setup that includes
| features above and beyond a "regular" server to provide
| what most people tend to associate with a cloud. That is,
| you're getting APIs to spawn virtualized compute and
| storage, rather than having to install a hypervisor and
| management APIs etc. and combine the resources into a
| cohesive whole yourself.
|
| My guess is that most of them will end up being sold to
| either hosting providers to provide cloud services to their
| customers or companies large enough to operate their own
| data centres where the IT department will use them to offer
| cloud services to other parts of the business, where the
| term really is not misleading anyone.
|
| It's too big even for most "private clouds" in smaller
| companies, because they tend to be too small to be able to
| order by the rack even when there are APIs etc. offered to
| developers to provision compute and storage (e.g. years ago
| I used to operate a hybrid cloud setup with a ~1000 or so
| VM instances across several countries, and while we had
| several racks worth of gear we physically owned in
| _aggregate_ , we didn't have any full racks in any
| individual location).
| andrewstuart wrote:
| Weird there's no mention of GPU at all when you'd think that's
| what would prick up the ears of the hosting companies .....
| stack a pile of GPUs in a rack, surely you can sell time on
| that.
|
| The Oxide machines seem to be aimed at 2020.
| 7thaccount wrote:
| I don't think they're trying to be AWS. They're trying to
| sell a product that greatly simplifies doing on-site cloud
| for companies. So they sell the physical hardware/software
| bundle and not time.
| axelthegerman wrote:
| This seems to be around selling the whole piece of hardware
| not just "time on that".
|
| That being said I'd still expect a monthly service fee for
| networking, electricity and service in general.
| depr wrote:
| They want a lot of control over all the hardware in their
| servers. NVIDIA isn't interested in that, so they can't
| provide those types of servers.
| andrewstuart wrote:
| AMD? Intel?
| throw0101a wrote:
| AMD helped them write their own bootloader:
|
| > _"Oxide is a strong believer in the need for open-
| source software at the lowest layers of the stack --
| including silicon initialization and platform enablement.
| With the availability of AMD openSIL, AMD is showing that
| they share this vision. We believe that the ultimate
| beneficiaries of open-source silicon initialization -- as
| it has been for open-source revolutions elsewhere in the
| stack -- will be customers and end-users, and we applaud
| AMD for taking this important and inspiring step!"_
|
| * https://community.amd.com/t5/business/empowering-the-
| industr...
|
| See Bryan Cantrill's (Oxide CTO) presentation on their
| adventures in this space:
|
| * https://www.osfc.io/2022/talks/i-have-come-to-bury-the-
| bios-...
|
| * https://news.ycombinator.com/item?id=33145411
| (discussion at the time)
| nickik wrote:
| There is still a huge amount of stuff not using GPU and never
| will. Claiming that's only for 2020 is pure nonsense.
|
| They are likely looking at future racks with GPU as well, but
| as a first product getting the basics right makes more sense.
| loudmax wrote:
| Oxide's goal is that they, and by extension their customers,
| have as much visibility and control over the software stack
| in these racks as possible, and that includes firmware. They
| started developing these systems before the current wave of
| interest in machine learning led by ChatGPU and Stable
| Diffusion really got underway.
|
| Nvidia GPU drivers are very proprietary, which means that
| admins and developers have limited visibility into them if
| they misbehave in any way. This goes against Oxide's
| philosophy of full visibility into a system that you
| purchase.
|
| Nvidia's CUDA software has a significant lead ahead of AMD
| and Intel GPUs, and they're not going to open source it any
| time soon. But this is a rapidly changing landscape, and AMD
| and Intel and others are pouring an enormous amount of
| research into getting their hardware and software to match
| what Nvidia has going. Nvidia is in pole position, but
| they're not guaranteed to stay there.
|
| There's still a large market for the CPU workloads that Oxide
| is offering. For now, Oxide will be concentrating on meeting
| this traditional compute demand. But you're right to point
| out that in 2023, the absence of a top tier GPU in these
| racks is noticeable. I suspect Oxide will want to include
| some form of GPU or TPU into the next version of their
| system, but they won't just grab whatever hardware happens to
| be in fashion. It needs to work with their system as a whole.
| svnt wrote:
| Now to see how their development timeframes and
| synchronization efforts with big hardware companies go.
|
| This is where they will enter the real "hard" part of
| hardware, with an exec team from software. Can they respond
| to the market while making hardware?
|
| They seem to have presented as generally thoughtful about
| their approach. If they can release major variants about
| annually or even sub-annually that is what I think will
| enable them to win.
| emmelaich wrote:
| Here's a bit of history:
| https://bcantrill.dtrace.org/2019/12/02/the-soul-of-a-new-co...
| NKosmatos wrote:
| I'll second that, they should make it clear what they're
| selling and what they have to offer. Every company should have
| this in their main/landing page... If I can't understand what
| you're selling/offering in 30sec, then I'm not interested :-)
|
| If you take some time and read around their site you'll see
| that they're offering a ready to run (turnkey) server. They
| have everything packed together, they've integrated everything
| that is needed (CPU, disk, networking...) into a nice looking
| cabinet, with not too many wires and they're selling it as a
| complete package.
|
| If you're in need of a server (cloud computer) and you don't
| want to but separate components and unpack them and connect
| them yourself, then this looks like a possible solution.
| nickik wrote:
| Literally on the main page there is a picture of a big
| computer. And then it says:
|
| Oxide Cloud Computer
|
| No Cables. No Assembly. Just Cloud.
|
| Contact Sales
|
| How much easier can they make it? They clearly want to sell
| computers.
| rob74 wrote:
| Yes, but... what is a "Cloud Computer"? Is it a "computer
| in the cloud", like e.g. an AWS EC2 instance? Or is it a
| fancy name for a good old fashioned server rack (on which
| you will probably want to run your own cloud, because
| everybody's doing that nowadays, hence the name), like in
| this case? And if it's a server rack, how come you don't
| need any cables? And what do you do with this "cloud
| computer"? Do they host it in their data center or deliver
| it to you? So there is _some_ potential for confusion -
| nothing that can 't be mostly cleared up by reading a bit
| further, but still...
| FridgeSeal wrote:
| > like e.g. an AWS EC2 instance? Or is it a fancy name
| for a good old fashioned server rack
|
| I mean, the former is just the latter with some of the
| setup done for you no? Anyways, it's a full server rack,
| with tightly vertically integrated hardware and software.
| Not sure if you've poked around the rest of their site,
| but it seems like their whole software stack is designed
| with some really nice usability and integration in mind:
| there's a little half-snippet there suggesting that
| provisioning bare-metal VM's out of the underlying
| hardware could as trivial as provisioning an EC2 with
| Terraform, and if that's the case, that's _massive_.
|
| > And if it's a server rack, how come you don't need any
| cables?
|
| Because they've gone to great lengths and care to design
| it to not need anything extraneous IIUC. I think the
| compute sleds all automatically mount into some automatic
| backplane that presumably gives you power, cooling and
| networking, and then, as above, you presumably configure
| all that via software, as you would your AWS setup. Not
| an expert here though, happy to be corrected by anyone
| who _actually_ knows better.
|
| > Do they host it in their data center or deliver it to
| you?
|
| Presumably the latter, given they're a hardware company,
| but if their software is even a 10th as good as it seems,
| I fully believe there'll be a massive market for renting
| bare-metal capacity from them.
| NKosmatos wrote:
| That's what some of us are saying, it's not crystal clear
| what they sell.
|
| You use words like: it seems, I think, presumably, I
| believe... This is what we're arguing. A company that has
| raised $44 million Series A for sure can afford to
| clearly write what they offer.
|
| I understand, you can't have all the people happy and no
| matter what you do there will always be "weirdos" that
| don't like your page/design/wording, but hey at least
| recognize it :-)
| steveklabnik wrote:
| > there's a little half-snippet there suggesting that
| provisioning bare-metal VM's out of the underlying
| hardware could as trivial as provisioning an EC2 with
| Terraform,
|
| We have a terraform provider, yes
| https://github.com/oxidecomputer/terraform-provider-oxide
| steveklabnik wrote:
| > Or is it a fancy name for a good old fashioned server
| rack
|
| In a sense. Yes, it is a rack of servers. You're buying a
| computer. But we've designed the rack of servers as a
| full rack of servers, rather than an individual 1U. Comes
| with software to manage the rack like you would a cloud;
| you don't think of an oxide rack as individual compute
| sleds, you think of it as a pool of capacity.
|
| > And if it's a server rack, how come you don't need any
| cables?
|
| Because you are buying an entire rack. The sleds are
| blind mated. You plug in power, you plug in networking,
| you're good to go. You're not cabling up a bunch of
| individual servers when you're installing.
|
| > Do they host it in their data center or deliver it to
| you?
|
| Customers get them delivered to their data center.
|
| Happy to answer any other questions.
| tryauuum wrote:
| Yeah but this just describes AWS UI. I even clicked on
| their demo, saw some dashboard for creating VMs, pretty
| unimpressive.
|
| It wasn't obvious to me you can own the stuff where it runs
| (I hope I understood it correctly)
| cmiles74 wrote:
| It does look pretty cool.
| hulitu wrote:
| > Literally on the main page there is a picture of a big
| computer. And then it says:
|
| So it is big, it looks good, but no specs. What
| architecture ? Some "cinebench" numbers ...
| steveklabnik wrote:
| Specs are here:
| https://oxide.computer/product/specifications
| throw0101a wrote:
| Designing a hardware-software combination to allow for the
| managing of compute (and networking and storage) to occur at
| the rack-level rather at the individual-device (server, switch,
| _etc_ ) level, so that large(r)-scale operators can manage
| cattle more easily (rather than herding cats/pets).
| subarctic wrote:
| It's a rack of servers you can buy for a lot of money and put
| in a datacenter. And once you plug it in and turn it on and do
| whatever setup is required, you're supposed to be able to spin
| up vms on it.
| pmontra wrote:
| The site is really hard to navigate. I eventually looked at the
| footer and found a link to the technical specifications
| https://oxide.computer/product/specifications They give an idea
| of the beast, especially the dimensions and weight
| Dimensions H x W x D 2354mm (92.7") x 600mm (23.7") x
| 1060mm (41.8") Weight Up to ~2,518 lbs (~1,145
| kg)
|
| It's a rack.
|
| BTW, they are following some no pictures policy. I found only a
| few pictures of boards but no picture of the product as a
| whole.
| nickik wrote:
| Literally on the main page: https://oxide.computer/
| aeyes wrote:
| Can you link the picture? I only see renderings on the main
| page.
| brucepink wrote:
| https://x.com/arjenroodselaar/status/1690986149161144320?
| s=2...
| pmontra wrote:
| Indeed, which demonstrates that an explicit link to Home in
| a visible place is never a bad idea. I didn't click on
| 0xide in the top left even if that's a common shortcut to
| the home and I didn't notice the link in the footer, which
| is where you bury the least interesting stuff. I kept
| clicking on the text links in the top bar.
| monkeydust wrote:
| Also this. Looks technically great but how do you sell this to
| business? Whats the price range? Is this supposed to be an
| acceptable solution between on-prem and cloud as we start to
| realise the true costs of being hooked on cloud when your a
| large org and not a startup?
| cstross wrote:
| They've reinvented the IBM Mainframe. Big rack-sized box with
| lots of redundant hardware that serves guest VMs. This is
| basically a zSeries in drag.
|
| The key difference is the price structure -- IBM leases the
| hardware wherever they can get away with it, and uses a license
| manager to control how many of the machine's resources you can
| access (based on how much you can bear to pay). This, however,
| is like a mainframe you own.
| 0xbadcafebee wrote:
| This is exactly what it is, plus custom "cloud" software
| panick21_ wrote:
| Mainframes also do lots of other things that this doesn't
| really do. Like allowing you to pretend its a bunch of multi-
| core machines, or even a single one.
| pravus wrote:
| In a data center you have racks of computers performing all of
| the workloads. At this point these racks are fairly
| standardized in terms of sizing and ancillary features. These
| are built-out to solve the following: *
| Physical space - The servers themselves require a certain
| amount of room and depending on the workloads assigned will
| need different dimensions. These are specified in rack "units"
| (U) as the height dimension. The width is fixed and depths can
| vary but are within a standard limit. A rack might have
| something like 44U of total vertical space and each server can
| take anywhere from 1-4U generally. Some equipment may even go
| up to 6U or 8U (or more). * Power - All rack
| equipment will require power so there are generally looms or
| wiring schemes to run all cabling and outlets for all powered
| devices in the rack. For the most part this can be run on or
| in the post rails and remains hidden other than the outlet
| receptacles and mounted power strips. This might also include
| added battery and power conditioning systems which will eat
| into your total vertical U budget. Total rack power
| consumption is a vital figure. * Cooling - Most
| rack equipment will require some minimum amount of airflow or
| temperature range to operate properly. Servers have fans but
| there will also be a need for airflow within the rack itself
| and you might have to solve unexpected issues such as
| temperature gradients from the floor to the ceiling of the
| rack. Net heat output from workloads is a vital figure.
| * Networking - Since most rack equipment will be networked
| there are standard ways of cabling and patching in networks
| built into many racks. This will include things such as bays
| for switches, some of which may eat into the vertical U budget.
| These devices typically aggregate all rack traffic into a
| single higher-throughput network backplane that interconnects
| multiple racks into the broader network topology.
| * Storage - Depending on the workloads involved storage may be
| a major consideration and can require significant space
| (vertical Us), power, and cooling. You will also need to take
| into account the bus interconnects between storage devices and
| servers. This may also be delegated out into a SAN topology
| similar to a network where you have dedicated switches to
| connect to external storage networks.
|
| These are some of the major challenges with rack-mounted
| computing in a data center among many others. What's not really
| illustrated here is that since all of this has become so
| standardized we can now fully integrate these components
| directly rather than buying them piece-meal and installing them
| in a rack.
|
| This is what Oxide has to offer. They have built essentially an
| entire rack that solves the physical space, power, cooling,
| networking, and storage issues by simply giving you a turn-key
| box you plant in your data center and hook power and
| interconnects to. In addition it is a fully integrated solution
| so they can capture a lot of efficiencies that would be hard or
| impossible in traditional design.
|
| As someone with a lot of data center experience I am very
| excited to see this. It is built by people with the correct
| attitude toward compute, imo.
| barrkel wrote:
| The way I interpret it: an integrated stack of compute, storage
| and networking hardware and monitoring software that you can
| plug in to your data center (owned or colo) and can e.g. deploy
| Kubernetes to, and then use as a deployment target for your
| services.
|
| The main USP: you own it, you're not renting it. You're not
| beholden to big cloud's pricing strategies and gotchas.
|
| Secondary USP, why you buy this rather than DIY / rack computer
| vendor: it's a vertically integrated experience, it's
| preassembled, it's plug and play, compatibility is sorted out,
| there's no weird closed third party dependencies. Basically,
| the Apple experience rather than the PC experience.
| nunez wrote:
| The Oxide has everything you need to stand up your own private
| cloud, and everything works together down to the software. It's
| a great alternative to buying and managing servers, switches,
| storage, power management, KVMs, all of the software in
| between, and the tens of professional services contracts
| required to glue it all together.
|
| It's the iPhone of hyperconverged infrastructure.
|
| (Sorry; three sentences.)
| cryptonector wrote:
| Low risk of vendor lock-in.
| fuzzylightbulb wrote:
| This thing is the definition of vendor lock in. It's a rack
| full of non-commodity hardware using non-standard connectors
| and a bunch of custom software.
|
| The industry moved to commodity x86 for compute for a reason.
| andrewstuart wrote:
| I wonder how effective AMD 7840 would be as a cloud CPU. 8 cores,
| fast GPU, AV1 video encoder. Tiny SBC machines .... you could
| probably stack a bunch of them in a rack.
|
| https://www.asrockind.com/en-gb/4X4-7840U-1U
|
| https://youtu.be/WCRK-Uwb0EA?si=zMYn2Gf3vTe-qMl8
|
| https://www.amd.com/en/products/apu/amd-ryzen-7-7840u
| wmf wrote:
| "Microserver" designs have failed several times. AMD Epyc is
| basically twelve 7840s in an even smaller space and you can
| partition it any way you want.
| fefe23 wrote:
| This is actually a pretty big deal.
|
| They sell servers, but as a finished product. Not as a cobbled
| together mess of third party stuff where the vendor keeps
| shrugging if there is an integration problem. They integrated it.
| It comes with all the features they expect you to want if you
| wanted to build your own cloud.
|
| Also, they wrote the software. And it's all open source. So no
| "sorry but the third party vendor dropped support for the bios".
| You get the source code. Even if Oxide goes bust, you can still
| salvage things in a pinch.
|
| Ironically this looks like the realization of Richard Stallman's
| dream where users can help each other if something doesn't work.
| throw0101a wrote:
| > _They sell servers, but as a finished product._
|
| They sell rack-as-compute.[0] Their minimum order is one rack:
| You plug in power and network, connect to the built-in
| management software (API), and start spinning up VMs.
|
| [0] With built-in networking and storage.
| steve1977 wrote:
| So just hyperconverged infrastructure with a cute name?
| danpalmer wrote:
| "Just" is doing a lot of lifting in that sentence.
|
| This achievement is clearly worth a lot to people.
| throw0101a wrote:
| With rack- and multi-rack-level management of all your
| hardware infrastructure (including networking and storage,
| along with VMs) using an API:
|
| * https://docs.oxide.computer/api/guides/responses
| steve1977 wrote:
| Yeah including networking and storage together with
| virtualization is what makes hyperconverged
| infrastructure hyperconverged. Otherwise it's usually
| just called converged infrastructure.
|
| It's nice, it's just nothing new.
| intelVISA wrote:
| oh, you tease.
| freeopinion wrote:
| So, you're saying this is just ho-hum, and you're
| sticking with your IBM 700 from the 1950s?
| sanderjd wrote:
| It's not a new _concept_ , it's a new _product_. Ideas do
| not become uninteresting the moment the first person
| instantiates the idea. Further iterations on a possibly-
| good idea are also interesting.
| zeckalpha wrote:
| Mainframes!
| throw0101c wrote:
| With source code availability:
| https://github.com/oxidecomputer
| zozbot234 wrote:
| "Real" mainframes have RAS (Reliability, Availability,
| Servicing) features such as hotswapping for all hardware
| components and automated HA/workload migration across
| physical racks. They can also do SSI (single system
| image), i.e. run a _single_ workload across physical
| nodes /racks as if it was just multiple 'cores' in a
| single shared-memory computer. Oxide computers will
| probably end up doing at least some of this (namely
| workload migration across racks for HA) but saying that
| it can comprehensively replace mainframe hardware as-is
| is a bit of a stretch. In terms of existing hardware it's
| closer to a midrange computer.
| NortySpock wrote:
| The Oxide and Friends podcast had an episode on
| virtualizing time, specifically for the purpose of live-
| migrating a container from rack to rack without the VM
| being aware, and allowing operators to take the rack
| offline on their schedule. Otherwise, apparently, you end
| up having to leave racks running because you cannot
| evacuate all of the containers currently running on it.
| (e.g. perhaps your contracts or SLAs are such that you
| cannot afford even the few seconds of downtime a shut-
| down-here-and-spin-up-elsewhere would cause)
|
| I believe the episode name was "Virtualizing Time"
|
| https://pca.st/episode/c10ce39c-1348-407f-b9c2-a36ced4e6b
| e8
| cratermoon wrote:
| https://en.wikipedia.org/wiki/PureSystems
| dustingetz wrote:
| way better explanation than the website copy
|
| "no cables no assembly just cloud" wtf is that
| steveklabnik wrote:
| Obviously different marketing copy speaks to different
| people. But that is referring to how, when you buy a rack
| from us, you don't need to put everything together and
| cable it all up: you pull it out of the box, plug in
| networking and power, boot the thing up, and you're good
| to go. Installation time is hours, not days or weeks,
| which is the norm.
| MichaelZuo wrote:
| Will you be selling services as well, such as taking care
| of installation, boot up, testing, validation, etc., for
| a fee?
| steveklabnik wrote:
| At this stage of the company, everyone gets a white-glove
| installation process. I suspect that will change over
| time but I don't work on that part of things, so I don't
| personally know the details.
| MichaelZuo wrote:
| Thanks, are the specific details standardized and
| available in writing? Or is it more tailored to each
| customer?
| steveklabnik wrote:
| Sorry to be slightly obtuse, which details are you
| referring to here? Help upon installation? At the moment,
| we are helping customers individually, yeah. But we do
| have a documented process we are following
| https://docs.oxide.computer/guides/system/rack-
| installation-... (and more on other pages there)
| MichaelZuo wrote:
| The details would be things such as the requirements
| associated with the white glove installation process:
|
| Size of doorways, weight bearing capacity of floors,
| electrical service parameters, environmental conditions,
| etc.
|
| e.g. Does it actually handle electrical voltage
| fluctuations of +/- 1V, or whatever is advertised?
|
| The guaranteed parameters of a fully set up machine:
|
| Minimum performance metrics, software compatibility with
| whatever the sales department promised, maximum power
| draw, etc.
|
| e.g. Can it reliably hit X metric (FLOPS, IOPS, Integer
| calculations, etc.)?
|
| And so on.
| steveklabnik wrote:
| Ah yeah, so the "facilities" section of
| https://oxide.computer/product/specifications has some of
| these things, probably the closest we have to publicly
| publishing that in a general sense.
| MichaelZuo wrote:
| Yes I understand, but will your included service actually
| verify that everything is set up correctly, meets
| advertised parameters, and sign off on it? (Such that the
| customer can start using it immediately afterwards.)
|
| Or does the customer need to take on some risk and hazard
| associated with installation, configuration, initial boot
| up, etc.?
|
| e.g. If someone buys with the intention of using it up to
| X FLOPS, and the machine only delivers Y FLOPS once it's
| all said and done, what happens?
| steveklabnik wrote:
| It's not the area of the company I personally work on, so
| I don't know those details, to be honest. We certainly
| make sure that everything is working properly.
| MichaelZuo wrote:
| So I assume there's no guarantee that it will be plug and
| play after the white glove installation?
|
| Otherwise I would imagine it would be a major selling
| point and be advertised publicly.
| steveklabnik wrote:
| I mean, we absolutely sell support. I just don't know
| anything about the details personally. You shouldn't take
| my lack of knowledge as a "no," just a "steve doesn't
| personally know."
| MichaelZuo wrote:
| I had assumed a simple 'yes'/'no'/'maybe sometime later'
| answer was just a quick phone call away with the relevant
| person.
|
| But if it is a complicated enough internal issue for that
| not to be the case, then my apologies for pestering.
| gleb wrote:
| I wouldn't be dismissive of people telling you that the
| product description can be improved. My opinion is that
| the description of the product in this thread will
| outperform your site 10 to 1.
| steveklabnik wrote:
| I am not trying to be dismissive, I was just explaining
| since there was some confusion.
| gleb wrote:
| I'll try to explain, not in the spirit of being
| argumentative, but with the hope of being useful.
|
| The comment you replied to was not questioning the value
| of integrated cabling. It was pointing out that the
| product description on the site does not make sense.
|
| "Cloud computer" sounds like a server you rent from AWS.
| It's kind of like calling Rust "cloud compiler."
|
| If you choose to use words that your audience doesn't
| understand, or even worse understands to mean the
| opposite of what you want them to mean, it's a good idea
| to explain these words immediately using conventional
| words with conventional meaning. The comments by
| throw0101a did that.
|
| The product seems really cool, but there is no way I
| would've understood what it was from the website.
| steveklabnik wrote:
| I understand that's what you're saying, and I understand
| what the parent is saying. I chose to explain what that
| alluded to, in case anyone in this conversation is also
| finding it hard to understand what is meant by that
| specific copy. That doesn't mean I don't understand the
| broader point, or that I think the website copy is
| perfect.
| twicetwice wrote:
| Perhaps if you don't understand what the copy means, then
| that is a sign that you are not the target audience,
| rather than that the copy is bad? From what I've gathered
| from reading other comments in this thread, that copy
| will make perfect sense to Oxide's target audience, as it
| uses words in a way that will be very familiar and make
| perfect sense to the kind of person who might make a
| purchasing decision for a system like this.
|
| And for what it's worth, I don't think you need to
| explain what's happening to Steve, it seems to me that he
| understands perfectly well. To me you come across as
| being rather condescending and in my opinion Steve is
| being commendably polite in response.
| gorjusborg wrote:
| > "Cloud computer" sounds like a server you rent from
| AWS. It's kind of like calling Rust "cloud compiler."
|
| Cloud as a term is pretty undefined/overloaded. It makes
| sense to me if I think about 'cloud' computing as API-
| based provisioning on pooled resources.
| cdchn wrote:
| >plug in networking and power
|
| No cables, except for a few cables.
| steveklabnik wrote:
| Yes, those are two different things.
|
| To be super clear about it, this is referring to not
| needing to cable up all of the individual sleds to the
| rack upon installation. It doesn't mean that we recommend
| connecting a rack of compute to your data center via
| wifi.
| troupe wrote:
| Cdchn was hoping that it had a Starlink antenna built
| into the rack. :)
| 0x457 wrote:
| powered by tesla coil
| cozzyd wrote:
| and solar panels!
| freeopinion wrote:
| Wait, these things are weather proof, too?!!
| aidenn0 wrote:
| They mean no intra-rack cables, which are the
| overwhelming majority of cables on a typical rack.
| pclmulqdq wrote:
| This is pretty big, as someone who has deployed servers
| to datacenters before. Remote hands are very good at
| plugging in the network uplink and the PDUs. Doing a
| complete leaf-spine 25GbE network with full redundancy is
| something they are pretty much guaranteed to screw up at
| some point.
| troupe wrote:
| Would anyone who has actually set up a rack assume that
| they meant these racks were wireless with a self-
| contained nuclear generator?
|
| I think their description conveys what it does just fine
| for the target audience.
| Eduard wrote:
| "no cables no assembly just cloud" is completely
| misleading to any kind of people - tech or marketing or
| not.
|
| When people hear cloud, it means that aspects such as
| electricity costs, electricity stability, Internet,
| bandwidth, fire protection, safety, etc etc are
| abstracted away.
|
| Oxide IS on-premise, right? The website is very vague and
| wishy-washy.
| steveklabnik wrote:
| It is on premises. You interact with the rack the same
| way you interact with the public cloud: as a pool of
| resources. The specifics are abstracted away. "Private
| cloud" is pretty well established terminology in this
| space, and that's what we're doing.
| the__alchemist wrote:
| Hypothesis: "no cables" and "no assembly" here is
| analogous to the term "serverless". Or, more abstractly,
| the word "literally".
| rcxdude wrote:
| Available to buy turnkey, not just rented out.
| unixhero wrote:
| There is no just in hyper
| quickthrower2 wrote:
| Have they basically done for the data centre what iMac did
| for computers in about year 1999 (or whenever!)
| repelsteeltje wrote:
| > what iMac did for computers in about year 1999
|
| Ehrm. What is that exactly?
|
| Are you alluding to cute design, different user interface?
| Or ditching then common PC component modularity? "Thinking
| differently"?
|
| One difference is that Oxide development was done in the
| open and they don't seem hell bent on creating a closed
| ecosystem. (Yet, at least)
| steveklabnik wrote:
| I think Oxide shares one idea with Apple: hardware and
| software should be created for each other. In that sense,
| your parent is correct.
|
| You are also correct that we diverge from Apple in other
| ways, such as our commitment to openness, rather than
| secrecy.
| quickthrower2 wrote:
| Yes. I imagine I got downvoted because it sounds like a
| snark but it wasn't meant to be.
| sangnoir wrote:
| > Ehrm. What is that exactly?
|
| The first iMac famously made it easy to connect to the
| Internet; The 'i' in iMac was for "Internet". Its setup
| manual was a couple of pages long, mostly pictures and
| IIRC, just 37 words.
| nonameiguess wrote:
| Brutal downvotes. I guess people forget the old Jeff
| Goldblum commercials? "There is no step 3."
|
| https://www.youtube.com/watch?v=rX4PzvWgu6c
| cryptonector wrote:
| It would be interesting to sell a data center in a container.
| Cooling, power supply, compute, storage, and network, all in
| a box. You supply power, a big network pipe, and the piping
| to external heat exchangers.
| dthul wrote:
| I know one company who offers that (there might be more):
| https://www.grando.ai/en/container
| anewlanguage wrote:
| AWS does that already for the defense industry:
| https://aws.amazon.com/blogs/publicsector/announcing-aws-
| mod...
| jjav wrote:
| > It would be interesting to sell a data center in a
| container.
|
| Sun did that experiment:
|
| https://en.wikipedia.org/wiki/Sun_Modular_Datacenter
| cryptonector wrote:
| Yeah, I'm aware, but I didn't think they were serious
| about it.
| c_o_n_v_e_x wrote:
| There are quite a few companies selling "modular data
| centers" now.
| throw0101c wrote:
| See SSExamples:
|
| * https://en.wikipedia.org/wiki/Modular_data_center
|
| Also:
|
| * https://www.deltapowersolutions.com/en/mcis/data-center-
| solu...
|
| * https://atos.net/en/solutions/high-performance-computing-
| hpc...
|
| * https://www.zelladc.com/zella-max
|
| Search for "shipping container data centre" or
| "containerized datacenter".
| bayindirh wrote:
| Huawei also does this. Built to order, single container.
| Completely isolated, plug'n'play.
| numpad0 wrote:
| There are whitebox Windows laptops, OpenWRT routers,
| Arduino boards, ArduPilot drones, etc. It almost sounds
| strange that there are no prepopulated 12U racks intended
| for OpenStack(is that still a thing?)
| rtpg wrote:
| IIJ has a project like this, data center in a container,
| just add power. They build it all up in Japan, ship it to
| rural areas across the world to basically jumpstart a local
| data center (I imagine mostly for industrial sites). They
| had a fun project where they had a half rack, powered by
| solar and connected to the net via Starlink.
| regularfry wrote:
| No idea how they do things today, but v0.1 of Azure (before
| it was called Azure) was a bunch of containers in a field.
| I remember seeing an aerial photo at the time.
| socrates137 wrote:
| 100%.
|
| I'm actually extremely impressed. I want one. I haven't worked
| in a data center in years, but I'd be tempted to do it again
| just to get my hands on one.
| EvanAnderson wrote:
| Same here. I really want to work on one of these. I got in
| the industry at the tail end of the time when people used Sun
| and DEC gear. I got to use just a little bit of it and it
| seemed so much more "put together" then PC stuff is even now.
|
| Oxide feels like it'll be that "integrated" experience, but
| with the added benefit of software freedom.
| fiddlerwoaroof wrote:
| I wish they'd sell a tabletop version for hobbyists, but
| realize this is probably a distraction. But... the problem
| with a lot of these systems (including the old Sun boxes and
| things like ibm mainframes and the AS/400) is that they sound
| cool but there's no real way for the typical new developer to
| "get into them" for fun and, as a result, you lose the chance
| for some developer selling it to their company based on his
| experience with the things.
| fiddlerwoaroof wrote:
| For example, this form factor looks really nice for a
| "hobbyist edition" or "evaluation edition":
| https://zimacube.zimaboard.com/. I would probably buy an
| Oxide rack like this as soon as pre-orders were announced.
| vhodges wrote:
| Not the same in any meaningful way but
| https://turingpi.com/product/turing-pi-2/ might interest
| you.
|
| Also https://artemis.sh/2022/03/14/propolis-oxide-at-home-
| pt1.htm...
|
| Does illumos run on ARM?
| Fnoord wrote:
| I own a Turing Pi 2 but the hardware it is running on is
| proprietary. The switch isn't managed. The manament
| software is very archaic. Yes, it is modular and
| stackable and probably thousands of times more hobbyist
| friendly than Oxide but so is edge computing in general.
| Xymist wrote:
| They won't even tell you how much a rack will cost.
| Infuriating typically B2B "talk to Sales so we can decide
| exactly how much we can get out of you and segment the
| market on the fly" approach persists even here, it seems.
| fiddlerwoaroof wrote:
| I wouldn't expect anything else for a full rack in this
| segment: it's going to be tens or hundreds of thousands
| of dollars, and big enough that there will be some
| inevitable negotiation about prices.
| Xymist wrote:
| That doesn't mean you can't have a thin spec builder and
| a pricing page, even if what that mostly gets used for is
| devs putting together a comparison of that to a cloud
| deployment or similar and taking that to the procurement
| department to argue it's worth opening the conversation.
| pid-1 wrote:
| While working in telecom data centers circa 2016 I've seen many
| single rack computers from Dell, IBM, HP, Huawei... Not sure
| that's a new ideia, ex. the open source bits.
| yencabulator wrote:
| I think Dell, IBM & HP all went through a "blade" era where
| they built cableless systems that plugged into a backplane.
| ollybee wrote:
| Existing vendors will provide rack integration services and
| deliver a turn key solution like this. Also vendors of
| virtualization management software have partnerships with
| hardware suppliers and be happy to deliver fully integrated
| solutions if you're buying by the rack. The difference is in
| those cases you have flexibility in the design which seems to
| be missing here.
|
| Proxmox and a full rack of Supermicro gear would not be as
| sophisticated, but end result is pretty much the same, with I
| imagine far far better bang for buck.
|
| I like it, but it doesn't seem like a big deal or revolutionary
| in any way.
| growse wrote:
| Those of us who've bought large "turn-key" solutions from
| Dell etc. have often discovered that it's actually just a
| cobbled-together bunch of things which may or may not work
| well together on a good day, depending on what you're trying
| to do. Just because it's all got the word "Dell" written on
| it, doesn't mean that the components were all engineered by
| people who were working together to build a single working
| system.
|
| When it breaks, good luck!
| EvanAnderson wrote:
| Total agreement. Another point: Having the "Dell" name on
| the front doesn't give you a "throat to choke" as so many
| people seem to think is important. Unless you're very large
| scale then, at best, you can threaten them that they don't
| get your next business. You're certainly not going to get
| help.
|
| You're no worse-off with Oxide from that perspective. Their
| open source firmware means that thr opportunity to pay
| somebody else to support you at least exists.
| tomnipotent wrote:
| > that they don't get your next business.
|
| Even small shops can use bad experience as leverage for
| credits and discounts, especially if the vendor has
| account managers. This is one of the (few) benefits of
| having a human involved in invoicing vs. self-serve.
| yencabulator wrote:
| Same is true of Oxide, it'll be up to actual experience to
| see how well it works. Oxide seems to have written their
| own distributed block storage system
| (https://github.com/oxidecomputer/crucible), have their own
| firmware, kernel and hypervisor forks, etc -- when any of
| that breaks, good luck!
| jjav wrote:
| > Oxide seems to have written their own ...
|
| > when any of that breaks, good luck!
|
| The premise is that you don't need luck, you can call
| Oxide. As you said, they wrote all of it, so they own all
| the interaction so they can diagnose all of it.
|
| When I call Dell with a problem between my OS filesystem
| and the bus and the hardware RAID, there's at least three
| vendors involved there so Dell doesn't actually employ
| anyone that knows all of it so they can't fix it.
|
| Sure, Oxide now needs to deliver on that support promise
| but at least they are uniquely positioned to be able to
| do it.
| yencabulator wrote:
| That's the same premise as with all "turn-key" solutions.
| If it didn't come with software support, it wasn't really
| turn-key.
|
| The rest comes down to execution. Sure, we all have high
| hopes for Oxide. Sure, we all hate established players
| like Dell.
| jjav wrote:
| > That's the same premise as with all "turn-key"
| solutions. If it didn't come with software support, it
| wasn't really turn-key.
|
| Just about any company will sell your company a support
| contract.
|
| The more interesting question is, can they back it up
| with action when push comes to shove? I suspect most
| people have plenty of stories of opening support tickets
| with big name vendors that never get resolved. And
| through the grapevine you find out that they won't fix it
| because they can't fix it. They might not even have
| access to the source code or anyone on staff who has a
| clue about it because it came from who knows where. Sales
| is happy to sell you the support contract but it doesn't
| mean your problems can be fixed. BTDT.
|
| From listening to the Oxide podcasts, my impression is
| that Oxide actually can technically fix anything in the
| stack they sell, which would make them vastly different
| from Dell et.al.
| yencabulator wrote:
| Skill-wise, yes for sure (except perhaps for storage -- I
| haven't heard them talk about that much). Bandwidth wise,
| though?
|
| I used to work for a company targeting Fortune 500s. At
| that level of spend, when a client had a problem,
| somebody got on a plane. Only a fraction of those
| problems escalated all the way to R&D, which is where
| Oxide skills are. That's where VMWare etc are hard to
| beat.
| dasil003 wrote:
| The premise is that the bandwidth needed will be orders
| of magnitude less, because the engineering will be orders
| of magnitude better. The opportunity makes sense as we've
| long been climbing up the local maximum peak of
| enterprise sales driven tech behemoths built on a cobbled
| together mix of open source and proprietary pieces held
| together with bubblegum.
|
| Can an engineering first approach break into the cloud
| market? Hard to say as enterprise sales is very powerful,
| and the numerous "worse is better" forces always loom
| large in these endeavours. That said, enterprise sales
| driven companies are fat, slow and complacent. Oxide is
| lean and driven, and a handful of killer use cases and
| success stories is probably enough to sustain them and
| could be the thin end of the wedge on long-term success.
| We can hope anyway.
| JeremyNT wrote:
| > Proxmox and a full rack of Supermicro gear would not be as
| sophisticated, but end result is pretty much the same, with I
| imagine far far better bang for buck.
|
| I think the question is how well they can do the management
| plane. Dealing with the "quirks" of a bunch of grey box
| supermicro stuff is always painful in one way or another. The
| drop shipped, pre-cabled cab setups are definitely nice but
| that's only a part of what Oxide is doing here. No cables and
| their own integrated switching sounds nice too (stuff from
| the big vendors like UCS is closer to this ballpark but also
| probably closer to the cost too).
|
| I suspect cooling and rack density could be better in the
| Oxide solution too, not having to conform to the standards
| might afford them some possibilities (although that's just a
| guess, and even if they do improve there these may not be the
| bottlenecks for many).
| throw0101c wrote:
| > _I think the question is how well they can do the
| management plane._
|
| Docs:
|
| * https://docs.oxide.computer/api/guides/responses
|
| See perhaps "This repo houses the work-in-progress Oxide
| Rack control plane."
|
| * https://github.com/oxidecomputer/omicron
| civilitty wrote:
| Classic hacker news!
| https://news.ycombinator.com/item?id=9224
| ollybee wrote:
| It's a fair point! I would certainly trust the opinion of
| Bryan Cantrill over my own as well.
| foobiekr wrote:
| Also future datacenter builds are going to be focusing on
| specific applications which means specific builds. I think
| Nvidia has a much better chance here with their superpod than
| Oxide. The target use case is pretty unclear.
|
| On-prem buyers are doing cost reduction and cost reduction
| targets things like, as one example, the crazy cost of GPU
| servers on the CSPs. Your run of the mill stuff is very hard
| to cost reduce.
|
| You can see their sort of lack of getting it by using Tofino2
| as their switch. That's just a very bad choice that was
| almost certainly chosen for bad reasons.
| hderms wrote:
| can you elaborate a bit? What you're saying sounds pretty
| interesting but I'm too ignorant to read between the lines
| foobiekr wrote:
| You don't build a new greenfield compute pod because you
| want to, you do it because it makes sense. Making sense
| is about cost and non-cost needs like data gravity and
| regulatory issues.
|
| The cost case only works for GPU heavy workloads which
| this isn't - wrong chassis, wrong network, etc.
|
| Tofino2 is the wrong choice because even when they made
| that choice it would have been clear that it's doa. Intel
| networking has not been a success center in, well, ever.
| That's a selection that could only have been made for
| nerd reasons and not sensible business goals alignment or
| risk mitigation.
|
| When you make an integrated solution you'd better be the
| best or close to the best at everything. This does not
| seem to be the best at anything. I will grant that it is
| elegant and largely nicer than the hyper converged story
| from other vendors but in practical terms this is the
| 2000s era rack scale VxBlock from Cisco or whatever Dell
| or HPE package today. Marginally better blade server is
| not a business.
|
| They also make a big deal and have focused on things no
| one who actually builds data center pods cares about.
|
| I actually hope they get bought by Dell or HPE or
| SuperMicro. Those companies could fix what's wrong here
| and benefit a lot from the attention to detail and
| elegance on display.
| la64710 wrote:
| AWS outposts have been there in the market for a long time ..
| though I am sure there are differences but to say extisting
| cloud vendors were blind to on prem requirements is a
| stretch.
| jjav wrote:
| > Existing vendors will provide rack integration services and
| deliver a turn key solution like this.
|
| My experience with the likes of Dell is that they'll deliver
| it but they won't support it.
|
| Sure, there's a support contract. And they try. But while
| they sell a box that says Dell, the innards are a hodgepodge
| of stuff from other places. So when certain firmware doesn't
| work with something else, they actually can't help because
| they don't own it, they're just a reseller.
| silverlake wrote:
| How is this different from AWS Outpost?
| ale42 wrote:
| I guess Outpost is not open source?
| samcat116 wrote:
| This is a one time cost, AWS is a rent only model
| skywhopper wrote:
| You own this. AWS Outpost is leased and you still also pay
| for the resource usage on top of the outpost unit itself. And
| this would not be integrated with your AWS account.
| electroly wrote:
| It's mostly not true that you still pay for resource usage
| on top of the Outpost unit. That's only true, AFAIK, for
| EBS local snapshots and Route 53 Resolver endpoints. The
| big boys--EC2 instances, S3 storage, and EBS volumes--are
| all "free" on Outposts. That is, included in the cost of
| the unit and not double-charged.
|
| Charging for EBS local snapshots on your own Outpost S3
| storage and Route 53 Resolvers on your own compute is a
| weird one. I don't know how they defend that. To me, it
| seems indefensible.
| willglynn wrote:
| AWS Outposts are leased:
|
| > You can purchase Outposts servers capacity for a three-
| year term and choose between three payment options: All
| Upfront, Partial Upfront, and No Upfront. ... At the end
| of your Outposts servers term, you can either renew your
| subscription and keep your Outposts server(s), or return
| your Outposts server(s). If you do not notify AWS of your
| selection before the end of your term, your Outposts
| server(s) will be renewed on a monthly basis, at the rate
| of the No Upfront payment option corresponding to your
| Outposts server configuration.
|
| https://aws.amazon.com/outposts/servers/pricing/
|
| > You can purchase Outposts rack capacity for a 3-year
| term ... either renew your subscription and keep your
| existing Outposts rack(s), or return your Outposts
| rack(s)
|
| https://aws.amazon.com/outposts/rack/pricing/
|
| There is no permanent purchase option.
| linklocaladdr wrote:
| Route 53 on outposts doesn't charge for your own compute.
| The outpost resolver is free. The oupost resolver
| endpoint must be backed by an in-region resolver
| endpoint, which doesn't become free just because you own
| an outpost. What you're paying for is the ability to
| sustain high query volume from instances in your VPC to
| an on-prem DNS server.
| electroly wrote:
| Thank you for this explanation!
| kristianpaul wrote:
| This is a turn-key solution, ready to use without eventually
| dealing with multiple devices with its own firmware and caveats
| revealing after where put to work together. The closest to that
| is that AWS managed rack that works with the web APIS you know
| already
| ec109685 wrote:
| AWS only rents their racks and are very expensive.
|
| I do wonder if AWS will eventually go down market if oxide
| gets any scale.
|
| It seems like AWS has all the pieces to compete with Oxide if
| they care to.
| hlandau wrote:
| >Even if Oxide goes bust, you can still salvage things in a
| pinch.
|
| Is this true? Can you set your own root of trust for the
| firmware signing key and build and deploy it yourself?
| mlindner wrote:
| I would assume so. They've said before you can make
| modifications to the firmware and deploy it yourself if you
| so wish. That's one of the major reasons that making the
| firmware open source is so useful.
| la64710 wrote:
| On site backup inventory
|
| DC build out cost and effort
|
| Power cooling requirements
|
| Dark fiber bandwidth requirement
|
| New headcount to support all this
|
| No thanks , I have widgets to make and sell and a business to
| run.
| monocasa wrote:
| It's meant for orgs running at a certain scale, but you'd be
| surprised how early that starts making sense. AWS isn't
| exactly paying the economy of scale savings on to you.
| la64710 wrote:
| True but at for companies operating at scale they not only
| operate on AWS but on other cloud providers as well as
| legacy data centers ... but business wise it's a hard sell
| , it may sell for couple of cycles to build a new dc or use
| an existing ones but then it will be back to the cloud
| again for many more cycles.
| monocasa wrote:
| I've found that the scale these start making sense is
| only a handful of racks. We're not talking full DCs, but
| a room in a building or some colo space.
|
| And beyond that, there's all sorts of weird environments
| that need a lot of local compute. You'd be shocked how
| many servers are on a cruise ship for one example among
| many.
| skywhopper wrote:
| Your business model is not the only one in existence.
| gigatexal wrote:
| Anyone know specs or prices?
| steveklabnik wrote:
| Specs are here: https://oxide.computer/product/specifications
|
| Prices are here: https://oxide.computer/sales ;)
| sorenbs wrote:
| I know you are half joking, but it would really be helpful
| to have ball-park pricing available. Are we talking Sun-
| level markups here, or how should we think about it? Given
| the enterprise sales contact form, I'm thinking yes, but
| I'd love to know for sure.
| steveklabnik wrote:
| > how should we think about it?
|
| I am not in sales and so I hesitate to speak on it in
| case I am incorrect, but the way that I personally think
| about it is that it is true that it is not an
| _inexpensive_ product: there 's a LOT of computer here.
| But the goal is to be competitively priced.
| vhodges wrote:
| The last time (one of them) Oxide hit HN there were some
| ballpark estimates based on the CPUs in use, switches
| etc. Someone else said 500K and up.
|
| I wish there was a 4U version (10-25K but I don't think
| that they could come close to that price point -
| regardless even that is out of reach for me to ever get
| to play on one :-/ )
| glandium wrote:
| I guess you're thinking 4U blade enclosures? That would
| certainly be interesting.
| Q6T46nT668w6i3m wrote:
| Awesome. This would be especially useful in science but the
| lack of GPUs is a non-starter. :(
| goldinfra wrote:
| Coupling vs Decoupling is not some one-sided thing. It's a
| major trade-off.
|
| One of the most obvious examples of the problem with this
| approach is that they're shipping previous generation servers
| on Day 1. One can easily buy current generation AMD servers
| from a number of vendors.
|
| They will also likely charge a significant premium over
| decoupled vendors that are forced to compete head-to-head for a
| specific role (server vendor, switch vendor, etc).
|
| Their coupling approach will most likely leave them perpetually
| behind and more expensive.
|
| But there are advantages too. Their stuff _should_ be simpler
| to use and require less in-house expertise to operate well.
|
| This is probably a reasonable trade-off for government agencies
| and the like, but will probably never be ideal for more savvy
| customers.
|
| And I don't know how truly open source their work is but if
| it's truly open source, they'll most likely find themselves
| turned into a software company with an open core model. Other
| vendors that are already at scale can almost certainly assemble
| hardware better than they can.
| lostdog wrote:
| If they standardize and open the server shape and plug
| interface then it gets really cool. Then I could go design a
| GPU server myself and add it to their rack. The rack is no
| longer a hyperconverged single-user proprietary setup and
| becomes something that can be extended and repurposed.
| foobarian wrote:
| "The Framework Cloud"
| dangoor wrote:
| > They will also likely charge a significant premium over
| decoupled vendors
|
| It seems like they're trying to hit a middle ground between
| cloud vendors and fully decoupled server equipment companies.
|
| Using Oxide is likely cheaper over the life of the hardware
| than using a cloud vendor. A company who already has in-house
| expertise on running racks of systems may be less the target
| market here than people who want to do cloud computing but
| under their own control.
| sangnoir wrote:
| > A company who already has in-house expertise on running
| racks of systems may be less the target market here than
| people who want to do cloud computing but under their own
| control.
|
| True, but Oxide may find themselves competing against Dell
| or HP if they adopt Oxides software for their respective
| servers. Additionally, Oxide may find itself competing
| against consultants and vendors in specialized verticals
| (e.g. core Banking software + Oracle DB + COTS servers +
| Oxide software). Oxide, and their competitors are going for
| people who used to buy racks of Sun hardware.
| ahhfgsado6698 wrote:
| HP and Dell would have to fundamentally change the way
| they design hardware and software to be that kind of
| threat, and if that ever happens I think I would be
| pretty okay with that outcome.
| mlindner wrote:
| > One of the most obvious examples of the problem with this
| approach is that they're shipping previous generation servers
| on Day 1. One can easily buy current generation AMD servers
| from a number of vendors.
|
| > Their coupling approach will most likely leave them
| perpetually behind
|
| This is a startup that took years to get their initial
| hardware developed. The time between this version and the
| version using the next version of AMD chips will be shorter
| than the time it took to develop this product. This is not an
| inherent issue with coupling vs decoupling.
|
| Also, most servers are rarely running on the most recent cpus
| anyway. At least in companies I've worked at with on-site
| hardware they're usually years (sometimes even a decade) out
| of date getting the last life sucked out of them before too
| many internal users start complaining and they get replaced.
| goldinfra wrote:
| Coupling requires more integration work, including writing
| and testing custom firmware. Oxide will be a tiny market
| player for a long time, even if things go very well. Are
| AMD and Broadcom really going to spend as much time helping
| Oxide as they do helping Dell? Of course not, Oxide's order
| volume will be a rounding error.
|
| I'm sure they'll improve their processes over time but the
| lag will probably always be a non-zero value. Hopefully
| they'll be able to keep it low enough that it's not an
| important factor but as a customer it's certainly something
| one should consider.
|
| It would be surprising if they _don 't_ run into some nasty
| issue that leaves their customers 6+ months behind on
| servers or switches at some point.
| mlindner wrote:
| From listening to their talks they've actually gotten
| pretty good direct responses from AMD and AMD likes them
| quite a bit. They've done what no other system integrator
| has done and brought up the CPU without using AMD's AGESA
| firmware bootloader. By simplifying the system they've
| reduced the workload on what they need to handle.
|
| The talk here talks about that from about 32:15 :
| https://www.osfc.io/2022/talks/i-have-come-to-bury-the-
| bios-...
|
| As to your second point, unless AMD somehow becomes
| supply constrained and only wants to ship to their most
| important customers first I don't see a future where
| there would be any lag. Again, the delay this time is
| from how long it took from company start until product
| release. Future delays will be based on the time it takes
| from them getting early development parts to released
| products, which they could even possibly beat Dell to
| market on given the smaller company size and IMO more
| skilled employees.
|
| > It would be surprising if they don't run into some
| nasty issue that leaves their customers 6+ months behind
| on servers or switches at some point.
|
| I mean they've already hit tons of nasty issues, for
| example finding two zero-day vulnerabilities in their
| chosen security processor. They've shown they can work
| around issues pretty well.
| DrScientist wrote:
| > it would be surprising if they don't run into some
| nasty issue that leaves their customers 6+ months behind
| on servers or switches at some point.
|
| I just think your premise is wrong - most customers don't
| care about not having the absolute latest and greatest.
| Indeed they will often avoid them because
|
| 1. They are new so more likely to have as yet
| undiscovered issues ( hardware or drivers ).
|
| 2. If you buy top end, they sell at a premium well above
| their performance premium.
|
| ie the customers who are perennially chasing the latest
| hardware are in the minority.
| goldinfra wrote:
| Most customers care about having the best of the
| available options. Rarely would any company deliberately
| choose to be behind where their competitors can be.
|
| 1. The way to run into undiscovered issues is to choose a
| completely custom firmware/hardware/software stack that
| almost no one else in the world is running.
|
| 2. Not sure where you're getting this from. There is
| almost always a price:performance calculation that
| results in current generation smashing the previous
| generation with server and switch hardware. Often this
| means not buying the flagship chips but still the current
| generation.
|
| And a major reason to get off old generations of hardware
| is that they become unavailable relatively quickly. It's
| always easier to buy current generation hardware than
| previous generation hardware, especially a couple years
| into the current generation. This has nothing to do with
| chasing the latest hardware.
| DrScientist wrote:
| > And a major reason to get off old generations of
| hardware is that they become unavailable relatively
| quickly.
|
| That's not in the customers interests per se- in fact
| it's a pain. Having control of their own stuff could mean
| they could offer a much longer effective operational
| life.
|
| > The way to run into undiscovered issues is to choose a
| completely custom firmware/hardware/software stack that
| almost no one else in the world is running.
|
| What breaks stuff is _change_ - sure when they are
| starting up it 's higher risk - but again if they can
| manage the lifecycle better, not have change for changes
| sake, then they could be much more reliable.
|
| > Not sure where you're getting this from.
|
| I was talking about not taking the flagship stuff - which
| is typically a few months ahead of the best
| price/performance stuff.
| xp84 wrote:
| > will probably never be ideal for more savvy customers
|
| IDK about every use case, but slightly older generations of
| CPUs would affect me roughly zero. I'm sure there are things
| so compute-intensive that one would care very much, but a lot
| of people probably wouldn't bat an eye about that, and not
| because they're unsavvy.
|
| To the extent that these things are supported as a whole by
| the vendor rather than a bunch of finger pointing though,
| that could be massive, specifically in terms of how many
| staff members you could "not hire" compared to if you had to
| employ someone to both build and continually maintain it.
|
| I'm posting this not to invalidate what you're saying, just
| to say that a little predictable upfront amount of money (the
| premium) will be spent very happily by lots of people who
| value predictability and TCO over initial price.
| SoftTalker wrote:
| Indeed, I'm still using a cluster of Haswell processors to
| run VMs for appropriate workloads and it's all fine.
| goldinfra wrote:
| If you're not rapidly scaling it probably doesn't matter.
| But if you're still buying (and maybe even using) Haswell
| CPUs in 2023, you may be missing out in a big way.
|
| A moderately large Haswell cluster is equivalent in power
| to a moderately powerful modern server.
| geraldhh wrote:
| replacing when and if energy, resources and effort
| required to run it surpass those required to replace it
| is efficient
| SoftTalker wrote:
| No not buying new, just using what was bought years ago.
| It still works, it does the job. Is it the best
| performance per watt, clearly no but the budget for
| electricity and the budget for new capital expenses are
| two different things.
| seabrookmx wrote:
| If you go on Google cloud and select an E2 instance type
| (atleast in `us-central1` where my company runs most of
| it's infra) you'll usually get Broadwell chips.
| pxc wrote:
| > Ironically this looks like the realization of Richard
| Stallman's dream where users can help each other if something
| doesn't work.
|
| How is it ironic?
| Upvoter33 wrote:
| I don't see it as a big deal - rather, I see it as a huge
| amount of venture cap spent on some very bright people to build
| something no one really wants, or, at best, is niche.
|
| Also, it has little to do with the cloud; it is yet another
| hyperconverged infra.
|
| Weirdly, it is attached to something very few people want:
| Solaris. This relates to the people behind it who still can't
| figure out why Linux won and Solaris didn't.
| Voultapher wrote:
| Right, who wants or benefits from open source firmware
| anyway.
|
| Also there are many situations where renting, for example a
| flat makes a lot of sense. And there are many situations
| where the financials and or enabled options of owning
| something make a lot of sense. Right now, the kind of
| experience you get with AWS and co. can only be rented, not
| bought. Some people want to buy houses instead of renting
| them.
| Always_Anon wrote:
| >Right, who wants or benefits from open source firmware
| anyway.
|
| Their competition has open source firmware as well:
|
| https://www.dell.com/en-us/blog/enabling-open-embedded-
| syste...
| bcantrill wrote:
| So OpenBMC is fine (happy for them!), but having open
| firmware is much deeper and broader than that: yes, it's
| the service processor (in contrast to the BMC which is a
| closed part on Dell machines) -- but it's also the root-
| of-trust and (especially) the host CPU itself. We at
| Oxide have open source software from first instruction
| out of the AMD PSP; I elaborated more on our approach in
| my OSFC 2022 talk.[0]
|
| [0] https://www.osfc.io/2022/talks/i-have-come-to-bury-
| the-bios-...
| Always_Anon wrote:
| Dell now ships with OpenBMC iDRACs and such. How does
| what you mention differ from the RoT in Dells?
|
| https://www.dell.com/en-us/blog/hardware-root-trust/
| c_o_n_v_e_x wrote:
| Dell uses trusted platform modules (TPM). It's a separate
| chipset than the BMC chipset.
|
| For a mostly open source solution, not only would you
| need open source BMC firmware, you must have an open
| source UEFI/BIOS/boot firmware like CoreBoot, LinuxBoot,
| Oreboot, Uboot, etc.
| necovek wrote:
| Well, you can buy your own hardware and set it up with
| OpenStack and use it as a private cloud. Companies like
| Canonical or Redhat make a lot of money by providing
| software (mostly open source) to support exactly that use
| case.
|
| And Canonical played with a cluster-in-a-box all the way
| back in 2013-2014:
| https://www.zdnet.com/article/canonicals-cloud-in-a-box-
| unde...
|
| You could turn it into an OpenStack cloud in ~20 mins with
| an automated Juju OpenStack install.
| jjav wrote:
| > Well, you can buy your own hardware and set it up with
| OpenStack and use it as a private cloud. Companies like
| Canonical or Redhat make a lot of money by providing
| software (mostly open source) to support exactly that use
| case.
|
| Sure you can, but then who will diagnose and fix your
| hardware/OS interaction problems when you have parts from
| five vendors in the mix?
|
| If you haven't lived through this, the answer is: nobody.
| Everyone points fingers at the other 4 and ignore your
| calls.
|
| Back in the day you could buy a fully integrated system
| (from CPU to hardware to OS) from Sun or SGI or HP and
| you had a single company to answer all the calls, so it
| was much better. Today you can't really get this level of
| integration and support anymore.
|
| (Actually, you probably can from IBM, which is why
| they're still around. But I have no experience in the IBM
| universe.)
|
| This is why Oxide is so exciting to me. I hope I can be
| in a company that becomes a customer at some point.
| Always_Anon wrote:
| >Sure you can, but then who will diagnose and fix your
| hardware/OS interaction problems when you have parts from
| five vendors in the mix?
|
| Dell is a single vendor that will diagnose and fix all of
| your hardware issues.
|
| With Oxide you're locked into what looks like a Solaris
| derivative OS running on the metal and you're only
| allowed to provision VMs which is a huge disadvantage.
|
| I run a fleet of over 30,000 nodes in three continents
| and the majority is Flatcar Linux running on bare metal.
| Also have a decent amount of RHEL running for specific
| apps. We can pick and choose our bare metal OS which is
| something you cannot do with Oxide. That's a tough pill
| to swallow.
| wmf wrote:
| _Dell is a single vendor that will diagnose and fix all
| of your hardware issues._
|
| And you'll be down for weeks or months while they do it.
| Always_Anon wrote:
| >And you'll be down for weeks or months while they do it.
|
| Off by a few orders of magnitude. Dell on-site SLA with
| pre-purchased spares was about 6 hours.
|
| With Oxide, you'd be lucky to get same day service.
| jjav wrote:
| > Off by a few orders of magnitude. Dell on-site SLA with
| pre-purchased spares was about 6 hours.
|
| You're talking about replacement parts. Yes Dell is good
| about that.
|
| The discussion above is asking them to diagnose and fix a
| problem with the interaction of various hardware
| components (all of which come from third parties).
| Always_Anon wrote:
| Oxide also has various hardware components from AMD,
| Intel, Samsung, etc. They are not manufacturing every
| component.
| mlindner wrote:
| But they _are_ writing the firmware that runs most of
| them and need to understand those devices at a deep level
| in order to do that, unlike Dell. Dell slaps together
| hardware and firmware from other vendors with some high
| level software of their own on top. They don't do the low
| level firmware and thus don't understand the low level
| intricacies of their own systems.
| Always_Anon wrote:
| No they're not unless I'm mistaken. They're not writing
| the firmware that runs on the NVMe drives, nor the NICs
| (they're not even writing the drivers for some of the
| NICs), etc.
|
| There's a line in the sand that you must cross when it
| comes to understanding the true nature of the componentry
| that you're using. At the end of the day, your AMD CPUs
| _may_ be lying to you, to all of us, but we just don 't
| know it yet.
| wmf wrote:
| I'm not speaking hypothetically. If you hit a "zero-day"
| bug that Dell has never seen it's going to take time. And
| somehow every large customer finds bugs that Dell
| certification didn't.
| Always_Anon wrote:
| >I'm not speaking hypothetically.
|
| Neither am I.
|
| >If you hit a "zero-day" bug that Dell has never seen
| it's going to take time.
|
| If you hit a "zero-day" bug that Oxide has never seen
| it's going to take time.
|
| >And somehow every large customer finds bugs that Dell
| certification didn't.
|
| Yes, happens. And I'm sure the exact same will happen
| with Oxide, so it's not a differentiator.
| pests wrote:
| > And somehow every large customer finds bugs that Dell
| certification didn't.
|
| It's a law of computer engineering.
|
| In the Apollo 11 decent sequence the Rendezvous Radar
| experienced a hardware bug[0] not uncovered during
| simulation. They found it later, but until then, the
| solution was adding a "turn off Rendezvous Radar"
| checklist item.
|
| [0] The Rendezvous Radar would stop the CPU, shuttle some
| data into areas it could be read, and woke the CPU back
| up to process it. The bug caused it to supuriously do
| this dance just to tell it "no new data", which then
| caused other systems to overload.
| jjav wrote:
| > Dell is a single vendor that will diagnose and fix all
| of your hardware issues.
|
| I've been a Dell customer at a previous company. I know
| for a fact that's not true.
|
| I had a support ticket for a weird firmware bug open for
| two years, they could never figure it out. I left that
| job but for all I know the case is still open many years
| later.
|
| Dell doesn't know how to fix things like that because
| they don't design and engineer the systems they sell.
| Dell is a reseller who puts components together from a
| bunch of vendors and it mostly works but when it doesn't,
| there's nobody on staff who can fix it.
| Always_Anon wrote:
| I've been a Dell customer for decades at this rate and I
| know for a fact it's true.
|
| I've had support tickets open for all kinda of weird
| firmware, hardware, etc. bugs and they've been well
| resolved, even if it meant Dell just replaced the part
| with something comparable (NIC swap).
|
| >Dell doesn't know how to fix things like that because
| they don't design and engineer the systems they sell.
|
| Of course they do. That's like saying Oxide doesn't know
| how to fix stuff because they don't design the CPU, NVMe,
| DIMMs, etc. Oxide is still going to vendors for these
| things.
| bcantrill wrote:
| Ironically, it was Dell's total inability to resolve a
| pathological rash of uncorrectable memory errors very
| much is part of the origin story of Oxide: this issue was
| _very_ important to my employer (who was a galactic Dell
| customer) and as the issue endured and Dell escalated
| internally, it became increasingly clear that there was
| in fact no one at Dell who could help us -- Dell did not
| understand how their own systems work.
|
| At Oxide, we have been deliberate at every step,
| designing from first principles whenever possible. (We --
| unlike essentially everyone else -- did not simply
| iterate from a reference design.)
|
| To make this concrete with respect to the CPU in
| particular, we have done our own lowest-level platform
| enablement software[0] -- we have no BIOS. No one -- not
| the hyperscalers, not the ODMs and certainly not Dell --
| has done this, and even AMD didn't think we could pull it
| off. Why did we do it this way? Because all along our
| lodestar was that problem that Dell was useless to us on
| -- that we wanted to understand these systems from first
| principles, because we have felt that that is essential
| to deliver the product that we ourselves wanted to by.
|
| There are plenty of valid criticisms of Oxide -- but that
| we don't understand our system simply isn't one of them.
|
| [0] https://www.osfc.io/2022/talks/i-have-come-to-bury-
| the-bios-...
| mlindner wrote:
| As a side question, what's the name of your custom
| firmware that is the replacement of the AGESA bootloader?
| I tried searching on the oxide github page but couldn't
| find anything that seemed to fit that description.
| bcantrill wrote:
| (The AGESA bootloader -- or ABL -- is in the AMD PSP.) In
| terms of our replacement for AGESA: the PSP boots to our
| first instruction, which is the pico host bootloader,
| phbl[0]. phbl then loads the actual operating system[1],
| which performs platform enablement as part of booting.
| (This is pretty involved, but to give you a flavor, see,
| e.g. initialization of the DXIO engine.[2])
|
| [0] https://github.com/oxidecomputer/phbl
|
| [1] https://github.com/oxidecomputer/illumos-
| gate/tree/stlouis
|
| [2] https://github.com/oxidecomputer/illumos-
| gate/blob/stlouis/u...
| mlindner wrote:
| Thanks, are the important oxide branches of illumos-gate
| repo (and any other cloned repos) defined anywhere? I
| definitely wouldn't have found that branch without you
| mentioning it here.
| Always_Anon wrote:
| Interesting enough I also ran into something somewhat
| related with Dell that they were not able to resolve so
| they ended up working in a replacement from another
| vendor.
|
| Nonetheless, it is quite interesting what you've built,
| but as the end user I'm not quote convinced that it
| matters. Sure you can claim it reduces attack vectors and
| such but we'll still see Dells and IBMs in the most
| restricted and highest security postured sites in the
| world. Think DoD and such. Core/libreboot with RoT will
| get me through compliance the same.
|
| The software management plane y'all built is the
| headlining feature IMHO, not so much what happens behind
| the scenes that the vast majority of the time will not
| have a fatal catastrophic upstream effect.
|
| >There are plenty of valid criticisms of Oxide -- but
| that we don't understand our system simply isn't one of
| them.
|
| That's not what I said. There's a line in the sand that
| you must cross when it comes to understanding the true
| nature of the componentry that you're using. At the end
| of the day, your AMD CPUs _may_ be lying to you, to all
| of us, but we just don 't know it yet.
| samcat116 wrote:
| The vast majority of people only need to deploy VMs.
| Always_Anon wrote:
| It's ironic coming from a company who's CTO has harped
| about containers on bare metal for years. Maybe a large
| swath only need to deploy VMs, but the future will most
| definitely involve bare metal for many use cases, and
| oddly Oxide doesn't support that currently.
| pseg134 wrote:
| I run a battalion of 78,000 nodes and I disagree with
| you.
| Always_Anon wrote:
| I used to run over 150,000 nodes and I agree with me.
| jryle70 wrote:
| See the pattern? Dell only care about the big guys.
|
| Set aside the childish tone ...
|
| > Dell is a single vendor that will diagnose and fix all
| of your hardware issues.
|
| There are two anecdotes here disagreeing with you, and
| frankly that's enough to say what you said above isn't
| true, not universally so. I doubt Odixe is targeting big
| deployment like yours, but more like theirs. Whether they
| will succeed is another matter, but they do have a valid
| sales pitch and the expertise to pull it off.
| jeffrallen wrote:
| > you had a single company to answer all the calls, so it
| was much better
|
| Huh, then why did Sun, Oracle, and Veritas have to set up
| a shared tech support center in San Jose?
|
| "Accelerated finger pointing", said a friend who had to
| do business with them.
| NexRebular wrote:
| The fact that it's not on linux is one of the great things
| about it. There is too much linux on critical infrastructure
| already and the monoculture just keeps on growing.
|
| At least with Oxide there is a glimmer of hope for a better
| future in this regard.
| eduction wrote:
| When you're deploying VMs, which is the use case here, the
| substrate OS becomes significantly less important. Those VMs
| will mostly just be linux.
|
| Yes they are using illumos/Solaris to _host_ this but they
| don 't sell on that, they sell on the functionality of this
| layer -- allowing people to deploy to owned infra in a way
| that is similar to how they'd deploy to AWS or Azure. How
| much do you ever think about the system hosting your VM on
| those clouds? You think about your VMs, the API or web
| interface to deploy and configure, but not the host OS. With
| Oxide racks the customers are not maintaining the illumos
| substrate (as long as Oxide is around).
|
| You could be right about demand, there is risk in a venture
| like this. But presumably the team thought about this - I
| think folks who worked at Sun, Oracle, Joyent, and Samsung
| and made SmartOS probably developed a decent sense of market
| demand, enough to make a convincing case to their funders.
| yencabulator wrote:
| > When you're deploying VMs, which is the use case here,
| the substrate OS becomes significantly less important.
| Those VMs will mostly just be linux.
|
| Now you need to know both the OS they chose and the OS you
| chose...
|
| (No, I don't believe it'll be 100% hands-off for the host.
| This is an early stage product, with a lot of custom parts,
| their own distributed block storage, hypervisor, and so
| on.)
| tinco wrote:
| This true for other hypervisors too. Enterprises are
| still paying hundreds of millions to VMware, who knows
| what's going on in there?
|
| I wouldn't have picked Opensolaris, but it's a lot better
| than other vendors that are either fully closed source,
| or thin proprietary wrappers over Linux with spotty
| coverage and you're not allowed to touch the underlying
| OS for risk of disrupting the managed product.
| sgt wrote:
| What's more important is that the team actually knows
| Illumos/Solaris inside out. You can work wonders with a
| less than ideal system. That said, Illumos is of high
| quality in my opinion.
| Always_Anon wrote:
| Seems risky considering how small of a developer pool
| actively works on illumos/Solaris. The code is most
| definitely well engineered and correct, but there are
| huge teams all around the world deploying on huge pools
| of Linux compute that have contributed back to Linux.
| tinco wrote:
| They had a bug in the database they are using that was
| due to a Go system library not behaving correctly
| specifically on illumos. They've got enough engineering
| power to deal with such a thing but damn..
| mbakke wrote:
| If everyone had this mindset we would be running our
| workloads on Microsoft Windows by now.
|
| GNU/Linux was also "risky" at some point.
| yencabulator wrote:
| Linux grew up in the bedrooms of teenagers. It was risky
| in the era of 486 and Pentiums. The environment and
| business criticality of a $1-2M rack-size computer is
| quite different.
| mbakke wrote:
| I had similar thoughts about VMware (large installations)
| back in the day. Weird proprietary OS to run other
| operating systems? Yet they turned out fine.
|
| This appears to be a _much_ better system than VMware, is
| free as in software, and it builds upon a free software
| operating system with lineage that predates Linux.
|
| I say this in the most critical way possible, as someone
| who has built multiple Linux-based "cloud systems", and
| as a GNU/Linux distribution developer: I love it!
| usefulcat wrote:
| Completely tangential, but this reminds me of an
| interview I had for my first job out of college in 1995.
| I mentioned to the interviewer that I had some Linux
| experience. "Ah, Linux" he said. "A cool little toy
| that's gonna take over the world".
|
| In hindsight of course it was remarkably prescient. This
| from a guy at a company that was built entirely around
| SGI at the time.
| kawhah wrote:
| It was totally a risky choice for companies in the 1990s
| and early 2000s to put all their web stuff onto Linux on
| commodity hardware instead of proprietary Unix or Windows
| servers. Many did it when their website being up was
| totally mission critical. Lots did it on huge server
| farms. It paid off very quickly but it's erasing history
| to suggest that it didn't require huge amounts of guts,
| savvy and agility to even attempt it.
| sanderjd wrote:
| I have no clue what OS runs my VMs on EC2.
| speed_spread wrote:
| I have a feeling they knew exactly from the start who their
| customers would be: People who have the budget to care
| about things like trust and observability in a complex
| system. But these would also be the kind of customers who
| require absolute secrecy and so this why you don't hear
| about them even though they might have bankrolled a sizable
| portion of the operation. Just like the first Cray to
| officially be shipped was actually serial number 2...
| jeffrallen wrote:
| Can you imagine trying to investigate Bryan for a
| security clearance?
|
| "Sir, if you have nothing to hide, why do you talk so
| fast, and pronounce words like a foreigner who learned
| English from a book?"
|
| :)
|
| We love you Bryan, never change.
| eduction wrote:
| Oh to be a fly on the wall for the first Oxide<>Oracle
| partnership discussion...
| jeffrallen wrote:
| Bryan: "Steve, I think you should take this meeting with
| Larry..."
| cashsterling wrote:
| Back in the day... Sun Micro was a GOAT and pushed the
| envelope on Unix computing 20-30 years ago. Solaris was
| stable and high performing.
|
| I don't run on-prem clusters or clouds but know a couple
| people who do and, at large enough scale, it is a constant
| "fuck-shit-stack on top of itself" (to quote Reggie Watts).
| There is almost always something wrong and some people upset
| about it.
|
| The promise of a fully integrated system (compute HW, network
| HW, all firmware/drivers written by experts using Rust
| wherever possible) that pays attention to optimizing all your
| OpEx metrics is a big deal.
|
| It may take Oxide a couple more years to really break into
| the market in a big way, but if they can stick it out, they
| will do _very_ well.
| icedchai wrote:
| I used to love Sun and Solaris. Then the dot-com bubble
| burst, and Linux ate its lunch. I haven't seen a new
| Solaris system deployed in over 20 years.
| steveklabnik wrote:
| Just to be clear, Illumos (it hasn't been Solaris in a very
| long time) is an implementation detail. It's not customer
| facing.
| yencabulator wrote:
| It'll become customer facing the moment something doesn't
| work right.
| ahl wrote:
| It won't. In the same way that AWS customers aren't
| debugging hypervisor, or Dell customers aren't debugging
| the BIOS, or Samsung SSD customers aren't debugging the
| firmware. Products choose where to draw the line between
| customer-serviceable parts and those that require a
| support call. In this case, expect Oxide to fix it when
| something doesn't work right.
| jjav wrote:
| When Apple supports OSX for consumers, they don't exactly
| surface the fact that there's BSD semi-hidden in there
| somewhere.
|
| That's because they own the whole stack, from CPU to GUI
| and support it as a unit. That's the benefit of having a
| product where a single owner builds and supports it as a
| whole.
|
| My impression of Oxide is that that's the level of single
| source of truth they are bringing to enterprise in-house
| cloud. So, I strongly doubt the innards would ever become
| customer-facing (unless the customer specifically wants
| that, being open source after all).
| yencabulator wrote:
| Apple is a horrible example, with Apple when you have a
| problem, you often end up with an unfixable issue that
| Apple won't even acknowledge. You definitely don't want
| to taint Oxide's reputation with that association.
|
| As for why I think Helios will become customer facing:
| Oxide is a small startup. They have limited resources.
| Their computers expensive enough to be very much business
| critical. You'll get some support by Oxide logging in
| remotely to customer systems and digging around, but
| pretty soon the customer will want to do that themselves
| to monitor/troubleshoot the problems as they happen.
|
| Imagine you're observing a recurring but rare I/O
| slowdown that seems to trigger under some certain
| conditions, and tell me a competent sysadmin wouldn't
| want to log in on all the related boxes (client Helios,
| >=3 server Helioses for the block store) and look at the
| logs & stats.
| jjav wrote:
| > Apple is a horrible example,
|
| Apple is a great example of the benefits of an integrated
| system where the hardware and software are designed
| together. There are tons of benefits to that.
|
| What makes Apple evil (IMO, many people disagree) is how
| everything is secret and proprietary and welded shut. But
| that doesn't take away from the benefits of an integrated
| hardware/software ecosystem.
|
| Oxide is open source so it doesn't suffer from the evil
| aspect but benefits from the goodness of engineered
| integration. Or so I hope.
| nijave wrote:
| Android is potentially a better example. Compare Android
| to trying to get Linux working on <some random laptop>.
| You might get lucky and it works out of the box or you
| might find yourself in a 15 page "how to fix <finger
| print reader, ambient light sensor, etc>" wiki where you
| end up compiling a bunch of stuff with random patches.
|
| Afaik Android phones tend to have a lot more hardware
| than your average laptop, too (cell modem, gps, multiple
| cameras, gyro, accelerometer, light sensors, finger print
| readers)
| freedomben wrote:
| In practice I don't think it's as good as in theory. I
| had Apple Macbook Pro with Apple Monitor, and 50% of the
| time when unplugging the monitor the laptop screen would
| stay off. Plugging back in to the monitor wouldn't work
| at that point so all I could do was hold the power button
| to force it off and reboot. That's with Apple controlling
| the entire stack - software, hardware, etc.
|
| I think the real benefit is being able to
| move/deprecate/expand at will. For example, want an app
| that would require special hardware? You can just add it.
| Want to drop support for old drivers? Just stop selling
| them and then drop (deprecate) the software support in
| the next release.
|
| I fully agree about the evilness, and it baffles me how
| few people do!
| Always_Anon wrote:
| Exactly right, Apple is actually a poor example. Watch
| enough Louis Rossmann and you'll grasp just how bad some
| of their shit can be.
| snuxoll wrote:
| > As for why I think Helios will become customer facing:
| Oxide is a small startup. They have limited resources.
|
| Have you looked at the pedigree of many of the people
| behind the project? I don't say this because "these guys
| smart", but because these guys bent over backwards for
| their customers when they were Sun engineers. Bryan
| didn't write dtrace for nothing.
|
| > Imagine you're observing a recurring but rare I/O
| slowdown that seems to trigger under some certain
| conditions, and tell me a competent sysadmin wouldn't
| want to log in on all the related boxes (client Helios,
| >=3 server Helioses for the block store) and look at the
| logs & stats.
|
| I think you're simultaneously over-estimating and under-
| estimating the people who will deploy this. There's a lot
| of companies who would want a "cloud in a box" that would
| happily plug hardware in and submit a support ticket if
| they ever find an issue, because their system engineers
| either don't have the time, desire, or competence
| (unfortunately common) to do anything more. The ones who
| are happy to start debugging stuff on their own would
| have absolutely wonderful tooling at their fingertips
| (dtrace) and wouldn't have any issue figuring out how to
| adapt to something other than Linux (hell, I've been
| running TrueNAS for the better part of a decade and being
| on a *BSD has never bothered me).
| pjmlp wrote:
| Apple is the survivor of 16 bit home micros integration,
| PC clones only happened as IBM failed to prevent Compaq's
| reverse engineering to take over their creation, they
| even tried to get hold of it afterwards via PS/2 and MCA.
|
| As we see nowadays on tablets and laptops, most OEMs are
| quite keen in returning back to those days, as otherwise
| there is hardly any money left on PC components.
| throw0101c wrote:
| > _When Apple supports OSX for consumers, they don 't
| exactly surface the fact that there's BSD semi-hidden in
| there somewhere._
|
| Or Linux running underneath all the Java-y Android stuff.
| Fnoord wrote:
| Funny how you mentioning BSD got me to thinking of Sony
| Playstation and Nintendo Switch. Which are proprietary
| and not user serviceable. A Steam Deck, Fairphone, or
| Framework laptop is each less proprietary and more FOSS
| stack, and user serviceable. Which a user may or may not
| want to do themselves; at the very least they can pay
| someone and have them manage it.
|
| Also, Apple is just the one who survived. Previously I'd
| have thought of SGI, DEC, Sun, HP, IBM, Dell some of whom
| survived some not.
|
| Those three consumer products I mentioned each provide a
| platform for a user and business space to floroush and
| thrive. I expect a company doing something similar for
| cloud computing to want the same. But it will require
| some magick: momentum, money, trust. That kind of stuff,
| and loads of it. (With some big names behind it and a lot
| of FOSS they got me excited, but I don't matter.)
| pjmlp wrote:
| Contrary to urban myths, Nintendo Switch OS is a
| microkernel OS, not something based on BSD.
| nosequel wrote:
| If you have a bug in how a lambda function is run on AWS,
| do you find yourself looking for the bug in firecracker?
| It is open source, so you technically could, but I just
| don't see many customers doing that. Same can be said
| about KNative on GCP.
|
| Their choice in foundation OS (for lack of a better term)
| really should not matter to any customer.
| yencabulator wrote:
| I am _unable_ to do so.
|
| Now imagine a multi-million dollar mission critical pile
| of computers running on premises, and your sysadmin being
| able to do so.
|
| Oxide is closer to a rack of Supermicros than AWS.
| sanderjd wrote:
| Ok but then that is purely additive then, right? Like,
| "have to find someone with Illumos expertise to fix
| something that was never intended to be customer-facing"
| may not be easy, but is still easier than the
| _impossibility_ of doing the same thing on AWS / Azure /
| Google Cloud.
| burnte wrote:
| > Just to be clear, Illumos (it hasn't been Solaris in a
| very long time) is an implementation detail. It's not
| customer facing.
|
| Solaris is still Solaris, as of the latest release last
| month. OpenSolaris hasn't been OpenSolaris in a while and
| is Illumos, yes.
| steveklabnik wrote:
| Yes, thanks. I didn't even realize my comment could be
| read that way, but I was speaking of Illumos only,
| Solaris is still Solaris :)
| sanderjd wrote:
| > _something no one really wants, or, at best, is niche_
|
| Could be! Seems too early to tell though, and remains to be
| seen whether it pencils. Which is the whole idea of starting
| a new venture, no?
| ChuckMcM wrote:
| Its a huge deal. I'm biased though because my own takes on how
| things should evolve were very similar. I was however
| completely unsuccessful in getting those ideas into production!
| And that, that is a huge deal. Through out my career it has
| been interesting to meet people with great ideas and then they
| are unable to get them into production, and when the idea does
| come into production everyone feels like "Wow, this is so
| obvious why didn't we do it sooner?" and some folks are banging
| their head against the wall :-).
|
| One of the more interesting discussions I had during my tenure
| at Google was about the "size" of the unit of clusters. If you
| toured Google you got the whole "millions of cheap replaceable
| computers" mantra. Sitting in Building 42 was a "rack" which
| had cheap PC motherboards on "pizza dishes" without all that
| superfluous sheet metal. Bunches of these in a rack and a
| festoon of network cables. What are the "first class" elements
| of these machines? Compute? Networking? Storage? Did you
| replace components? Or a whole "pizza slice" (which Google
| called an 'index' at the time). Really a great systems analysis
| problem.
|
| FWIW I'm more of a "chunk" guy (which is the direction 0xide
| went) and less of a "cluster" guy (which is the way Google
| organized their infrastructure). A lot of people associated
| with 0xide are folks I worked with at Sun in the early days and
| during that period the first hints of "beowulf" clusters vs
| "super computers", was memory one thing (UMA) or did it vary
| from place to place (NUMA). I have a paper I wrote from that
| time about "compute viscosity" where the effective compute rate
| (which at the time largely focused on transactional databases)
| scaled up with resource (more memory more transactions/sec for
| example) and scaled down with viscosity (higher latencies to
| get to state meant fewer transactions/sec) Sun was invested
| heavily in the TPC-C benchmarks at the time but they were just
| one load pattern one could optimize for.
|
| These guys have capitalized on all that history and it is
| fucking amazing! I just hope they don't get killed by
| acquisition[1].
|
| [1] KbA is a technique where people who are invested in the
| status quo and have resources available use those resources to
| force the investors in a disruptive technology to sell to them
| and then they quietly bury the disruptive technology.
| mgh95 wrote:
| Can you clarify a bit on what you mean by "chunk" guy? Are
| you alluding to the ability to distribute work by an
| isolation mechanism such as cgroups vs machine a-la
| borg/google?
| ChuckMcM wrote:
| More on infrastructure composition software is an
| abstraction above that.
|
| Is the unit of composition a rack (chunk), a server
| (smaller chunk), or a blade (smallest chunk)? In what I
| think of as classic systems architecture you've got a
| 'store' (storage), 'state' (memory), 'threads'
| (computation), and 'interconnect' (fabrics). In the 90's a
| lot of folks focused on fabrics (Cray, Connection Machine,
| Sun, etc) somewhat on threads (compute blades), and state
| came along for the ride. How these systems were composed
| was always a big thing, then along came the first Beowulf
| clusters that used off the shelf motherboards (a "chunk" of
| threads/state/store) with a generic fabric (Ethernet).
| Originally NASA showed that you could do some highly
| parallel processing on these sorts of systems and Larry and
| Sergei at Stanford applied it to the process of internet
| search.
|
| Collectively you have a 'system resource' and with software
| you can make it look like anything you want. When you do
| compute with it, its performance becomes a function of its
| systems balance and the demands of the workload. Its all
| computer sciencey and yes there is a calculus to it. This
| isn't something that most people dive into (or are even
| interested in[1]) but it was one of the things that
| captured my imagination early on as an engineer. I was
| consumed with questions like what was the difference
| between a microprocessor, a mini-computer, a workstation,
| and a mainframe? Why do they each exist? What does one do
| that they other can't? Things like that.
|
| [1] At Google I worked in what they called 'Platforms'
| early on and clearly _most_ of the company didn 't really
| care about the ins and outs of the systems
| bigtable/gfs/spanner/etc ran on, they just wanted APIs to
| call. But they also didn't care about utilization or costs.
| By the time I left some folks had just figured out (and one
| guy was building his career on) the fact that utilization
| directly affected operational costs. They still hadn't
| started thinking about non-uniform rack configurations for
| different workloads.
| TheRealDunkirk wrote:
| > I just hope they don't get killed by acquisition.
|
| Private equity in the US has collectively determined that no
| company shall exist outside of investment ownership. I don't
| know what the ownership structure looks like, but generally
| speaking, it seems that nearly everyone has a "fuck you"
| number. Now that Oxide is venturing into Dell and HP's turf,
| I worry someone will get a fix on what Brian's number is.
| ThinkBeat wrote:
| IBM invented this a long time ago. Mainframes.
|
| >They sell servers, but as a finished product. Not as a cobbled
| together mess of third party stuff where the vendor keeps
| shrugging if there is an integration >problem. They integrated
| it.
| zemo wrote:
| > Ironically this looks like the realization of Richard
| Stallman's dream where users can help each other if something
| doesn't work.
|
| that's only true if you think that "users" means "people who
| operate cloud computers", which is about as far from
| understanding what Stallman is talking about as is possible.
| Someone who makes SaaS and runs it on an Oxide computer is no
| less of a rentier capitalist than someone who makes SaaS and
| runs it on AWS.
| dewey wrote:
| I always thought it's just about the hardware but it seems like
| it also includes it's own kind of virtualization / provisioning
| interface for VMs, firewalls and also an overview over running
| software.
|
| Does this "lock" you into the Oxide platform and you just buy
| into the whole thing instead of buying some server from Dell,
| then running some Proxmox like software and a Docker host?
| depr wrote:
| You run VMs on their platform, so it doesn't lock you in more
| than any cloud host locks you into their provisioning
| interface.
| wpm wrote:
| The VMs are bhyve on illumos. They're about as hard to move off
| as moving a datacenter sized Proxmox cluster to anything else.
| api wrote:
| I saw a demo from them and it looked nice but then they told me
| they are stopping at the VM and storage blob level of
| abstraction.
|
| Virtually all the value we get from cloud is in their managed
| offerings of complex and difficult to admin systems like
| Kubernetes, Postgres, and managed storage layers.
|
| That's where all the value is but it's also where all the cost
| is. If you just want compute, storage, and bandwidth all those
| can be had at commodity prices. Look at Hetzner, Hivelocity,
| FDCServers, DataPacket, OVH, or VPS providers like Vultr. You can
| get dozens of cores, terabytes of SSD, and gigabits of unmetered
| bandwidth for a few thousand dollars or less. It's very cheap.
|
| Without the high level managed stuff we would be off cloud five
| minutes from now.
|
| I'm sure there is a market for this among people who run on
| premise data centers, but I think they would have a much larger
| market if they went further up the software stack. Right now they
| just look like they are competing with Dell and Supermicro, not
| Amazon or Google Cloud.
| weystrom wrote:
| > Everyone at Oxide makes $201,227 USD, regardless of location.
|
| Man I would love to work for a company like that. I don't know
| why more people don't set their startups like that.
| keepamovin wrote:
| Exactly, man! That's the way it should be done. It's not a
| company's business to decide how much your life should cost.
| It's its business to reward you for the value you provide, and
| that value is not tethered to your location, so neither should
| be your salary!
| dewey wrote:
| This really is one of these things where every employee in a
| cheap cost of living area will say "Exactly, man!" and people
| in an expensive area will see it differently because it might
| cap them lower than what they could get.
|
| I always find that a very naive point of view, of course I
| would want to earn a US tech salary while living somewhere in
| the country side, who wouldn't. But I'm also aware that this
| is not how the world works in reality, there's different tax
| systems, different expense costs and we don't live in a
| global one-market world.
|
| I find the strategy of defining different "zones", like most
| of the remote first / salary transparency companies much more
| realistic.
| throw0101a wrote:
| > _This really is one of these things where every employee
| in a cheap cost of living area will say "Exactly, man!" and
| people in an expensive area will see it differently because
| it might cap them lower than what they could get._
|
| Well the co-founders live in the Silicon Valley area, with
| their physical HQ being in Emeryville:
|
| * https://en.wikipedia.org/wiki/Emeryville,_California
| dewey wrote:
| Early stage VC-backed startup compensation is very
| different from later stage companies or bootstrapped
| companies. It's very common for founders or early
| employees to receive a lower salary and receive equity
| instead.
| throw0101a wrote:
| Holding paper-gains equity doesn't provide monthly
| cashflow to buy groceries. :)
| dewey wrote:
| I'm sure 200k will cover that, even in the SV.
| cdchn wrote:
| I got some bad news for you.
| OkayPhysicist wrote:
| The Bay Area is expensive, but it's not THAT expensive.
| cdchn wrote:
| If you've got a family to support and you need to rent or
| try to own a home you're going to have a tough time with
| 200k.
| OkayPhysicist wrote:
| I don't know how you're coming to that conclusion. 200k's
| not going to buy you a large house, but it'll comfortably
| pay rent/food/savings for a 2-3 bedroom house or
| apartment for a family of 2-4 in all but the very most
| expensive parts of the Bay.
|
| For mortgages, you'd need to be looking in the cheaper
| parts of the Bay, but that still means "dense, boring
| suburb" as opposed to "crime-ridden slum".
| cdchn wrote:
| They didn't say "Bay Area", they said SV.
| almost_usual wrote:
| If you owned a home in the Bay Area prior to 2020 and
| refinanced down to a sub 3% mortgage 200k is plenty. If
| you're trying to buy now on that salary it will be
| challenging.
| tock wrote:
| It's just crab mentality unfortunately.
| wuliwong wrote:
| Hmmm...I actually had to look up what crab mentality
| meant but I don't see how it applies here. From my quick
| lookup, crab mentality apparently refers to a reactionary
| state of mind where a person wants to sabotage another's
| success even when it doesn't directly impact their own.
|
| In the case of everyone making the same salary, you could
| certainly still sabotage someone's success but I don't
| see how having the same salary makes this type of
| mentality _more_ likely than at a company with a more
| typical salary distribution.
|
| My concern with everyone having the same salaries is
| that, potentially, employees have less motivation to
| excel in their individual work and are more likely to do
| the minimum to just stay in good standing with their
| employer and not get fired. Maybe a company can offset
| the lack of direct financial motivation with more of a
| team motivation that the financial success of the company
| as a whole results in financial reward for the individual
| or some other way of recognizing individual success
| inside the company.
|
| I am doubtful this flat salary structure will result in a
| more successful company overall but I do think it's good
| to try new things. And yes this has probably been tried a
| number of times before but maybe not exactly like this.
| Or maybe some other external variables have changed
| w.r.t. other attempts in the past and this time it works.
| The typical salary structures we see in US companies
| today are the result of a large number of trials and
| errors and learning.
| tock wrote:
| Ah I think you misunderstand. I think employees in the
| west being pissed that someone in a third world country
| makes the same salary as them is crab mentality.
|
| I totally understand why companies want to pay less
| though. It's massive cost savings and it makes sense for
| them to hire for less money.
| wild_egg wrote:
| > of course I would want to earn a US tech salary while
| living somewhere in the country side
|
| It's not about what you want, it's about knowing your
| value. If your work is worth a SF salary then that's what
| you should be getting.
|
| Moving from Idaho to SF doesn't magically make you more
| productive. The company knows it's still getting more value
| from you than what you're being paid. They just want to
| keep more of that value for themselves whenever possible.
|
| Have some respect for yourself and know your worth
| carbotaniuman wrote:
| Or, it could be that being San Francisco has
| agglomeration effects (granted most startups/companies
| uterrly fail at this part).
| spacemadness wrote:
| I am so confused by this. I demand more in the Bay as Bay
| area landlords and their Nimby pals are exploitative
| jerks and California taxes and fees add up quick. Why is
| that cleanly separated from the discussion? I see it as
| more "I will demand more here than other places" vs. "I
| know my worth and it's exactly X"
| OkayPhysicist wrote:
| Being in SF makes the market for your labor more
| competitive. If I'm living on a ranch in rural Idaho, I
| would interact with very few people on a given day, and
| most of them would be the same people I interacted with
| yesterday. In SF, I'd be interacting with far more
| people, and far more new people, with a much higher
| probability of those people working in tech, some subset
| of whom will be willing to offer me a job.
| Aurornis wrote:
| > Moving from Idaho to SF doesn't magically make you more
| productive.
|
| If your entire world consists of staying home and
| interacting remotely with a company, then you are
| correct: Location doesn't change anything.
|
| However, moving to a high-energy city with a high density
| of experienced engineers and tech companies can increase
| your rate of learning, career advancement, and experience
| much more rapidly than living in a smaller city. You have
| to actually branch out and interact with local companies
| and people, but it does happen.
|
| But this is all beside the point. Hiring is a labor
| market. Developers who live in SF have more high-paid job
| options to choose from than someone living in Idaho. As a
| result, you need to bid more to get them into your
| company. Hence, the higher salary.
|
| The discussion about cost of living misses this point.
| The real reason developers from places like SF get paid
| more is because if you don't pay them wages that are
| competitive with their local companies, they're just
| going to walk away and take any number of higher paying
| jobs they have access to.
| osti wrote:
| I'm only worth as much as what people are willing to pay
| me. There is no inherent value in whatever I'm doing.
| keepamovin wrote:
| Is that why you went to Berlin? How does Berlin compare to
| SV for cost of living?
| dewey wrote:
| This site is usually helpful:
| https://www.numbeo.com/cost-of-
| living/compare_cities.jsp?cou...
| KomoD wrote:
| No, that should not be trusted, any random person can go
| mess with the numbers.
| dewey wrote:
| Just like Wikipedia and many other information sources.
| Don't trust one source blindly, do your research and
| you'll be fine. It gives a good enough general
| indication. A comparison like this will always depend on
| too many factors to make it applicable to everyone in any
| case.
| keepamovin wrote:
| Interesting, it's almost double in SV the cost of Berlin!
|
| How much salary do you think should allocate to base and
| how much toward any location adjustments, in general?
| dewey wrote:
| I can't answer that, but there's some companies that do
| all that in public:
|
| - https://handbook.gitlab.com/handbook/total-
| rewards/compensat...
|
| - https://www.checklyhq.com/blog/open-sourcing-our-pay-
| calcula...
|
| - https://buffer.com/salary-calculator
| keepamovin wrote:
| Hey, nice resources, man! Thank you. No worries that you
| can't decide on the amounts, it's OK :)
| glitchcrab wrote:
| My employer takes cost of living into account by
| multiplying your salary by the numbeo factor for where
| you live. Base salaries are all calculated for Cologne
| (HO) and then adjusted by the relative cost of living
| factor for your locale. (we're entirely remote)
| a_imho wrote:
| Assuming interchangeable human resources there are
| basically 2 options, either the company is extracting more
| value from everyone than the salaries they pay, in that
| case paying fairly would eat into the company's bottom
| line. Poor CXOs would not turn extra profits by keeping
| less fortunate employees on low salaries but just the
| regular one. Or the highest paid employees are not pulling
| their weight and their salaries are already subsidized by
| the rest of the company, which is also not quite fair.
|
| Nevertheless, I agree everyone looks at this problem from
| their own POV, however it should not be the norm to provide
| equal compensation for equal work.
| keepamovin wrote:
| Can you explain more about your view that it should not
| be the norm to provide equal compensation for equal work?
| a_imho wrote:
| It was a bad edit, it (should not be _controversial_ |
| should be the norm) to expect equal compensation for
| equal work.
|
| In slightly more detail
|
| https://elsajohansson.wordpress.com/2017/09/13/what-does-
| a-w...
|
| https://elsajohansson.wordpress.com/2022/09/16/the-wage-
| gap-...
| keepamovin wrote:
| I understand! That's what I suspected, but wasn't sure. I
| tried understanding it for a minute then thought I'll
| just ask you! :) haha
|
| Also, thank you for the links. I'll read this because I'm
| interested in knowing more about thinking behind this. My
| instinct is in this direction regarding salaries, but I
| want to develop more clarity. :)
| keepamovin wrote:
| If that's your blog, well done! I am totally on board
| with everything you say there. I've been thinking this
| way for years, thanks for articulating some more reasons
| why this is really important! Love your work! :)
| a_imho wrote:
| Thank you for taking a look. Be warned though, I'm
| personally affected and just as biased as everyone else
| on both sides of the issue.
| azinman2 wrote:
| > of course I would want to earn a US tech salary while
| living somewhere in the country side, who wouldn't
|
| I wouldn't. I like cities and center of culture and human
| activity. And the stats generally show cities growing
| globally, so I'm not the only one.
| dewey wrote:
| Replace "somewhere in the country side" with "some other
| place where the cost of living is lower than the highest
| one in the country" if it makes you happier.
| azinman2 wrote:
| The same argument applies. Places with high costs of
| living are high because they're desirable. There are no
| cheap cities I'd want to live in.
| keepamovin wrote:
| Your perspective on egalitarian salaries is intriguing, but
| it's important to remember you can't speak for others
| regarding their intentions or their _whys_ , or what it is
| for them. You can't pretend to define that for someone who
| is not you.
|
| While you argue it's naive and cite different tax systems
| and costs, companies are successfully adopting this
| approach even in high-cost areas like the Valley. It might
| be hard to argue that the world doesn't work like that in
| reality, given that it's already happening.
|
| While it may not be widespread, it doesn't mean it's
| without merit or unrealistic. After all, remember that
| today's 'unrealistic' could be tomorrow's norm! It's
| tempting to think that people in less expensive areas would
| be the main proponents of a uniform salary, but the reality
| is nuanced. Preferences are likely influenced by a variety
| of factors, not just cost of living.
|
| Your later points about the differences between
| compensation at different company stages are well taken,
| however it could be difficult to assert how much this
| dynamic affects preferences given the practice is not
| limited to early stage companies and equity vests often
| fail to yield returns.
|
| In addition, your suggestion that only people in poor
| places want egalitarian salaries, could be seen as
| disrespectful of other people, because it seems to ignores
| the totality of an individual while preferring to try to
| reduce them to simplistic motivations. In that way, it's
| also considered abusive. And can also be seen as
| disrespectful of others experience, and maybe arrogant:
| "Only people in poor areas want such naive, unrealistic
| salaries."
|
| Looking deeper, this aspect of your comment, combined with
| its narrow focus on a single explanation, might be
| interpreted as your attempt to express your personal
| frustrations at your own salary performance, or justify and
| rationalize why you may not be making more. This might
| occur because you may find it easier to view something you
| don't have as unrealistic and naive, rather than the result
| of choices you could change.
|
| In short, while you mention that egalitarian salaries and
| enthusiastic support of them is naive and unrealistic, it
| could be argued that the view espoused in your comment is
| naive and unrealistic because: it lack awareness of complex
| dynamics; ignores the totality of an individual while
| preferring to try to reduce them to simplistic motivations,
| and dismisses real practices as unrealistic, which might
| also be seen as out of touch. Overall, your views
| unfortunately could be interpreted as narrow and an
| expression of personal frustration, instead of a reflection
| of underlying real dynamics.
|
| To conclude, while it's likely there's some truth to the
| correlation you propose, it's also likely true that even if
| some correlation exists, location is not the only factor at
| play. People may have various salary preferences,
| independent of their location, just as the value they
| provide is also independent. Finally, indeed, your view
| could be expressed more respectfully of others.
|
| Anywho, it's understandable you may have that perspective,
| given what your background might be. Yet it's always good
| to remember that you can adapt your view over time, can
| grow and can include more data to expand that awareness of
| reality which you value! :)
| caymanjim wrote:
| Except if everyone gets paid the same, you're not being
| rewarded for the value you provide. I don't think salary
| should be tied to location, but it should be tied to
| experience, ability, and effort.
| keepamovin wrote:
| You raise an important point. There's certainly multiple
| ways to look at it.
|
| The egalitarian way where a business can divide the share
| of revenue that is allocated for salary equally, no matter
| the role. This makes sense from a philosophy of everybody
| being a team and contributing equally to the results of the
| company. It can foster an esprit de corps and and a sense
| of fairness. On the negative side it could encourage
| companies to have more burdensome measures of fairness and
| contribution, and lead to resentment towards colleagues who
| don't pull their weight.
|
| Then there's the other method, where a value-based salary
| is allocated to each employee taking into account their
| experience, ability and effort. Crucially, however, this
| salary is not adjusted for location. That's the case to
| which I was speaking, specifically, even tho the type used
| by 0xide is clearly the egalitarian one.
| svnt wrote:
| Presumably their equity is not evenly split.
| dgb23 wrote:
| > I don't think salary should be tied to location, but it
| should be tied to experience, ability, and effort.
|
| That's the labor theory of value (see: Smith, Marx), which
| in theory sounds meritocratic but it can't really be
| measured or assessed.
|
| In reality compensation either becomes a function of power,
| social currency and negotiation skills, which is the
| general norm in professions, or you have an
| institutionalized, perhaps even democratic process to
| determine salaries. Both of these variants generate
| overhead and are only approximations to what anyone would
| see as fair.
|
| The variant here where everyone gets the same, generous
| piece of a pie seems refreshingly simple and honest. I
| would also assume that it attracts the right kind of
| people, who are intrinsically motivated (at least after the
| threshold of a very high level of comfort is reached.)
| slibhb wrote:
| Saying that people should be paid according to
| experience, ability, and effort is absolutely not the
| labor theory of value.
|
| The idea that contribution "can't really be measured" is
| a cop-out. Contribution can't be measured perfectly but
| it can be estimated with some accuracy by people who are
| involved in day-to-day work. "Some accuracy" is really
| all that's required: as long as contribution is
| correlated with compensation _to some extent_ , you have
| a functioning meritocracy.
|
| > The variant here where everyone gets the same, generous
| piece of a pie seems refreshingly simple and honest.
|
| I bet it works great if you have a small team, are
| extremely picky about hiring, and quickly fire bad hires.
| Otherwise it will be awful.
| PaulDavisThe1st wrote:
| > Contribution can't be measured perfectly but it can be
| estimated with some accuracy by people who are involved
| in day-to-day work
|
| This is handwaving in the extreme. Anyone involved in
| software development knows that a single line of code can
| be critical to success or failure, as can a blob of 100k
| LOC, so product-quantity metrics are of almost no use.
| The "estimate" you're talking about generally comes down
| to general "feelings" about who works hard, which have
| repeatedly been shown to be poor metrics for actual
| contributions.
| slibhb wrote:
| Life is full of handwaving. In almost any workplace, it's
| very simple to know who's doing the work (and it's
| usually a shockingly small number of people). It's the
| idea that we can reduce this to a mathematical formula
| (the opposite of handwaving) that's odd.
|
| > The "estimate" you're talking about generally comes
| down to general "feelings" about who works hard, which
| have repeatedly been shown to be poor metrics for actual
| contributions.
|
| How has this been "shown"? Anyway, you're begging the
| question that there's some way to determine "actual
| contributions" that we can compare to "feelings".
|
| If you actually work with a group of people on a daily
| basis and can't rank order them in terms of usefulness, I
| find that astonishing. And remember, rankings don't have
| to be perfect, they just have to more accurate that
| random.
| PaulDavisThe1st wrote:
| > And remember, rankings don't have to be perfect, they
| just have to more accurate that random.
|
| No, they have to better than both random and "everyone
| is, on average, and over an extended period of time
| contributing roughly the same". That's quite a challenge.
|
| Do you go to customers and ask them which features
| provide the most value to them, and then follow the code
| back to the people who implemented them? Do you go to the
| customers who paid the most, and repeat the question to
| them only?
|
| We're not talking about some award-prize ceremony speech
| in which we acknowledge that Dmitri and Aneka led the
| work to get version 8.0 which has been a huge success.
| We're talking about actual salaries, which are presumably
| linked in some way to actual sales, and I'm insisting
| that connecting individual _developer_ efforts to the
| sales numbers is extraordinarily hard. "More" and "less"
| are not enough to come up with actual numbers.
| slibhb wrote:
| > No, they have to better than both random and "everyone
| is, on average, and over an extended period of time
| contributing roughly the same". That's quite a challenge.
|
| This is something virtually all functional companies do
| when they decide raises. The fact that they don't do it
| perfectly isn't a huge problem, they just need to be more
| right than wrong.
|
| > Do you go to customers and ask them which features
| provide the most value to them, and then follow the code
| back to the people who implemented them? Do you go to the
| customers who paid the most, and repeat the question to
| them only?
|
| Does productivity equal sales? I don't think so. If
| someone does a good job implementing a feature that
| doesn't drive sales, that should count toward their
| productivity. Equally, imagine a task that could be
| assigned to anyone that drive sales: it doesn't make
| sense to reward the person who happened to be assigned
| this task when anyone could have done it.
|
| You're demanding way too much here because you're
| unwilling to get "handwavy" and instead want some
| rigorous way to quantify productivity. Instead, embrace
| subjectivity! Imagine you're in charge and ask yourself
| questions like:
|
| 1. If I need to organize a meeting to address some
| problem that needs to be solved as soon as possible, who
| would I invite to the meeting?
|
| 2. If someone tells me he plans to quit, how much would I
| be willing to offer to convince him to stay?
|
| 3. If someone quits, how hard are they to replace? In
| terms of hiring a replacement and/or transferring their
| responsbilities to someone else.
|
| I suppose there are some workplaces where it's genuinely
| hard to rank people. But my sense is that they're rare,
| small, and careful about hiring. Everywhere I've worked,
| this is not the case and I'm fairly sure this is the
| norm.
| PaulDavisThe1st wrote:
| The 2nd and 3rd questions you pose are all about sales.
|
| If your company brings in $10M a year, and somebody plans
| to quit (or has quit), how much will your sales drop
| (immediately, and over a period of time). That's the
| answer to how much you can afford to offer them to stay,
| and that's how much you should offer their replacement.
| Suppose that says drop by $1M and you can be satisfied
| that the drop is 100% a consequence of the departed (or
| soon to depart) developer - that's how much value they
| bring to the company, and by the logic of capitalism
| (which I don't play by, btw), you should pay them some
| amount less than that.
|
| The problem is: you can't determine the value before they
| quit, and you can't be sure that their replacement will
| provide that value after they are hired.
|
| Look, I understand that in an organization of any size,
| there are likely to be slackers that feel like a
| deadweight, and others who feel like the contribute far
| more than the average.
|
| The question is: does tying salary to this perception
| actually bring the benefits you think it does (which are
| intimately connected to the notion of incentive) ?
| There's some good evidence that for developers and other
| "head-based" employees, it does not, and that flat pay
| scales create an environment in which you get different
| kinds of benefits.
|
| I've worked exclusively in a distributed FLOSS project
| for the last quarter century, so in many respects, I'm
| not well positioned to talk about what happens inside
| traditional corporations.
| dgb23 wrote:
| The problem is that in the end, power, negotiation and
| social currency often dominate over merit (such as
| effort, experience etc.) when it comes to compensation.
|
| Even if you actually do measure and agree on metrics,
| then the measurement can easily become the goal for those
| who are not intrinsically motivated. Work ethic can't be
| taught by dangling carrots in front of people, because
| acquiring the carrot becomes the goal instead of moving
| the cart. This can be detrimental in a highly
| collaborative workplace.
|
| Having a flat, generous salary might solve this problem,
| because you filter out carrot hunters and get cart
| movers.
|
| > I bet it works great if you have a small team, are
| extremely picky about hiring, and quickly fire bad hires.
| Otherwise it will be awful.
|
| Finding the right people to work with is difficult
| regardless. The same worker can be miserable in one place
| and flourish in another.
| iamawacko wrote:
| That is not the labor theory of value. The whole point of
| LTV, at least in Marxian economics, is that workers can't
| be payed according to their socially necessary labor,
| because a surplus labor is extracted.
|
| Besides, LTV as a theory is meant to be a description of
| the world as Smith, Marx, etc. see it, not a prescription
| for how things should be done.
| dgb23 wrote:
| You're right that it is descriptive and not normative.
|
| But the underlying belief of paying someone according to
| their effort, is very much based on the same premise.
|
| What I'm saying is that nobody is _really_ paid according
| to their effort, experience etc. because those things
| cannot be reasonably measured.
|
| The typical process of determining compensation is based
| on negotiation and power. In some places the process is
| more democratized and rules based. Both of these are only
| to some degree related to actual effort, experience and
| so on. This discrepancy becomes larger the more people
| are involved as well.
| closeparen wrote:
| Location independence is not the interesting part of Oxide's
| compensation strategy. It's that everyone makes exactly the
| same. There are no negotiations, no levels, no promotions,
| and consequently no promo packets or promo projects. This is
| extremely enticing to FAANG types who are tired of a certain
| kind of bullshit, at the cost of a certain level of ambition.
| cdchn wrote:
| Selecting for lack of ambition may have some negative
| consequences on your businesses' ambition.
| closeparen wrote:
| But the problems of engineers optimizing architectures
| and project plans for their career trajectory over
| business need are also well-known.
| systemvoltage wrote:
| Yeah this I've seen everywhere, but more related to the
| company size. In a 50 employee company, it's hard to pull
| that off. As organizations get larger, you get this as
| inevitable part of human nature. If the salary table is
| flat, then highly ambitious people will ask for more
| skin-in-the-game.
|
| The skin-in-the-game bit can be promotions, equity, stock
| compensation, etc.
| sunshowers wrote:
| As a FAANG to Oxide refugee, I think what we're doing is
| orders of magnitude more ambitious, especially considering
| our small size.
| systemvoltage wrote:
| I wonder if they offer equity. Hierarchies always form,
| they tried this with Gore Associates in Arizona (totally
| flat company structure) and has major problems.
| steveklabnik wrote:
| We are compensated with equity, yes.
| Aurornis wrote:
| Do all employees get the same equity?
|
| I've worked at two companies where everyone got the same
| base salary, but the variation came in equity.
| steveklabnik wrote:
| I don't know to be honest, but I assume that the equity
| is variable.
| zensavona wrote:
| One of the many reasons I like working at Crazyegg.
| aabhay wrote:
| Works until you have to hire a designer
| nfriedly wrote:
| Why would that be a problem? Surely there are some designers
| willing to work for $200k.
| aabhay wrote:
| But very few founders willing to pay a designer that much.
| dcre wrote:
| We have two!
| paryhin wrote:
| We've got designers, me included, and a few others on the
| team who aren't engineers. I also hail from what people like
| to call a "third-world country". So far it's working great,
| and we're hiring more folks from all over the world, not just
| the USA.
| apexalpha wrote:
| Damn that's insane. As someone from Europe these salaries are
| just extreme.
|
| If ya'll are looking for a remote security engineer from Europe
| hit me up. :)
|
| For those numbers I'll walk the servers into the customers
| myself.
| DonnyV wrote:
| Salary is that high in the US because we have no social net
| or price caps here. Your on your own for everything.
| Healthcare, retirement, overpriced homes, out of control
| rent, etc
| throwaway1777 wrote:
| Also because you can be fired at any moment.
| bornfreddy wrote:
| You can be fired in EU too, it just costs a few months'
| salary. And you can't quit without notice too so it goes
| both ways.
| vasco wrote:
| They say they offer healthcare too, not sure how it works
| for their employees outside the US:
| https://oxide.computer/blog/benefits-as-a-reflection-of-
| valu...
|
| With this in mind, it's just a good salary. Until they have
| competition squeezing them on margins it looks like an OK
| approach. At some point their board most likely won't agree
| with paying some people below market rates for important
| roles and other more fungible roles paying 2-3x market
| rates but while they can keep doing it, it's great free
| marketing.
| Yajirobe wrote:
| Overpriced homes, out of control rent, retirement are
| extremely problematic in a lot of European countries. And
| in the US, healthcare is usually covered by the company
| (talking about big tech corps).
|
| So the extremely high salary is still a net positive
| compared to EU
| cedws wrote:
| >overpriced homes, out of control rent
|
| Plenty of that in London unfortunately and our salaries are
| half.
|
| >Healthcare
|
| And not so much of that... despite paying tax for it
| anyway.
| troupe wrote:
| Wouldn't an employee making that $201k salary in another
| country be responsible for paying all the taxes to support
| the socialized healthcare, etc? In other words the take
| home pay may be significantly less.
| robocat wrote:
| Here's the calculations for New Zealand in USD:
|
| Income tax: 34% total (top marginal rate of 39%).
|
| Leaving $133k.
|
| Other taxes (GST~VAT 15%, city rates, high petrol excise,
| etcetera) will easily take another $10k. Interest on your
| home is not deductible (NZ has very few deductables).
| People earning six figures will often pay for private
| health insurance and medical fees on top of the
| socialised healthcare - maybe another few thousand.
| rwiggum wrote:
| retirement is covered by social security healthcare by
| insurance (offered by the state if you can't afford it
| yourself)
|
| homes, etc is true though and we just need to build more
| inventory IMO. but there are tons of areas with affordable
| homes, they just aren't near the big cities like NYC or SF
| or LA
| DonnyV wrote:
| You can not live on Social Security, no way. If you don't
| have a job your not getting healthcare and even if your
| job provides healthcare. Its too expensive to actually
| use. Also you need to be living below poverty wages
| before a state will give you healthcare.
| cryptonector wrote:
| That's probably [EDIT: decidedly] on the low side for the bay
| area for rock star devs, and Oxide has lots of rock star
| devs. I haven't looked but I assume they pay bonuses,
| probably differential bonuses.
| steveklabnik wrote:
| We are not paid bonuses. We do receive stock.
|
| I took a base-pay cut to work at Oxide. Zero regrets.
| pighive wrote:
| Hi Steve, bit of a tangent, I stumbled upon your blog
| from intro section and went down the rabbit hole. Wanted
| to ask if you have any recommendations on documentaries.
| TIA!
| steveklabnik wrote:
| I am unsure why you are curious about documentary
| recommendations from me, haha, but here's one I enjoyed
| quite a bit recently:
| https://www.youtube.com/watch?v=BKP1I7IocYU
| sunshowers wrote:
| Like Steve, I also took a pay cut (from around 400-600k a
| year TC at a FAANG) to work at Oxide. I'm very passionate
| about what we do and was attracted to the values-oriented
| culture.
| weystrom wrote:
| Forget the numbers, just knowing that CEO is not fucking me
| over to build a second vacation home is Aspen would be enough
| for me.
| outside1234 wrote:
| Well, they didn't say the stock was the same for every
| employee...
| keepamovin wrote:
| While that may be true, it's also unfortunately true that
| stock in startups has a high probability of ending up
| worthless. It's important to compare real money now, not
| just theoretical money in future.
|
| I doubt my Aspen vacation home developer will accept
| payment in startup stock.
| loeg wrote:
| Well, it's an immediate stumbling block to hiring anyone
| currently making more than that, such as senior engineers in
| the US.
| YorickPeterse wrote:
| That is only a problem if you insist on hiring people from
| such expensive areas. Given Oxide hires remotely, I doubt
| this is an actual problem for them.
| mtlynch wrote:
| According to their LinkedIn[0], 28 of their 52 listed
| employees live in SF Bay Area, so their salaries wouldn't
| be that high for that area.
|
| [0] https://www.linkedin.com/company/oxidecomputer/people/
| paxys wrote:
| No company wants to hire in expensive areas, but that's
| just where engineering talent is concentrated.
| YorickPeterse wrote:
| Ah yes, because good engineers can only be found in San
| Francisco and similarly expensive areas.
| macintux wrote:
| Hence the word "concentrated". Your snark seems
| misdirected.
|
| I live in flyover country, and there are no shortage of
| talented people here, but I wouldn't dispute that the
| per-capita software developer talent in SF is going to be
| higher.
| clpmsf wrote:
| I think it's mostly societal perception. For whatever
| reasons, a lot of people seem to think of professionals
| in SF and NYC as smarter or more capable than
| professionals from Boise or Salt Lake City. Just like a
| lot of people assume those who got into Harvard or
| Stanford as 18yr olds are smarter or more capable than
| those with degrees from the large public universities.
| jatins wrote:
| That's just base salary. For Staff+ engineers making 600-700k
| at BigTech, it is probably still a block.
|
| But for senior engineers at these companies, 200k is pretty
| competitive with the base salaries offered at those places
| loeg wrote:
| I don't understand why you would compare total compensation
| at Oxide with base salary at other employers? Total
| compensation is the relevant figure across the board, and
| senior (as in, experienced, not some specific levels.fyi
| tier) engineers often make significantly more than
| $201,000.
| jatins wrote:
| > That's just base salary
|
| I meant 200k is just base salary at Oxide. It does not
| include equity
| loeg wrote:
| Oxide equity, like for all private companies, isn't worth
| anything until and if they go public; it has no cash
| value. For total compensation purposes, it's $0.
| jatins wrote:
| > Oxide equity, like for all private companies, isn't
| worth anything until and if they go public
|
| Open AI just had a tender offer for employees this month.
| It's not public. Clearly the equity is worth something
| even though the company is not public.
|
| Some companies facilitate secondary transactions for
| their employees.
|
| There are ways many employee friendly companies provide
| early liquidity without IPO.
| nosequel wrote:
| user: sunshowers has been replying in this thread and
| mentioned going from FAANG-> Oxide and taking what seemed
| like about a 60% paycut.
|
| Yes it is somewhat of a stumbling block, but I'm betting they
| are getting extremely high quality employees who are doing it
| for the passion and not just for the money.
| hnthrowaway0315 wrote:
| I can work for half of that in Canada. But I don't have half of
| the skills :(
| rglover wrote:
| Brilliant way to keep your team focused. No drama or in-
| fighting about who's getting more or less.
| jpdb wrote:
| I really wish oxide had a Homelab/consumer centric offering!
|
| Spec wise, some low power systems like an Intel NUC, LattePanda
| Sigma, or Zimaboard. You could fit 3/4 of them in a single 1u
| with a shared power supply. They could even offer a full 1u with
| desktop grade chips on the same sleds.
|
| I have thought about building one myself, but it's a large
| investment of time that I can't seem to find lately.
| EvanAnderson wrote:
| Even just a medium business offering would be great. I'd love
| to not have to use Dell or HP gear-- anything to get away from
| the cobbled-together stack of legacy IBM PC compatibility and
| third-party ODM/OEM stuff glue-and-taped together by the
| vendor.
| siscia wrote:
| I am missing how AWS/GCP/Azure does not solve this for you.
|
| Price point?
| EvanAnderson wrote:
| On prem. Reliable and inexpensive network connectivity they
| has any resemblance to a 10G LAN doesn't exist where I am.
|
| I work with some businesses who need very, very reliable,
| high-bandwidth, and low latency connectivity to their data.
| The amortized cost of on-prem beats the cost of any off-
| prem offering as soon as the cost of the necessary
| connectivity is factoted-in.
| MichaelZuo wrote:
| Isn't that exactly what this Oxide rack is for?
|
| Your not going to find any serious hardware product with
| reliability guarantees, in writing, for much less than
| half a million anyways.
| EvanAnderson wrote:
| I'm talking shops who spend $200-$500K on servers and
| storage, not north of $1M (which is where this Oxide gear
| lives). Something like a 1/4 scale Oxide rack, perhaps.
| bobthecowboy wrote:
| I work at SoftIron, another startup in this space. Our
| HyperCloud product might be interesting for you. I'm not
| in sales, so I can't comment on the prices, but I'd guess
| we're much more competitive since you don't actually need
| to buy an entire rack of our gear at a time.
|
| That said, where this product-space gets tough is
| actually scaling it down. It's pretty challenging to
| create something that is remotely stable/functional in a
| homelab (space/power/money) budget. Three servers and a
| switch would probably be the bare minimum. We (and I'm
| sure Oxide :) scale up like a dream.
| grappler wrote:
| This all has me wondering, if I just want to play with
| stuff in this space as an individual homelabber who earns
| a tech salary and wants a nicely designed rack-mounted
| alternative to a mess of unorganized NUCs and cables and
| whatnot, what are my best options?
| bobthecowboy wrote:
| If you're willing to spend money on rack-mounted gear you
| definitely have options, and what you get sort of depends
| on what you're interested in playing with.
|
| A lot of homelabbers (and even some small businesses) go
| for Proxmox as a virtualization distribution. I don't use
| it myself, but IIUC it's effectively a Debian distro
| packaged to run KVM/LXC, with support for things like
| ZFS, Ceph, etc. It has some form of HA, an API used by
| standard open source devops tools, handles live
| migration, etc.
|
| So buy some used rack-servers on Ebay (or new, if you're
| ballin'). A lot of businesses sell their old stuff, so
| you can pick up a generation or two out of date for a
| good price. If you want to do fancy stuff like K8s, Ceph,
| etc you'll probably want at least three nodes, ideally
| more, and a bunch of disks in them. Networking gear is a
| sort of pick your poison thing. A lot of people love
| Ubiquiti gear; a lot of people hate it. TP-Link is
| another that's good and budget friendly. StarTech sells
| smallish racks (including on Amazon), if you want to
| start there.
|
| It won't look exactly like SoftIron's HyperCloud or
| Oxide's Cloud Computer, but you can certainly get pretty
| sophisticated.
|
| Not sure if this answers your question, but other great
| spaces to explore are the 2.5 Admins and Self-Hosted
| podcasts.
| grappler wrote:
| I'm really thinking mostly about the hardware part here,
| and maybe just enough layers of the stack to feel like an
| integrated hardware setup. Let the nerds play with
| whatever software they want above that.
|
| To go ahead and dream a bit:
|
| I'd hope for an online configurator like the one
| SoftIron's HyperCloud has [1] but instead of "talk to a
| sales rep", show a price for what you just configured,
| like you're configuring a macbook.
|
| Relatedly, there should be a standard rack form factor in
| the size category of NUCs and Mac Minis, rather than
| having to go all the way to the 19 inch monster racks
| that medium to large businesses use. If it were nailed
| down to the point of being able to blind mate (just
| learned that term from Oxide's article here!) gear into
| it, that would be kind of perfect. 1.
| https://softiron.com/hypercloud/configure/
| electroly wrote:
| AWS Outposts is the solution. I like Oxide but people
| seem to be blind to the actual competition when they
| focus on Dell as the competitor. AWS has been shipping
| Outposts racks for years. All prices are public on their
| website and you can order it today. Nearly every
| configuration is sub-$500k. Fully managed and AWS
| supports the entire stack; no buck-passing among vendors,
| same as Oxide.
| linklocaladdr wrote:
| I'm not sure where his customers are, but Outpost
| up/downlinks are supposed to to be at least 1gbit, and
| they don't behave well in situations where the latency to
| the paired region is high. EBS lazy loading blocks is
| great in region but awful when your ping is 300ms.
| electroly wrote:
| You can use EBS Local Snapshots, can't you? It stores in
| your Outpost S3 storage.
| zeckalpha wrote:
| This sounds like Synology to me.
| FridgeSeal wrote:
| Same here!
|
| I've not personally used it, but their stack of software is
| open source, and according to some commenters in the thread,
| super high quality.
| ren_engineer wrote:
| I'd imagine they'll get to that eventually, these types of
| companies generally start at enterprise level because that's
| the most profitable and requires closing smaller numbers of
| deals. Once the product is proven and their support
| infrastructure is in place they can go for other market
| segments to try and maximize revenue
| xur17 wrote:
| It's not just about maximizing revenue, it's also about
| getting it into developer hands early (homelabs, side
| projects, college students, etc) so they can become familiar
| with it, and become an advocate for it within their company.
| Cloudflare is a good example of this.
| campbel wrote:
| I also like Tailscale as an example of this. Really great
| platform for devs to jump on for home labs and build
| experience.
| e12e wrote:
| It would be great if Oxide had something like Canonical's
| "Orange Box"/cloud-in-a-box for homelabs, evaluation, training
| (in the management bits) - and hobby work loads!
|
| https://canonical.com/blog/jumpstart-training-with-the-orang...
|
| https://arstechnica.com/information-technology/2014/06/hands...
| FromOmelas wrote:
| seconded. it would provide an on-ramp to get familiar with the
| software without forking over 500k
| newsclues wrote:
| Not 1U but perhaps a box design that isn't noisy like a pizza
| box server.
|
| Don't know if oxide would want or be able to compete in the low
| cost market but a bigger a more expensive desktop/workstation
| as a mini homelab cloud could be a great option to get people
| trained on the oxide platform.
| sgloutnikov wrote:
| Unfortunately they are not planning home lab things anytime
| soon, per a recent podcast episode [0].
|
| If you want to play around with their Hubris OS: "You wanna buy
| an STM32H753 eval board. You can download Hubris, and then
| you've got - you've got an Oxide computer. You have it for 20
| bucks."
|
| [0] https://changelog.com/friends/8
| cprecioso wrote:
| Maybe somebody can help me find the $20 one, but the one I
| can see is $460
|
| https://estore.st.com/en/stm32h753i-eval2-cpn.html
| mkeeter wrote:
| Look for the "Nucleo" boards rather than full-fledged eval
| kits, e.g. https://www.st.com/en/evaluation-
| tools/nucleo-h743zi.html is $26.
| taink wrote:
| I don't know if this is a reputable source in any way but a
| quick search shows this:
| https://www2.mouser.com/c/?processor%20series=STM32H753
| robertlagrant wrote:
| I'm a bit confused about them calling it a cloud computer. The
| primary benefit of cloud is you spend opex instead of capex.
| Isn't this moving back to capex again?
| hotpotamus wrote:
| If they lease it to companies, would it be considered opex?
| robertlagrant wrote:
| It would be interesting to buy a load of them and set up as a
| cloud vendor :)
| subarctic wrote:
| Ya, in the past the messaging was more like "get the ergonomics
| of the cloud with the economics of owning your own hardware", I
| think it's confusing to just call it a "cloud computer"
| robertlagrant wrote:
| Yes - that makes loads more sense to me. It's cool that you
| can spin stuff up and it's all integrated with the hardware -
| having seen fairly large VMWare installations and all the
| faff it takes to make them work, this seems really useful. I
| just don't think it's the same as the cloud, and the article
| seems to discount the main reason for the cloud: opex,
| letting you get started (and stopped) quickly and cheaply.
| throw0101a wrote:
| Private cloud is a thing too: lots of folks run VMware,
| Hyper-V, or OpenStack in-house.
| steve1977 wrote:
| And it's as much of a misnomer there.
| throw0101a wrote:
| > _And it 's as much of a misnomer there._
|
| Dude, why are you even in this thread? Trying to be a
| Debbie Downer is just going to ruin the rest of the day.
|
| * https://xkcd.com/386/
|
| Trying to get the last word in on the Internet is a quick
| and easy way to have a bad time. Let it go and move onto
| something else.
| wmf wrote:
| This has definitely been true, but what if Oxide is the
| first actual private cloud?
| jbiggley wrote:
| That was the line we were sold by cloud vendors. There is a
| reason that significant portions of both native cloud and
| migrated workloads are shifting back to self-hosted; the cost
| savings never materialized.
|
| My google-fu failed to find any articles for or against my
| statement that weren't paid advertising or lightweight tech
| summaries. StackOverflow will have to do.
| https://stackoverflow.blog/2023/02/20/are-companies-shifting...
| robertlagrant wrote:
| Of course - every pay as you go service that goes above a
| certain amount of utilization will be better replaced with up
| front investment. But that doesn't mean that the opex model
| is bad in general; you've just only picked cases where it's
| bad.
| wpm wrote:
| So, what you're saying is, that whether opex or capex
| models are better or not...depends? That there is no silver
| bullet, golden gun, one-size-fits-all platitude anyone can
| post in an HN comment?
|
| If the opex model doesn't work for an organization, that
| they perhaps just went above that utilization threshold
| where it doesn't make sense anymore, and would be a
| potential 0xide customer?
| solatic wrote:
| There's two independent issues: capex vs. opex, and
| flexibility.
|
| When opex is a primary benefit of the cloud, that's
| specifically for start-ups and other businesses that have
| little working capital. The _actual_ primary benefit of the
| cloud for _most_ cloud customers is flexibility - prior to AWS
| adoption, getting a server provisioned could be a multi-week if
| not multi-month affair negotiating between devs or operators
| and on-prem infrastructure workers to get the servers through
| capacity planning and provisioning. AWS made provisioning so
| simple, that you could start to set up auto-scaling, because
| you had extraordinarily high confidence that the capacity would
| be immediately available when your autoscaler tried to scale
| up.
|
| But opex is _not_ a benefit of the cloud for heavy /established
| businesses. Cloud opex is a financial expenditure every month
| that you can't get rid of and counts directly against your
| profits. Indeed, the desire for capex in the cloud is so high
| that businesses routinely purchase Reserved Instances and other
| forms of committed usage, which allows accountants to treat the
| cost of the RI as capex and then discount the expenditure
| through depreciation (to zero, since there will be nothing to
| sell when the RI expires) over the lifetime of the RI. It is
| normal and frequent for businesses to make capital expenditures
| to reduce their operational expenditures over time, thus
| increasing their monthly/quarterly profits.
|
| Oxide's unique value proposition is to give customers,
| particularly those with high monthly cloud bills that they have
| difficulty reducing, the operational flexibility of cloud
| computing with the profit-improving benefits of capital
| expenditures.
| quickthrower2 wrote:
| The way you put it, this capex thing though is sounding like
| just sacrificing cashflow for some accounting sophistry.
|
| Surely the main benefit of reserved instances is lower TCO
| and if you can show you can afford AWS for 3 years a bank
| would surely loan you the money to pay upfront if you can
| save say 50% it is simply cheaper even with interest.
| solatic wrote:
| Again this goes back to flexibility. RIs necessarily take
| away from your flexibility. AWS and others try to grant you
| the flexibility anyway, by allowing you to shift RI credits
| between physical instances, but lower TCO is definitely not
| guaranteed. If you buy an RI for a server type you're not
| actually using, you're spending money on servers that's
| getting wasted compared to not actually buying the RI.
| e12e wrote:
| Private Cloud allows infrastructure team to run big server
| parks efficiently, while product teams can "buy" resources
| easily. It's essentially why Amazon made aws, why Google made
| Borg.
|
| For a regional hospital for example, there might be a desire
| for in-house network and hardware - but perhaps the system for
| digital patient journals run on Kubernetes and managed
| databases.
| axelthegerman wrote:
| Sure but some businesses care about how much they spend and
| then about capex vs opex.
|
| I'd rather have $1 in capex than $10 in opex
| fuzzylightbulb wrote:
| Really depends on cash flow.
| tibbydudeza wrote:
| Reminds of what they worked on before at Sun ZFS storage
| appliance.
|
| Can it run Linux or Windows Data Center as ex-Sun folk they seem
| to have loved the last open source release before Oracle but now
| is still community developed variant of SunOS ???.
| rhinoceraptor wrote:
| The hypervisor OS is based on Illumos, which was forked from
| OpenSolaris, and it uses Bhyve from FreeBSD for virtualization.
|
| I would imagine the system architecture is different enough
| that running Linux as the base OS would take some work, for
| example it doesn't have an AMIBIOS.
| rwmj wrote:
| Haven't blade computers been doing this for a while?
| EvanAnderson wrote:
| Sort of. The big deal with Oxide is that all the legacy
| compatibility with the IBM PC platform is gone, and the whole
| stack, top-to-bottom, is built by then (including firmware).
|
| It's not like commodity x86 gear with black-box (often buggy)
| firmware and layer-upon-layer of hacks and compatibility
| kludges to present the hardware interface of a late model IBM
| PC AT.
| steve1977 wrote:
| What makes you think so? According to their website, the
| compute bit is based on AMD EPYC 7713P CPUs.
|
| I can buy these at my local electroncis retailer. So pretty
| much commodity x86.
| nickik wrote:
| The CPU is commodity, nothing else is. Costume Mainboard
| and firmware without BIOS and their own BMCish thing and
| their own Root of Trust. Same for their router. Standard
| chip, everything else is costume.
| steve1977 wrote:
| So similar to a IBM mainframe then?
| nickik wrote:
| Similar in some ways different in others. But in terms of
| not being a PC architecture. Yes it is. But in many other
| ways its not at all like a Mainframe.
| steve1977 wrote:
| I'm not sure if "not PC architecture" is really an
| advantage for many customers.
| repelsteeltje wrote:
| It's similar to hyperscale infrastructure -- it doesn't
| matter as long as it looks like a PC architecture from
| the OS running inside a VM. The layers and layers of
| legacy abstraction firmware, BMC, drivers BIOS,
| hypervisors you get with a typical on premise
| Dell/HP/SuperMicro/... server motherboard are responsible
| for a cold start lasting 20 minutes, random failures,
| weird packet loss, SMART telemetry malfunctions, etc.
|
| This is the type of "PC architecture" cruft many
| customers have been yearning to ditch for years.
| steve1977 wrote:
| I'm not in the bare metal/data center business anymore at
| the moment, but I was for more or less the last 25 years.
| I never had such issues. Maybe I was just lucky?
| repelsteeltje wrote:
| Maybe you were. :-) And maybe this is not for you or me
| (I haven't contacted their sales, yet) it's not for
| everyone.
|
| Personally, I have always been annoyed that the BIOS is
| clunky and every change requires a reboot, taking several
| minutes. As computers got faster over the years, this has
| gotten worse, not better. At the core of cloud economics
| is elasticity: don't pay for a service that you don't
| use. Wouldn't it be great to power down an idle server,
| knowing that it can be switched on _seconds_ before you
| actually need it?
| steve1977 wrote:
| > Wouldn't it be great to power down an idle server,
| knowing that it can be switched on seconds before you
| actually need it?
|
| considering you would still need to boot the VMs then,
| once the Oxide system is up, I'm not sure if this is such
| a big win.
|
| And at a certain scale you'd probably have something like
| multiple systems and VMware vMotion or alternatives
| anyway. So if the ESXi host (for example) takes a while
| to boot, I wouldn't care too much.
|
| And, economics of elasticity - you'd still have to buy
| the Oxide server, even if it's idle.
| repelsteeltje wrote:
| > considering you would still need to boot the VMs then,
| once the Oxide system is up, I'm not sure if this is such
| a big win.
|
| To be honoust, I'm using containers most of the time
| these days but even the full blown windows VMs I'm
| orchestrating boot in less than 20s, assuming the
| hypervisor is operational. I think that's about on par
| with public cloud, no?
|
| > [...] vMotion [...] ESXi.
|
| Is VMware still a thing? Started with virsh, kvm/qemu a
| decade ago and never looked back.
|
| > And, economics of elasticity - you'd still have to buy
| the Oxide server, even if it's idle.
|
| That's a big part of the equation indeed. This is where
| hyperscalers have an advantage that Oxide at some point
| in the future might enjoy as well. Interesting to see how
| much of that they will be willing to share with their
| customers...
| steve1977 wrote:
| Re VMware, it's certainly still a thing in enterprise
| environments. Can kvm do things like live migration in
| the meantime? For me it's the other way round, haven't
| looked into that for a while ;)
|
| How do you mean Oxide might have that advantage as well
| in the future? As I understand, you have to buy hardware
| from them?
| repelsteeltje wrote:
| Ah yes live migration, off course. We design "ephemeral"
| applications that scale horizontally and use load
| balancer to migrate. With 99% of traffic serviced from
| CDN cache updates and migrations have a very different
| set of challenges.
|
| As to your question, I meant to say that as volumes and
| scales economies increase they can source materials far
| cheaper than regular shops. Possibly similar to AWS, gcs,
| Azure, akamai etc. It would be nice if they were able and
| willing to translate some of those scales economies into
| prices commensurate with comparable public cloud
| instances.
| brucepink wrote:
| If they care about security, it certainly is.
|
| If you want more insight into all of the things that
| normally run on "PC architecture" - the 2.5 other
| kernels/operating systems running underneath the one you
| think you're running -
| https://www.youtube.com/watch?v=mUTx61t443A
| wpm wrote:
| Someone should tell the investors!
|
| I imagine they are aware that this isn't a solution for
| many customers. A John Deere tractor makes a poor
| minivan. This isn't for you. That's fine. It's not for me
| either. That's ok. I don't need to poo-poo their efforts
| and sit and moan about how it's not for me.
| dist1ll wrote:
| The difference is that when you're buying a x86 system, the
| entire CPU bringup (incl. AMEGAS/openSIL on AMD) runs
| proprietary and poorly documented firmware. You're entirely
| at the mercy of the vendor.
|
| Oxide has put immense effort into writing open-source
| platform initialization code, and built their own open-
| source BMC/RoT solution.
| steve1977 wrote:
| So effectively I'm at the mercy of Oxide, at least as
| long as their system does not become some kind of
| standard.
|
| Not in theory maybe, but in practice. Because as a
| customer, I would probably also need to put in immense
| effort to understand and maintain that software myself.
| EvanAnderson wrote:
| Their firmware is open source. You can pay whoever you
| want to maintain it. You can't do that with Dell, HP,
| Supermicro, and the unknowable rabbit hole of ODMs and
| sub-suppliers and contractors who actually make the
| hardware and firmware for these companies.
|
| Until you've dealt with a malfunctioning Dell or HP
| server and have to live with being told "we don't know
| why it acts that way, we'll try to get the ODM to repro"
| I don't think you can appreciate how cool Oxide's
| offering seems.
| steve1977 wrote:
| If I have server under maintenance with Dell or HP, they
| would replace the server or component for me in such a
| case.
|
| Which would probably be a lot faster than trying to find
| someone who could maintain some non-standard firmware (as
| good as it might be).
|
| Even if I had to replace the server on my own cost it
| would probably still be cheaper. And it would be easy to
| replace because it's commodity hardware, that was kind of
| my point.
| EvanAnderson wrote:
| I have had experiences with tens of Dell servers with the
| same model NIC having the same fault. The servers were
| absolutely under maintenance. I fought with tech support
| for weeks before I was finally told it was a
| driver/firmware issue and that I had to work around it
| (and lose performance for the sake of reliability).
|
| Maybe if I had hundreds of servers Dell would have helped
| me out. At the scale of tens they told me to take what I
| got. The Customer got a lower performance solution and
| nobody anywhere could help them for any amount of money,
| short of replacing the gear.
|
| That's just a performance issue. I've heard horror
| stories about reliability-- All the way down to disk
| firmware and RAID controllers. I consider myself lucky.
| steve1977 wrote:
| Not saying things like that don't happen.
|
| But how much effort (or money) do you think it would have
| taken to fix this issue if the NIC firmware was open
| source?
|
| And with standard hardware, depending on the model, you
| might have had the option to add dedicated PCIe NICs for
| example. Not great, but at least something. Now try that
| with something proprietary (as in non-standard) like this
| Oxide system.
| dist1ll wrote:
| Replacing hardware? Sure, they'll help. What about
| debugging firmware though? I'm curious how much help you
| would get from Dell fixing and patching complicated
| firmware errors. A side benefit of the openness is that
| firmware issues can be discussed publicly, and the
| patches can be upstreamed into the main repo and made
| available to every customer (and even competitor). This
| gives you the kind of network effects that you'd never
| see in a locked-down ecosystem.
| steve1977 wrote:
| > What about debugging firmware though? I'm curious how
| much help you would get from Dell fixing and patching
| complicated firmware errors.
|
| If they replace the broken component with a working
| component, then I don't care how they fix their firmware
| errors.
| speed_spread wrote:
| The CPUs are x64 but the architecture is not that of a PC,
| there is no BIOS, etc. You couldn't boot Windows or Linux
| on the bare metal. The hardware, firmware and hypervisor
| are custom built for control, safety and observability. On
| top of that, the application OS all run on VMs which _do_
| have a (virtual) PC architecture.
| mwcampbell wrote:
| To me as a casual outside observer, the fact that they're
| using hardware virtualization at the top of the stack,
| after bcantrill gave so many talks about running
| containers on bare metal, is the most disappointing part.
| They could have had unbroken control and observability
| from the bottom of the stack all the way to the top. They
| got so close!
| speed_spread wrote:
| It's possible that the hypervisor can reserve you a full
| CPU or full cores for the guest OS to work with, so you
| still get most of that bare metal goodness.
| Aeolun wrote:
| But there's no pricing! How will I know if this could potentially
| save me money compared to AWS if I cannot make a sensible
| comparison until I contact sales (or probably after I contact
| sales).
| twoodfin wrote:
| To be fair, if you're in the market for a system like this,
| you're probably not paying rack rate for AWS or Azure, either.
| quickthrower2 wrote:
| Should be getting reserved instances at least
| Aeolun wrote:
| Most people that would be able to buy these would not, but
| that makes it more relevant to know how things compare.
|
| If you already get a 50% discount on your on-demand prices
| for AWS, does buying your own cloud make economical sense?
| DoingIsLearning wrote:
| Weird criticism, for anything involving CapEx it will always
| require an RFQ and quote.
|
| You're not buying a bag of peanuts and they will likely price
| it differently if you're buying 1x or 500x.
| SanderNL wrote:
| Why though? Quantity based pricing is not hard to express in
| a table.
|
| What is so different about CapEx and a bag of peanuts that
| defies even providing an approximate range? Number of zeroes,
| no?
|
| I don't like it, but it is the norm sadly. It's just a signal
| for "it's expensive and if you even care about the number we
| don't want to talk with you".
| darkwater wrote:
| > Why though? Quantity based pricing is not hard to express
| in a table.
|
| It is not, but when it's written publicly you cannot send
| different prices to different customers, depending on who
| the customer is
| gizmo wrote:
| In fairness, the blogpost argues that owning is more
| economical than renting. Everybody knows what cloud compute
| costs and if this more economical then maybe there should be
| some price indicator?
| wmf wrote:
| Yeah, the price has to be somewhere between a Supermicro
| rack and an Outpost rack.
| scott00 wrote:
| If you are genuinely in the market for multiple racks of
| servers you (a) know how much a rack of hp/dell gear costs,
| which gets you within an order of magnitude of what this is
| going to cost, and (b) would not buy one of these without a
| sales call even if you could.
| Aeolun wrote:
| I wouldn't buy these without a sales call, but costing it out
| to less than an order of magnitude would make it a lot easier
| to determine economic viability.
|
| I just hate calling people to determine if we're going to
| match, when they could have given me that information up
| front.
| steve1977 wrote:
| Sorry, but this is quite a bunch of b... IMHO.
|
| Let's start with their beliefs: 1. Cloud computing is the future
| of all computing infrastructure.
|
| Yeah, no. There's still gonna be things like mobile phones which
| will do some stuff locally. And probably also desktops. And
| probably also on-prem servers.
|
| Which brings us to their belief no. 2: The computer that runs the
| cloud should be able to be purchased and not merely rented.
|
| If you buy a computer and run virtualized loads on it, then
| you're not doing cloud computing. As simple as that.
|
| Basically, as others have commented already, this is just
| something like a blade server system or a hyperconverged system
| like for example Nutanix is offering. The only real difference
| seems to be the open source approach.
| parasubvert wrote:
| The ship has sailed many years ago on your definition of cloud
| computing. Fight it all you want, you're just going to be
| downvoted.
| albert_e wrote:
| Is the name of the company "Oxide" or "0xide" (with a leading
| zero)?
|
| All text seems to indicate the former while the logo seems to
| spell the latter?
| nopcode wrote:
| Yeah the logo is a bit confusing, but the full name is "Oxide
| Computer Company".
| steveklabnik wrote:
| The name is "Oxide Computer Company." But since "0x" is often
| used for hexidecimal numbers, and "0x1de" happens to be one, we
| play around with it from time to time.
| axelthegerman wrote:
| > The computer that runs the cloud should be able to be purchased
| and not merely rented.
|
| This 100%!
|
| Not everything needs to be a subscription. Sure there are running
| costs for operating that cloud computer as well as ongoing
| software development costs. But having photo albums in the cloud
| shouldn't cost me $30/month just because of storage - let me buy
| that hard drive (and maybe compute why not) and pay $10/y for
| operational costs
| gizmo wrote:
| $10 per year, minus credit card fees and taxes. If you send a
| single email to customer support in 5 years the company already
| loses money. The company can't just sell you the HDD and put a
| warning in the FAQ that you are responsible for data loss.
| Because that will result in angry emails and bad reviews.
|
| How many engineers and how many years does it take to build the
| infrastructure, write all the software, and deal with all the
| hardware to provide a photo service that "just works"? How many
| customers do they need before they break even? And obviously
| they can't raise VC because a business model that is predicated
| on making basically no money per customer can never have an
| exit. How would you bootstrap such a thing?
|
| It feels strange that physical storage costs almost nothing on
| amazon but the storage attached to a cloud machine is
| expensive. But run the numbers and you'll see that running a
| cloud service isn't about hardware costs at all.
| dewey wrote:
| I guess I don't understand how that's a radical statement if
| colocation was a thing even before you could rent cloud
| resources.
| dusted wrote:
| > core beliefs as a company:
|
| > Cloud computing is the future of all computing infrastructure.
|
| Oh god no!
|
| Anywhere I can donate to have this not happen? I want my
| computing on premise, preferably under my table.
|
| But well, if it _HAS_ to happen, 0xide is probably a lesser evil.
| ThrowawayTestr wrote:
| AMD and Intel so they keep making desktop CPUs?
| Fnoord wrote:
| No worries, it is just their sales pitch. They'll co-exist.
| eddyg wrote:
| If you don't listen to their "On the Metal" podcast, you're
| missing out. So many great stories from legends in the industry.
| Just start with the first episode.
|
| https://oxide.computer/podcasts/on-the-metal
| jjice wrote:
| And they currently run a podcast weekly called "Oxide and
| Friends" where they talk about miscellaneous things in the
| software space.
|
| The most recent few episodes have been about corporate open
| source and they've had excellent guests, like Kelsey Hightower.
| Definitely the best computer related podcasts out there. Bryan
| and Adam are great hosts and their humor is always a delight.
| repelsteeltje wrote:
| +1 Agree
|
| Other podcasts I'd recommend: ADSP [1] (if you're into
| programming), 2.5 admins [2] (if you're into computers). But
| I have no recommendations about hardware design because AFAIK
| the podcasts you mention and what Oxide is doing are pretty
| unique.
|
| [1] Algorithms + Data Structures = Programs
| https://adspthepodcast.com/
|
| [2] Allan Jude, Jim Salter, Joe Ressington
| https://2.5admins.com/
| vineyardlabs wrote:
| The Amp Hour is really the only popular
| hardware/electronics design podcast as far as I can tell,
| and it's pretty decent, especially when they have
| interesting guests.
| doublepg23 wrote:
| I've learned some very good tips from 2.5 admins but I have
| to take breaks often because their egos are a little much
| at times...
|
| The shows at https://www.jupiterbroadcasting.com/ are my
| personal go-to.
| ahl wrote:
| Much appreciated!
| fbdjdjfb wrote:
| Added to my podcast player - Thanks!
|
| In a similar vein, Embedded.fm is a great podcast for embedded
| SW. Though I bet Oxide's take on embedded rust is a lot
| different than the hosts of that show!
| benpacker wrote:
| I know price will vary wildly based on how many you're buying,
| but does anyone have the roughest ballpark for how much it would
| cost to buy one (1), or like two?
| wg0 wrote:
| I would assume a million dollar. Ballpark.
| Aachen wrote:
| I mean, the raw hardware surely costs around 100k, and no way
| that it costs more than ten million, so you're always going
| to be right with that "ballpark" qualification
| wg0 wrote:
| I vaguely remember listening to their pod cast and my
| impression was that it starts around 500k.
|
| Add few add ons, service contract, support etc so would be
| there.
|
| I want such a rack but power draw on average is listed
| around 12kwh. Unbelievable.
| yardie wrote:
| They mentioned it really quickly in their Oxide an Friends
| podcast but, IIRC, prices start at $500k. Some of the audience
| asked if they were going to do a smaller configuration like
| half or quarter rack. And they said they were looking into it
| but weren't sure the of the business case.
| MichaelZuo wrote:
| So the real question is whether 1 Oxide rack can outcompete 2
| or 3 racks of normal commodity hardware.
|
| Or provide enough white glove after-sales support and written
| guarantees to peel away low end mainframe customers at a
| fraction of the price.
| throw0101c wrote:
| > _So the real question is whether 1 Oxide rack can
| outcompete 2 or 3 racks of normal commodity hardware._
|
| Given their management plane/API:
|
| * https://docs.oxide.computer/api/guides/responses
|
| the performance may be about the same, and CapEx as well
| (or maybe a little higher), but OpEx could be where you
| make it up in large(r)-scale operations.
|
| And space efficiency is also not to be sneezed at: for some
| operations DCs/compute can be place anywhere because
| latency is _that_ big of a deal, but in other places you
| need to be close to certain things (trading), and real
| estate can get expensive.
| dmix wrote:
| I don't know anything about buying servers, is that
| expensive?
| darkwater wrote:
| It depends on how many servers you put in a rack. It's been
| years now I did this kind of work but I would say that an
| average rack with 20-25U in computing, 5-10 in storage and
| 5 in networking will cost you 300k$ easily. I'm pretty sure
| that Oxide will be more an Apple-esque experience, also on
| the price side, so a "normal" rack giving the same
| performances will be cheaper but if you want Oxide you are
| looking for other features beside the pure HW.
| gorkish wrote:
| At the leading edge, the configuration you just described
| is probably more like $800k
| yardie wrote:
| It's very expensive hardware. I think they are trying to
| bring TCO down on the operation side with better control
| plane. I work in hyperconverged systems and it's a bunch of
| tradeoffs. Nothing I've configured has approached $500k so
| their control plane and OS has to make a really good show
| of why this 2x expensive cabinet is better than rack and
| stack Dell.
| Kranar wrote:
| Let's just say I hope I am interpreting something
| incorrectly, because if 500k is the minimum price and you
| match it to the minimum configuration found here:
|
| https://oxide.computer/product/specifications
|
| Then yeah, it's ridiculously expensive.
|
| That said compared to competitors it's in the right
| ballpark, but I have no idea how companies manage to spend
| so much money for this stuff. I am the founder of my own
| tech startup and I remember when I was looking at storage
| solutions and building out computing clusters there were
| companies charging absolutely insane prices.
|
| I literally just spent about a week of my own time studying
| and learning as much as I could about it on my own and
| ended up building out my own custom solution for about
| 20-25% of the price these other companies were charging. I
| remember hearing people trying to scare me out of it saying
| if I did my own solution I'd need to hire full time
| operations people, and I'd always have to worry about
| things breaking and maintenance or headaches and nightmares
| etc...
|
| It's been over 10 years now and absolutely no headaches, no
| nightmares, and very very minimal maintenance is needed.
| PTOB wrote:
| Tell me more!
| Kranar wrote:
| Yeah sure, my process to learn was literally I went down
| to a local computer store, I bought a cheap desktop
| computer and 6 cheap hard drives, an HBA, a RAID
| controller, a bunch of cables, went back to the office
| and installed Ubuntu Server onto it and practiced several
| "skills", like how to setup automated backups using
| rsync, how to physically install these components, how to
| install mdadm for software RAID etc... comparing software
| RAID to using a RAID controller. I setup several drills
| for each task involving various types of failures and how
| to manage them to the point that it was part of muscle
| memory.
|
| Some drills I practiced were setting up RAID 10 and on
| hardware failure having an email sent to me, so I went
| through the process of getting RAID 10 working using 4 of
| the drives, and then I would physically pull a hard drive
| out of the system as it was running to simulate a
| failure, and then I swapped in a new hard drive and
| ensured that the RAID rebuild process took place.
|
| Once I was confident enough, I went to thinkmate.com and
| bought two JBOD expansion chassis each of which supported
| 78 drives and filled each of them up with 5 TB Seagate
| drives to get a total of 390 TB. The JBODs are managed by
| a 1U server running Ubuntu Server and software RAID using
| mdadm. I also bought 8 compute servers that were
| considered high density and could fit in 4U. I got a
| Cisco router for Internet and I networked everything
| within the data center together using a used Infiniband
| switch that I bought off of ebay. I also got a KVM so I
| could remotely access all of the systems. If there's one
| thing that I would change today, it would be to use ZFS.
|
| I remember comparing the cost of doing this DIY setup to
| some other premade solutions like EMC and the cost was
| astronomical, like 3-4x what I ended up having to pay. I
| also even remember watching Linus Tech Tips and
| Level1Techs on Youtube and they both had good content
| about how they managed their storage that was fairly
| reasonable but still slightly on the pricey side,
| nevertheless I learned quite a bit from it.
|
| At any rate, the bottom line is that it's not trivial to
| learn all this stuff by any means and I remember having
| some serious frustrations due to just how bad and
| demoralizing some error messages can be, but it's not
| thaaaat difficult either and in my situation my company
| is self funded, no venture capital or outside investors
| of any kind so every dollar my company spends is a dollar
| out of my own pocket. You better believe when I'm
| starting my business I'm not going to just blow 100s of
| thousands of dollars extra unless I absolutely have to.
|
| About 5 years ago I managed to use my own storage system
| to ditch Dropbox in favor of Nextcloud, which is just
| leaps and bounds superior. I remember I got so frustrated
| with Dropbox because I wanted to just do something as
| simple as create a sub-directory and grant only read only
| permissions to some accounts. I also remember wanting to
| do some simple things like create a fake account with
| very limited permissions that could be used in our
| conference room for presentations or demos but the only
| way to do that with Dropbox would be to create an actual
| account and have to pay the full price for it.
|
| Nextcloud works amazingly well, has all kinds of cool
| plugins, and gives our organization a lot of flexibility
| that Dropbox doesn't and I can make as many accounts as I
| feel like for whatever reason I feel like.
| ThePowerOfFuet wrote:
| 400 TB but no ZFS, and lots of hardware and software but
| no support.
|
| And not a single mention of backups.
|
| Good luck with that.
| Kranar wrote:
| It's since grown over the past 10 years to about 2 PB of
| storage.
|
| I mentioned I use rsync for backups.
| ThePowerOfFuet wrote:
| rsync to where?
| aeyes wrote:
| In the spec sheet it looks like they have options with 16, 24
| or 32 "compute sleds" (servers?).
| JeremyNT wrote:
| > They mentioned it really quickly in their Oxide an Friends
| podcast but, IIRC, prices start at $500k. Some of the
| audience asked if they were going to do a smaller
| configuration like half or quarter rack. And they said they
| were looking into it but weren't sure the of the business
| case.
|
| That strikes me as being in the right ballpark, but it's
| going to be tough to swallow since that's the lowest level of
| granularity.
|
| For most orgs you'd be left paying for a _lot_ of excess
| capacity you couldn 't immediately put to use as you migrate
| workloads in. I guess in ~4 years once you reach steady state
| and you're retiring / replacing these things it all works
| out, but if you're migrating from vmware or something else in
| a traditional blade/chassis world it's not like you can just
| wave a magic wand and move $500k worth of compute over to
| this thing at once.
|
| If you're green fielding something, that's a lot of cash to
| sink in on compute you may not need for some time in the
| future. Never mind your DR site(s) also needing that much...
| dbancajas wrote:
| Follow up question would be how much are equivalent solutions
| from established rack providers?
| chrisweekly wrote:
| Ambitious / audacious -- and compelling.
|
| Pricing??
|
| Also "Facilities" link in footer: 404
| benjaminleonard wrote:
| Sorry about that, fixed - you can find that information on the
| specs page now
| TrueDuality wrote:
| I'd like to add that their open source code is also EXTREMELY
| high quality. If you're an embedded developer go take a look at
| Hubris and Humility. I ended up using those to GREAT effect for
| this custom one-off aerospace device and it was a fantastic
| experience to integrate with. Definitely a change from what I was
| used to that took a bit of getting used to.
| throw0101c wrote:
| > _I 'd like to add that their open source code is also
| EXTREMELY high quality._
|
| * https://github.com/oxidecomputer
| doublepg23 wrote:
| MPL even, that should satisfy about anyone who'd want to
| contribute.
| csomar wrote:
| I agree. I stole some of their stuff from here
| https://github.com/oxidecomputer/third-party-api-clients/tre...
| when I needed a SendGrid integration. High quality code and
| proper use of Rust types.
| jeffrallen wrote:
| Their ring logger was really enlightening to me about the
| value of Rust enums. Heterogeneous log events are dropped
| into the ring with some holding only the fact that something
| happened, and others holding additional data about what
| happened. Then Humility is able to print out the contents of
| the ring either online or in crash dumps. This is how you get
| logging in nostdlib Rust without ending up without half of a
| badly implemented printf. Instead, Humility, which has the
| full stdlib available, formats the enums for the firmware.
| vlovich123 wrote:
| Link?
| mkeeter wrote:
| The embedded side is here: https://github.com/oxidecomput
| er/hubris/blob/master/lib/ring...
|
| And the debugger is here: https://github.com/oxidecompute
| r/humility/blob/master/cmd/ri...
|
| I also gave a talk at this year's Open Source Firmware
| Conference that covers a bunch of debug strategies:
| https://www.osfc.io/2023/talks/unplugging-the-debugger-
| live-...
|
| (conference videos aren't online yet, but should be
| posted early next week)
| behnamoh wrote:
| On a side note, I really liked their website. The ASCII
| animations are interesting -- wish there was a video game that
| had those.
| ranger207 wrote:
| Cogmind (https://www.gridsagegames.com/cogmind/) has those
| sorts of animations
| jefurii wrote:
| The developer also released the ASCII art editor he used to
| make all those nifty designs:
| https://www.gridsagegames.com/rexpaint/
| LoganDark wrote:
| Woah , definitely going on my wishlist
| mlindner wrote:
| I've been interested in Cogmind for a while, unfortunately
| it only works on Windows (which seems an odd choice for a
| text-based game) so I have no ability to play it.
| rileyphone wrote:
| It appears to work under Proton in Linux.
| https://www.protondb.com/app/722730
| benjaminleonard wrote:
| They are very fun to make as well! I've built my own mini-lib
| on top of this ASCII rendering library
| (https://github.com/ertdfgcvb/play.core).
|
| I design them in Monodraw, pass it through a janky converter
| I wrote that converts text into a json grid of characters. I
| then render a number of layers that get combined, which is a
| mix of the static art layer, and others generated from
| functions that spit out a similar cell based frame.
|
| If you're interested: https://gist.github.com/benjaminleonard
| /c913ddbf23fe7a70f9c2...
|
| And for what it's worth there's this ASCII game:
| https://twitter.com/StoneStoryRPG
| throw-DO-178C wrote:
| > ... code is also EXTREMELY high quality. If you're an
| embedded developer go take a look at Hubris and Humility.
|
| So, Humility is like MC/DC?
|
| https://en.wikipedia.org/wiki/Modified_condition/decision_co...
| steveklabnik wrote:
| Humility is a debugger, not a code coverage tool.
| throw-DO-178C wrote:
| HUMILITY
|
| 1. The state or quality of being humble; freedom from pride
| and arrogance; lowliness of mind; a modest estimate of
| one's own worth; a sense of one's own unworthiness through
| imperfection and sinfulness; self-abasement; humbleness.
|
| -- Webster's 1918 Dictionary
|
| Our avionics software was originally spec-ed to not NEED a
| debugger it was to be of such high quality, but the
| designer added break point op-codes in the VM anyway. I
| guess he was aware of the concept of humility, too. ;) Like
| humor, humility seems to be in short supply these days.
| throw-DO-178C wrote:
| > Like humor, humility seems to be in short supply these
| days.
|
| Hmm...maybe I should have referenced:
|
| https://en.wikipedia.org/wiki/Sacred_cow_(idiom)
|
| > The motto of the satirical magazine The Realist was
| "Irreverence is our only sacred cow".
|
| Humor does seem to _exist_ today, it just has a narrow
| band-pass filter, with apparent focus on "organic"
| product marketing.
| danbruc wrote:
| Tangent. In my admittedly limited experience, embedded code
| seems to have a tendency to be some of the worst code you can
| come across. The stuff is already low level and not the most
| easy to follow, but then embedded developers seem to despise
| names with more than two letters and a number. Without the
| datasheet at hand it is impossible to figure out what any of
| the code does because everything is just an abbreviation or
| acronym from some block or pin-out diagram.
| otteromkram wrote:
| What's the point of your comment?
| teraflop wrote:
| Yeah, I just skimmed through the reference docs for Hubris and
| was very impressed with what I saw. No "rocket science", just
| (apparently) solid technical decisions that are extremely well
| justified and documented.
|
| https://oxidecomputer.github.io/hubris/reference/
| o11c wrote:
| Some interesting rare choices there (some of which would be
| applicable to normal userlands or even to language VMs):
|
| * Memory _access_ is protected, but _addresses_ are not
| virtually mapped. The lack of paging of course ties strongly
| to the inability to dynamically add tasks.
|
| * They wish they could have read-only shared libraries
| (viable since there's only one address space) so they could
| truly be shared, but the current ecosystem assumes mutability
| is possible (even though in Rust mutable globals are rare).
| taink wrote:
| The only big thing missing for now is their OS (or rather,
| their Illumos distribution), Helios[1].
|
| Can't wait to read its source code, I was curious since reading
| this thread[2].
|
| [1] https://github.com/oxidecomputer/helios (if this is not
| found for you, we're in the same boat)
|
| [2] https://news.ycombinator.com/item?id=33337086
| cryptonector wrote:
| > their open source code is also EXTREMELY high quality
|
| I would expect nothing less from that crew. They did amazing
| things at Sun Microsystems, Inc. (RIP), and they continue to do
| even greater things now.
| throwawaaarrgh wrote:
| Thought I had heard the cringiest random names for an oss
| project, and then...
| otteromkram wrote:
| Is that the same extremely high-quality open-source code that
| currently has a failing build?
|
| https://github.com/oxidecomputer/hubris
|
| And, that one underscore-delimited folder name in this repo
| just catches the eye, huh?
|
| https://github.com/oxidecomputer/humility
| mbStavola wrote:
| If those are your complaints, I think they're probably doing
| okay.
| mgkimsal wrote:
| I met a couple of their folks on site last week in Raleigh at the
| All Things Open conference. Saw a couple quick demos, and... it's
| not for me. But I can see the benefits, and many other folks
| stopping by the booth seemed to get the benefits as well. The
| folks at the booth were really nice too. Granted, that's sort of
| your job when being in a vendor booth at a conference, but it's
| surprising how often that's not the case (booths staffed by
| people who don't know the product, or are simply indifferent to
| the company, etc).
|
| EDIT: "it's not for me" - I'm not working with organizations that
| have that sort of need directly. Re-reading that phrase, it came
| across as a bit dismissive of what they've built (which is
| undoubtedly impressive).
| steveklabnik wrote:
| Thanks for stopping by! All Things Open is one of my favorite
| conferences, glad we were able to be there.
|
| And yeah it's totally chill: this product is not for everyone.
| No slight taken!
| mgkimsal wrote:
| Nice to have been able to put a real life face to a name I've
| seen for so long!
| wg0 wrote:
| This is beautiful, Apple kind of elegant but the guys taking care
| of server racks are not the same crowd as those into
| Typescript/Rust/Go I guess?
|
| But yes, if multiple teams and they have direct access this rack
| to provision VMs, maybe yes.
|
| Also - as a developer you don't provision VMs on daily basis. You
| start a project, you need those resources at the very outset and
| that's about it?
| nu11ptr wrote:
| > The computer that runs the cloud should be able to be purchased
| and not merely rented.
|
| How is this different than colo space that has been predominant
| for years? Perhaps simply that it can be purchased more easily
| and standardized like a typical cloud VM?
| drzaiusx11 wrote:
| That it's a turnkey, vertically integrated and open platform
| with everything included. Seems big enough differentiator for
| me. It's like going to going to a restaurant with table service
| vs a BYOB pizza shop.
| FridgeSeal wrote:
| I believe-as a few other commenters have pointed out and you
| allude to, the advantage is that it can be configured and
| operated like a massive set of cloud VM's. There's monitoring,
| provisioning, network, etc etc all setup and fully integrated.
|
| I imagine colo's have something like this, but I _suspect_
| operationally it's a lot more powerful, easy to use and
| functionally closer to the API's and behaviour users are used
| to on current cloud providers.
|
| Now, if someone who actually knows for sure feels like chiming
| in, I'm happy to be corrected.
| Fnoord wrote:
| It is the combination of things. Full FOSS stack, only cables
| being power and networking, and it being AIO package instead of
| conventional server rack.
| flakiness wrote:
| I love they do _not_ mention "AI" in the article.
| Always_Anon wrote:
| Because they can't. Their product currently has zero GPU
| compute.
| jjice wrote:
| Huge fan of the people behind Oxide as well as the concept behind
| the product. Can someone tell a layman like myself where a 44M
| Series A funding round lands among other Series A rounds? I was
| at a startup during 2021 and when we raised 25M, it was a big
| deal for us and seemingly huge. Compared to that 44M is insane.
| Is this notably high or is my frame of reference just small?
| Either way, I really look forward to seeing Oxide's progress
| going forward.
|
| It's a company run by people who care and have cared for a long
| time, and they have all my faith.
| benjaminwootton wrote:
| People raise those sorts of numbers for web apps all the time.
| It actually doesn't sound that big for such a broad hardware
| and software company with long and high touch sales cycles
| ahead of them.
| twicetwice wrote:
| Do they raise those sorts of numbers in series A, though?
| Other comments in this thread seem to suggest they've raised
| a surprisingly large amount of money for such an early stage
| of the typical fundraising process. If it is atypically high
| it would make sense to me--it's hardware which requires more
| up-front capital, and it's a rockstar team so I understand
| why investors would have confidence in them--but from what
| little I know about fundraising it does seem remarkable to
| me.
| vb-8448 wrote:
| Like too much the idea.
|
| If they manage to do get an IBM mainframe level after-sales
| support, they will rock.
|
| Does someone know how much the lowest level box costs?
| FridgeSeal wrote:
| Some comments up-thread suggested 500k.
| vb-8448 wrote:
| Yeah, I read ... just "wow". I hoped something more
| affordable, or even something more flexible like "you get the
| full rack, but you are enabled to use only x % of the
| capacity for a lower entry price".
| caskstrength wrote:
| They seem to be cagey about their networking. Nothing besides
| vague "100GbE to the switch". Is this some home-brew asic or
| fpga? What offloads does it support? Is the driver upstream?
| Youden wrote:
| No, there's just a lot more that can be said than they wanted
| to fit on this one page.
|
| Specs of the switch are available here:
| https://oxide.computer/product/specifications
| caskstrength wrote:
| I was asking about their NIC, not switch.
| brucepink wrote:
| They use Chelsio NICs - I've heard them mentioned on one or
| other of their podcasts.
| eaasen wrote:
| The NIC is a chip-down Chelsio T6 ASIC with 2x100GbE ports,
| one going to each Ethernet switch in the rack.
| wpm wrote:
| https://docs.oxide.computer/guides/system/network-preparatio...
| pseudoshikhar wrote:
| I think it's something like - shipping management software with
| bare metal ?
| samuell wrote:
| Now I only wonder if it makes sense to install a SLURM HPC
| cluster on one, and how much hassle that would be?
| remeq wrote:
| I always had a feeling that Bryan and people around kinda took
| the tech first approach and it never really came to a proper
| fruition. I think the ambition behind the engineering marvels
| they did over time was certainly bigger. Dtrace, Fishworks,
| Joyent, then this... so why is that? Perhaps the fierce refusal
| of joining the mainstream side (Linux)? Fingers crossed, but...
| mathverse wrote:
| This is too startup-y and niche and too small to attract enough
| companies on the market.
|
| Best case this gets acquired by HP,IBM or Dell and will die out
| because talent will leave.
|
| I know a 4bn/y revenue company that could greatly benefit from
| this but they will never even consider buying from a company like
| this.
| klysm wrote:
| The value proposition might be worth the downside risk
| throw0101c wrote:
| > _This is too startup-y and niche and too small to attract
| enough companies on the market._
|
| From their press release:
|
| > _Oxide customers include the Idaho National Laboratory as
| well as a global financial services organization. Additional
| installments at Fortune 1000 enterprises will be completed in
| the coming months._
|
| * https://www.bloomberg.com/press-releases/2023-10-26/oxide-
| un...
|
| From an interview:
|
| > _And what was the reaction of lucky rack customer #1? "I
| think it's fair to say that the customer has appreciated the
| transformationally different (and exponentially faster!)
| process of going from new rack to provisioned VMs."_
|
| > _The same day Cantrill appeared as a guest on the Software
| Engineering Daily podcast.[1] ("The crate, by the way? Its own
| engineering marvel! Because to ship a rack with the sleds, it's
| been a huge amount of work from a huge number of folks...")
| Cantrill wouldn't identify the customer but said that
| "Fortunately when you solve a hard problem like this and you
| really broadcast that you intend to solve it... Customers
| present themselves and say, 'Hey, we've been looking for --
| thank God someone is finally solving this problem'."_
|
| * https://thenewstack.io/in-pursuit-of-a-superior-server-
| oxide...
|
| Niches can be profitable. Not every company has to be "web
| scale".
|
| > _I know a 4bn /y revenue company that could greatly benefit
| from this but they will never even consider buying from a
| company like this._
|
| What does "like this" mean? Small? Every company starts out
| that way.
|
| I think its cool that someone is trying to do startup hardware
| things, as in recent years it seems like its mostly software. I
| don't a use for the product in my area of IT, but I wish them
| luck.
| PoignardAzur wrote:
| How can you be so sure? I'm sure GP did a ton of market
| research before making that comment. This company is probably
| doomed.
| sambull wrote:
| feels a lot like those vblock /converged infrastructure in a box
| things they used to push. Turnkey virtualization environment - I
| guess the advantage/? here is a open source stack.
| ls65536 wrote:
| As it appears Intel Capital is one of their Series A investors, I
| wonder if this has some bearing on the chances that there could
| also be an Intel CPU-based Oxide rack (or maybe individual
| sleds?) in the future.
|
| It seems that Oxide's product is rather tied to AMD's CPU
| offerings for the time being (which perhaps will serve them very
| well for now), and given what they've accomplished so far, I
| imagine it would take quite a bit of effort on the platform
| initialization side (and other lower level stuff) to get things
| working with Intel CPUs instead. Surely the ability to have
| vendor diversity for many of their components should be an
| advantage for Oxide (and their customers downstream as well), so
| maybe there's something to think about there. Of course, this is
| all interesting only once Intel actually gets competitive again
| on the datacenter CPU side of things, where they seem to have
| really dropped the ball in recent years.
|
| On the other hand, their networking switch hardware uses Intel
| components (Tofino), so maybe that'll be the extent of Intel's
| integration in their rack for the foreseeable future?
| flumpcakes wrote:
| This is interesting as I think Intel have stopped developing
| Tofino, after buying Barefoot networks back in 2019.
|
| AMD recently purchased Pensando, I'm not sure if their network
| chips are similar to Tofino but they seem like they are P4
| compatible DPUs so they might be a good choice when migrating
| off of the dead Tofino platform.
|
| So if anything Oxide will be moving further away from Intel!
|
| I would love to work on this kind of stuff, not sure how to get
| started to end up working somewhere like Oxide..
| bilekas wrote:
| This looks really interesting, but please can we get some idea of
| the prices involved without needing to contact sales teams ?
| osti wrote:
| They use Tofino as the networking switch. But Intel is
| discontinuing Tofino so I'm wondering what does future upgrade
| path look like for Oxide? Would they consider something like a p4
| programmable DPU as the replacement?
| flumpcakes wrote:
| Oxide racks curren use AMD CPUs and and after buying Pensando
| AMD now offer P4 DPUs. I would hazard a complete guess and say
| maybe a move from Tofino to Pensando is on the cards for future
| networking revisions.
| osti wrote:
| Company where I work at is evaluating Pensando DPUs and they
| are very powerful.
| wmf wrote:
| DPUs can't really replace switches. Maybe they could just
| keep using Tofino for the next N years until something better
| comes along.
| osti wrote:
| It kind of can if you want to add programmability to your
| switches. So on top of traditional switches, you can add a
| cluster of DPUs in order to add networking functionalities
| to your switches. Microsoft has done just that.
| https://www.usenix.org/system/files/nsdi23-bansal.pdf
| wkoszek wrote:
| I regret missing the Open Source Firmware Conference--I really
| wanted to meet them and learn some more on what's the story here.
|
| I know it's a brilliant team, and since ZFS/OpenSolaris/Dtrace
| I've known about Bryan. The product is a nicely looking product
| too. One thing I don't get is the target market. Who is it?
| Banks? Cloud vendors? Insurance companies? I think there's a part
| of the computer business that I don't get yet.
|
| One guy in one of the previous HN threads here said that it's
| convenient to run 1 RFP for 1 box costing $1M-$2M vs running 30
| RFPs for isolated components to put on the rack. Ok, I can see
| that. That's some argument.
|
| But I know that Supermicro has rack assembly service.
| https://www.supermicro.com/en/products/rack and I'm sure Dells
| and HPEs have the same. Does it mean that it's impossible to call
| them and tell them: "Excuse me, I want 2 racks, 15 systems each,
| top of the line AMD CPU with RAM, NVMe storage, all maxed out,
| and whatever fastest Juniper switches you can find. Put me VMWare
| vFusion on it. Ship this thing to Infomart in Texas. I have $2M
| here with your name on it". They won't do this for me?
|
| Its either this, or Oxide is going to e.g.: Bank of America, and
| along with RFP the bank is asking: "Show me complete supply chain
| records, including the source code to your boot loader, drivers,
| and any firmware that any component on this motherboard is
| running". And maybe that's ... impossible these days?
|
| I'm the startup CTO, so not the consumer base, but after DHH
| started posting his thing about cloud exit, I decided to explore
| this, and I walked the floor of 3 DCs (in SV/TX). People who walk
| you around the DC don't appear to care about cables, fans or
| noise etc. If it's longer than 1hr on the floor, they'll put
| Airpods in and they're done. I've seen cages that look unified,
| pristine, with love and affection put into cable layout, and I've
| seen cages that look like a total mess. It's your cage--you only
| go there if things break.
|
| My last guess is that maybe Oxide rack will end up being sort of
| what an Apple Macbook is among cheap HP/Acer laptops from
| Walmart. And it'll be a shiny toy of bold bearded IT dudes who
| work for GEICO IT by day, but by night they scavenge eBay for
| used server deals and get excited about the idea of running their
| own private rack in their basement. They can't afford it at home,
| but at work they'll want their own Oxide. If you're reading this,
| do know I know who you are.
| steveklabnik wrote:
| > One thing I don't get is the target market. Who is it?
|
| Fortune 1000, government. Large organizations that want to own
| their own hardware, yet want the cloud experience for deploying
| their software to it. First two customers to have received
| racks are Idaho National Lab, a Department of Energy
| laboratory, and a global finances firm. I think that gives a
| characterization to at least part of the market.
|
| > They won't do this for me?
|
| You are correct that they will do that for you. There are big
| differences though. From a hardware level, the largest one is
| that what you'll get in that case are individual 1U servers, in
| a rack, built from a bunch of reference designs, pieced
| together from various different organizations. We designed this
| as a whole rack. From scratch. This has a number of benefits,
| like for example, we use 80mm fans, at a very low RPM. Our fans
| draw less power and operate more quietly than the usual fans
| you'd get in that case. (I know you said people don't care
| about noise, and it's true that it's not a feature of the
| product to sell, just an interesting aspect of the design
| process: we didn't set out to make the rack quiet, it just is,
| thanks to other decisions that are more meaningful, like
| cooling efficiently) On the software side, we have written an
| enormous amount of software from scratch, designed for this
| specific hardware. Including a control plane, so you can think
| of the whole rack as a pool of resources, not as individual
| servers you manage yourself. And since we have done all of this
| in-house, we can take responsibility for the full quality of
| the product. If you have a firmware bug, it's not "oh sorry,
| we'll file a ticket with our firmware vendor and let you know
| when that's sorted," we will fix it ourselves. Everything is
| integrated and works together, because we built it for purpose
| that way, not because we installed a bunch of things from
| different organizations, ran some test, and said "looks good to
| me."
| cheez0r wrote:
| Congratulations Bryan and the Oxide team! Well done on your
| achievement.
| VikingCoder wrote:
| Next, I form a company that rents out time on an Oxide rack...
| tivert wrote:
| That website is interesting. Maybe it's a Firefox bug, but when I
| load the page with NoScript on, the text is very weird, like
| this:
|
| > TODay WE aRE aNNOUNCING tHe
|
| It's actually _much weirder_ than that, as some of the capitals
| in my quote are actually big lowercase letters, not capitals
| (i.e. the opposite of smallcaps).
| dcre wrote:
| Can't repro in FF with JS off (not exactly NoScript). Could it
| be an extension doing something weird? Curious if you see
| normal text when you view source on the page.
| autoexec wrote:
| I saw the same thing and wondered why nobody in the comments
| mentioned how painful this site was to read. Copy/paste the
| text and suddenly it's all back to normal. Saved the page to
| disk and loaded the local copy in the browser - also perfectly
| normal. Very strange. Maybe something to do with their CSS?
| Fnoord wrote:
| That is an odd bug I have seen before with a browser (not sure
| when or when). It freaked me out back then though.
|
| Open it in a private window and see if you can reproduce. By
| default, a private window uses no extensions though I prefer to
| have uBO enabled by default there, too.
|
| Another peculiar detail I saw on landing page is screenshot of
| management interface. It must be old as it shows various VMs
| including FreeBSD 12.1 and Ubuntu 20.10. Ubuntu 20.10 is EOLed
| long ago and also wasn't a LTS release. Not something you
| wanted running in enterprise 3 years ago, nevermind recently.
| wiradikusuma wrote:
| So what's the use case for this? Can I buy this and start a
| hosting company?
| apexalpha wrote:
| I work at a Telco that needs the teams to be able to create
| VMs, networks etc.. But due to legal constraints we need to
| host it on prem.
|
| For those companies this would probably be a cost effective
| solution since now we spend a lot of resources managing varying
| Openstack installations with different hardware from HP / Dell
| etc...
| alberth wrote:
| > "Today we are announcing the general availability of the _WORLD
| 'S FIRST_ commercial cloud computer"
|
| (emphasis mine)
|
| I'm having a hard time getting past the first sentence of this
| blog post.
|
| Maybe I'm missing the obvious.
| paxys wrote:
| Just marketing fluff. It's a rack mounted server. Nothing
| "world's first" about it.
| mbakke wrote:
| No wireless. Less space than a nomad. Lame.
| LoganDark wrote:
| They're selling a rack that hosts an entire virtual cloud.
| Nobody has done that yet. There have been servers with
| hypervisors preinstalled, but nothing like this.
| alberth wrote:
| Oracle, AWS & Azure all have "Cloud at Customer" offerings.
|
| And these offerings have existed for years.
|
| And from hardware manufacturers, Dell/VCE, Nutanix, and more
| "hyper-converged" infra has existed.
|
| Note: I'm not being a hater. I'm just genuinely confused.
|
| ---
|
| https://www.oracle.com/cloud/cloud-at-customer/
|
| https://azure.microsoft.com/en-us/products/azure-stack
|
| https://aws.amazon.com/outposts/
| reffaelwallen wrote:
| maybe its about allowing you to create your own cloud, not
| using a third party vendor's software? its not a computer
| you hook up to a cloud, it is its own cloud? i only know
| about aws outpost tho, so I might be wrong
| alberth wrote:
| > "not using a third party vendor's software"
|
| Wouldn't you still be using 3rd party vendor software,
| it'd be Oxide software now?
| LoganDark wrote:
| > Oracle, AWS & Azure all have "Cloud at Customer"
| offerings.
|
| AFAIK they manage it for you, as if you're just a colo.
| Whereas oxide just hands the entire rack + software over,
| but with no self install of any software stacks required
| (such as with azure)
| CyberDildonics wrote:
| This is literally what people would do long before 'the
| cloud' was ever a buzzword.
| LoganDark wrote:
| Yeah because oxide.computer isn't someone else's machine
| anymore. It's yours.
|
| That's literally why it's so exciting lol
| CyberDildonics wrote:
| Why would it be exciting for a company to be literally
| doing what lots of companies have been doing for
| literally 25 years?
| LoganDark wrote:
| because they do it better...?
| lopkeny12ko wrote:
| > We're really excited to have the first commercial cloud
| computer -- and for it to be generally available! If you yourself
| are interested, we look forward to it making its first impression
| on you -- reach out to us!
|
| Can we please stop with the "the only way to get our pricing is
| by booking a sales call with us." This is a 100% surefire way for
| me to never pay for your product, and instead go to competitors
| who provide straightforward, no-nonsense pricing on the website.
|
| This is ironic given the amount of self-praise they give
| themselves in this article about how much they care about
| shipping something you can buy once instead of renting from the
| traditional cloud. Great, so then tell me how much it costs...
| paxys wrote:
| Their pricing starts at $500K (but realistically will be $1-2M
| per order at a minimum). This is not intended for people who
| browse their website, click "buy" and fill in their credit card
| info. If you don't want to talk to a sales rep you were never
| their target customer, and I'm sure they aren't sweating the
| hypothetical lost sale.
| rwmj wrote:
| I still don't get why they can't put a price on the website.
| layer8 wrote:
| It gives them more flexibility and reduces competition.
| It's quite common in B2B, for products/services in that
| pricing range.
| lopkeny12ko wrote:
| How in the world is reducing competition a good thing?
| layer8 wrote:
| Making it harder to compete with them is a good thing for
| Oxide, obviously, and therefore why it provides negative
| incentive for them to publicly advertise pricing.
| omarfarooq wrote:
| It's not rocket computer science to spin up a pseudo-
| consulting entity [that works with larger enterprises] to
| get on sales call for competitive intelligence.
| layer8 wrote:
| It's still harder, and you'll only get a single price
| point at a time, which also might change three months
| later.
| paxys wrote:
| Why would they? What advantage do they get from it?
| cozzyd wrote:
| Presumably everybody's quote is going to be a bit different
| depending on various factors...
| layer8 wrote:
| I wonder how much the premium is over traditional server
| hardware with the same capabilities. You save some
| integration work and need less know-how, but it'd be
| interesting to compare.
| carbotaniuman wrote:
| Their entire point is to be different then that - if you want
| servers just go to HPE and spec out some servers. (Which really
| does beyond small scale, but at the rack level it's unheard of)
| bigironcto wrote:
| I am CTO of a large global data center provider posting with
| throwaway account.
|
| As a technologist, I really appreciate what they have done.
| Impressive work, high quality, however I don't understand who
| this is for.
|
| The meaningful market for Data Center hardware is pretty well
| defined in two clusters. People that build/make custom gear (such
| as Hyperscalers) and people that buys HP/Cisco/IBM/Dell...
| (blades or hyper-converged). To scale, you obviously want your
| DCs as standardized as possible.
|
| Until this company has a certain/size and scale, no one serious
| will trust their black boxes at any type of scale.
|
| Beyond the tech, how would support services really work? We can
| have a technician from any of the large vendors on-site in less
| than 2 hours. In some of our DC clusters we actually have vendor
| support personnel 24x7 on-site with vendor paid spare parts
| inventory. How would they provide that level of service?
|
| Maybe I am not the target audience for this offering.
| apendleton wrote:
| I think they're mostly targeting customers who want an AWS- or
| GCP-like experience from a developer perspective (compute is
| abstracted and you can provision it with an API, etc.), but
| want to own their own compute infrastructure and have it on-
| prem. That market has mostly had to cobble together consumer-
| inspired HP/Cisco/whatever stuff historically (like, one of the
| early talks about the Oxide value proposition was complaining
| about why every server in the rack needs a CD drive, which was
| the norm from Dell), because the kinds of stripped-down, super-
| efficient hardware designs the hyperscalers were building
| weren't available to the general public, so this is that:
| hyperscaler-like technology for people who want to own it
| themselves.
|
| I think the motivations for why people would want to own their
| own are probably a mix of financial (at a certain scale there's
| a tipping point and it gets cheaper), and
| regulatory/compliance/whatever, like if it's healthcare data,
| or defense, etc.
| bigironcto wrote:
| Thank you for the response. The problem you described has
| been solved by the large vendors with Hyper-converged
| offerings for many years so it sounds like Oxide might be a
| bit late to the party.
|
| I do understand well the rational of running your own servers
| vs hyperscalers, as well as the repatriation trend but I see
| Oxide at best as a niche player.
| liotier wrote:
| Lack of local support does make them a niche player, but
| everything starts from a niche and those who believe they
| don't start from a niche disperse their efforts. So, with
| the hypothesis that Oxyde is smart, the question therefore
| is: what niche is Oxyde focusing on ?
| Voultapher wrote:
| Genuine question, you mention Hyper-converged, can you
| point to anything that comes even close to the experience
| you presumably get from the Oxide offering.
| sbarre wrote:
| Part of the problem here is that the people who make the
| purchasing decisions (like this CTO guy) don't care about
| "the experience" because they're not the ones unpacking
| boxes and plugging in cables.
|
| They pay other people to do that and they don't really
| care if it's a miserable time. And if it takes days
| instead of hours, who cares? Rarely is someone setting up
| a data center under the gun (unless you're Elmo and we
| all saw how that went).
|
| Factors like scalability and ongoing support are much
| more top of mind.
|
| Not saying that Oxide can't address this, and I _love_
| Oxide 's focus on the experience, but I think this
| bottom-up approach to convincing customers is going to be
| a steep climb..
|
| But they seem to be up for steep climbs, so I wish them
| all the best!
| dfc wrote:
| Who is Elmo?
| romanhn wrote:
| Elon Musk
| gregw2 wrote:
| Elon Musk, Twitter/X CEO, went to the data center and
| started unplugging things. You can't make this stuff up!
| https://www.techdirt.com/2023/09/12/the-batshit-crazy-
| story-...
|
| Why "Elmo"? https://www.businessinsider.com/twitter-
| insiders-users-calli...
| mlindner wrote:
| > we all saw how that went
|
| I mean... it went quite well all considering? There was
| some site instability for a brief period of time and now
| it's back to working normally. The initial hypothesis was
| that the data center was vital and so couldn't be shut
| down quickly. Turns out the hypothesis was incorrect. So
| I'm not quire sure that makes the point you're trying to
| make.
|
| Of course Twitter is definitely not all use cases, so
| trying to generalize from one data point isn't a great
| idea in general.
| sanderjd wrote:
| Yeah I think this is the interesting question. My (naive,
| uninformed) interpretation is that Oxide is betting that
| there is some area in the overlap of the venn diagram
| between "people who have reasons not to run their
| businesses on the public clouds" and "people who are not
| suits but technologists and care about the user
| experience of setting up and maintaining their
| infrastructure because they want to do it themselves /
| without hiring lots of people and service contracts".
|
| I'm not sure how big that space is, or whether it is
| likely to grow or shrink over time, but I'm intrigued by
| the proposition they're testing here!
| parasubvert wrote:
| Dell's VxRail has been very popular and successful. Not
| perfect, but pretty good, and I think the market leader.
|
| HPE Simplivity has done well.
|
| Also, Nutanix.
| ricardobeat wrote:
| These seem to solve the 'host your own cloud' problem,
| but are still standard server blades requiring a ton of
| surrounding hardware and maintenance. Oxide is entirely
| integrated.
| hollerith wrote:
| Although it might be true that the existing vendors don't
| provide a good experience, I'm still a little miffed that
| the OP does not even mention the existence of the
| existing vendors.
| sanderjd wrote:
| This I don't get. It's marketing copy, from a small
| upstart. They don't need to link to their established
| competitors...
| dboreham wrote:
| This might boil down to "why does anyone need Stripe when
| there's Visa"?
| mixdup wrote:
| Stripe and Visa are two different parts of the same
| ecosystem though. You can't really replicate Stripe with
| entirely Visa services
| jeffrallen wrote:
| Oxide has customers who have been waiting for real
| integration and innovation from HP, Dell, and Cisco and are
| ready to take a risk on something new.
|
| I set up some Cisco server hardware a few years ago, and
| only by the time I'd managed to order it I was already
| wishing I had a better choice. When it arrived and the
| remote serial was unusable to fix the BIOS ("American
| Megatrends copyright 1984" at 9600 baud? No thanks.) I was
| ready to give up and go back to AWS.
|
| This is a market ready for a kick in the ass, which Oxide
| plans to do.
| yjftsjthsd-h wrote:
| > The problem you described has been solved by the large
| vendors with Hyper-converged offerings for many years
|
| All Oxide has to do to win that market is ship software and
| firmware that doesn't suck, because there are incumbents
| but the incumbents are clearly incapable of doing so.
| count wrote:
| The bar really is sooooo low
| SamuelAdams wrote:
| It's a bit like Apple entering the Bluetooth headphone
| market. Tons of players, but they all suck for a variety
| of reasons. Apple announced the AirPods, which pair
| extremely well with your phone, and they really don't
| suck.
| sanderjd wrote:
| I mean, I don't know anything about this space like you do,
| I'm just an intrigued observer, but my impression is just:
| They are going to try to _compete_ with those large
| vendors. Maybe they won 't be able to! Such is life, and
| indeed is the most common outcome for new ventures.
|
| But precious few new ventures have ever entered a market
| where nobody said some form of "I don't get it, there are
| already established products in this space, how will this
| new thing ever get traction?". And yet, lots of new
| ventures turn out to be competitive with the established
| players in their market, for one reason and another.
| delfinom wrote:
| >I think the motivations for why people would want to own
| their own are probably a mix of financial (at a certain scale
| there's a tipping point and it gets cheaper), and
| regulatory/compliance/whatever, like if it's healthcare data,
| or defense, etc.
|
| Yea, there's definitely a market in defense here. Because
| even though Azure/AWS offer Govcloud, its inadequate for non-
| civilian connected infrastructure. This offers benefits of
| writing "modern software" and deploying it in similar modern
| fashion while keeping it completely running isolated. Imagine
| being able to make your command and control operations
| actually decentralized and not vulnerable to a missile strike
| on a single datacenter.
| chx wrote:
| Maybe you are not. This offering definitely sounds like
| something for on prem and not a large data center. Basically,
| if your core competence is hosting stuff you don't need the
| extra value they provide. But if your core competence is
| basically anything else and just need more than a single server
| under the IT guys' desk then this begins to look very exciting.
| bigironcto wrote:
| You might be right but if a customer won't have the
| size/scale, it won't value the unique proposition from Oxide.
| I hope I am wrong because it would be great to see a new
| player with a fresh perspective in the hardware market.
| Voultapher wrote:
| Bryan Cantril claims the cost of running these could be
| worth it for many small companies, and if you compare it to
| EC2 costs for running let's say a CI I find that easy to
| believe. Maybe it won't be cheaper than Hetzner and co. but
| that's not what they are competing against on a product
| level.
| itomato wrote:
| If you could drop ship a rack of gear to the Colo before,
| with the puny compute and bandwidth potential in that number
| of Rack Units, didn't it just become massively more
| appealing?
| soulofmischief wrote:
| I wonder if they aim to target small operations and startups
| initially.
| chubot wrote:
| You're writing like the status quo is a law of nature. At best
| it's been that way for a decade or two
|
| How many times has computing hardware changed in response to
| the economics of the parts and the economics of the businesses
| buying hardware?
|
| There are downsides to new models, but money solves a lot of
| problems
|
| So I don't know about Oxide in particular, but it seems short
| sighted to bet on stagnation
|
| Also Oxide is doing what Google did 20 years ago, and Facebook
| open sourced ~10 years ago, so it's not exactly unproven
| qaq wrote:
| They literally had CTOs of F100 companies that want to buy this
| gear as part of VCs pitch. Because as you can imagine your
| question was the first question VC's asked.
| candiddevmike wrote:
| Purchasing a ton of hardware from a startup seems extremely
| risky for a F100. It's one thing to be left holding the bag
| when a SaaS startup goes under, but when you just spent
| millions on gear that is now completely unsupported... eek.
|
| I'd be curious to see what companies are interested, Oxide
| doesn't have any logos on their website which is a little odd
| given the space.
| throw0101c wrote:
| > _Purchasing a ton of hardware from a startup seems
| extremely risky for a F100._
|
| If you're working for a megacorp nothing tends to happen
| quickly, so there will be a slow roll-out over a multi-year
| period as old hardware gets phased out and new hardware is
| brought in for a refresh.
|
| If there's a hiccup at any point they'll simply keep the
| previously purchased stuff running and start a new roll-out
| with another vendor next fiscal.
|
| > _I 'd be curious to see what companies are interested,
| Oxide doesn't have any logos on their website which is a
| little odd given the space._
|
| 1. They're just starting out. 2. Some of their customers
| want to be (or start-out initially) discreet:
|
| > _Oxide customers include the Idaho National Laboratory as
| well as a global financial services organization.
| Additional installments at Fortune 1000 enterprises will be
| completed in the coming months._
|
| * https://www.bloomberg.com/press-
| releases/2023-10-26/oxide-un...
| guhidalg wrote:
| F100 waste so many millions already on projects that never
| see the light of day, why couldn't they throw money at
| Oxide and see if it works better than their AWS contract?
| cdchn wrote:
| Megacorp Boondoggles sounds like a lucrative market.
| bee_rider wrote:
| I wonder how to break into it.
|
| I bet the trick is to keep the perfect balance of high
| profile wins and and losses. You want to be defensible as
| an expert to the non-technical folks while obviously a
| fall guy to the technical ones I guess.
|
| I think these guys aren't that, though, they seem to be
| selling a real, cool product.
| dpflan wrote:
| Yes, an industry where FUD and being SOTA is coveted,
| like cybersecurity can really get enterprise interest
| with the right connections. Few contracts of sorts to
| trade for logos on a slide deck and high profile
| references can get you further. Does your product
| actually do anything y is useful? Maybe, not yet, maybe
| never, but you were convincing enough and hey 1/10
| startups fail.
| brundolf wrote:
| > completely unsupported
|
| That equation changes with the entire software stack being
| open-source
| sbarre wrote:
| What about the hardware?
| brundolf wrote:
| In terms of physical hardware repairs or replacement,
| sure, that could be a risk when the supplier is an early-
| stage startup. Though I wonder if the open nature would
| also make it easier to create eg. third-party sleds
| spamizbad wrote:
| The fact that they're willing to absorb that risk is a
| strong signal they're solving a real problem. It's been a
| while since I've worked somewhere with on-prem hardware,
| but I remember long build-outs, unhelpful vendors: A RAID
| card firmware bug bricked our SAN. Our extremely expensive
| support contract gave us front-row seats to finger-pointing
| between the card manufacturer and Dell but ultimately no
| solution was provided to us. Our IT director, who was
| absolutely furious, basically had to twisted Dells arm to
| get them to send us replacement hardware. Whole thing was a
| giant fiasco.
|
| This is the secret none of those existing vendors (Dell,
| Lenovo, HP, etc) are willing to tell you: They have very
| limited technical expertise on what they sell you and
| outside of some specialized troubleshooting they can do,
| they'll defer to their vendors. The understanding is that
| you've got the intellectual horsepower on staff to cope
| with their various shortcomings.
| sophacles wrote:
| I doubt any F100 would go all-in on new vendor like Oxide
| for anything at first. I bet they could spend millions on a
| a few racks as a trial and have some groups work with it. A
| couple years down the line maybe they start expanding usage
| if they like it.
|
| Of course that framing itself is bad - F100 companies
| aren't usually quite that monolithic. By the time they get
| that big there's a heterogeneous set of processes,
| equipment, systems, etc. Some parts of the company may use
| Oxide right away because they see it as a solution, and
| others may keep using the IBM mainframes, and other still
| will keep using racks/blade servers from Dell for eternity.
| aeyes wrote:
| Maybe something like Dell VXBlock didn't exist when they
| pitched their idea?
|
| Any hardware contracts are very long term and you'll have a
| hard time getting me to switch to a different vendor,
| especially when they also want to come in with an unknown
| operating system which I have to run.
| steveklabnik wrote:
| Dell's hyperconverged solutions did exist when the company
| started. We believe we can compete.
| spamizbad wrote:
| When my company looked into it VXBlock looked less like
| fully integrated hardware and software and more like a
| smattering of pre-selected components wired into a box
| ready to go with VMWare with a support contract. If you're
| already in deep with VMWare they're probably great. But the
| software side made my head spin. It looks like Oxide is a
| better fit for orgs that are more IaC.
| qaq wrote:
| I guess time will tell VXBlock just looks like amalgamation
| of SKUs Dell has. Oxide was built as "clean slate" from
| firmware to every minute detail to offer a compelling
| product for companies that want to have a hyperscaler style
| systems for their on-prem workloads.
| aeyes wrote:
| Is it now bad to have a few offerings that you can tailor
| to your needs?
|
| With compute one size doesn't fit all, maybe you need
| more disk space or maybe you need GPUs... I'm sure Oxide
| will come out with different spec modules over time.
|
| The idea is similar: It's a rack which runs virtualized
| workloads and you don't have to think much about
| individual machines.
| strgcmc wrote:
| Essentially this same sentiment, applies to any number of
| things:
|
| - Why would anyone buy the Framework laptop, they don't have
| nearly the support/pedigree that Dell, HP, etc. has?
|
| - Why would anyone use iPhones in the enterprise/IT world, they
| don't have nearly the support/pedigree that Blackberry,
| Microsoft, etc. has?
|
| - Why would anyone use Google Fiber, they don't have nearly the
| network or support that AT&T, Spectrum, etc. has?
|
| - Why would anyone ever use Linux (in enterprise, let's say),
| compared to the support and adoption that Microsoft/Windows
| offers?
|
| - ...
|
| I'm purposely picking different examples with varying degrees
| of success or adoption. I am not claiming that Oxide will be an
| instant category-dominating success. I don't think Oxide
| expects to replace HP/Cisco/Dell/etc. overnight, and I don't
| think a business has to launch with that ambition from the
| start, to prove that it's worth launching.
|
| But this take is so repetitive as to be bordering on cliche --
| I don't know if you're self-aware enough to realize, you are
| literally just a living embodiment of the "Innovator's Dilemma"
| right now...
| endisneigh wrote:
| Your examples are strange.
|
| Framework _is_ niche.
|
| iPhones do have the pedigree.
|
| Google Fiber _is_ barely used.
|
| Most folks _do use_ a supported Linux distribution, they
| don't roll their own.
| leetrout wrote:
| > iPhones do have the pedigree
|
| Not in 2007-2008 which is equivalent to Oxide today.
| endisneigh wrote:
| Apple was an incredibly well established company and the
| initial iPhone was not used in enterprise... it didn't
| really start to take off there until iPhone 4
| (2010-2011).
|
| Not to mention the comparison is inane to begin with.
| Using an iPhone for your enterprise and moving your tech
| infra to a relatively unknown company are not equivalent
| at all.
| mixmastamyk wrote:
| You must have missed the "beleaguered" era.
| slikrick wrote:
| it's so funny to me that you're harping on this analogy
| when it's clear you are so stubborn you won't accept
| anyone else's answer anyways.
| mfer wrote:
| iPhones became popular through the bring your own device
| movement. You aren't going to see that with racks in a
| data center
| FullyFunctional wrote:
| It think it was the other way around. The success of the
| iPhone (and Android) was a large factor behind the BYD
| movement.
| uecc wrote:
| iPhone was truly transformative.
|
| This just seems like a twist on hyper-converged
| infrastructure + open source.
|
| I mean, I love their design. I just don't think it's
| special enough to warrant mass adoption. And I certainly
| wouldn't be deploying at scale in an enterprise
| environment with zero widespread adoption.
|
| Small entities will just use cloud same as always. Large
| companies have a multitude of unique needs that won't all
| be catered for by a single box. The big vendors will
| clean up on that front as usual.
| simonebrunozzi wrote:
| They were not immediately adopted by enterprise
| customers, in fact. It took years for that.
| sanderjd wrote:
| > _Framework is niche._
|
| "Niche" is not a dirty word. Framework is niche, but if
| they can make their cost structure work (which I have no
| idea whether or not they can), _that 's ok_. New products
| don't need to dominate their market in order to be
| succesful.
|
| (But I do agree that Google Fiber is a bad example / a good
| example of a new product in an established market _not_
| working.)
| TimTheTinker wrote:
| You may not be aware of the pain that many large, non-software
| companies currently have on AWS. Gigantic monthly bills
| (hundreds of thousands per month) coming from subdivisions that
| aren't capable or motivated to reduce their AWS budget or
| usage. To the office of the CTO, Oxide's value proposition (buy
| instead of rent) could be _very_ motivating.
|
| "Hey subdivision A, could we buy a few Oxide racks and move
| your workload there from AWS? It looks like they would have all
| the storage and compute you need. Yes? Ok, in 36 months we'll
| pay your current IT department employees a bonus of 50% of
| whatever it has saved us vs your current AWS budget."
| dewbrite wrote:
| Having worked somewhere with an AWS spend of $30k/mo on
| _virtually nothing_, I can attest to this. I think most of it
| was sales demos that never got cleaned up.
| cdchn wrote:
| Converting your OpEx to CapEx is not a remedy for sloppy
| bookkeeping.
| thfuran wrote:
| But if it significantly reduces total cost, it could
| still be a useful bandaid.
| cdchn wrote:
| Most people who can do this (aren't as entrenched in AWS) end
| up moving to a cheap VPS provider so that they don't end up
| having to pay for all the internetworking, facilities, throw
| redundancy out the window, and then still have to pay the IT
| burden to heavy-lift all their workloads to this whole new
| "Oxide" system.
| omarfarooq wrote:
| > end up moving to a cheap VPS provider
|
| Like which?
| cdchn wrote:
| Hetzner, Linode, Rackspace, take your pick.
| bjackman wrote:
| If those subdivisions aren't capable of reducing their AWS
| usage how do you imagine that they are capable of migrating
| to an Oxide rack?
|
| Or in other words, is migrating to Oxide somehow assumed to
| be easier than migrating to some other non-locked-in cloud
| infrastructure?
| TimTheTinker wrote:
| Yeah, my example communication was a bit contrived.
|
| But the point still stands. There's a lot of AWS spend
| happening (even after being optimized) that is frustrating
| when you look at the raw numbers and consider how much
| server capability you could outright _buy_ for the same
| amount. And Oxide would make it so much easier to run a
| bunch of VMs (and infrastructure-as-code) than standard
| racked x86 servers.
|
| Oxide appears to be a complete shoo-in for companies that
| _used_ to run a bunch of VMs on racked Dell /HP servers,
| migrated their VMs/storage to AWS, hate their monthly AWS
| bills, and still have the old server rooms available.
| Nilithus wrote:
| I guess there must be a largish market for this since AWS
| introduced Outpost to provide the "cloud" to onprem industries.
| I feel like this is competing with that market.
|
| Since many of those use cases probably already run extensive
| on-prem infrastructure this could appeal to them. AWS outpost
| talks about industries like healthcare, telecom, media and
| entertainment, manufacturing, or highly regulated spaces like
| financial services. I've heard of media companies that process
| through things like IMAX cameras that have just tons of TB's of
| data sometimes just for 5 minutes worth of footage. That would
| simply be too cost prohibitive - in bandwidth alone - to try
| and move around in the cloud and you don't want to have to wait
| for things like AWS snowball or whatever.
|
| While I think the space is "niche" those niche spaces are not
| small. Big companies with big budgets.
| panick21_ wrote:
| I think they are a competitor to the 'HP/Cisco/IBM/Dell...
| (blades or hyper-converged)' part of this. They basically
| saying 'we will do it better'.
|
| Their marketing and story is supposed to convince you that you
| could save money running their things rather then Dell. And
| instead of paying for VMWare you get Open Source Software for
| most of it.
|
| > Until this company has a certain/size and scale, no one
| serious will trust their black boxes at any type of scale.
|
| I guess that a risk they are willing to take. Some costumers
| might wait for a few years until they see Oxide being big
| enough.
|
| Other costumers might be sick of HP/Dell and might take a Risk
| on a smaller company.
|
| Since they seem to have some costumers, some organizations are
| willing to take the risk to get away from Dell and friends.
|
| So I think you are the target audience but you are not willing
| to risk it until they are larger and less likely to fail and
| they have a good story in regards to support. I assume they
| have a support story of some kind, no idea what it is 'Contact
| Sales' ....
|
| In terms of 'trusting they will continue exists' all they can
| do is survive for a few years until they are pretty
| established, then more people will be willing buy their
| product. And hopefully in that time their existing costumers
| rave about how amazing the product is.
|
| Lets hope they don't go bust because all potential costumers
| are just waiting. Then again, you can't anybody for not buying
| from a startup.
| throw0101c wrote:
| > _Their marketing and story is supposed to convince you that
| you could save money running their things rather then Dell.
| And instead of paying for VMWare you get Open Source Software
| for most of it._
|
| As someone who has dealt with mostly Debian and Ubuntu in
| recent years, every time I had to deal with even small
| numbers of RHEL licenses I often asked myself " _Why do put
| up with this?_ " (I know why, but still... such overhead.)
| johncowan wrote:
| I remember the first time I heard of someone being fired for
| buying IBM, a thing that many people thought would never
| happen.
| panick21_ wrote:
| I think that ship has sailed in the 90s. From the 80s to
| the 90s IBM dropped like 50% of the people that worked
| there and lots of companies were moving away from
| mainframes in droves. I'm sure some people got fired for
| sticking to mainframes to long and wasting money. And
| likely some companies went bust in the late 90s for having
| a IT infrastructure based on IBM Mainframes.
|
| That's mostly speculation but it seems like it has to be
| true.
| algas wrote:
| I agree, it seems the same to me. They have the allure of "we
| do it different" along with the promise of "we do it well".
|
| Unrelated, but a "costumer" makes costumes. I think you were
| looking for "customer".
| itomato wrote:
| What about from an Integrator or VAR? Would you buy it then?
| cdchn wrote:
| I really wonder what their mental image of a product market fit
| is. They're undeniably doing some cool stuff but myself and it
| seems like everybody else has to do some serious mental
| gymnastics to figure out who they sell this to and what needs
| it fulfills or niches it can fit into.
| carapace wrote:
| > Beyond the tech, how would support services really work? We
| can have a technician from any of the large vendors on-site in
| less than 2 hours. In some of our DC clusters we actually have
| vendor support personnel 24x7 on-site with vendor paid spare
| parts inventory. How would they provide that level of service?
|
| Forgive me for being naive, but that sounds like a great way to
| differentiate their offering.
|
| E.g. the famous Maytag Repairman, who sits at his desk doing
| nothing because Maytag washers are so reliable that he has
| nothing to do.
|
| > In a time in which the laundry appliances of major
| manufacturers had reached maturity, differing mostly in minor
| details, the campaign was designed to remind consumers of the
| perceived added value in Maytag products derived from the
| brand's reputation for dependability.
|
| https://en.wikipedia.org/wiki/Maytag_Repairman#Ol'_Lonely
| cryptonector wrote:
| Bryan Cantrill is famously against vendor lock-in. He wrote a[n
| in]famous blog about the "FYO point" while at Sun. Oxide may be
| going for customers that also have the same aversion to vendor
| lock-in.
|
| One thing that Bryan understands is that you can "lock" the
| customer in with great products and services, as well as
| continuing development, while also making the customer feel
| secure in having a way out should you turn into a company that
| treats locked-in customers as cash cows. The open source
| strategy (it is a strategy for Bryan and Oxide) is there
| precisely to do this: make the customer feel they can leave
| you, but then not.
|
| For your deeply technical staff, having source code access is a
| big deal too, since it enables them to better understand the
| products they use.
|
| How big is the market of sufficiently-vendor-lock-in-averse
| customers? I don't know -- that's not my remit. But there's the
| size of that market _right now_ , and whether Oxide (and any
| other companies with similar visions) can grow that market by
| sheer willpower. I make no predictions.
|
| What if Oxide can get the next Netflix to use their stuff
| instead of a public cloud?
| latchkey wrote:
| Oxide is the definition of vendor lock in. All of their
| hardware is unique... even down to the choice of fans. Fan
| burns out? Now you've got to buy another one... from them.
|
| One of the amazing shifts in the last 20 years was realizing
| that commodity hardware, when deployed correctly, could do
| the job.
| cryptonector wrote:
| If their SW and FW source code is MPL 2.0, that's good
| enough to limit the extent of vendor lock-in. Sure, it
| would take time to take over maintenance of that code and
| then add support for different HW and so on, but there can
| be a cottage industry of consultancies that can help if
| ever Oxide vendor lock-in or bankruptcy becomes a problem.
| latchkey wrote:
| No it isn't good enough. This is a hardware play because
| you could theoretically take that software and run it on
| whatever hardware you want. You're not going with this
| business because of their open source software though,
| you're going with it because they are making innovative
| hardware.
|
| If you're buying millions of this stuff, what says that
| you're going to get support for it in 5 years. Who
| knows... maybe Cisco wakes up and gives them an offer
| they can't refuse and then shuts down the company.
|
| By the way, people endlessly gripe about Google
| deprecating things and that's just software...
| sanderjd wrote:
| Their hardware isn't really innovative though... Even the
| hardware integration isn't really innovative, as others
| here have pointed out.
|
| My two cents is that this isn't really an innovation
| business model, it's an _execution_ business model. Their
| proposition seems to be that they can execute a a server
| rack with integrated hardware and open-source software so
| well that customers will love their product.
|
| Maybe they'll just end up with tons of nerds our there
| thinking, "I really _wish_ I could justify investing in
| Oxide racks, but it just doesn 't make sense for my
| business / I just can't sell it to my CIO", but hey,
| maybe not!
| latchkey wrote:
| Solution looking for a problem.
| bcantrill wrote:
| Not sure if we're talking metaphorical fans or literal ones
| but assuming the latter: replacement is covered under
| warranty. And for me personally, the "amazing shift" that
| you describe was in fact a decade-long experiment that left
| me with an inescapable conclusion: commodity hardware
| cannot, in fact, do the job -- and not for lack of trying!
| latchkey wrote:
| Hi Brian, thanks for the response.
|
| Metaphorical fans, but also fans, and everything else. It
| was an example from the blog post. I looked at that super
| innovative and cool backplane. Sure, warranty covers it
| for the first N years, but then what? What happens when
| things turn into a Tesla situation and you get horror
| stories of delays and poor results?
|
| I think 'commodity' is generalized at this point. Given
| the choice of Supermicro and Oxide. I'm going to pick
| Supermicro. They can deliver me a clean rack of machines
| too. Why would I go with Supermicro? Big company, lots of
| products, lots of choices, I can work with them to get
| what I want.
|
| Oxide is too singular. It is one rack, one design, one
| set of specifications. Don't get me wrong, there is some
| value in that... and I'm sure you'll sell plenty of
| hardware, but I'm not finding the value in it for myself.
| That's the part that I mean by 'commodity'.
| ThePowerOfFuet wrote:
| You are ignoring the software stack, which is where a
| huge portion of the value-add is found.
| latchkey wrote:
| I'm not ignoring the software stack, I just don't see any
| value in the additional vendor lock in on it. I'd rather
| use open source stuff developed by a large community of
| people and not a single small vendor tied to a very
| specific and limited hardware stack.
| sanderjd wrote:
| I don't think they're planning on forcing you to buy
| their products :)
|
| As far as I can tell, there exists no "open source stuff
| developed by a large community of people" to run
| integrated server rack hardware. But if you're using such
| a thing, then I'd like to know what it is!
| SamuelAdams wrote:
| > How big is the market of sufficiently-vendor-lock-in-averse
| customers?
|
| Very. Just look at the USA defense spending budget. If you've
| ever worked on AWS-govcloud or secret, you know there's a
| market here.
|
| This has huge use for military too. Imagine having a black
| site or off-grid location but still needing a rack of things.
| What if you could spin up an entire enterprise infrastructure
| by just loading up this rack?
|
| If this team manages to get this thing government certified,
| there's a lot of profit to be had.
| fuzzylightbulb wrote:
| > What if you could spin up an entire enterprise
| infrastructure by just loading up this rack?
|
| This is already being done quite successfully with existing
| offerings. Flexpod, VxRack, Nutanix, there are a ton of
| options.
| sanderjd wrote:
| And now there's another competitor with a different take
| on it!
|
| "Other products already exist" is never a strong
| counterargument to the existence of new products.
| foobiekr wrote:
| You are strongly misinterpreting both how real-world
| customers perceive vendor lock in and how DoD procurement
| works. Everything here is so far off from reality I don't
| even know where to begin.
| kubanczyk wrote:
| > "FYO point"
|
| https://web.archive.org/web/20080705140230/http://blogs.sun..
| ..
|
| And "FYO" stands for Fuck You Oracle.
| say_it_as_it_is wrote:
| Every single one of the companies you listed started off with
| unverified hardware, some that failed in the field and
| continues to today. Why _wouldn 't_ you try something else,
| considering the status quo? This is a nothing-to-lose situation
| as long as you don't put all your chips into the bet. Every
| single datacenter has capacity and a need to diversify. It's
| not even a heroic feat of risk taking.
| alberth wrote:
| Re: who is this for.
|
| My guess, someone who buys hyper-converged infrastructure today
| (e.g. Nutanix, VCE, etc) ... but that market is getting smaller
| and smaller by the day.
| INTPenis wrote:
| To be completely honest, this is for the idealists out there.
| Those of us who are itching to replace our vSphere with oVirt
| because 1) we have the time and skill to do it, 2) we believe
| in open source and 3) we believe we can make huge savings by
| using open source.
|
| I expect the oxide supporters to have a hard few years ahead of
| them of finding bugs in high throughput environments. But at
| the end of the day it will be worth it just to have another
| competitor in a pretty boring playing field.
| user_7832 wrote:
| > however I don't understand who this is for.
|
| I can't think of a lot of examples right now but I can already
| imagine one type of customers for such a product - universities
| wanting high performance computing. At my alma mater, the HPC
| cluster/server was in a different slightly distant location.
| Using something like AWS wouldn't be liked by almost any uni
| admin, and running a server on premises isn't a great idea in a
| place that gets the occasional (but rare) power or internet
| cut. Outsourcing some of these responsibilities may have been
| nice for our admin.
| lucideer wrote:
| > _you obviously want your DCs as standardized as possible.
| Until this company has a certain /size and scale, no one
| serious will trust their black boxes at any type of scale._
|
| This gets to the crux of my first thoughts when I read the
| marketing copy: can they deliver (reliability).
|
| They do admit very clearly that what they're doing is _hard_
| and that at many points during development they were reluctant
| to be too ambitious (for obvious reasons), but at each stage
| they did just that: proceeded with the most ambitious option.
| That takes a huge amount of self belief that might be warranted
| or might be hubris. As you point out, untimely the success of
| Oxide won 't only hinge on ability to deliver on that self
| belief, but more on their ability to convince prospective
| customers that they have competency to do what noone else can.
|
| I fully support every part of their approach in theory but my
| wizened traveled self thinks it smells a little too good to be
| true.
| dclowd9901 wrote:
| I might not be some brilliant CTO, but this product obviously
| seems to fit _very well_ between two very large product
| offerings: fully cloud being one and fully on-prem being the
| other. I've often thought it would be cool if you could _buy_ a
| physical space in a server farm, like literally an area of
| square feet where the racks are all serviced and provided, but
| meets all of the needs a company who doesn't want fully cloud
| has. This is actually way more clever than that.
|
| A lot of people didn't understand the need for the iPad mini or
| Mac mini or myriad other products without an obviously existing
| market. I think this is extremely keen of Oxide to fine the
| borders between these offerings and fit very snugly between
| them.
| lulznews wrote:
| It's an acquihire play. It might be possible to beat the hypers
| with $100 billion and some really good engineers. But building
| custom racks ain't the angle to do it.
| eep_social wrote:
| I think you actually said it without recognizing.
|
| The current state of the art is a fucking train wreck. Choose
| any layer of abstraction and start picking at it and you'll
| find mostly gaffer tape. Scaling up sucks. Rewriting old
| monoliths to micro services wasn't a panacea except maybe for
| cloud vendor profits. You said the hardware market is well
| defined, which is another way of saying ossified. Particularly
| when you start comparing it to the expected pace of software.
| How's Open19 going? How much liquid cooling do you have in
| customer racks?
|
| In my opinion this is ultimately a software offering that
| happens to come on vertically integrated hardware. It offers a
| complete, highly polished API to a minuscule-scale DC. If they
| can find market fit and make a little bit of money, the next
| step is to start making deep improvements to things behind the
| abstraction. From where you sit, you are well aware that you
| could make huge improvements inside the DC demarc if you could
| do it without disrupting customers. But you're probably limited
| by the terrible, terrible APIs you would be forced to use, that
| don't offer the capabilities you need, from vendors who would
| be happy to chat but would provide timeframes in the 6-18 month
| range for even a modest improvement.
|
| So this, IMO, is about defining that interface to a DC on a
| single hardware platform with vertical integration, and then
| scaling up from there. The current hyperscalers will adopt the
| suite of APIs and capabilities directly, make lower quality
| copies, or die.
|
| Of course all assuming they're successful enough to get the
| revenue machine going in the first place which seems likely to
| me, given the absolute dogshit state of the cloud world today,
| the trend towards multi cloud, and the business case for moving
| certain loads back on prem or to a bare metal colo++ offering.
| paxys wrote:
| Regardless of how good the product is, are companies really going
| to trust a startup with 100% of their hardware, software and
| support dependencies with no outside ecosystem and no
| interoperability with any of their existing infra?
| bobthecowboy wrote:
| I work for a competitor that has actually been shipping a
| similar product for quite some time now.
|
| Yes, companies are really going to trust a startup for a stack
| like this. Will they go all in for 100% of their infra? Of
| course not, but as a test against a similar infra stack at a
| lower cost with an appealing feature set? Why not?
| kristianpaul wrote:
| I definitely want to know more on their Kubernetes, this a
| software not designed for on-prem so i'm curious on their stack
| choice
| digitalsanctum wrote:
| Congrats to the Oxide team. I'm not a potential customer but
| still get a thrill from the open source projects from these
| folks.
|
| My favorite is the automated CIO repo:
| https://github.com/oxidecomputer/cio
| TrueDuality wrote:
| Wow! Congratulations! I've been following you guys for a while
| and have loved what you've been doing. Really want to get my
| hands on some of this hardware to play with.
|
| I'd love to see a video exploring the rack and showing the first
| time setup.
| deweywsu wrote:
| For someone who's not familiar with what you get from AWS, this
| appears to be similar, on the surface anyway. Can someone explain
| how it's different?
| steveklabnik wrote:
| AWS rents you time on their servers. We sell you servers.
|
| The difference is you own the hardware.
| fuzzylightbulb wrote:
| You can hire people that actually know how to drive business
| outcomes using AWS.
| nkko wrote:
| I am supper impressed with how well-rounded they are. Wherever
| you look from git repo to the website and everything in between.
| rvz wrote:
| I'm very skeptical on this despite so much praise yet no-one has
| used it.
|
| It seems the only reason why people are getting too excited is
| because of the team members and the name.
|
| What if this venture becomes yet another VC pump and acquire scam
| to get you to purchase a massive room heater?
|
| They better not get themselves acquired like the rest of the
| hyped up VC scams out there.
| breatheoften wrote:
| I discovered their podcast a month ago and binged through the
| backlog -- it's one of the best technology podcasts of all time!
| Really like the way these folks think and communicate!
| dilippkumar wrote:
| As a long time follower from the first "On the metal" podcast and
| as someone who has been cheering for you silently from the
| sidelines - Congratulations!
|
| You all have arrived. Wishing you decades of prosperity and good
| fortune.
| flumpcakes wrote:
| I think a lot of commenters here have little experience with
| hyper-converged infrastructure if they don't see this is
| different. This looks, by far, the simplest way to run workloads
| outside of the big cloud providers (AWS, Azure, GCP).
|
| It seems they only support up to virtual machines but this is
| still miles ahead of VMWare, Nutanix, or CISCO ACI to my eyes.
| This honestly looks simple enough that a developer/DevOps team
| could manage it. With the other 'hyperconverged' infrastructure
| offerings you will still need a dedicated team of trained ops
| professionals to manage it.
| layer8 wrote:
| It may be a limited market. You need to be large enough for a
| private cloud to make sense, but above a certain size it also
| should be a no-brainer to have a competent ops team making you
| less dependent on a singular infrastructure provider.
| dmix wrote:
| Maybe 37signals type businesses, SMB tech startups with a
| serious, stable, non-high growth business model and a tech
| savvy team. If you're selling $500k boxes you don't need to
| sell a TON of them to make a hundred million dollar business.
| Unless their VCs expect Oxide to be a billion dollar company
| with a broader market.
| whizzter wrote:
| With 44m in financing they probably expect a billion dollar
| company. https://oxide.computer/blog/oxide-unveils-the-
| worlds-first-c...
| Nilithus wrote:
| Perhaps this changes the calculus. If you don't need as much
| of an ops team to go private cloud. Then you might not need
| to be as large before it makes sense.
| layer8 wrote:
| By a private cloud making sense, I mean the lower bound for
| being an Oxide customer, i.e. having the budget to buy the
| infrastructure with enough headroom for usage spikes and
| expected near-term growth. The reasons why you otherwise
| would go public cloud when you're still small. The reduced
| need for an ops team with Oxide was already factored in.
| novok wrote:
| So uber did private cloud, then went to public cloud
| recently because the balance was not worth it ( and they
| probably had enough scale to negotiate a good price, like
| netflix ) . If they were on something like oxide which
| reduced infra management costs a lot staffing wise and
| negotiated a bulk deal, would've they gone the public
| cloud route?
| Guvante wrote:
| The hardware only lasts five years at most. That means you
| have a decent cadence of repurchases.
| tryauuum wrote:
| I hate all the big clouds and never use them. Yet it was hard
| to me to get their selling point, I had to read like for 2
| minutes
| parasubvert wrote:
| I have a lot of experience with hyper-converged infrastructure.
| "Miles ahead" how? Oxide's control plane barely competes on
| basic functionality that OpenStack had 13 years ago, let alone
| VMware which is miles ahead of OpenStack. The hope is that the
| simplicity in hardware and network/storage/compute architecture
| will drive dividends as they improve their control plane
| software.
|
| I don't discount that it's a great achievement towards
| simplicity in the racking approach for rack-scale use cases
| (having lived through Dell VxRack's nonsense), but let's not
| kid ourselves that any DevOps team could manage this - do they
| understand BGP peering? Three phase power requirements?
| Cooling? ZFS? How about basic maintenance migrations ala
| Vmotion or DRS? (kidding! they can't do it)
|
| If you don't think Oxide will require an ops team, you may be
| wish projecting.
| omneity wrote:
| This point is understated. One of the main reasons cloud is so
| attractive to executives is the smaller devops footprint
| compared to a traditional on-premise deployment.
|
| A solution as lightweight on operations as Oxide challenges
| this premise at its core and assuming the capex is not
| outrageously high, it might suddenly get very attractive.
| fishtoaster wrote:
| > This honestly looks simple enough that a developer/DevOps
| team could manage it. With the other 'hyperconverged'
| infrastructure offerings you will still need a dedicated team
| of trained ops professionals to manage it.
|
| That seems like the best explanation of the value here I've
| heard so far! I could definitely see a number of SMBs whose use
| cases are expensive in AWS/etc and _would_ be a good fit for a
| few racks in a datacenter, but who don 't want the cost of a
| team to manage them. If this significantly lowers the skill
| threshold necessary to run servers in a datacenter, I could see
| that being a huge value prop.
| mstade wrote:
| It bugs me more than it should that they flirt with hexadecimal
| numbers in their branding with the whole 0x thing, but the logo
| is 0xide when clearly it ought to be 0x1de. Designer came up with
| a clever idea but didn't understand hexadecimal, and no one had
| the heart to tell them? :o)
|
| I'm kidding, mostly. The engineer in me is bothered by it, but
| the part of my brain that cares more about design and branding
| understands that 0x1de would cause inconsistencies in the
| branding elsewhere. (E.g. 0xDocs: https://docs.oxide.computer)
| nwsm wrote:
| 0x1de would be great branding for some kind of integrated hex
| editor (hence IDE)
| brucepink wrote:
| You'll like https://admin.pci-ids.ucw.cz/read/PC/01de then.
| mstade wrote:
| Haha indeed I do! :o)
| nicwolff wrote:
| https://0x1de.com redirects to https://oxide.computer so they
| did think of it.
| benjaminleonard wrote:
| You may be happy (or unhappy) to know that was definitely
| considered - it felt a little more trouble than it was worth,
| and dare I say a little too on the nose?
|
| We did manage to get a neat PCI vendor ID:
| https://pcisig.com/membership/member-companies?combine=01de
| mstade wrote:
| Haha yes, definitely a little too obvious. I love your
| branding by the way, great work!
| kayge wrote:
| "we needed a name -- and once we hit on Oxide, we knew it was
| us: oxides form much of the earth's crust, giving a connotation
| of foundation; silicon, the element that is the foundation of
| all of computing, is found in nature in its oxide; and (yes!)
| iron oxide is also known as Rust, a programming language we see
| playing a substantial role for us. Were there any doubt, that
| Oxide can also be pseudo-written in hexadecimal -- as 0x1de --
| pretty much sealed the deal!"
|
| from https://bcantrill.dtrace.org/2019/12/02/the-soul-of-a-new-
| co...
| throwaway892238 wrote:
| There's a fundamental misunderstanding of what The Cloud is that
| seems to pervade discussions like this. Too many technical people
| don't understand what they're looking at.
|
| The Cloud is not "a computer". The Cloud is a public utility. You
| don't rent the transformers at an electric utility company -
| because why the hell would you? It's just one component of a much
| larger system that is valueless to the consumer without all the
| other components.
|
| What these people are selling is more like a battery that you can
| purchase that plugs into the electric grid. But why would you
| want to purchase a battery? It degrades over time, losing value,
| and eventually needs to be recycled, and somebody has to deal
| with all that. Renting is the perfect way to push all of that
| time-consuming and complex maintenance out to someone else who
| can lower cost with volume.
|
| The idea that renting servers is not sustainable would suggest
| that somehow computers are not like housing, cars, shop vacs, or
| any of the million other things you can rent. A computer is an
| expensive commodity like any other, and renting makes perfect
| sense a lot of the time.
|
| If you need to purchase, for economic, load, tax, regulatory, or
| other reasons, there are already ready-made computers with (or
| without) service contracts and supported OSes that can be plopped
| into any colo, and colos sell dedicated machines already. This
| pitch doesn't include anything new other than buzzwords. Yeah I'm
| sure they did a bunch of engineering - that nobody will ever
| need. I'm calling shenanigans.
| 0xbadcafebee wrote:
| Oh. I get what this is. It's a "cloud" mainframe. They're trying
| to be the Apple of mainframes.
|
| Reinvent OpenStack, slap it on some 8Us and storage arrays, shove
| it in a rack, and ship it to a colo with a professional
| installer. So basically one of the larger server vendors but with
| integrated "cloud" software, minus the 2-hour service turnaround
| and spare parts.
|
| The fact that they're writing the software from scratch is going
| to add years of lead time until they reach parity with other
| solutions. My guess is they're hoping they can get sticker price
| or TCO low enough that it outweighs the lack of functionality and
| uncertainty of a brand new _everything_. If you just need some
| VMs in a lab in the office closet, might work.
|
| They only have $44M to build that company? I hope the next
| funding round is better :/
| steveklabnik wrote:
| > slap it on some 8Us and storage arrays,
|
| This does not accurately represent the amount of hardware
| design we have done, let alone the software work.
|
| > The fact that they're writing the software from scratch is
| going to add years of lead time
|
| What lead time are you referring to?
| singhrac wrote:
| Will Oxide build a GPU rack as well? Asking for a friend.
| steveklabnik wrote:
| We do not currently have plans for a GPU-focused product, but
| it is on our radar.
| dilyevsky wrote:
| Hey Steve, what is Oxide solution for multi-rack deployments?
| If i need to build spine-leaf do i have to byo spine gear or
| is it taken care of?
| steveklabnik wrote:
| Details forthcoming :)
| cj wrote:
| Wow, they've raised a total of $78 million, and they're only
| Series A!
|
| I remember a time when a Series A was rarely above $10m. Feels
| like at that capital level they should be further into the
| alphabet.
| 0xbadcafebee wrote:
| The business they're trying to build is capital-intense, and
| they're basically writing an OS from scratch, in addition to
| hardware from scratch. 78 million is actually really low
| Fnoord wrote:
| OS is not from scratch but based on proven tech like Solaris,
| Bhyve, Dtrace, Rust, etc. Stuff they either wrote themselves
| or are otherwise very familiar with.
| 0xbadcafebee wrote:
| I meant the "cloud" components, which typically represent a
| quasi distributed OS (because real distributed OSes don't
| exist) and thus are complex to create and maintain. But
| that's, uh, concerning, that they're also using Illumos. On
| the one hand, they're guaranteed to understand the system
| well, because they're probably the only people on the
| planet who do... On the other hand, makes it harder to hire
| for, port software to, or later sell the company due to the
| risk of maintaining such a one-off stack. But Joyent was
| able to pull off a big exit after almost a decade, so maybe
| these folks can too.
| SteveNuts wrote:
| Does anyone know if they plan to support bare metal machines in
| the future?
| vasco wrote:
| Not having a picture is criminal.
| bittermandel wrote:
| I have been waiting for this for a while. Their OSS code is of
| the highest quality and I'm hoping the hardware is at the same
| level!
| BD103 wrote:
| I follow a person on Github who works here! (cbiffle[0]) They
| wrote a fantastic series[1] on learning Rust from a C perspective
| and I highly recommend it.
|
| [0]: https://github.com/cbiffle [1]:
| https://cliffle.com/p/dangerust/
| brynet wrote:
| Congrats bcantrill, et al.
| cmdrk wrote:
| send one my way to evaluate for HPC workloads, thanks.
| __xor_eax_eax wrote:
| I feel like this company misses the point of cloud computing. Its
| mostly cost elasticity, not where your servers are physically
| located.
|
| This feels like the rackspace approach
| alberth wrote:
| The irony is that Oxide caters to the HN crowd, but the HN crowd
| would _not_ be the buyer of their offerings.
|
| Old school CIO's, who make enterprise decisions without developer
| support, are typically the buyer of hyper-converged
| infrastructure (or Utility companies who struggle with CapEx vs
| OpEd).
|
| I wish them luck. There's a bunch really good folks over there.
| asadm wrote:
| Depends. HN crowd will likely prefer this over existing ones in
| their day job.
| Fnoord wrote:
| > The irony is that Oxide caters to the HN crowd, but the HN
| crowd would not be the buyer of their offerings.
|
| Rust, Oxide, and iron-y.
| vercantez wrote:
| Brian Cantrill is a legend. We're gonna need a sequel to
| https://www.youtube.com/watch?v=tDacjrSCeq4. Very happy to see
| them ship.
| cashsterling wrote:
| I've been following Oxide since they formed and I really hope
| they crush it.
|
| Semi-sort-of related... there was an automation company, Bedrock
| automation, that went defunct about a year ago. Their PLC
| hardware ideas were dope but I always felt like they were missing
| the boat a little bit by supporting stale PLC programming
| languages. I used to wonder if supporting Rust and Ada on these
| PLC's would be been a good idea to diversify/specialize into
| complex control system domains. Also, iirc, Bedrock didn't
| support EtherCAT which I felt was a mistake.
|
| Anyhow... one of these days it would be cool for a forward
| thinking company like Oxide to tackle what a modern complex,
| distributed control system hardware/software stack should look
| like using Rust/Hubris.
| PaulWaldman wrote:
| The industrial automation industry is highly risk adverse. In
| some systems, like safety, this mentality is justified without
| proper migration plans. In other areas, it's alarming, like
| choosing to run a system on hardware that has been out of
| support for 30+ years without adequate spare parts.
|
| Operation/Shop floor technologies (OT) are treated like
| mechanical equipment, "When it fails, we'll swap it out for a
| new one." Well, this isn't a motor, it has programming, and it
| interfaces with I/O sensors and devices.
|
| The main challenge is the lack of knowledge and skills in
| modern technologies among technical staff and decision makers
| in industrial organizations.
|
| A an aside, in my early days I hopped on a 5-hour flight and
| drove 6 hours to replace a failed hard drive in a Windows NT
| machine used as an HMI. Then a year later, replaced it and all
| the other clients with a vSphere stack. The local resources,
| both internal and external, were too intimidated to touch it.
|
| I'd be in favor of a "reset" in automation, it feels like
| fighting city hall.
| cashsterling wrote:
| I agree with everything you said... but there are a lot of
| areas in control system engineering that would benefit from
| having really robust hardware designs coupled with guaranteed
| correctness that Rust/Ada provide, and then all of the upside
| of all of the advanced control programming that is relatively
| easy in a modern programming language.
|
| Beckhoff & B&R Automation are part of the way there. They
| both have some really slick portfolios. IMO, on paper, these
| are the two most advanced control system companies (with
| large sales volume).
| Aerbil313 wrote:
| What is the real solution for highly complex technology like
| IT, as end times are coming?
|
| It's a critical issue. Resources are wasted, and soon we
| won't be able to waste as much as we did, as the abundance
| era is coming to an end. The desire and need for higher
| efficiency through highly complex technology is only to
| increase, as the population gets older and older.
|
| I can easily see mandatory IT training (like the current
| mandatory army training in some countries) and states
| standardizing certain technologies to avoid fragmentation and
| thus freezing progress, for good reasons. Like forbidding any
| database to be used in production except 3 most complete ones
| or banning all web frameworks except React and Svelte
| (arbitrary). It's not that they won't want progress, but they
| won't be able to afford progress and the resulting skill
| fragmentation in the IT workers while people are hungry and
| money is ever tighter.
| vineyardmike wrote:
| I feel like a lot of people miss the point on PLC stuff.
|
| "Stale PLC programming languages" might be stale and in need of
| a rework, but "rewrite in rust" entirely missed the value
| proposition. Memory safety is important, sure, but these
| systems aren't usually ever allocating memory or taking actions
| that aren't deterministic. PLCs usually operate in _strict
| realtime_ environments. Like each operation contains a known
| number of clock cycles, and is timed to match physical "stuff"
| moving in the real world. To rewrite in rust, you'd need a
| flavor of rust where you can deterministically know the timing
| of ops and calibrate the timing of each branch of any control
| paths.
|
| That said, there is a market for softer realtime PLCs and
| companies like Rockwell sell Windows IoT hardware- but they
| still contain a strict-realtime companion controller. Arduino
| is now selling a mountable PLC that can be programmed using
| their tools, but even they support traditional PLC programming
| languages.
| cashsterling wrote:
| Yep... I programmed PLC's for periods of my career. I could
| write about this until my fingers fall off, but in short:
|
| Rust and C code can also operate in strict real-time embedded
| environments provided some basic rules are followed like "no
| dynamic memory allocation". This done all the time.
|
| If one follows hard real-time coding standards like Misra C
| or "JPL real-time C" (and these standards can also be applied
| to Rust, Zig, Ada, etc.) the code will run deterministically
| on a given target... just like PLC code running is not
| necessarily any more deterministic than C code (no dynamic
| allocation) running on an embedded processor. In fact, many
| PLC's today run a "PLC code execution" engine on top of a
| real-time operating system like QNX (which I think is mostly
| C code). Even in some older PLC's, it is still C firmware
| that's interpreting and running the PLC code.
|
| PLC programming "languages" (ladder logic, FBD, CFC) were
| designed for their programming audience, who are not software
| engineers. It is difficult to represent complex logic and
| numerical code in these languages and that limits the
| sophistication of algorithms that can be implemented on these
| systems. For instance: try writing a model predictive control
| routine in ladder logic; I think it could be done but I'd
| lose my remaining hair doing it.
|
| PLC Structured Text is very similar to Pascal and pretty
| capable, but folks generally don't write model predictive
| control algorithms in Structured Text either.
|
| It is hard to orchestrate multiple PLC to run as a cohesive,
| deterministic unit. The interfaces between PLC often need to
| be kept simple and sometime this communication is less real-
| time. Things are a lot more advanced in cluster computing.
|
| The limitations of current PLC architectures is already a
| pain point for complex control system, like large robotic
| manufacturing lines or optimal control of HVAC in large
| buildings)... and it's going to get worse as performance
| demands increase. Again, as mentioned above, I think Beckhoff
| and B&R are further along the evolutionary path than others
| in the industry.
| gigel82 wrote:
| That's an interesting juxtaposition to Microsoft's move to "cloud
| computer" which is a thin client with everything (including the
| OS) streaming from the cloud.
|
| Or maybe it'll be a curious unintended match instead, with
| whatever "streaming" devices Microsoft pushes next being
| connected to one's own "cloud computer" instead...
| siliconc0w wrote:
| I really appreciate the vision and the technology (also the
| dedication to opensource).
|
| The difficulty is that HP/Dell/NetApp/etc are well established
| and maybe if you're a SME you can appreciate the benefits of an
| end to end integrated solution but most of the people making
| purchasing decisions largely don't. It's easy to market on CapEx
| but difficult to market on OpEx - you're asking customers to
| (likely) pay more for a similar performance invariably less
| featureful solution from a small company which they're then
| locked into largely leaning on the promise it'll be easier to
| operate (where is your GPU integration, object storage, shared
| filesystem, integrated kubernetes, etc). They then have to
| convince all the incumbent software vendors to support their
| software (e.g SAP/oracle/etc) on this specialized hardware.
| addisonj wrote:
| I have been following oxide for a bit, and really don't have to
| add to the tech conversation, but do want to say:
|
| Congrats to the team on reaching this big milestone and (in my
| eyes at least) just as much congratulations on doing it in a way
| that has been unique and sticking to values that seems to drive a
| strong positive culture (at least from the outside looking in).
|
| Shipping products is _hard_ , and only getting harder. IMHO, one
| of the big drivers of that is just how complex every market has
| became. Building and selling software alone is so much more
| multi-disciplinary than it was 10 years ago and adding hardware
| to the mix is upping that by a huge factor. As I look around, I
| see so many companies struggle to build teams that can handle the
| huge range of required tasks. To see a company like Oxide that
| (once again, from the outside a least) seems to have things
| together on so many fronts, especially while doing it while
| sticking strongly to some core values, is pretty inspiring.
|
| Not to get overly cynical, but I don't think it is an extreme
| opinion to say that current start-up culture feels like you have
| to make big compromises in what you believe in order to be
| successful. Whether that be open-source, how you value and pay
| employees, or even just rushing things to deliver that aren't
| ready.
|
| While I acknowledge Oxide has some well-connected, experienced
| founders that I am certain enabled them to get the resources and
| trust to do things their way, I really hope they kill it so that
| other founders and builders can learn that you still can build
| not just financially successful products, but great organizations
| that truly care about their values.
| bcantrill wrote:
| Thank you very much for the kind words! It has been important
| for us to do things the right way and to be a model for others
| -- so it's really meaningful for us to hear that that's
| appreciated; thank you!
| J_Shelby_J wrote:
| > Shipping products is hard
|
| And shipping full cabs is harder!
| hn_throwaway_99 wrote:
| Yeah, I don't honestly know much about the hardware side of
| things, nor the business/economic rationale for why one would
| prefer an Oxide rack over a more conventional setup, but
| reading through their posts, what they've done is just
| singularly _so impressive_ that I really, really, really want
| them to succeed.
|
| I've worked in startups most of my life, but it's tough feeling
| like you've got to take a bunch of shortcuts ("MVP"-it before
| you run out of runway, etc. etc.), so to see a startup just put
| a ton of real engineering prowess into a great product is a
| sight to behold.
|
| Also, while there are obviously amazing products out there in
| the world, it's so often a "race to the bottom" that it feels
| like decently made products are just prohibitively expensive.
| I'm really into antique espresso machines, and there is a
| reason that seeing rebuilds of beautiful old lever espresso
| machines get so many oohs and ahhs online - they truly don't
| make them like they used to, so it's so cool to see these
| beasts brought back to life. For some reason I have a similar
| feeling looking at these Oxide racks - they just didn't cut
| corners to build an awesome machine. Again, not qualified to
| say whether that makes economic sense long term, but it's still
| a thing of beauty regardless.
| awoejtraor wrote:
| Correct me if I'm wrong, but it seems like these guys just
| reinvented mainframes.
| husamia wrote:
| I was reading until I saw green and some type of rack looking
| thing, wait I thought it was not a rack! seriously, great job.
| let me own it first.
| daviddever23box wrote:
| Love it.
| anyoneamous wrote:
| As a HPC guy, I really like the idea of one or two of these racks
| serving as the cluster front end, providing the login nodes,
| controllers, etc. You would still need generic servers for the
| main bulk of the compute for density and cost reasons, but it
| would basically make the back-end part of the cluster into an
| interchangeable blob of cores, allowing us to focus on the
| interesting bits of running a service.
| novok wrote:
| With the rack being so quiet, I wonder if they would ever sell
| these in smaller versions for more local installations. Would be
| pretty niche, but it would be interesting. Same kinds of markets
| that ui.com tries to serve.
| Fnoord wrote:
| Yeah the rack is custom designed to be very quiet. And it was
| early mentioned in the article. I'm sure they thought it
| through... but why? Do they want people to store this in their
| office workspace? It seems like a niche to me? For smaller
| embedded devices like home appliance and the like, fanless or
| the option to have low or no fan speed (like modern GPUs) seems
| to make sense. Because they're stored in the household or
| office. But servers are not and if they are they're still
| stowed away or fans are replaced with less noisy ones. You want
| the fans as noisy as possible to generate as cold as possible
| hardware which helps to increase heat dissipation for improved
| performance before any throttling.
| wpm wrote:
| >That the rack is quiet wasn't really deliberate (and we are
| frankly much more interested in the often hidden power draw
| that blaring fan noise represents).
|
| They optimized for power. That roar of fan noise doesn't
| necessarily mean they're doing a better job cooling.
| Fnoord wrote:
| So for power and therefore noise. Why though? I can see how
| it is interesting given energy crisis plus not going to
| squeeze out max W for performance at the cost of energy. If
| that is the reason, then it is going to be more cost
| efficient than competitors on the longer term, at the cost
| of perhaps requiring more hardware to possibly scale up.
| josephcsible wrote:
| They say this:
|
| > While the software is an essential part of the Oxide cloud
| computer, what we sell is in fact the computer. As a champion of
| open source, this allows Oxide a particularly straightforward
| open source strategy: our software is all open.
|
| But their homepage says this:
|
| > As soon as power is applied to an Oxide rack, our purpose-built
| hardware root of trust - present on every Oxide server and switch
| - cryptographically validates that its own firmware is genuine
| and unmodified.
|
| That sounds like tivoization to me. The value of their software
| being open source is heavily diminished by their hardware
| refusing to run anything but their "genuine and unmodified"
| software.
| avhception wrote:
| Not sure about the tivoization. You could never send a pull
| request to the Tivo repo on Github, because it does not exist.
| And maybe the Oxide hardware allows you to change the
| certificates so you can roll your own?
| meithecatte wrote:
| I suppose we need to know if it's secure boot (tivoization), or
| verified boot (remote attestation).
| mlindner wrote:
| From my own understanding its neither (unless by "remote" you
| don't mean Oxide). From my understanding there's no
| involvement of Oxide in running the computer. You should
| never need to talk to them to configure anything nor should
| you ever need to talk to them if you want to flash the
| firmware with something else entirely.
| necovek wrote:
| Canonical did something similar back in 2014 but mostly for demo
| purposes with "Orange Box": the promise was that you could easily
| do it with any available hardware.
| lxe wrote:
| Am I the only one who's confused by the offering? Are they
| selling cloud hosting or servers?
| steveklabnik wrote:
| We are selling servers. You interact with those severs like you
| would a cloud, but it is hardware you own.
| lxe wrote:
| Thanks for clarifying. Do I physically connect the server to
| the Internet or is it colo'd somewhere you provide?
| syntheticgate wrote:
| We ship you the whole rack you bought, either to your own
| on-prem datacenter or your preferred colo partner etc. Once
| on-site, assuming the infrastructure is already in place,
| it's ~3hrs to uncrate, connect power and network, and do
| the site-specific install configuration (like DNS, time
| servers, identity etc etc).
| exposition wrote:
| Has anybody else used Rust for embedded?
|
| Would be interested to hear your experience.
|
| So far, I've seen mixed reviews. Some say that you can end up
| using unsafe a lot and so it's better to stick with C or even use
| Zig.
|
| Wondered if there was any merit to this.
| Aurornis wrote:
| > Some say that you can end up using unsafe a lot and so it's
| better to stick with C or even use Zig
|
| Using unsafe in Rust isn't inherently bad. If you're doing
| embedded work, using unsafe is mandatory once you get to the
| level of interacting with the hardware.
|
| That doesn't mean the code is literally unsafe, just that the
| interactions are happening in a way that the compiler can't
| guarantee. That's completely expected when you're poking at
| hardware registers.
|
| You still get most of the benefits of Rust, the language. I've
| had good success with it.
| exposition wrote:
| Yep I get that- it's just that using unsafe reduces the
| benefits you get from Rust's model.
|
| And so I was wondering whether using it is still worth it.
|
| Obviously, it probably still has many benefits over C but
| something like Zig might be simpler and better suited.
| Aurornis wrote:
| > it's just that using unsafe reduces the benefits you get
| from Rust's model.
|
| No, you still get the benefits where it matters.
|
| The "unsafe" is basically marking a boundary where you're
| doing things outside of what the compiler can verify. If
| you're poking at hardware registers, that's expected and
| normal.
|
| Putting "unsafe" in a program at the hardware boundary
| doesn't reduce the benefits of Rust elsewhere in the
| program.
| musha68k wrote:
| Shining beacon of hope.
| _a_a_a_ wrote:
| Without JS this site's text is a weird mix of upper and lower
| case letters, even within the same word. Even large lowercase
| mixed in. The underlying text is perfectly normal. Can any web
| devs suggest why this is? TIA
| wmat wrote:
| What am I missing here? Hasn't SoftIron been building this exact
| thing for around 5 years? Heck, they design and manufacture all
| of the hardware as well.
|
| https://softiron.com/hypercloud/
|
| https://softiron.com/blog/run-bmc-why-we-decided-to-build-ou...
| qaq wrote:
| Not being familiar with SoftIron but I would imagine there can
| be more than one company working in a given niche? Why would it
| be surprising?
| Aachen wrote:
| The blog says they're the first
| bobthecowboy wrote:
| Correct. I work at SoftIron. We have several HyperCloud
| systems in production right now. SI has been shipping
| purpose-built storage systems for years as the root-comment
| suggests, but HyperCloud (which is closer to Oxide's
| product) has been in production systems in defense,
| banking, internationally for well over a year now.
| glandium wrote:
| It looks like SoftIron is not shipping a rack in a single
| box, though.
| bobthecowboy wrote:
| I'm not sure exactly what you mean.
|
| I believe our smallest configuration is 5 servers and a
| few switches.
| Always_Anon wrote:
| Oxides biggest selling point is the API, which can be
| manipulated with Terraform. I see no SoftIron Terraform
| plugin.
| bobthecowboy wrote:
| We do have a Terraform plugin in active development. It's
| good enough that I use it every day, but in fairness it
| isn't released yet.
|
| Our API is also under active development, but it is open
| source today:
|
| https://github.com/SoftIron/hypercloud-api
|
| Go bindings included, other languages are under
| discussion.
| jrexilius wrote:
| Thanks for pointing this out! I wasn't aware of SoftIron. I
| think its a big deal that there are two vendors on this path.
| And both seem to be doing it the right way. I think it makes
| the whole, stronger than the sum.
| stonogo wrote:
| When Hewlett-Packard released a similar product (rack-integrated
| compute), it was hard to tell if the backplane bandwidth killed
| it or the terrible Java management software killed it. It looks
| like this has a better design for each.
|
| There doesn't seem to be a way to provision a bare metal
| operating system, so HPC is out, and the networking is previous-
| generation, so there are two opportunities for progress right
| there.
|
| Now that they're VC-funded I expect an OEM to snap them up before
| either opportunity can be pursued.
| fuddle wrote:
| Do they support GPU's?
| steveklabnik wrote:
| Not currently, maybe someday.
| 867-5309 wrote:
| are they selling something? can we buy it?? it's anyone's guess..
| steveklabnik wrote:
| We are selling racks of computers, and you can buy them, yes.
| ianceicys wrote:
| Dumb question: How is Oxide different than AWS Outposts or Azure
| Stack HCI?
|
| https://azure.microsoft.com/mediahandler/files/resourcefiles...
| mad_vill wrote:
| iirc azure stack hci is only 1U and 2U configurations, not a
| full 32U rack.
| mlindner wrote:
| I don't know enough about Azure Stack HCI, though I expect it's
| similar. For AWS Outposts you can't actually own the hardware.
| It's hardware leasing on your own premises. More so, it's a
| black box that you can't inspect, while Oxide's firmware is all
| open source. You can even flash your own modified firmware
| without going through Oxide.
| abbbi wrote:
| just fooling around with their demo web console ..
|
| "A disk cannot be added or attached unless the instance is
| creating or stopped." "A network interface cannot be created or
| edited unless the instance is stopped."
|
| really?
| alberth wrote:
| It's a nitpick but I thought its strange in the web console,
| main left navigation bar, it saying "Access & IAM".
|
| Because IAM = Identity & _Access_ Management
| abbbi wrote:
| well, i just wanted to get a feeling what experience it would
| be using their web console and it seems they still have some
| limitations in place that have long been solved by other
| "hyperconverged vm-deploy" players.
|
| Want to spin up not only 1 but 20 instances via the console?
| Nope. Any kind of guest agent in the virtual machine? For
| direct interop? How is the guest filesystem freezed during
| snapshot creation? Checkpoints for incremental vm backup?
| They seem to use raw images, not qcow2? Why?
|
| etc..
| alberth wrote:
| They have 14-years of catch up to do, to be parity with
| Nutanix and the likes.
|
| Not being negative ... just calling out this is offering is
| super late to the game.
| wpm wrote:
| You are being negative though. You could have phrased
| that as "they are still new, but I hope they can catch
| up", which doesn't imply they messed up by being late to
| the game.
|
| Disruptors never have feature parity to established
| players in their first offering.
| PeterCorless wrote:
| Congrats, Bryan and team! I've been following your evolution for
| a while now. People might just look at it and say "It's just a
| rack of CPUs and storage." And I can only imagine just how much
| you might be tempted to throttle them (or at least flame them in
| online posts).
|
| Over the decades the separation between "software" people and
| "hardware" people has grown. With "cloud" people have grown
| comfortable papering over poor basic performance by abstracting
| away even your visibility into how a system is running. "You
| don't need to worry about that! It's ~serverless~." But you'll
| worry when that bill comes due at the end of the month.
|
| With systems like Oxide, you are allowing users to once again
| actually see what they get, and ensure they get what they paid
| for, from a high-end cloud server.
|
| It should be putting other systems providers on-notice that the
| days of flaky, non-performant, and poorly-integrated components
| are behind us. People want _beasts_ from their servers.
|
| And software designers, this is also your wake-up call that you
| can't just put lousy-performing software with poor CPU
| utilization and memory hogging on big metal and hope that it's
| "good enough." You really need to design your software to run
| ~efficiently~ in such systems.
| throwawaaarrgh wrote:
| > People might just look at it and say "It's just a rack of
| CPUs and storage." And I can only imagine just how much you
| might be tempted to throttle them
|
| Lol. Bro.
| mlindner wrote:
| I'm not even the target market for this and I completely
| understand the feeling there. There's a lot of people who
| just have no conception of computing hardware anymore. People
| who are tied to AWS and have spent their entire careers
| working with AWS and now simply believe with a passion that
| "that's just how it is" as AWS continues to raise prices
| while their own cost of compute continues to get cheaper and
| cheaper. This is a race to the bottom.
|
| This is a hot take, but I think Moore's law/Wright's Law has
| actually been disastrous for the entire field of software
| engineering even while it's been an amazing boon to software
| businesses.
| christophilus wrote:
| > CPUs ... throttle
|
| The guy's "lol" was at the (possibly unintentional) joke.
| tevon wrote:
| How do you think this will affect companies building single-
| tenant apps?
|
| I can imagine SaaS businesses offering their product as a "box":
| making on-prem significantly easier. Deliver your software along
| with the hardware, hook it into their network.
|
| Done.
|
| Anyone with more experience have thoughts on if this will
| potentially become a common option for SaaS?
| wmf wrote:
| Appliances were a big thing 20 years ago; they've been replaced
| by virtual appliances. You _do not_ want to try to get hardware
| support from a SaaS company.
| unclejack wrote:
| How can I build something for the Oxide Cloud Computer as a
| developer? Will there be something to simulate the APIs and so
| on?
|
| Similar hardware for home labs and for personal computers with
| open firmware would be great. I'd also love to see servers with
| hardware which can be upgraded (rather than completely recycled).
| steveklabnik wrote:
| We publish an OpenAPI spec for the API. The console and CLI
| tools are built on top of this. We have a mock server for the
| console to test against, you'd have to adapt that if you wanted
| to go that route, but you could.
| nickdothutton wrote:
| People that shun engineered systems generally don't understand
| the value. Have probably never experienced it for themselves.
| TaylorAlexander wrote:
| I have to say, as someone who will never buy one of these, I just
| want to see pictures of all the cool features they are talking
| about. And yet, their website seems strangely devoid of product
| photos! Or I have been unable to find them.
|
| I worked photojournalism in university, and it really strikes me
| when people talk about how cool their novel interconnect is and
| how clean the design is, but their long writeup doesn't feature
| any pictures?? Would love to see what it looks like.
| punkspider wrote:
| I feel the same. I'm also amazed at how much traction this is
| getting and I'm skimming through the comments to understand why
| this is so important. Pics would have helped.
| mhio wrote:
| This podcast on the backplane has the best set of photo's I've
| found so far:
|
| https://oxide.computer/podcasts/oxide-and-friends/1281318
| prideout wrote:
| Do they have GPU's?
| mkoubaa wrote:
| To people who agree with the stated premise: "Cloud computing is
| the future of all computing infrastructure"
|
| Do you think people want to design games with unreal engine on
| the cloud? Is there no desktop application that's safe? I don't
| fully accept this premise and I wonder if I'm the crazy one
| sometimes.
| steveklabnik wrote:
| Game development and desktop applications aren't
| infrastructure.
| mkoubaa wrote:
| Sounds like a reincarnation of the mainframe
| Waterluvian wrote:
| "Cloud computing is the future of all computing infrastructure."
|
| My dad, who worked for IBM through the 90s and 00s always points
| out how amusing this is. We started with the cloud. Then went
| PCs. Then are going cloud again.
|
| Doesn't mean this is wrong. It's just amusing.
|
| If we do it properly, it should be far more optimal than all the
| local computers not doing anything most of the day.
| Animats wrote:
| "Everyone at Oxide makes $201,227 USD, regardless of
| location."[1]
|
| Do they actually assemble and build their own racks of hardware,
| or is that outsourced? Somewhere, there must be an assembly
| plant. If this stuff actually exists. It's hard to even find
| pictures of their products. Do they have production
| installations?
|
| [1] https://oxide.computer/careers
| nickik wrote:
| Manufacturing is done by Benchmark in Raleigh, NC. Its
| outsourced.
|
| First rack shipped to costumer Jul 1, 2023:
| https://twitter.com/oxidecomputer/status/1674901883130114048
|
| Here is a picture an incomplete rack:
| https://pbs.twimg.com/media/FfT7MHoUoAE90QZ?format=jpg&name=...
|
| You can find other pictures on twitter and other places.
| choppaface wrote:
| That comp number is apparently as-of 2021 and it was explained
| in a post by the CTO https://oxide.computer/blog/compensation-
| as-a-reflection-of-...
|
| Certainly odd but not out of the ordinary for a small Bay Area
| start-up where the Founders have a ton of cash and the focus is
| mostly on what they personally want to do. Posts like these are
| b/c the Founders want to hire 1-3 people who fit exactly into
| alignment with them.
| steveklabnik wrote:
| That number was just updated, it was less in 2021.
|
| We're about 60 people, currently.
| kag0 wrote:
| It would have been wild if you'd built what you did with
| founders + 3
| ahl wrote:
| It's wild that we built it with 60 people!
| aks_tldr wrote:
| in other words. no support ops for customers
| modeless wrote:
| No GPUs? I hope they have a plan for GPUs.
| Fnoord wrote:
| GPU/NPU.
|
| Would need CUDA for anything serious. Bound to Nvidia, for now.
| Even AMD and Intel cannot compete in this space.
| Always_Anon wrote:
| And it'll be tough to provide this because Oxide does not
| have a bare metal instance type (VMs only) and the hypervisor
| is running on illumos.
| otteromkram wrote:
| I might be missing something, but I thought the whole point of
| cloud was to get rid of on prem setups.
|
| If users wanted to save some money, why not just implement a
| hybrid cloud system?
| steveklabnik wrote:
| Cloud is a deployment model. Rent be own is an ownership model.
| You can do all four combinations.
|
| If you're trying to implement a hybrid cloud solution, with
| part of it being on-prem, you would be in the market for
| hardware that to use in that implementation. We are now vendors
| of such.
| zmmmmm wrote:
| What's the interface point by which I can utilise this "cloud" -
| is it providing an AWS like API, is it something standard I can
| tap into so I'm not "locked" into this just as much as I am my
| cloud vendor? Or is it all in on Kubernetes and I have to use
| that?
| steveklabnik wrote:
| There is an API, on which we have also built a console, so you
| can use the API directly, use various CLI clients we've
| provided, or the console. Or you can sling JSON via curl, if
| you prefer.
|
| If you want kubernetes, you'd run it yourself on top of what we
| offer, think EC2, conceptually.
|
| Here's our API docs if you'd like to check them out:
| https://docs.oxide.computer/api/disk_list
| LarsDu88 wrote:
| Didn't Sun Microsystems used to do this? Sell servers in shipping
| crates?
|
| Didn't that still come with the weeks of overhead that compelled
| people to adopt cloud in the first place?
| steveklabnik wrote:
| Sun certainly sold computers, yes.
|
| I'm not sure what "weeks of overhead" you're referring to,
| specifically, as it could mean a few different things. Happy to
| elaborate if you will :)
| zemo wrote:
| their whole "AWS is bad because THEY are engaging in rentier
| capitalism, we are good because we sell you the tools to let YOU
| engage in rentier capitalism" shtick rings empty to me
| steveklabnik wrote:
| I don't know what is rentier capitalism about owning compute
| that your organization uses? Could you walk me through that?
| zemo wrote:
| the vast majority of organizations buying this product will
| use it to build SaaS. Now comes the part where you pretend
| otherwise ("but hospitals/universities need this too!"),
| press me for more evidence ("how do you know that's what
| people buy it for?"), or feign ignorance on the topic
| ("what's inherently rentier capitalist about SaaS?").
| steveklabnik wrote:
| Nope, I appreciate the elaboration. Neither of our first
| two customers are SaaS companies, but there's no point in
| arguing about it: I have no idea if most of our customers
| eventually will or not. Thanks for elaborating, I at least
| grok what you mean now.
| LTL_FTC wrote:
| I really like the company name! It's like hexadecimal notation
| with IDE!
|
| It's high time the datacenter takes a more holistic approach. For
| years I have been thinking that it's silly to stick a bunch of
| tiny fans in individual servers when one large impeller pulling
| air from an inclosed rack would be more efficient. Imagine large
| fans with ducts that come down to each rack or row of racks.
| Current in == current out; so a large diameter fan (5 feet)
| coupling down to small hose (10 inch) means the air moving
| through the small end will be really moving fast through those
| components!
| dizzydes wrote:
| Can someone clue a greenhorn like me in on why this is a big deal
| vs prior dedicated storage providers like Hetzner etc?
| steveklabnik wrote:
| I wrote this a while back, it's still true:
| https://news.ycombinator.com/item?id=30678324
| astrea wrote:
| I think I'm missing something here. Do they sell servers or are
| they selling some interesting service where you can buy a server
| AND they physically host the machine in their own data centers?
| If the former, what value add do they provide over
| IBM/SuperMicro/[INSERT COMPUTER COMPANY HERE]? The custom OS? You
| could get 99% of the way there with bare-metal Kubernetes.
| steveklabnik wrote:
| We are selling servers. The value is in building the entire
| thing, hardware and software, for purpose. This has many knock
| on effects, quality, debugability, the Buck stops with us, etc.
| oedemis wrote:
| sustainability considerations?
| oldbbsnickname wrote:
| Anyone recall Egenera? Compute only, virtualization, and fault-
| tolerance with the Ferrari of servers only MAANG-equivalents
| could afford in small numbers.
|
| [PDF]
| https://web.archive.org/web/20050325213243/http://www.egener...
| acyou wrote:
| Couple questions:
|
| 1. Are there integrator companies that take these building blocks
| and build datacenters for other companies?
|
| 2. Are these racks intended to be bought in large quantities and
| put into new datacenters?
|
| 3. Is liquid cooling an option?
|
| 4. Do you sell datacenters as well as just the racks? Do you have
| any of your own datacenters built out?
|
| 5. Tough question: if these are so competitive and good for your
| customers, why aren't you also making lots of them for yourself
| and also selling cloud compute?
| steveklabnik wrote:
| 1. Not yet, we'll see what the future holds
|
| 2. Eventually, right now we are just starting, so working
| closely with customers and in small quantities, but eventually
| that is the idea.
|
| 3. No. We are already doing very well with fans, the design
| means a very tiny power draw. The racks are quite quiet.
|
| 4. No.
|
| 5. Selling hardware and running a private cloud are two
| different businesses. The public clouds don't also sell
| servers. We don't plan on running a cloud, even if doing so
| would be a nice experience, we have to focus on what we're
| doing.
| acyou wrote:
| Thanks for the answers. Some followups:
|
| 1. Seems like you would need quite a bit of capital to be one
| of these integrator companies, to float the cash to buy land,
| build a facility and buy the racks. Does your own vs rent
| philosophy extend so far to also say that companies should
| also own the land and facility on and in which a server rack
| sits? Early on, Amazon talked about purpose-built datacenters
| being important, with huge power and internet connectivity
| requirements, earthquake/fire/flood protection and HVAC
| requirements. Sounds like the best way to go, if you have a
| huge amount of cash, seems like unit economics are now
| favoring data center costs in the billions of USD.
|
| 2. For now, would it make much sense to buy one or more of
| these and co-locate it with a cloud company's existing
| facilities in an office building? Or, if you had some space
| in a "good" datacenter where you leased space and sublet
| space, connection, power and maintenance costs to your
| customers, would that reduce the friction for buying one of
| these? Or does that essentially defeat the purpose and
| advantage of owning your own racks? If you remove the price
| of renting the rack, what's the ongoing cost to park one of
| these in an existing datacenter? Is that even an option?
|
| 3. From a purely mechanical perspective, the performance and
| efficiency advantages with liquid cooling vs. air cooling
| seem great. Seems like the problem is mostly around
| installation, maintenance, application requirements and
| reliability. Would be great to eventually see an integrated
| rack with liquid cooling plumbing hookups going directly to
| heat exchangers. At the cost of massively increased initial
| cost, maintenance cost, and with a built-in severe,
| catastrophic failure mode... Understood that this doesn't
| really make sense if these racks are being put into existing
| air-cooled datacenters or office buildings, which seems like
| it's going to be the case for awhile?
|
| 5. Too bad, seems like your prospective customers are in a
| similar sales funnel as people who want to rent capacity in a
| private cloud (and it's always nice to be your own customer!)
| What prevents your competitors from also selling servers? Do
| you think that the reason that public cloud companies don't
| sell servers is that they think their competitors will get
| ahold of the hardware and copy their technical advantages? Or
| is it that they have the deep pockets for building
| datacenters that their customers don't necessarily have, and
| they would rather have monthly revenue anyways?
| steveklabnik wrote:
| For 1 and 2, to be honest, I don't personally know much
| about all of that. I'm just an engineer :) I think there's
| a whole lot of "it depends."
|
| 3. Software engineer, not hardware, haha. I don't know if
| people have tried liquid cooled racks, personally.
|
| 5. "What prevents your competitors from also selling
| servers?" I mean, like I said, they're just totally
| different businesses. Doesn't mean it's impossible, but it
| feels like there's not a ton you'd be sharing there, other
| than one half knowing where they'd be getting some supply
| from.
| killjoywashere wrote:
| People who think cloud computing is the future of all things
| don't understand where their chips come from.
| garfieldnate wrote:
| I haven't listened to Oxide & Friends, but I absolutely love On
| The Metal. The guests are eminent and extremely interesting, the
| hosts are passionate and fun, and there's always a wonderful
| chemistry among the hardware geeks on the show that just makes me
| so excited to get into hardware and see what the future brings
| for us. Highly recommended.
| sinzone wrote:
| The Apple of Servers cloud. Fully vertically integrated.
| Fischgericht wrote:
| I know that many might have by now forgotten how an actual data
| center works.
|
| "One size fits all" will never work.
|
| Here are the main headaches that someone who actually runs a data
| center will run into:
|
| - How many power circuits does the DC provide you? What voltage?
| AC or DC?
|
| - How many amps do the circuits have? Is the PDU provided by the
| DC, or do you provide it?
|
| - How many upstreams will you have? Dual core routers? BGP?
| Static routing? Failover?
|
| - Upstream provided via Fiber? Copper?
|
| - Given that for the last couple of years being able to re-buy a
| typical server model (say: Supermicro), CPU, Storage for more
| than a couple of weeks has been a challenge, and given that rack
| hardware is supposed to run for years - how can I be sure to be
| able to scale?
|
| - What happens if that Oxide startup goes bust 6 months from now?
|
| - Would any sane DC operator really buy "we just built our own
| switch"? I wouldn't. Those who have been in that business for 20
| years still f*ck it up on a regular basis. I want a proven vendor
| for my mission critical stuff.
|
| I am sorry, I don't see them addressing ANY of the real life pain
| points you have operating your own rack. Unless they ship a
| couple of data center engineers within the box.
|
| The whole cloud stuff is simply designed for managers who believe
| that in a job market where it's hard to find DC/server engineers
| that things will be magically done in the background. With this,
| it's worse - it's pretended that "stuff just works" if you ship
| it in a single box. And that's simply not a case. A DC
| installation not constantly maintained by skilled engineer will
| go down in flames within months, one way or another.
| wpm wrote:
| Many of your questions have answers in TFM:
| https://docs.oxide.computer/guides/introduction
| Fischgericht wrote:
| Part of the power distribution question is answered, in a way
| that gives even more concerns, though. A single DC
| distribution without redundant PSUs? What if one of the
| servers trips a fuse?
|
| By the way, the list of questions would go on. At least over
| here, DCs offer different rack footprints. They don't even
| list their footprint.
|
| But most importantly: The TFM doesn't say "There is an
| experienced network engineer in the box that you will own and
| that will do the maintenance". Unless this is provided, this
| will blow up.
|
| Again: I can imagine that they find a hand full of managers
| buying into this. Up until the date the first few of their
| racks have burned down a DC, or got hacked and all customer
| data stolen.
|
| Come on, this is hacker news. Everybody here should know that
| nobody so far has found a way (not even via AI) to replace
| the competences many of us here have. Again, I understand
| that management would love to get rid of those expensive and
| annoying nerds, but the ACTUAL REALITY is: An unmaintened
| rack in a DC will go down in flames, and management people
| will be in tears.
|
| So, let's just be realistic: Oxide is simply a terrible idea
| that will blow up. But I congratulate them for VCs falling
| for this.
|
| Now, let's find a VC that is funding someone doing an IPMI
| system that is not shit, does not have access to all data of
| the server, and does not get hacked within 5 minutes after
| deploying it. That would address an ACTUAL need.
|
| Sometimes it makes sense to ask those who you hope your
| future customers could be.
| hypfer wrote:
| > Come on, this is hacker news.
|
| Yep, exactly. That's precisely the reason why this
| "solution" to a problem that doesn't exist is currently
| being praised as the soon to be heir of the throne of
| computing
| mutant wrote:
| You know what's cool? Pricing information.
| kjellsbells wrote:
| I see lots of comments debating this technology choice or that
| one, as if the economic buyer actually cares. I dont think they
| do as much as the HN commentariat might think.
|
| All that really matters is whether Oxide's guesses about who'll
| buy and what they'll pay are right.
|
| Lets say that Oxide are guessing that their lead customers are
| telco and oil/gas: conservative industries, regulated, very
| strict about uptime and data sovereignty, etc. Oxide might spend
| 1 to 2 years running the gamut of all the people in those types
| of customers that can say no: ops, swcurity, network teams etc.
| But if Oxide can reach the finance team and get them to believe
| that a one time chunk of $$ is better than opex in the current
| interest rate environment, them it really doesnt matter what the
| tech stack is. (And bear in mind that financial models are f(t)
| so things might turn upside down as the economic picture changes
| over time)
|
| Assuming they have done this math, (surely a yes, or else VCs
| would not be lending them money) Oxide must need sufficent
| funding for a long sales cycle, and plausible enough bench
| strength to support customers who are going to hold them to
| ruthless SLAs and penalty clauses. All of that take serious coin
| up front, eg theres no outsourcing your 24x7 support to some kid
| reading from a script.
| emsign wrote:
| > Cloud computing is the future of all computing infrastructure.
|
| I absolutely dread that future. I want all my computation back to
| be local, doesn't matter to me if I own the cloud computer or not
| as long as I also don't own the environment it is embedded in. If
| the connection to it isn't mine and neither is its source of
| power then I don't really control "my" cloud computer.
| Uptrenda wrote:
| I hate websites that aren't transparent about pricing. Like, how
| much is this crap going to cost? What are they offering exactly?
| I hate having to speak to some normie in sales just to get access
| to something. I swear - with how hard it is to buy from some of
| these companies its a wonder that they sell anything at all.
| laniakean wrote:
| I feel the same. I am interested in the product, but I want to
| know how much it would cost. Without knowing at least the range
| of the prices, I find it unproductive to contact the sales
| representative, since the price could end up way beyond what I
| can afford.
| unixhero wrote:
| I want one. But what is the price?
| tgtweak wrote:
| Kick ass project, been following bcantril since before Joyent and
| as a (server/infra + sysadmin nerd this really hits home): Are
| there plans to build some public-cloud-esque software add-ons to
| offer cloud-native equivalents to s3, serverless-functions,
| message queues, etc?
|
| I feel like the "running your own datacenter" complexity of on
| prem is a big part of why companies go to the cloud in the first
| place - but that a good amount of cloud lock-in is actually in
| the software/service stack and reliance on it. I know there is a
| competent open source option for most of these services (and in
| fact most are repackaged open source software...), but they do
| seem pretty disparate and require quite a fair bit of knowledge
| and expertise to deploy and run at scale.
|
| Secondary question in the same thread: What about "accounting"?
| Running infrastructure at large-scale companies (such that would
| look at a rack-level solution) usually entails budgeting and
| accounting of resources - how does oxide simplify this for
| infrastructure ops trying to keep the various IT consumers
| honest?
| talent_deprived wrote:
| I can't read the dark theme for more than a couple of minutes
| without a headache and eye strain starting up.
| slowhadoken wrote:
| "The cloud" has been a buzzword for twenty years but I feel like
| it's finally taking shape. This is promising.
| joby_surge wrote:
| Sooo basically it's an OSS version of Egenera's BladeFrame or
| VCE's vBlock.
|
| It's going to be tough sledding getting these into enterprise
| accounts.
|
| Best of luck.
___________________________________________________________________
(page generated 2023-10-27 23:01 UTC)