[HN Gopher] Oxide Computer: Docs
       ___________________________________________________________________
        
       Oxide Computer: Docs
        
       Author : avrong
       Score  : 284 points
       Date   : 2023-07-20 20:48 UTC (1 days ago)
        
 (HTM) web link (docs.oxide.computer)
 (TXT) w3m dump (docs.oxide.computer)
        
       | EvanAnderson wrote:
       | Everything about Oxide's gear sounds like fun. I imagine it must
       | be a bit like what working with minicomputers in 70s thru the 90s
       | was like.
       | 
       | I did a little work in the late 90s with Alpha-based machines. I
       | was impressed at those machines didn't seem like the hack-job
       | crap that PC-based stuff was (with simulated chips from the early
       | 1980's hiding out in dark corners because "compatibility") and
       | still is today. I'm betting working with Sun gear felt similar,
       | though I never got to work with it. Just having an honest-to-God
       | serial console, as opposed to crappy bag-on-the-side things that
       | scrape video memory and pretend to be "legacy" PC input devices,
       | would be an amazing thing.
       | 
       | I'll never be able to work with their stuff because I don't work
       | with Customers at that scale. I'm also vastly unqualified to work
       | for them at their current stage. I suppose maybe someday they'll
       | need field service technicians... I can hope, I guess.
        
         | soneil wrote:
         | I really wish they'd do a tour of it, hardware & software.
         | They've gotta be proud as hell of what they've pulled off, it
         | boggles my mind that they're not more eager to show it off.
        
           | mst wrote:
           | Given it was only relatively recently they were talking about
           | getting their first deployment out the door, I'd suspect that
           | staying mostly quiet is strategically wise in terms of
           | avoiding having more prospective customers come in the door
           | at once than they can (as yet) service to the standard they
           | want to.
           | 
           | So "they probably really really -want- to show it off, but
           | are showing remarkable restraint for good reason" seems at
           | the very least plausible to me.
        
           | EvanAnderson wrote:
           | I'd fly out to their factory on my dime and pay to watch a
           | dog-and-pony show and hear a Q&A. I'm that excited about this
           | stuff. Even if I just end up being a Customer of somebody who
           | hosts my VM on this gear it's plenty exciting. It actually
           | feels like a new computer, as opposed to the same old, same
           | old offerings from the incumbents.
        
           | newsclues wrote:
           | They need to get Linus Tech Tips to do a video on the
           | machines and people of Oxide.
        
             | wmf wrote:
             | And then void the warranty by water cooling it.
        
               | newsclues wrote:
               | Drop a sled first, then try to watercool one! It would be
               | fun content.
        
         | Aurornis wrote:
         | > I imagine it must be a bit like what working with
         | minicomputers in 70s thru the 90s was like.
         | 
         | I had a chance to talk to someone who worked at Oxide once. I
         | got excited because my experience matches up nicely with their
         | products on several levels.
         | 
         | The person got excited as well and started talking about how to
         | interview there, but the conversation got kind of weird. They
         | kept emphasizing how important it was to not talk about
         | compensation in the interview. Apparently they pay engineers
         | all the same comp (not bad, but would have been a significant
         | step down from every offer I received during that time of my
         | life) and they select for people who aren't interested in
         | getting paid a lot for their unique skills.
         | 
         | Probably not a bad deal for people who like working in that
         | domain with like minded people. At the time I got some very
         | uneasy feelings from being aggressively coached to not bring up
         | compensation or ask any questions about equity during the
         | interview like it was some unspoken rule that would get me
         | disqualified. Maybe the person was exaggerating, but I found at
         | least one other person with a similar story.
         | 
         | Honestly, their comp would have been awesome if I was a single
         | guy living in a low cost of living location and working remote,
         | but at the time it would have meant giving up quite a bit to
         | work for an early stage startup with high expectations and an
         | unspoken rule that I should never ask about compensation.
        
           | bcantrill wrote:
           | This conversation strikes me as unlikely on several levels.
           | First, no one would have coached you on "how to interview at
           | Oxide" because that's not where the process starts -- it
           | starts with you preparing your materials.[0] (Our review of
           | the materials constitutes ~95% of our process.) Second, we
           | have always been very explicit about compensation (that is,
           | we ourselves brought it up early in conversations); no one at
           | Oxide would tell you to "not bring it up" because everyone at
           | Oxide knows that it is a subject dealt with early in the
           | process. And finally, this is all assuming that you were
           | talking to someone before March 2021, when we published our
           | blog post on it.[1] After the blog post, compensation simply
           | doesn't come up: everyone has seen it -- and indeed, our
           | approach to compensation is part of what attracted them to
           | the company!
           | 
           | [0] https://docs.google.com/document/d/1Xtofg-
           | fMQfZoq8Y3oSAKjEgD...
           | 
           | [1] https://oxide.computer/blog/compensation-as-a-reflection-
           | of-...
        
             | Aurornis wrote:
             | > First, no one would have coached you on "how to interview
             | at Oxide" because that's not where the process starts
             | 
             | I was using "interview" as a generic term for applying to a
             | company, not literally referring to your internal process.
             | 
             | I didn't interview with or even pursue Oxide after the
             | conversation (and never said I did)
             | 
             | This all came up because I asked the person what the equity
             | compensation was like. Not an unreasonable question when
             | talking about a startup. That's when they started advising
             | me that I shouldn't bring it up and it's not something they
             | talk about. After that I got uncomfortable about pursuing a
             | company that discourages any conversation about
             | compensation to the extent that someone felt necessary to
             | warn me about it when I hadn't even applied.
             | 
             | > no one at Oxide would tell you to "not bring it up"
             | because everyone at Oxide knows that it is a subject dealt
             | with early in the process.
             | 
             | They were trying to tell me that it was important that I
             | avoid giving the impression that I cared about
             | compensation, as that would be a negative if I talked to
             | anyone else at the company. Just repeating what I was told.
             | 
             | > And finally, this is all assuming that you were talking
             | to someone before March 2021, when we published our blog
             | post on it.[1]
             | 
             | No, they told me to look up the blog post, but I had not
             | read the company blog before taking to this person.
             | 
             | > After the blog post, compensation simply doesn't come up:
             | everyone has seen it -- and indeed, our approach to
             | compensation is part of what attracted them to the company!
             | 
             | Or maybe your approach to compensation is what filters
             | people out of the application pipeline? I don't think it's
             | realistic to think that this compensation strategy is what
             | attracts people to the company rather than pre-selecting
             | people out.
             | 
             | I read the blog post, but I feel like I'm missing the
             | equity portion of the conversation still.
             | 
             | Regardless, is it so hard to believe that compensation
             | "simply doesn't come up" because potential candidates (like
             | me) are sometimes coached to not bring it up? Or that the
             | company's stance appears to discourage bringing it up? This
             | feels like some circular logic: Nobody brings it up because
             | we discourage people from bringing it up.
        
               | bcantrill wrote:
               | It definitely wouldn't be viewed as a negative to talk
               | about it, and in fact our transparency on this topic
               | makes it very easy to talk about directly. So we
               | absolutely don't discourage talking about it -- but it's
               | also true that people for whom the compensation is going
               | to make Oxide impossible do self-select out. And that's
               | okay! People have different needs at different stages of
               | their career, and there are different things that they
               | want; there is nothing wrong with optimizing for
               | compensation -- but it's also true that Oxide is very
               | unlikely to be a fit for someone optimizing for
               | compensation, for many reasons.
        
             | petesergeant wrote:
             | Hey Bryan, I enjoyed reading the cash compensation article,
             | but I'm curious about how it meshes with your equity
             | compensation? Is it awarded purely on when people joined?
             | Does everyone who wasn't a founder get the same amount? It
             | feels like you could run into much of the same problems the
             | article points out about transparency and so on if the
             | equity component isn't as straight-forward?
        
             | alexgartrell wrote:
             | The person to whom you are replying clearly meant (IMO)
             | that you shouldn't ask for more compensation or you will
             | make people act defensive. Frankly, your reply reads a
             | little defensive so maybe that's not awful advice?
             | 
             | It also seems like this was a spontaneous initial
             | conversation and not part of the process, so I'm not sure
             | why you are suggesting that they made it up.
        
               | gary_0 wrote:
               | I didn't realize there are two ways of looking at it
               | until your comment. It didn't occur to me that if you
               | randomly have Oxide on a list of 40 other companies to
               | apply to, and you expect to talk to them about
               | compensation as you would with most others, you're going
               | to have a weird time because they have an uncommon
               | policy.
               | 
               | But on the other hand, Oxide is very up-front about it,
               | and their CTO is happy to go on HN and chat about it. So
               | not knowing about it makes you look like you didn't do
               | your research before the interview, or knowing about it
               | and trying to force the issue anyways makes you look kind
               | of arrogant (if you don't agree with it, you can just not
               | apply).
        
               | Aurornis wrote:
               | > It also seems like this was a spontaneous initial
               | conversation and not part of the process, so I'm not sure
               | why you are suggesting that they made it up.
               | 
               | Yes, it was an initial conversation as I said. When I
               | asked about equity compensation I was told that
               | compensation discussions are to be avoided because
               | bringing it up could be considered a negative by the
               | company.
        
             | [deleted]
        
       | paywallasinbeer wrote:
       | After giving the "Known Behavior and Limitations" a scroll here
       | https://docs.oxide.computer/release-notes/system/1-0-0 It seems a
       | little... half baked? Especially for a company whose specialty is
       | the integrated management platform. Interested to see customer
       | reviews.
        
         | convolvatron wrote:
         | this looks like a pretty normal bug list
        
       | goalonetwo wrote:
       | Oxide is an HN darling. I have never seen anything even close to
       | negative about them here. I hope someone writes a case-study
       | about this. It seems to be a mix of the charisma of the
       | team/founders and their product that makes everyone love them.
       | 
       | I would put Wireguard/Tailscale in the same category as well.
        
         | panick21_ wrote:
         | It partly because they have a good podcast that is perfectly
         | aligned with what many people on HN like.
         | 
         | Other then that, the code is open source. They also help
         | sponsor both Rust and OpenFireware conferences.
        
         | azinman2 wrote:
         | But are people buying their machines?
        
           | intvocoder wrote:
           | They only started shipping this month... so, jury's out?
        
           | monocasa wrote:
           | They recently shipped their first rack IIRC.
        
           | LoganDark wrote:
           | no, companies are :)
        
           | newsclues wrote:
           | Given they shipped their first machine, it seems so.
        
         | HL33tibCe7 wrote:
         | fly.io too, although the scales have fallen from the eyes of HN
         | to some extent after their repeated outages
        
         | [deleted]
        
         | owenmarshall wrote:
         | > I would put Wireguard/Tailscale in the same category as well.
         | 
         | Sometimes the hype exists for a good reason - as a paying
         | enterprise Tailscale customer, I'll fight you if you _ever_
         | suggest I have to staff people to admin an IPsec or SSL-based
         | VPN again.
        
         | Tuna-Fish wrote:
         | Here's the big negative things I can come up with quickly:
         | 
         | 1. It's not clear the market case closes, because while there
         | are customers who would greatly benefit from their systems,
         | those same customers are also very averse to change, meaning
         | that most of them will probably sit out for the first few
         | generations to see if they have staying power. If everyone does
         | that, it becomes a self-fulfilling prophecy.
         | 
         | 2. ... but this would have been fine, a few years ago. It's not
         | fine in the current market situation, where there is much less
         | easy money looking for a place to go. I sure hope they are
         | sitting on a long runway.
         | 
         | 3. Speaking of, they are very late in their execution. A SM4
         | platform that only shipped in 2023 is not great. I really hope
         | they are far along on their SP5 development. (... But this also
         | dampens current sales. I bet a lot of potential customers are
         | thinking that a SP5 Oxide Rack seems a much more appealing than
         | a SP4 one, so why not wait?)
         | 
         | But the reason they are not discussed that much is that people
         | who understand the market are all really, really hoping that
         | they pull it off. Because the current situation in server
         | hardware is _dire_. In every thread people who don 't know much
         | about servers ask why not get a similar system from Dell or
         | Supermicro or whoever. The answer is that the commodity servers
         | are pieces of shit that requires significant local engineering
         | resources to manage. Software quality of firmware is generally
         | horrendous, and when this is pointed out to the vendors, their
         | answer is that they know, but everyone else is just as bad,
         | wontfix. A significant draw of the cloud is that all that pain
         | goes away, because the hyperscalers realized how shit
         | everything was and fixed it in their systems. It just never
         | trickled down to the market below them.
        
           | panick21_ wrote:
           | There are just very few people regularly buy 500k-1M in
           | Server hardware and have a good understanding of the market
           | and the finances involved in the alternative.
        
       | wmf wrote:
       | It's cool to see Oxide shipping RIFT routing well before
       | networking vendors.
        
       | ConanRus wrote:
       | [dead]
        
       | mwcampbell wrote:
       | I'm a little surprised that, according to the docs, "the Oxide
       | rack does not come with any preloaded machine images" [1]. But I
       | guess it makes sense that the initial, early-adopter customers
       | wouldn't have a problem with rolling their own VM images. I'm at
       | least glad that Oxide didn't decide to start by only allowing VMs
       | that use a pre-defined set of pre-made images in some custom
       | format.
       | 
       | [1]: https://docs.oxide.computer/guides/creating-and-sharing-
       | imag...
        
         | wmf wrote:
         | The preloaded images would all be Solaris anyway. ;-)
        
           | jclulow wrote:
           | We don't call it Solaris anymore! :P
        
           | 59nadir wrote:
           | Surely Illumos? Solaris is probably dead to most people
           | involved, a husk of what it was and could have been.
        
       | [deleted]
        
       | engagthe wrote:
       | I'd love to see someone provide a turnkey managed bare metal
       | container platform, complete with L4 / L7 routing. I haven't
       | heard if oxide has a container play, and I suspect it may require
       | virtualization based on their choice of host OS.
        
         | doctorpangloss wrote:
         | By "L4 / L7 routing" do you mean specifically something that
         | works like Amazon's Application Load Balancer and Network Load
         | Balancer for Kubernetes? And by "turnkey managed" you mean, you
         | """just""" """configure""" """IP addresses""" and it """all
         | just works"""?
         | 
         | You can certainly install Ubuntu on a very powerful machine
         | with a WAN interface (e.g., a NIC connected to a residential
         | cable internet connection). Then, use something like k0s to
         | provision other bare metal workers. Those three steps, and
         | you've got a "bare metal container platform." You don't need a
         | "LoadBalancer", you can specify that nginx-controller runs on
         | the host network of specifically the machine with the WAN
         | interface and configure its service's external IP to the WAN
         | IP, and now you support Ingress.
         | 
         | But how do you imagine having multiple LoadBalancer resources
         | without multiple IPs? And how do you imagine having multiple
         | IPs without ARIN? The turnkey challenging part is the public
         | IPv4 addresses, not the platform.
        
           | dilyevsky wrote:
           | > You don't need a "LoadBalancer", you can specify that
           | nginx-controller runs on the host network of specifically the
           | machine with the WAN interface and configure its service's
           | external IP to the WAN IP, and now you support Ingress.
           | 
           | And if you need to service that machine or it goes down?
           | 
           | > But how do you imagine having multiple LoadBalancer
           | resources without multiple IPs? And how do you imagine having
           | multiple IPs without ARIN?
           | 
           | Most colos/transit providers will happily lend you their IPs
        
       | javajosh wrote:
       | I'm particularly impressed with their anti-tamper measures. "For
       | each server sled, shine a light into the cubby to look for any
       | physical tampering or damage."
       | 
       | https://docs.oxide.computer/guides/system/rack-installation-...
        
         | soneil wrote:
         | That's pretty standard? Anything that's going straight into
         | 400V (I assume) deserves a good eyeball before it's energised.
         | I doubt their "root of trust" knows how to protect the system
         | from a dead mouse in the bus bars.
        
           | Tuna-Fish wrote:
           | > Anything that's going straight into 400V (I assume)
           | 
           | I think the sleds blind mate to a 55V DC common bus.
        
           | javajosh wrote:
           | I'm not worried about dead mice; I'm worried about tampering.
           | I would have hoped there'd be more guidance than "just look
           | and see if anything looks weird". A determined threat actor
           | can easily side-step that check with, for example, careful
           | desoldering/resoldering technique.
        
             | gorkish wrote:
             | If this is seriously your level of supply chain scrutiny,
             | being an Oxide early adopter probably isn't the best idea.
        
             | scq wrote:
             | I suspect this step is intended to protect you from FedEx,
             | not the NSA.
        
         | rhinoceraptor wrote:
         | The rack also comes with built-in security features to ensure
         | all hardware and software are genuine Oxide products:
         | purpose-built hardware root of trust (RoT) - present on every
         | Oxide server and switch - cryptographically validates that its
         | own firmware is genuine and unmodified         encryption of
         | data at rest via internal key management system built on the
         | RoT         trust quorum establishment at boot time to ensure
         | the cryptographically-derived rack secret is verified before
         | unlocking storage
         | 
         | https://docs.oxide.computer/release-notes/system/1-0-0
        
       | corysama wrote:
       | Oxide's "On the Metal" podcast is an incredibly fun deep dive
       | into technical issues no one should have to deal with as told by
       | people who lived them. "Deep" as in: software DRAM drivers, Ring
       | -1 security, bespoke motherboard designs... I only wish there
       | were more episodes.
       | 
       | https://oxide.computer/podcasts/on-the-metal
       | 
       | I just noticed they have a second podcast. I'll assume it's just
       | as good.
       | 
       | https://oxide.computer/podcasts/oxide-and-friends
        
         | bcantrill wrote:
         | For whatever it's worth, I think the reason the doc site was
         | submitted to HN recently is in fact because of our recent
         | episode on the frontend.[0] We have really enjoyed doing Oxide
         | and Friends, and if you're an On the Metal listener, we think
         | you'll find a lot to like!
         | 
         | [0] https://oxide-and-friends.transistor.fm/episodes/the-
         | fronten...
        
           | pjmlp wrote:
           | By the way, Robot was initially tested at Nokia Networks Oy,
           | and we on the NetAct business unit weren't that found of it.
           | 
           | Otherwise, another interesting episode.
        
         | rtpg wrote:
         | Oxide + Friends is very good. It's very motivating to listen to
         | and gets me excited to try and build new things. Way more
         | positive vibes being put out there than a lot of stuff.
        
         | [deleted]
        
         | helf wrote:
         | This sounds like a great listen. Thanks for mentioning it
        
           | howinteresting wrote:
           | [flagged]
        
             | mburns wrote:
             | looking at his (recent) comment history, I can't imagine
             | this is the one in particular that got him shadow banned.
        
             | usefulcat wrote:
             | For that one comment, probably not, but maybe for dozens of
             | previous flagged and/or dead comments
        
               | howinteresting wrote:
               | Once you're shadowbanned your comments start off as dead
               | by default and have to be vouched to be visible.
        
               | usefulcat wrote:
               | Ah, I wasn't aware of that--thank you for the
               | clarification.
        
             | dang wrote:
             | We didn't shadowban them, we told them we were banning
             | them: https://news.ycombinator.com/item?id=34916347, we
             | told them why, and it wasn't the reason you've presumed.
             | 
             | When people presume things about moderation, they almost
             | always get it wrong. That's a pity because all anyone has
             | to do to get the correct answer is ask us.
             | 
             | If you see an account that's banned and you don't think it
             | should be, please let us know at hn@ycombinator.com so we
             | can take a look. We've unbanned quite a few accounts that
             | way. In the meantime, you can vouch for their good comments
             | (see https://news.ycombinator.com/newsfaq.html#cvouch). But
             | please don't divert the thread with offtopic noise that is
             | likely to be incorrect.
        
               | stefantalpalaru wrote:
               | [dead]
        
               | howinteresting wrote:
               | Yes, you told them that you prioritize vibes over
               | justice:
               | 
               | > repeatedly breaking the site guidelines and using HN
               | for flamewar and ideological battle
               | 
               | helf's comment is absolutely correct and them being
               | heated was very reasonable given the context. I'd be
               | heated too if I had a daughter nearing puberty.
               | 
               | (And, in an attempt to be intellectually honest, I'll say
               | that their recent comment on the post about Kevin
               | Mitnick's death was undoubtedly crass.)
        
         | LoganDark wrote:
         | Are these available in text or article format?
        
           | gorkish wrote:
           | O+F is posted to youtube, I get the auto-generated transcript
           | from there. It's unscripted/unstructured, so if you are
           | primarily wanting to hear discussion of the oxide
           | hardware/software/progress it's pretty difficult to consume
           | as an audio program.
           | 
           | This is not a dig on the program at all; I'm glad they are
           | making the time to produce it, and I'd rather they spend
           | their effort getting racks out the door instead of generating
           | marketing hype.
        
             | realitythreek wrote:
             | I always miss the live Discord call and never realized they
             | were published to YT! Thanks for the notice.
        
         | yardie wrote:
         | O+F is really good especially if you can make it to the Discord
         | Server
         | 
         | https://discord.gg/gcQxNHAKCB
        
       | lwhsiao wrote:
       | Anyone know if a documentation framework is used for this site,
       | or whether it is bespoke?
        
         | dcre wrote:
         | It's bespoke. Remix + AsciiDoc + OpenAPI.
        
       | codetrotter wrote:
       | I wonder how much the smallest, cheapest configuration will cost.
       | I would really love to buy one of these. But I suspect it will
       | cost 100x more than I can afford heh.
        
         | wmf wrote:
         | Estimates in the previous thread were at least half a million.
        
           | codetrotter wrote:
           | Yeah that's about 100x more than I can afford exactly.
           | 
           | I think I will go back to my little collection of RPi Compute
           | Module machines. And maybe some day in the very distant
           | future I can buy big boy servers lol
        
             | masklinn wrote:
             | Depending on your exact wants, I think there was an LTT
             | video recently on how you can purchase older server blades
             | for very reasonable prices, and upgrade them using also
             | reasonably priced used server-grade CPUs and RAM. Obviously
             | you won't have the bang of new hardware, but if what you
             | want to do is play around with enterprise-class systems...
        
       | Roark66 wrote:
       | It sound nice, but I've been building systems like that since
       | 2012 (usually vmware vCloud director plus custom code). Using
       | hardware like Fujitsu's cx1000, Nexus 1000 distributed vswitch,
       | brocade network switches, and San storage. The systems I built
       | back then could dynamically provision separate
       | network/conpute/storage segments for PCI compliance and more. If
       | I were to build something like this today I'd probably consider
       | using Kubernetes too and integrating it with public cloud for
       | scale out. This way a business can have their "own" cloud that is
       | much more cost effective at certain scales and access to
       | "unlimited" public cloud resources at the same time (at the cost
       | of increase in system complexity).
       | 
       | Is this just more of the same, or is there some innovation there?
       | It is interesting that computing trends go in
       | centralise/decentralise cycles. We can observe this as far back
       | as 1980s. I can't wait for the next "decentralise" cycle as I'm
       | under an impression a lot more innovation happens during that
       | phase.
        
         | panick21_ wrote:
         | I think the vision is that a company doesn't need to heir and
         | pay you and not pay vmware and they can have that kind of thing
         | turn key.
         | 
         | The innovation is mostly in the integration between the
         | hardware and software of both the server and the switch.
         | 
         | This podcast goes into what that enables:
         | https://www.youtube.com/watch?v=AkWh2Sms3aw
         | 
         | > I can't wait for the next "decentralise" cycle as I'm under
         | an impression a lot more innovation happens during that phase.
         | 
         | I think we are beyond that cycle. Centralization and
         | decentralization happen at the same time. We have much more
         | much cheaper chips now everywhere that do a lot more, and then
         | we have local and distributed compute deployed everywhere and
         | we also have large datacenters at the center of it.
         | 
         | Neiter datacenters nor distributed compute is gone go away
         | anytime soon.
        
       | benrockwood wrote:
       | It's great to see Oxide shipping. They have an incredibly
       | talented team thats worked very hard for a long time in the best
       | tradition of Sun Microsystems.
        
         | dsies wrote:
         | Not being pedactic, I swear :)
         | 
         | Can you elaborate on the Sun comparison? I am a huge fan of Sun
         | and what they did for computing at large - designing hardware,
         | creating specs, their contributions to evolving unix and so on.
         | I'm not sure how Oxide compares. Unless you're talking about
         | "in the spirit of Sun".
        
           | jeffrallen wrote:
           | Sun was a combination hardware and software shop, which Bryan
           | appreciated and has tried to replicate at Oxide. The only
           | reason they have a chance today is because the
           | hardware/firmware interface in most servers is terrible
           | quality.
        
           | cbarrick wrote:
           | I'm not the one you replied to, so I don't know what they
           | mean specifically about the "tradition of Sun", but
           | 
           | Bryan worked at Sun where he helped create DTrace.
           | 
           | After the Oracle acquisition, he left to join Joyent as VP of
           | Engineering and then CTO. Steve was COO of Joyent. And I have
           | heard similar comparisons between Joyent and Sun.
        
       | ocdtrekkie wrote:
       | As an on-prem sysadmin, I'm not sure I will ever in my life work
       | on an environment that has the minimum specifications of an Oxide
       | system. Every entire datacenter I've worked in has less total
       | capacity than a single "sled" here.
       | 
       | Are there intentions to go smaller than this?
        
       | earthboundkid wrote:
       | Does anyone know an order of magnitude for the price of a rack?
        
         | sitkack wrote:
         | 500k-1M-1.5M
        
       | ThinkBeat wrote:
       | Can you configure your Oxide mainframe for different uses? Like
       | max it out with 90% being GPUs?
       | 
       | I like the idea, though it may be dumb, if you could fill it with
       | different types of processors for what you need.
       | 
       | Sort of like chiplets but at a different scale
        
         | sam_bristow wrote:
         | I don't believe they have any GPU option in this initial rack.
         | In one of the podcasts they said something about none of the
         | GPU vendors being open enough to allow the kind of deep
         | integration they're going for. It might change in future of
         | course.
        
       | ThinkBeat wrote:
       | I recently saw a horrible presentation by Microsoft on how they
       | built the OpenAI supercomputer on Azure. (with custom bits).
       | 
       | How would an Oxide computer systems look like for for an "OpenAI
       | super computer" compared to Azure?
        
         | pitaj wrote:
         | That kind of magnitude of compute is more datacenter level. But
         | either way, you're probably better off buying one of the all-
         | in-one tightly integrated solutions from Nvidia or Intel.
        
       | 0x6c6f6c wrote:
       | > Oxide Computer Model 0, also known as the "0x1"
       | 
       | So is it 1 or is it 0..
        
         | haimez wrote:
         | Hopefully they end up with a German market only SKU before the
         | next generation of hardware ships, otherwise it's a real missed
         | opportunity.
        
         | OJFord wrote:
         | Yes.
        
         | rafram wrote:
         | It was a 0 in both names, but cosmic rays flipped one of the
         | bits.
        
       | dist1ll wrote:
       | As someone who's only dealt with commodity server hardware, these
       | specs make me salivate. And all these boot/management TUIs are
       | just so satisfying to look at.
       | 
       | (Sorry to be that guy, but just a friendly suggestion: high
       | contrast dark themes are difficult to read for people with
       | astigmatism. Especially since this is technical documentation,
       | intended to be thoroughly read, you might want to consider a
       | light theme toggle.)
        
         | tiffanyh wrote:
         | > _As someone who 's only dealt with commodity server hardware,
         | these specs make me salivate_
         | 
         | Commodity servers have the same specs.
         | 
         | Here's a reseller of SuperMicro servers, where you can buy
         | similar compute on the cheap.
         | 
         | https://www.siliconmechanics.com/systems/servers/rackform
        
           | Aurornis wrote:
           | > Commodity servers have the same specs.
           | 
           | Probably not a coincidence. It would be interesting to know
           | which ODM they partnered with for the hardware.
           | 
           | I've done some work with SuperMicro in the past. Some of
           | their boards come with extensive headers and customization
           | options right out of the box. They're also happy to work on
           | board level customizations with the right contracts in place.
        
             | bcantrill wrote:
             | We didn't work with an ODM: the ODMs were unwilling to
             | contemplate some of the most basic things we needed (e.g.,
             | replacing the BMC with a much lighter weight service
             | processor, having a true root-of-trust, etc.) -- let alone
             | the more things we wanted to do (e.g., our own switch). The
             | compute sled and the switch are both of our own design and
             | look nothing like what you'll find from an ODM; if you're
             | curious in the details, we have discussed them quite a bit
             | in our Oxide and Friends podcast.[0][1][2]
             | 
             | [0] https://oxide-and-friends.transistor.fm/episodes/tales-
             | from-...
             | 
             | [1] https://oxide-and-friends.transistor.fm/episodes/the-
             | sidecar...
             | 
             | [2] https://oxide-and-
             | friends.transistor.fm/episodes/bringup-lab...
        
               | mst wrote:
               | Please consider generating some sort of automated
               | transcript from these.
               | 
               | I'd hope your target audience would understand the
               | limitations of such a thing, and I'm probably not the
               | only person who'd rather read than listen even with the
               | obvious caveats.
               | 
               | (these days automated transcripts seem to be no harder to
               | mentally fix up the errors in as I read them than
               | "somebody typing fast on a software keyboard and
               | suffering the inevitable tyop and autocorrupt related
               | issues" is, though of course others' mileage may vary)
        
               | actionfromafar wrote:
               | They could probably have someone proofread the
               | transcripts, there aren't that many episodes.
        
               | mst wrote:
               | I was going for "set up some code once and don't think
               | about it again" to maximise the odds of the idea sounding
               | tempting.
               | 
               | Proofreading would set up an expectation on the part of
               | readers that it -had- been proofread and corrected and
               | therefore a commitment to perform a repeated "boring but
               | important" task going forwards for whoever's doing said
               | proofreading.
               | 
               | That way would likely lie either delayed transcripts or
               | never getting to initial activation energy to provide
               | anything at all.
               | 
               | So I think "add a quick bit of code to your podcast
               | publishing workflow and a CAVEAT IN BIG LETTERS" is
               | better to do first.
               | 
               | If it turns out enough people care about the transcript,
               | doing it a more labour intensive nicer way later is
               | something they can decide, well, later.
               | 
               | Shipping is feature zero, as ever.
        
           | EvanAnderson wrote:
           | I haven't worked with SuperMicro (aside from having it inside
           | "appliance" devices that I've worked adjacent to), but I
           | assume my experience with Dell and HP commodity servers are
           | similar.
           | 
           | The thick layer of hardware contrivances necessary to
           | maintain IBM PC compatibility is unnecessary for the task of
           | bulk hosting of x86/x64 VMs. There's a lot of hardware and
           | software that just doesn't need to be there.
           | 
           | Bare metal out-of-band management ends up being bolted-on to
           | these "legacy" contrivances (scraping video memory for remote
           | consoles, faking being USB peripherals). A serial console or
           | SSH connection to a service processor would be vastly
           | superior. I can't begin to count how many times an iDRAC
           | "lied" to me about issues with a machine, or how many times
           | the solution was "upgrade the iDRAC firmware and reboot it".
           | 
           | I have been mostly unimpressed with the quality of firmware
           | for motherboards, baseboard management controllers, RAID
           | adapters, NICs, HBAs, power supplies, backplanes, front
           | panels, etc. Every new model of system or component ends up
           | being an exercise in fear / anticipation of problems. The
           | integrator has very little power over the firmware quality
           | and I can be assured that if I do have a firmware-induced
           | issue I'm many, many steps away from actually communicating
           | with somebody who can help.
           | 
           | Granted, maybe if I was buying at the scale of Oxide's
           | prospective Customers I'd have some pull with the
           | integrators, but I'm skeptical of that, even.
           | 
           | Oxide is actually building computers. Putting commodity
           | motherboards into boxes with other commodity components won't
           | ever have the level of attention to detail and integration
           | that Oxide can provide.
        
           | sjdmdlakziggy wrote:
           | They are not even close to the same specs.
        
             | upon_drumhead wrote:
             | https://www.siliconmechanics.com/system/rackform-a235.v8.1/
             | 2...
             | 
             | This is pretty darn close to the "Gimlet" Compute Sleds
             | 
             | One 225W TDP 64-core AMD Milan CPU 1 TiB of DRAM across 16
             | DIMMs 12 front-facing hot-swappable PCIe Gen 4 U.2 storage
             | devices 2 internal M.2 devices 2 ports of 100 GbE
             | networking
             | 
             | Point for point, the same hardware as specified.
        
           | intelVISA wrote:
           | As much as I wish Oxide the best, most of us here would
           | probably prefer our own libre, hand-crafted OS atop Coreboot.
           | 
           | (Or maybe I speak for myself only...)
        
         | jxf wrote:
         | > boot/management TUIs
         | 
         | Where are you seeing these in the docs? Or are those somewhere
         | else?
        
           | dist1ll wrote:
           | Here https://docs.oxide.computer/guides/system/initial-rack-
           | setup
        
         | all2 wrote:
         | > sorry to be that guy ... astigmatism
         | 
         | Darkreader is a lovely plugin that might make your life much
         | easier: https://addons.mozilla.org/en-
         | US/firefox/addon/darkreader/
         | 
         | I believe there is a Chrome extension, too.
         | 
         | It lets you set BG, FG, sepia, total contrast, etc. It is quite
         | a neat piece of kit.
        
           | bpye wrote:
           | I would never have thought to use an extension intended for
           | adding dark mode, to instead view dark mode websites in a
           | light mode, but that makes total sense!
        
           | pxc wrote:
           | I recommend Midnight Lizard as a backup for difficult sites.
           | It's a little heavier/slower, but sometimes works on pages
           | where Dark Reader doesn't. You can configure Midnight Lizard
           | not to apply to all sites by default, then selectively turn
           | it on where Dark Reader fails.
        
         | benjaminleonard wrote:
         | All of our sites and indeed the web console itself are driven
         | by the same design system and we are planning on a light theme
         | as soon as we can.
         | 
         | Thanks!
        
         | pxc wrote:
         | > high contrast dark themes are difficult to read for people
         | with astigmatism.
         | 
         | Are you yourself affected by this? I have astigmatism and I
         | keep hearing this, but I've never experienced it. If you are
         | affected, do you keep your screen at high brightness? I'm
         | wondering if it doesn't happen to me only because my
         | astigmatism is mild, or if the fact that I tend/have to keep my
         | screens at relatively low brightness plays a role.
         | 
         | Incidentally, for different reasons, high contrast dark themes
         | can be problematic for me as well (especially at high
         | brightness). Dark Reader and Midnight Lizard are essential for
         | me in keeping contrast in a comfortable range.
        
           | benjaminwootton wrote:
           | I was about to ask the same question. I think I have moderate
           | astigmatism and find dark mode easier on the eyes.
        
             | jna_sh wrote:
             | I have astigmatism and loathe dark mode. I find low
             | contrast and soft colours easiest.
        
       | whalesalad wrote:
       | Would be cool to play with this hypervisor on some lab hardware.
        
         | mkeeter wrote:
         | It's been done before!
         | 
         | See the excellent "Oxide at Home" blog post [1], and HN
         | discussion [2]
         | 
         | [1] https://artemis.sh/2022/03/14/propolis-oxide-at-home-
         | pt1.htm... [2] https://news.ycombinator.com/item?id=30671447
        
       | ChrisArchitect wrote:
       | What's new here
        
         | wmf wrote:
         | I think these docs were just published.
        
           | benjaminleonard wrote:
           | These have been online for a few months actually, though the
           | release notes have only just been added.
        
           | dang wrote:
           | OK, let's document that in the title above. Thanks!
        
           | slater wrote:
           | Yeah, few days ago:
           | 
           | https://docs.oxide.computer/release-notes/system
        
       | gorkish wrote:
       | As someone who has been attempting to smash computers together
       | into 'hyperconverged infrastructure' since before Y2K, I could
       | not be happier for the buzz around Oxide; hopefully we'll see
       | smaller scale versions of the concept out soon -- something in
       | the 3-10 node size for SMB and/or a small scale-out system for
       | SOHO. It seems insane to me that the tech to do this with off the
       | shelf free software exists, but there's no way to buy it ready to
       | go at small scale. Go get a quote for an HCI vmware buildout and
       | DISMAY.
        
       | drbawb wrote:
       | I am super interested in learning more about the storage
       | subsystem! I figured they'd be using ZFS, given the people
       | involved, but it appears they've also gone ahead and built a
       | clustered FS (crucible) on top of it? I figured something like
       | that would be necessary to handle fault tolerance at the gimlet
       | level. (Losing an entire shelf / drive controller, etc.) Getting
       | ZFS to go multi-node is surely a neat trick.
       | 
       | Second to that I just want to say the presentation of these docs
       | is top notch. (I so desperately wish I was the target customer
       | for these systems; reading these docs makes me want to do
       | terrible things to my electrical service and play with one of
       | these racks.)
        
         | kaliszad wrote:
         | They use Crucible on top of ZFS.
         | https://github.com/oxidecomputer/crucible I don't think they
         | have anything for S3-like service but there are other options
         | for that, e.g. https://garagehq.deuxfleurs.fr or MinIO. I am
         | not sure whether they have their own SSDs or use of the shelf
         | SSDs just with their firmware or something.
        
         | ori_b wrote:
         | They're not the first company to do that. https://panzura.com/
         | did something similar.
        
       | oxide wrote:
       | This is great, thanks for bringing this to my attention. Cool
       | name, too.
        
       | solarkraft wrote:
       | Finally I actually understand what they're building. Now I must
       | ask: Why?
       | 
       | On-prem servers aren't a new invention. The market seems pretty
       | saturated (and shrinking). Virtualization isn't a new invention
       | either. The market seems pretty well served, at least
       | commercially. Can the integration of both be a convincing enough
       | advantage?
       | 
       | The management UI certainly looks nice; it's something I'd like
       | to have on my KVM box at home (any good Proxmox alternatives?). I
       | don't see why it'd have to be bound to an enormous server.
        
         | rtpg wrote:
         | The short answer is that on-prem is used by a lot of companies
         | for many reasons that go beyond "legacy/people don't know about
         | cloud" (and even go beyond "regulatory environments"!), and
         | these boxes are meant for people who have serious requirements.
         | 
         | I'd recommend the O+F ep posted in sibling, but I think here
         | the pitch is "well you need this hardware anyways right? How
         | about buying one that's easy to use and doesn't take a month to
         | get working?" All built by people who are so obsessed with root
         | cause analysis that they've ended up writing their own
         | firmware, running on an OS where these people are common
         | contributors.
        
           | solarkraft wrote:
           | Thanks, I think _now_ I get the excitement (from the
           | engineering perspective more than the business one)!  "Fixing
           | everything" is probably everyone's dream.
        
         | faitswulff wrote:
         | One benefit is that they're competing in an industry where the
         | time to get a sled up and running for compute can be measured
         | in weeks (they say they've heard up to 90 days) whereas their
         | solution is basically plug and play - Bryan was trying to get
         | Steve to admit set up took "hours" whereas Steve was hedging
         | and saying customers could get started "within a week."
         | 
         | They go into more details on their podcast, and this section in
         | particular covers the bootstrap time:
         | https://youtu.be/5P5Mk_IggE0?t=3381 Pretty fascinating stuff.
        
           | throwawaylinux wrote:
           | Do I have it right that they ship their own hypervisor
           | (that's based on maybe Solaris, not KVM) as firmware? Let's
           | assume a very small team can compete on a technical level, it
           | still seems like it could cut out a lot of the potential
           | market.
           | 
           | I can't imagine that large cloud / "web scale" companies
           | would want that. Most want a fair bit of control of their own
           | hypervisor and management stacks based around KVM. And
           | "enterprise" type companies are going to have issues with
           | certification I would have thought -- will RedHat, Microsoft,
           | SAP, Oracle, etc certify their supported products on top of
           | this hypervisor? Seems like a difficult and expensive
           | process.
           | 
           | So what's left? Companies that support their own virtual
           | machine software but don't support their own hypervisor and
           | don't like what's available from vmware or Microsoft or
           | RedHat. A small niche. Or are my assumptions wrong?
        
             | panick21_ wrote:
             | I think quite often when we assume 'most want to fair bit
             | of control' is just not true. Enterprises want something
             | that just works, they want control if they can't have
             | something that just works.
             | 
             | If you have a team that is struggling building an internal
             | cloud with all this control (and problems) and all this
             | commodity hardware (and its problems) then maybe they would
             | be happy to switch to something that just works.
             | 
             | > And "enterprise" type companies are going to have issues
             | with certification I would have thought -- will RedHat,
             | Microsoft, SAP, Oracle, etc certify their supported
             | products on top of this hypervisor? Seems like a difficult
             | and expensive process.
             | 
             | If that was the case and nobody running any of these would
             | run their rack, then I wouldn't think they would not have
             | received any funding. But I don't know enough about these
             | certification process to really comment.
             | 
             | > Companies that support their own virtual machine software
             | but don't support their own hypervisor and don't like
             | what's available from vmware or Microsoft or RedHat
             | 
             | Non of these come with a fully integrated rack.
             | 
             | The competition would be somebody willing to buy a rack of
             | Dell servers with VMWare software. Or somebody willing to
             | buy a rack of Dell server and then use RedHat and set up
             | all their own cloud style infrastructure.
        
               | throwawaylinux wrote:
               | > I think quite often when we assume 'most want to fair
               | bit of control' is just not true. Enterprises want
               | something that just works, they want control if they
               | can't have something that just works.
               | 
               | That's not what I'm assuming here. Read carefully, I
               | divide the market into 3 categories. Those who support
               | their own VM image software and hypervisors, those who
               | support neither, and those who support VM image but not
               | hypervisor.
               | 
               | First is Amazon, Google, Facebook and the like (and it's
               | not an assumption we can see their public contributions
               | to KVM, QEMU, etc., and hear their talks about some of
               | what they use internally). Second is "enterprise" who
               | wants something that just works. Third is ? and would
               | they want to support their software on a niche
               | hypervisor?
               | 
               | > If that was the case and nobody running any of these
               | would run their rack, then I wouldn't think they would
               | not have received any funding. But I don't know enough
               | about these certification process to really comment.
               | 
               | Well it is the case that enterprise (supported) software
               | is not just supported on any hypervisor.
               | https://access.redhat.com/articles/973163 RHEL runs on
               | their own KVM as well as MS, VMware, some cloud vendors.
               | Some application software also gets certified to hardware
               | and hypervisors, not just operating system (e.g., SAP
               | does this).
               | 
               | > Non of these come with a fully integrated rack.
               | 
               | It's not fully integrated if it doesn't come with the
               | guest software though, is it?
               | 
               | > The competition would be somebody willing to buy a rack
               | of Dell servers with VMWare software. Or somebody willing
               | to buy a rack of Dell server and then use RedHat and set
               | up all their own cloud style infrastructure.
               | 
               | Right. And the problem for Oxide is that the competition
               | will have fully certified and supported operating system
               | and application software for their virtual machines.
        
             | kaliszad wrote:
             | The hypervisor is based on Bhyve from FreeBSD + Propolis in
             | user space. Illumos actually has/ had KVM and there is a
             | talk by Bryan Cantrill where he speaks about the porting
             | effort. All of that information is readily searchable.
        
         | [deleted]
        
         | throwaway2037 wrote:
         | Stupid question (no trolling, I promise): Could this product be
         | valuable to public cloud vendors, like AWS, Google, Oracle,
         | IBM, etc? Or even medium sized ones, like Linode? My thinking:
         | Could this be the cloud inversion moment akin to TSMC and
         | outsourcing semi-conductor manuf?
        
           | dbish wrote:
           | Probably not. These vendors have massive supply lines that
           | are designed and tuned to their needs.
        
           | littlestymaar wrote:
           | For big vendors, no. Because what oxide does is basically
           | sell the kind of server that the hyperscalers have been
           | building for themselves internally. But for smaller cloud
           | provider, who aren't running custom hardware made for
           | hyperscale and are instead using of the shelf servers with
           | all their flaws, then it could make a lot of sense.
        
         | chamakits wrote:
         | I'm not affiliated with them, but I recall this being marketed
         | at some point as giving you the flexibility and customization
         | powers that the Googles and Facebooks of the world have with
         | their on-prem infrastructure without needing to have as deep of
         | a dedicated staff as they do to just this which allowed them to
         | develop all their custom tooling in the first place.
         | 
         | Basically if you are on-prem, and you are dissatisfied with
         | what you are getting out of today's onprem sellers. Things like
         | bad firmware with slow update cycles, issues with rack/power
         | supplies/cabling/interconnecting systems. Closed down systems
         | that don't allow much customization, etc. They are open
         | sourcing a lot of their work along the way
         | 
         | Again, I'm not affiliated with them, and my info may be
         | outdated so take it with a grain of salt. But that's how I've
         | seen them for some time.
        
           | wmf wrote:
           | I would say Oxide is inflexible and non-customizable since
           | they have exactly one hardware configuration and few software
           | features at this point. Their claim is more that their rack
           | works and everything else on the market is full of bugs.
        
             | throwaway2037 wrote:
             | I'm not an infra engineer, but this claim "everything else
             | on the market is full of bugs" might be the killer app. Of
             | course, it needs to be true. What if they iterate to an
             | insanely stable embedded code base (BIOS, etc.)? Then,
             | continuously upgrade the hardware to use the latest
             | CPU/RAM/NVME. I could see that being very valuable.
        
         | strangemonad wrote:
         | A dumbed down interpretation. What most people can buy off the
         | shelf for servers more or less looks like a pc shoved into an
         | odd looking case (1u, 3U rack). To get anywhere near the cost /
         | performance of the big players you need something that's
         | designed for the data center server workloads.
        
         | bpye wrote:
         | It feels like they're trying to give people the benefits that
         | hyperscale cloud operators have, without being one?
        
           | superb_dev wrote:
           | That is exactly their goal. Hyperscale for the rest of us
        
         | jitl wrote:
         | Public cloud like AWS is a premium product. When your web scale
         | business wants to increase profits by cutting costs and you've
         | already done a few passes making your software run faster,
         | owning your own metal & renting colo space starts looking like
         | a big avenue for savings. Especially if you're doing something
         | bandwidth intensive, where AWS makes you pay through the nose.
         | I think we'll see a fair number of companies move back towards
         | owning metal, especially if the metal is super easy to manage.
         | I think the Oxide pitch is making owning racks sensible for
         | ~2000 eng companies instead of ~20,000 eng companies.
        
         | mhh__ wrote:
         | YOLO - sometimes it's worth paying more money for something
         | someone has actually sat down and designed properly.
        
           | throwawaylinux wrote:
           | So what has been designed properly and what hasn't?
        
             | mhh__ wrote:
             | The random piles of servers in most offices above a certain
             | headcount
        
         | panick21_ wrote:
         | Cars aren't a new invention either, and yet people build new
         | ones.
         | 
         | There is an existing market, it is large. Making a product for
         | that market that is just simply 'really good' has potential to
         | make money.
         | 
         | Almost all great computer companies started into markets where
         | things already existed that were comparable.
        
       ___________________________________________________________________
       (page generated 2023-07-21 23:02 UTC)