[HN Gopher] John Carmack on lead time to push fixes in startups
___________________________________________________________________
John Carmack on lead time to push fixes in startups
Author : patapong
Score : 58 points
Date : 2023-02-17 19:34 UTC (3 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| jobs_throwaway wrote:
| >The time to deliver a fix to VR users at Meta was often over six
| months, disregarding all the problems that were just ignored.
|
| I find it astounding that this could be true for someone as
| senior and respected as Carmack. Is it because he's working in a
| hardware division? What takes so damn long?
|
| Side note - its interesting how Carmack has come out and publicly
| bashed Meta so much. I guess you can do that when you're in his
| position.
| Trasmatta wrote:
| Maybe the regular, 50 people all bikeshedding every single tiny
| decision rather than just letting the people who know what
| they're doing get their job done?
| potatolicious wrote:
| > _" Is it because he's working in a hardware division?"_
|
| Likely? Certainly from all appearances from the outside there
| are many legitimate bones to pick about Meta's organizational
| sclerosis, but part of this is structural to the product
| itself.
|
| "I can fix the image quality issues with X relatively easily, I
| know exactly what needs to be done"
|
| elicits (rightfully!)
|
| "We operate on an intensely compute-constrained device where
| every cycle matters. Over-consumption of compute causes battery
| life to degrade significantly and the device to become
| uncomfortably warm to wear. How much compute will this solution
| cost?"
|
| Then you have to balance the additional compute with drawing
| down compute somewhere else. Maybe the tracking camera
| algorithms can be optimized to consume less compute, which
| opens room for additional usage elsewhere, but now you've
| introduced a blocking dependency to delivering this
| enhancement.
|
| Even in a well-functioning organization this prevent shipping
| as fast as a startup. Part of why startups ship fast isn't just
| organizational, it's because the products are simpler with
| fewer technical constraints.
|
| Hardware (particularly embedded hardware - which Meta Quests
| are not precisely, but are pretty close) presents _intense_
| levels of constraints that most people who write code on
| servers (or even PCs) don 't often appreciate.
| tormeh wrote:
| Meta Quest 2 has 6GB of DDR4 RAM. It has a 64bit CPU. I'd say
| it has more in common with a server than with an embedded
| device.
| ginger2016 wrote:
| Maybe his ego got hurt at Meta. Why would you publicly bash
| your previous employer. Isn't staying quiet the right thing to
| do?
| OnlineGladiator wrote:
| > Why would you publicly bash your previous employer.
|
| Maybe because they suck and deserve to be criticized?
|
| > Isn't staying quiet the right thing to do?
|
| It's certainly the easier thing to do. It's a sad world we
| live in where being 'professional' means being dishonest.
| tormeh wrote:
| He's a true VR believer. I think it pained him to watch it go
| less well than it could.
| MikeTheRocker wrote:
| Even software can move slowly because everything has to go
| through numerous levels of bureaucracy and review. Before you
| even start working on something, you often need to have
| meetings with stakeholders from various teams and disciplines
| to "align" (read: convince them to do the work on their part of
| the stack). Then on top of that, the Quest OS update shipping
| cadence is monthly.
|
| Source: worked at Meta on Quest system software with Carmack
| 2018-2021
| [deleted]
| skellera wrote:
| I'm sure things used to be this way but "we're like a startup" is
| so overused in big tech these days. It's usually an excuse for
| something wrong rather than the benefits of a startup.
|
| I wish we were building for customers instead of building for my
| or someone in my leadership chain's promotion. I think most
| companies start with that focus then lose it after a certain
| size. Most of the engineers working on projects don't even care
| about the product.
| rgblambda wrote:
| > "we're like a startup" is so overused in big tech these days.
| It's usually an excuse for something wrong rather than the
| benefits of a startup.
|
| I've only ever heard that phrase or something approximating it
| used to excuse bad practices.
| halvo wrote:
| My "we're like a startup" experience was all the uncertainty of
| a startup along with all the bureaucracy of big tech which
| resulted in pivoting to a new 12 month roadmap every 6
| months...
| allenu wrote:
| I worked in a "startup-like" team at a big company a few years
| ago. It was fun in that we moved quickly since we had fewer
| processes than typical teams at Big Co. Over time though, I
| realized we were just making stuff to make some higher-up
| happy, and not really chasing a customer.
|
| I also found out we were funded just to light a fire under
| another team that moved very slowly. Whatever we made would be
| taken back to that team and we'd have to sell them on using it,
| and also show them what was possible.
|
| Ultimately, we were just a cog in this larger machine. Nowadays
| I don't really trust anybody at big companies that says "our
| team is like a startup".
| simonw wrote:
| Maintaining that velocity inside a larger organization is a
| really interesting challenge.
|
| There are things you can do that can help. The biggest in my
| experience is robust automated testing combined with robust
| automated deployments
|
| Continuous deployment (any code that passes the tests gets
| deployed) is incredible if you can get there, but the next best
| thing is continuous deployment to a staging environment followed
| by one-click-deploys by trusted release managers to production.
| ginger2016 wrote:
| Startup probably worked on Carmack's request because they were
| afraid he will post a negative review or give negative review
| about the startup to his friends. A startup can't afford to piss
| off a person like John Carmack, they might go out of business,
| because it might impact their ability to get new customers and
| raise capital.
|
| Meta on other hand could care less about John Carmack's opinion
| as it has no impact to their business. In short John Carmack's
| opinion was ignored it had no impact for a large company.
|
| The mistake John Carmack is making is that he thinks his views
| are special, they are not.
| alexashka wrote:
| Is that why Meta never hired John Carmack? Because his opinions
| have no impact on their business?
| qbrass wrote:
| No, it's because he never applied for a job there.
| Sakos wrote:
| I work at a small company and customer issues or requests can
| be solved (or worked on) within a day because there's almost
| zero red tape. And it doesn't need to be Carmack pointing out
| the issue or delivering the request for it to be done.
| bombcar wrote:
| I don't know that's true, and he may not have even blathered
| his name out there.
|
| I know that I got some _really_ responsive work from startups
| we purchased from, and it was very nice, and we were just one
| of their customers.
|
| Sadly, once they pivot away from whatever it was then it all
| goes away.
| codetrotter wrote:
| > I was pretty sure I knew what most of the problems were, so I
| got in touch with the developers, and everything got sorted out
| in a few days (they were already aware of some issues).
|
| Yeah but like, I get that startups are better at listening to
| customers and adjusting quickly.
|
| But I can confidently say.. if John Carmack himself legit sent me
| an email pointing out some problem with something I made. You bet
| your ass I would respond and listen to what was on his mind.
| Regardless of the size of my company and the amount of work on my
| plate :p
| jayparth wrote:
| But he was doing that daily at Meta, and it still took 6 months
| for things to happen!
| [deleted]
| bmitc wrote:
| Do we know that the things he wanted to fix were important
| enough to fix?
| nh23423fefe wrote:
| Yes we know that.
| bombcar wrote:
| Something something ancient Chinese parable the only person
| not impressed by the General/Emperor/Mystic Sage is his wife.
| yuppie_scum wrote:
| I bet more than half the kids working there weren't even born
| when Quake came out
| Kranar wrote:
| But his entire point is that's not what happened at the big
| company he worked at.
| stonemetal12 wrote:
| Yeah but at a big corp it is more than just you. Even if you
| fixed it are you authorized to get a build out, out of cycle?
| If you work on an App, can you get it through the app store
| approval process?
| [deleted]
| rcme wrote:
| At my last company we had a Discord that our users would use to
| give use feedback, bug reports, and ideas. It was amazing. The
| feedback itself was really helpful in building the product, but
| you instantly acquire a user's loyalty the first time they report
| an issue and you fix it that day. Users are really grateful for
| that.
| bmitc wrote:
| Isn't this him just bragging, which gets extrapolated to some
| platitude about startups? By the same token of startups
| supposedly being able to fix things faster, which is debatable,
| it is more likely that they have more problems by moving faster
| without that many users. Things change a bit when you have
| millions, hundreds of millions, and even billions of users.
| Barrin92 wrote:
| You can't and really should not expect this kind of speed from a
| large company. In this case he used a service where you can pay
| 170$ to get a custom satellite image from space. My guess is they
| treat every customer individually and the technical issue
| reported was obvious.
|
| At Meta you have hundreds of millions if not more users and any
| issue largely isn't so obvious that you can just go and fix five
| lines of code. When someone comes around and says "hey I'm Bob
| xyz isn't working" you probably need to figure out first if
| that's a real issue at all and if fixing it doesn't break ten
| other things.
|
| The potential downside is much, much larger if you overreact to
| individual complaints at that scale.
| baby wrote:
| Honestly I don't know what to do with this tweet + HN thread
| because it leaves little room for actual discussions and a lot of
| room for misguided guessing.
|
| Unless someone from the org comments here, which I doubt
| considering the animosity HN has at VR or Meta, we'll never know
| why oculus didn't work like that, if it was a real issue there or
| if Carmack is exaggerating, if it was an issue are tweets like
| going to help the org rethink itself?
|
| The only signal we're getting, from the outside, are the
| releases. I had the go and the quest and the gap in technology
| between the two were amazing.
|
| I don't know about quest to quest 2, or quest 2 to quest pro, and
| I guess we'll see what the quest 3 delivers. Maybe other people
| know?
| hideo wrote:
| I have great respect for John Carmack but this feels like a very
| unnuanced take.
|
| Lead time isn't about "startups", it's about code complexity,
| side effects and blast radius.
|
| The time it takes to push code when you have 10 use-cases and
| 1,000 users, is very different from the time it takes to push
| code when you have 1,000 use-cases but 10,000,000 users. At that
| scale, it takes A LOT of effort to keep fix times small. It isn't
| going to occur naturally.
|
| Some times the effort to keep that fix time small may not be
| worth it, some times it might.
| duxup wrote:
| I work on a 4 person team. At one point it is was two of us.
|
| I love a quick fix where I tell the customer "try it again" done.
___________________________________________________________________
(page generated 2023-02-17 23:02 UTC)