[HN Gopher] Uber's crazy YOLO app rewrite, from the front seat
___________________________________________________________________
Uber's crazy YOLO app rewrite, from the front seat
Author : gregdoesit
Score : 188 points
Date : 2021-05-24 17:41 UTC (5 hours ago)
(HTM) web link (blog.pragmaticengineer.com)
(TXT) w3m dump (blog.pragmaticengineer.com)
| sneak wrote:
| > _I have no idea how, but two and a half-hours later, the fix is
| live. I 'm assuming the release team used all the goodwill
| accumulated over the years to make the fastest app update I've
| ever heard of happen. Either way, it's done, and done fast. Let's
| just hope it works._
|
| It's sort of amazing how we as an industry have all just adapted
| to the App Store industrywide censorship and gatekeeping.
|
| It's astounding that Apple has been able to pull this off, and
| even more astounding that so many companies are willing to work
| as sharecroppers for them.
|
| Find it on the Apple App Store under AOL keyword Uber!
| [deleted]
| superasn wrote:
| This whole blog post reads like The phoenix project book. I
| actually chuckled at the part where the legal got involved for
| the client-side fix like it so used to happen in the book.
| dvh wrote:
| >Two weeks before my starting date, my manager emails me to ask
| if there's any way I can do Android.
|
| I've solo made and published over 15 Android apps but I went full
| Dunning-Kruger and now I think I know nothing about Android
| development. It is too complex. If manager asks me, I don't know
| Android. I can't imagine Android development can be known.
|
| If you ask someone if they know Android development and they say
| yes, be skeptical.
| zibzab wrote:
| To be fair, this was in 2016 :)
| dudul wrote:
| I'm surprised to read that most people left _after_ the nightmare
| instead of during.
|
| I would have taken this shit for a week, maybe 2, but past that,
| I'm out of here. If you're gonna stick around during the whole
| thing to see it through, why leave once it's over?
| bogwog wrote:
| Burn out.
|
| This first time I did a game jam many years ago, I stayed up
| for nearly 72 hours to get it finished. I slept maybe less than
| 8 hours that entire time, I felt like absolute shit towards the
| end. Nausea, headache, exhaustion.
|
| The game was completed, but to this day, if I even look at a
| screenshot of that game or hear one of the sound effects, I
| start to feel nauseous almost immediately.
|
| I'd imagine some people on this project had a similar (although
| maybe not as extreme) reaction.
| meheleventyone wrote:
| A lot of people want to stick around to complete a project as
| it looks good on your CV. Particularly at a junior level.
|
| From my experience (in games) it's also quite intoxicating at
| the time and you get quite emotionally invested in supporting
| the team and fixing issues. In particular crunch is partly
| enabled because you get close through the adversity. You
| usually don't get much opportunity to take stock until
| afterwards being so caught up in the moment. It's pretty common
| to get a rash of departures right after it all calms down.
| silexia wrote:
| They didn't have a choice. Imagine their reference check for
| future jobs where Uber HR tells the new potential employer "X
| left during the start of crunch time for our new app". You have
| to suffer through, then leave.
| dudul wrote:
| Fair enough, but honestly, I would probably just be open
| about why I left Uber. I would want to make sure I would not
| end up in the same kind of toxic environment.
| psyc wrote:
| HR is not going to snipe an ex-employee in solidarity with
| the new employer. As a rule, they only confirm the person
| worked there when they said they did.
| mschuster91 wrote:
| > As a rule
|
| Rules are there to be broken, and even those mandated by
| law will get broken (see for example the non-compete cartel
| drama from a couple years back).
| tester756 wrote:
| >The pace of work is still excruciatingly slow. Tooling is a
| major issue: compiling the app from scratch takes 25-30 minutes
| on iOS and around 10 on Android.
|
| what the fuck?
|
| HaLf HoUr To CoMpIlE aPp?
| yboris wrote:
| I'm curious, what would have been the repercussions of an
| engineer saying "I'm comfortable with at most 40 hours of work
| per week". Would they have been fired?
|
| Also, were the people working longer hours getting paid overtime?
| didibus wrote:
| I kinda hate this engineer, he didn't push hard enough, he let
| the professional bar be crossed, there was seemingly no business
| value in the whole project either. It would be refreshing if the
| devs just didn't take this, those that quit after should have
| quit during the project instead.
|
| I think there's a saying that goes: "you get what you're willing
| to tolerate".
|
| This is often the reason why employees get screwed over, they're
| willing to tolerate far too much, for far too little reasons.
|
| Covering up bad management and planning is not something
| engineers should tolerate. You should say, no sorry, you missed
| the deadline to fund us and have engineering start early enough,
| that's that, you screwed up your planning, you failed at being a
| good manager. We're not going to cover for your screw up.
|
| Edit: Specifically, I don't personally hate them, but I hate that
| it played out this way, and I know it's a difficult position to
| be in and it's difficult to get out of, it just makes me mad, and
| I don't know what solutions there are.. maybe to fight to change
| the norm back to hourly employment, or have contracts with a per-
| week maximum hour cap that if breached warrants bonus pay on top
| of salaried employment.
| TazeTSchnitzel wrote:
| Not everyone wants to martyr themself to save management from
| hoisting themselves on their own petards.
|
| Especially when it's just a taxi app.
| Syntaf wrote:
| Props to the author for a great read!
|
| Just recently I had to pull a couple 12 hour days resolving a
| critical issue for an enterprise customer; I'd never been so
| ready to hang up my keyboard and move into a cabin in the woods
| afterwards. I can't imagine extending that crunch for _months_.
| nosefrog wrote:
| I have a lot of friends who worked at Uber around 2015. I didn't
| accept an Uber offer because they were all insanely burned out.
| calcsam wrote:
| Here's a Twitter thread about the same story, that dives a bit
| more into the internal politics --
|
| https://twitter.com/StanTwinB/status/1336890442768547845
| robocat wrote:
| I like this quote: "A bunch of folks burned out along the way.
| A ton of money was spent, hard lessons were learned, but still
| to this day most people insist the rewrite was all worth it.
| New engineers who joined up loved the architectural consistency
| and never knew the pain it took to get there."
| jgalt212 wrote:
| Celebrity board members in a nutshell.
|
| > Ariana Huffington joins Uber's board around the same time and
| releases her book on the importance of sleep right around this
| time, and piles of free copies are available in the office. Yes,
| sleep is important. So why am I at the office past midnight?
| psing wrote:
| My takeaway -- even the highest regarded engineering orgs are
| still shitshows behind the scenes.
| cobaltoxide wrote:
| Are you referring to Uber as "highest regarded engineering
| org"? :grimmace:
| whalesalad wrote:
| In my personal experience, the more the org gives off the
| impression that it is hot shit, the worse it usually is.
| cghendrix wrote:
| What about Netflix?
| screye wrote:
| Yep, In my experience :
|
| Very high standards = Slow shipping = Boring
|
| You can bet that working on the Google Ads teams is far more
| 'boring' than Stadia or Glasses. Similarly, at my FANGAM
| company, the most boring teams are also the most stable.
| toast0 wrote:
| > Very high standards = Slow shipping = Boring
|
| I don't think this is quite right.
|
| Slow shipping is tied to coordination more than standards
| (although if the standards are enforced by testing from
| another team, there's your coordination). If the standards
| are self enforced, you can still ship frequently, but maybe
| not a lot of big releases. Small teams and decoupled work
| where possible helps.
|
| I can only quibble about boring though. The code changes
| may not be exciting, but the environment can be. Some
| external event that spikes traffic can test your high
| standards, and may need remediations (in which case, I hope
| you can ship fast). Of course, boring is actually nice.
| tshaddox wrote:
| It took me a second to correctly analyze the sentiment of
| "hot shit." I realized it means "really good," as opposed to,
| for example, "steaming pile of shit" which means the
| opposite.
| fration wrote:
| It also generally has the connotation of someone having an
| inflated opinion of themselves.
| ChicagoBoy11 wrote:
| You'll enjoy this exploration on this precise topic:
|
| https://www.youtube.com/watch?v=ifNEgwGACEQ
| samuli wrote:
| Here's a link to the original video from the actual
| standup comedian in the video
|
| https://www.youtube.com/watch?v=igh9iO5BxBo
| robocat wrote:
| The expression "shit hot" is a superlatively good thing. I
| don't ever recall hearing "hot shit", although perhaps it
| is a US term or a modern usage. Urban dictionary defines
| both.
| lamontcg wrote:
| "Hot shit" is a US term, and it is old and was around in
| the 80s at least.
| reaperducer wrote:
| Longer than that. I saw it in a book from the 1950's.
|
| I expect it's a variation of "hot stuff" which I've heard
| in songs from the 1930's.
| lumost wrote:
| The more the org values external appearance, the more the org
| justifies projects based on external appearances.
| twic wrote:
| Sometimes i think about trying to get a job at Netflix just
| so i can find out what it's really like.
| ryanianian wrote:
| I had 6+ years at a FAANG. It was awful. I would never go
| back. I happily took a 20% pay cut to go elsewhere. This
| said, I learned a lot (both good and bad) and it has been a
| good reference point for my career.
|
| So I recommend all engineers to strongly consider a FAANG
| while they're young and can tolerate a bit of burnout.
| Learn how the big companies do it, pay attention to what's
| good and what's bad, and keep that perspective with you in
| your career.
|
| But be honest with yourself ahead of time about what would
| make you leave and follow through with that. "Golden
| handcuffs" of high pay is a real thing.
| notyourwork wrote:
| Not all teams in FAANGs result in burn out or require
| careful through about joining. I've been at one of them
| for 6 years and still find my work life balance to be
| sane. In fact I enjoy what I'm doing quite a bit because
| of the business space I'm working in and the quality of
| peers.
| silicon2401 wrote:
| what do you consider sane work life balance? I consider
| anything over 40h excessive
| notyourwork wrote:
| Yes. Aside from on-call week when I carry a pager 24-7
| for the week but I don't work outside business hours
| unless paged.
| silicon2401 wrote:
| Yes what? I asked "what do you consider sane work hours?"
| lol.
| notyourwork wrote:
| Because it was obvious, we know what full time is defined
| as. Anything more than 40h is more than a full time job.
| jefftk wrote:
| Another data point: I've never felt pressured to work
| more than 40 hours a week, and have probably averaged
| quite close to that over my FAANG career. Recently, with
| pandemic/WFH/less-to-do-outside-work, I started working
| some on evenings/weekends, and my manager asked me to
| stop because he doesn't want to have that kind of team
| culture.
| notyourwork wrote:
| +1 It can create a really toxic work culture. If I do
| work more its because I'm interested in a problem that I
| don't want to let sit till next week or the following
| day.
|
| In general, I try to do this discretely by not pinging
| people outside working hours or trying to give impression
| I'm working late. Working late for me is usually working
| through code or a design at a time when I'm caught in it
| and don't want to let it sit until I solve it. If this
| happens I'll start late next day or reduce hours
| following week and I make sure I call this out during
| standup so my team knows why I'm signing off early.
|
| It can become a snowball when one teammate sees another
| working and feels compelled to do same. Kudos to your
| manager for calling this out.
| enterdev wrote:
| what was wrong with it?
| notyourwork wrote:
| It likely depends on the team and area of company. Any
| company of reasonable size will have a distribution similar
| to bell curve of teams ranging from excellent to complete
| shit shows. Most fall somewhere in the middle with diverse
| pros/cons depending on the tech stack, connectedness to
| customer and where their software falls in its lifecycle
| from shiny new to dusty needs replaced.
|
| You get lucky rarely with an awesome team and usually fall
| in middle with some gripes. If you get into a shit show
| head for the door as fast as you can or stick around to
| learn a thing or two about what NOT to do.
| structural wrote:
| Yeah, but if you're not already in the job, the
| distribution of "teams that are currently hiring external
| candidates" can be quite skewed towards the worse side of
| things, and if you don't know this going in, you'll have
| a very bad time at most large organizations.
|
| A successful strategy is quite frequently to treat your
| first team at a company as just the second phase of an
| interview to get into one of the better teams in the
| company. Depends on the company, to be sure, but I've
| definitely seen many instances of "we're a good team, we
| know it, we'll happily poach good engineers from any team
| we like". It helps that the good engineers are going to
| be unhappy with their current team and looking to make
| the switch, anyways.
|
| If you don't switch teams within your first year or so,
| it usually means you a) got very lucky at random, b) got
| into a good team because you knew someone internally with
| good info and were able to short-circuit the process, or
| c) you failed to network/interview well enough internally
| to move up, and you should probably either be okay with a
| dumpster fire or move to another company.
| HowardStark wrote:
| Funny, that was my first thought as well. Recently read
| their engineering blog post from 2012 on the API gateway +
| UI system [1] and thought it was really cool and made a lot
| of sense (in their specific, truly massive use case).
|
| From recent memory, I can't recall Netflix really
| reinventing the wheel a-la-OP/Uber.
|
| [1] https://netflixtechblog.com/embracing-the-differences-
| inside...
| tomnipotent wrote:
| > I can't recall Netflix really reinventing the wheel
|
| Netflix has many private and public projects that have
| alternatives that could have been adopted. I'm glad they
| didn't, because Netflix releases quality product and I'm
| super appreciative of their contributions.
|
| https://github.com/netflix
| protonimitate wrote:
| I've personally never thought of Uber engineering as the
| pinnacle of contemporary engineering teams, is that a common
| sentiment for others? I know they're popular, but I thought it
| was mostly due to the pay scale / name recognition, not
| technical quality.
| sdljfjafsd wrote:
| IMO, I would not put Uber engineering in the "highly regarded"
| bucket.
| trevor-e wrote:
| If I'm reading this correctly, the whole fiasco started because
| the CEO decided they _have_ to release a new version of the app
| by some arbitrary date? And to meet this arbitrary deadline, the
| engineers had to roll out a ton of new features all at the same
| time inevitably causing a ton of fires/bugs/outages for the end
| users? Sounds like a nightmare and I'd be super pissed as an
| engineer having to support this mess. Is there more background on
| why Helix had to be released so aggressively?
| Tomte wrote:
| At my former employer deadlines were often not even ostensibly
| about some rational scheduling or roadmap, but simply "the
| birthday of the long-deceased husband who made the company big
| and successful".
| edoceo wrote:
| Slightly better than random tho right? Cause you can plan
| around predictable things.
| jccooper wrote:
| I've done the deadline-via-management-boast thing, and it's
| highly corrosive. Burnout's especially bad when you look back
| when it's all done and ask "why?" and there's no answer.
| thesausageking wrote:
| What I don't understand is why they did a complete rewrite. If
| the CEO wants a "new app", you get to work doing a redesign of
| the UX and a refactor of the code base. You usually end up with
| a better product and better code, and it gives you the
| flexibility of not being all or nothing.
| staplung wrote:
| Burnout was an obvious cost to this mess. A less obvious cost
| must surely have been all the people that read about it and
| decided to never work for Uber. But then it's always been easy to
| ignore the costs that can't be measured.
| raylad wrote:
| Another example of out-of-touch upper management dictating an
| unrealistic development schedule for no particular good reason
| (according to the article), and making everyone's life hell for
| months.
| PragmaticPulp wrote:
| I've heard about this rewrite multiple times now from different
| perspectives, both public and private. It's interesting how much
| the narrative changes based on the context of how it's being
| told. This author seems genuine and honest, but it's also clear
| that the story is being spun in the most positive way possible.
|
| The previous thread here has a different perspective on the
| situation: https://news.ycombinator.com/item?id=25373462
| jonas21 wrote:
| If you read to the bottom of the Twitter thread, it's pretty
| consistent with what we're hearing here.
|
| It sounds like they're both saying there were lots of near-
| disasters, but these were overcome, and the app successfully
| shipped - however at the cost of burnout in the team. I
| wouldn't say either is putting a more positive or negative spin
| on it than the other.
| plorkyeran wrote:
| Is it really a different perspective? This author is trying to
| do their best to give it a positive spin but the article is
| still a description of an unforced disaster that miraculously
| worked out for Uber.
| influx wrote:
| I worked at Uber and this whole shit show was typical and
| unnecessary. From the time, one of the things that TK was so
| proud of was the logo redesign and the design of one of the
| floors in the SF HQ that had a bunch of scifi elements.
|
| What was the benefit customers got out of any of this?
| golergka wrote:
| Theoretically, in a different company, great office design
| would make employees happy and more productive, for example.
| But I'm yet to see a better office design than a remote-first
| company.
| askafriend wrote:
| Building the company is just as important as building the
| product. The company is more than the product.
| noobermin wrote:
| I hope this is sarcastic because if not I'm going to frame
| this very characteristic Hacker News comment.
| dang wrote:
| " _Please don 't sneer, including at the rest of the
| community._"
|
| https://news.ycombinator.com/newsguidelines.html
| notyourwork wrote:
| Not everything has to benefit customers to stroke a founders
| ego.
| deegles wrote:
| Or to get engineers promotions.
| notyourwork wrote:
| POA: Promotion oriented architecture is a frustrating form
| of castle building.
| cancan wrote:
| i worked on the backend for this rewrite, and it was even more
| extreme than the other write-ups suggest. we not only rewrote the
| apps, we also 1. moved from email-based auth to
| phone-based auth 2. removed all the auth and onboarding
| logic from the clients and moved it over to server 3.
| changed the networking from a hacky OpenAPI/Swagger code to
| Thrift over HTTP
|
| i am not saying it was the most conservative call to do all this
| at once but it was necessary. the auth logic was basically
| untenable and no one really knew how it worked. some team in
| amsterdam could (and did) lock down uber for android users in
| russia because they changed some code.
|
| we also had to move to phone as a primary identifier, because
| honestly there are many countries people dont even use email. but
| how do you deal with phone numbers that were validated more than
| 7+ months ago, if you want to avoid things like this? [1] or how
| do you deal with promotions that required validating phone
| numbers not being VoIP that you could do out-of-band before but
| now have to do in-band?
|
| anyway, i am being scatter minded here but it was probably one of
| the riskiest things uber did, from my point of view, changing the
| top of the funnel + also moving users from one auth to another
| with a download, esp given with apple's rollout mechanism we had
| no way back.
|
| but it all worked! i should probably write this up one day.
|
| 1: https://arstechnica.com/information-
| technology/2016/02/when-...
| anirudhan wrote:
| deal with phone numbers that were validated more than 7+ months
| ago, if you want to avoid things like this? [1]
|
| Can you expand on how apps handle this usecase?
| cancan wrote:
| The short answer is, you need to validate phone numbers every
| 6 months (which is the time phone operators keep the number
| closed when people change numbers) if you need 100%
| confidence that they are the real owners.
|
| This is a problem if you are relying only on phone-numbers as
| a factor, which Lyft was (so was Uber, in some markets). In
| reality, a lot of the time you verify the phone number and
| present a password challenge (aka ask for password) so it's
| not a huge problem.
|
| But then, you'd need to handle the case when Alice signs up
| in with the phone number 123, then changes their number to
| 456, and 6 months later Bob signs up with 123 because they
| are the new owners. Now, Alice has to provide a new number
| (with some Grace period) and Bob has to be eligible for all
| the promos / signup goodies that were previously tied to that
| 123 number again.
|
| Software is hard.
| sneak wrote:
| > _we also had to move to phone as a primary identifier,
| because honestly there are many countries people dont even use
| email._
|
| And I'm over here without a SIM card in my devices, and not
| using a telephone number at all.
| cancan wrote:
| Well, it's a numbers game (pun intended)
| cghendrix wrote:
| Can you expand upon the team in Amsterdam locking down Uber for
| people in Russia?
| cancan wrote:
| i am being sorta vague here and taking some creative
| liberties with the markets involved. but at a high level,
| there was so much logic spread over various versions of the
| apps + geographies + some permutations of those. so you could
| add a check somewhere (app/server/both) like if (foo) and
| assume it'd not break in some markets, but lo-and-behold, if
| often did.
| tomnipotent wrote:
| Uber is in many markets, but has to serve them all from a
| single distribution. That leads to logic, screens, and
| workflows specific to each customers situation. I imagine it
| isn't difficult for one team to inadvertently affect another,
| given this constraint.
| gregdoesit wrote:
| Yeah, on my team (payments) we completely underestimated the
| backend complexity, assuming there would be little to no
| effort, and pulling in a backend engineer "just in case".
|
| We ended up scrambling as it became apparent how many things
| changed (from polling to push notifications, thrift changes etc
| etc) and making the backend ready became a heroic effort
| itself... especially that we had very few backend engineers in
| Amsterdam at the time.
|
| You should write up your experience!!
| cancan wrote:
| yeah, probably we annoyed the sh*t out of many payments folks
| as so much of the top of the funnel relied on things they
| should not have been doing. and the thrift changes, lol, that
| was something else.
|
| but it all worked out seems like. and honestly, if you
| consider how fast uber moves compared to its competitors, im
| willing to bet it was the right call now.
| tstrimple wrote:
| I wonder what the effective hourly rate was that these developers
| were paid for all of this work.
| gumby wrote:
| That was the year I sold my cars and switched to bike+rideshare.
| Even though my cc gave me an Uber benefit the new app experience
| was so flakey that I dumped uber for over a year, until I
| traveled to a place with poor Lyft service and found that the
| uber app was usable again.
|
| I've never seen why the app needed rewriting.
| Guest42 wrote:
| From a cost/opportunity cost standpoint, it's an enormous
| decision. I wonder what those numbers look like.
|
| In general though I'm pretty biased towards boring things that
| work correctly as long as possible.
| splatzone wrote:
| Why is it legal for businesses in the US to ask employees to work
| around the clock like this? It seems like an insane and unethical
| practice
| missedthecue wrote:
| because they pay through the nose for it
| JI00912 wrote:
| Because large companies have a huge amount of influence over
| the political parties.
| alecco wrote:
| Good luck being a large US corporation without giving money
| to politicians. They'll shake you down with threats of
| regulation and other tactics.
| raspasov wrote:
| At-will employment. If you don't like, go do something else.
| JI00912 wrote:
| > If you don't like, go do something else.
|
| Many people don't have many options and companies abuse the
| shit out of that because people don't want to be homeless,
| get kicked out of the country or dead when they can't pay
| medical bills. Etc. Thus the need for legislation. And the
| lack of such legislation partially explains fabulous areas
| like skid row.
| oh_sigh wrote:
| Skid Row would be better if employers were prevented from
| having their highly paid salaried employees from doing
| crunch time?
| volkk wrote:
| that doesn't explain the _why_, that only explains the how
| this keeps persisting
| awillen wrote:
| They're incredibly well compensated and for the most part have
| the financial freedom to quit at any time as a result. They can
| choose to work in terrible conditions for a lot of money - that
| seems totally reasonable.
|
| Now on the other hand, consider people who are stuck in part
| time jobs that treat them terribly for minimum wage. They don't
| have the freedom to quit because they can't afford to miss a
| paycheck.
|
| In general, I don't think people who freely opt into anything
| should get sympathy for it.
| jonny_eh wrote:
| How about if no one would get treated like this?
| gorbachev wrote:
| Money.
| coldcode wrote:
| My team and I were often the target for insane rewrite requests
| like this; I know this type of ridiculousness only too well. Sure
| you can succeed even under the most insane circumstances but that
| cost is not worth it for the people who are forced to do the
| impossible; in the end the people who suffered are mostly ignored
| while the top execs revel in the end result as if it was them
| personally doing it.
|
| I finally said enough of this madness and retired this year. No
| matter how many times you manage to pull off miracles the people
| demanding them don't give a shit about you or your team.
| xiphias2 wrote:
| I have no problem with leaders encouraging overworking, but the
| article could have detailed the compensation that people got
| during the work. My guess is that they were getting not much more
| hourly wage than a McDonald's worker for correcting for the bad
| leadership.
| gregdoesit wrote:
| OP here. We did not get any additional compensation for the
| long hours beyond you could expense food at late hours and on
| weekends. And at the end of the project, my director told
| everyone to take their SO to the most expensive restaurant,
| hire a babysitter if they have kids and expense the whole
| thing. I submitted a bill of EUR300.
|
| When I joined Uber, my compensation effectively doubled from my
| last - already pretty well-paying gig - so even if calculating
| the per-hour wages for those 3 months, I made good money, and
| nothing comparable to the hospitality industry.
| notyourwork wrote:
| That sounds ignorant of reality and abusive at best. In my
| former org when we go into q4 things get a bit crazy and
| chaotic. Once the dust settles my manager would let us take a
| week off and do whatever we wanted. No vacation needed, just
| don't come to office and do something fun.
|
| Its sad how many orgs still abuse salaried employees time.
| mschuster91 wrote:
| Let me guess, some kind of ad, marketing or e-government
| agency? The "clients have leftover budget they need to burn
| or next year's budget shrinks" problem is real.
|
| I wonder why no large company has ever discovered the
| _absurd_ waste this budget policy causes.
|
| Add to that the problems associated with tenders (=
| sometimes absurd amounts of money wasted for pitching) and
| the overhead of way too many stakeholders and feedback
| loops on client side... in many a project the single thing
| that makes the most savings is a competent (!) PO on client
| side that gets "dictatorship powers" from their C-level
| that can override petty politics and internal squabbles if
| they threaten progress too much.
| notyourwork wrote:
| Sorry, to be clear I don't and never have worked at Uber.
| My former org was a reference to a team I was on prior to
| current team at another company.
| caractacus wrote:
| So you work (at least) double hours, a bunch of employees
| quit, "extreme stress", everyone "giving 110%", and you get
| nothing but a dinner out of it and the salary you'd have got
| if you'd worked a normal role. Uber's stock, meanwhile, got
| as big as Travis's ego...
|
| What a fucked up world where this is deemed reasonable.
| You're even able to justify it as "hey it was tough and
| ridiculous and half the people didn't make it but look, I
| wrote some books and was able to become a manager."
| alasdair_ wrote:
| There is always gamedev, where after a year of 80 hour
| weeks, the game is launched and... half the people get
| fired because they are no longer needed.
| stale2002 wrote:
| > world where this is deemed reasonable
|
| I mean... the world where uber engineers are getting paid
| 300k a year, is probably that world where people might
| think that this is reasonable, given the compensation at
| companies like uber.
| xiphias2 wrote:
| Sure, I understand, it's a relationship that is still quite
| good, so if you are young, healthy and can manage the stress,
| it's probably not a problem.
|
| I have another question though that's somewhat unrelated: I
| use Uber quite often in latin america when I'm travelling,
| and my main issue is driver cancellations after 10-30 minutes
| of waiting for the Uber to start coming to my place (outside
| the city). It's quite frustrating to me to read about all the
| UI improvements over the year at Uber, and it seems that
| software engineers at Uber don't care about the real user
| experience (waiting a lot for nothing). I would happily pay
| more to the drivers if this behaviour would stop (they cancel
| because I'm outside the city usually), but nothing has
| changed in the last 5 years about it. I don't care about the
| nice icons or dark mode or other UI changes BTW, because
| that's not what I remember after a ride.
| PragmaticPulp wrote:
| > My guess is that they were getting not much more hourly wage
| than a McDonald's worker for correcting for the bad leadership.
|
| I agree that this crunch time seems unnecessary, but it's
| mathematically impossible for Uber engineers to work enough to
| reduce their wages to anything near fast food wages.
|
| Even someone working 80-hour weeks, 52 weeks per year at
| $15/hour would earn $62K. That's less than half the starting
| salary for entry level engineers at Uber.
|
| Entry-level compensation at Uber is around $160K, with senior
| software engineers earning $400K or more:
| https://www.levels.fyi/company/Uber/salaries/Software-Engine...
| dylan604 wrote:
| >80-hour weeks, 52 weeks per year at $15/hour would earn $62K
|
| Someone not salaried would be eligible for overtime and
| double time if working 7day weeks. That would significantly
| change the amount earned at $15/hour
| plorkyeran wrote:
| Doubling the wage for the 40 hours of overtime still
| doesn't get you to $100k, and the people working on this
| were all making much more than that.
| [deleted]
| j4yav wrote:
| Wow, what a nightmare! I remember a project or two like that in
| my career that I wish I could go advise my younger self to avoid.
| I have been around long enough to remember when people bore this
| sort or abuse as a mark of pride almost, like the scars made them
| who they are despite seeing peers burning out, becoming sick,
| quitting, or getting divorces as a result of overwork.
| the_local_host wrote:
| I'm surprised that "moral hazard" is rarely mentioned in these
| types of stories. By making their overtime effort available to
| the CEO as insurance against bad decisions, the engineers are
| in a way encouraging more bad decisions, since it's the
| engineers bearing the costs.
| Retric wrote:
| Until the entire team quits at the same time which has
| happened to many companies that do this stuff.
|
| However, if this update failed nothing significant would have
| happened to Uber. So, they might not have actually cared.
| jonplackett wrote:
| > Tooling is a major issue: compiling the app from scratch takes
| 25-30 minutes on iOS and around 10 on Android
|
| As a relatively naive to massive apps iPhone developer (I've made
| apps with millions of downloads, but nothing on Ubers scale) I
| find it confusing how a relatively straight forward iPhone app
| like Uber can be so complex. It's basically a map and 3 buttons?
| And an 'in the car' state. They weren't doing pool or bill
| splitting at this point I don't think. I'd have thought
| everything else is back-end complexity?
|
| Can someone do an explain like I'm 5 on this?
| martpie wrote:
| Where to start? A wild guess:
|
| - A/B testing capabilities
|
| - Different features per country/region
|
| - Different payment methods per country
|
| - ...
|
| I mean it's like e-commerce, when you stay in a single country,
| things are easy. When you go international, wild sh*t happens
| to cover all the spectrum of scenarios and respect the laws in
| place, while keeping it simple enough to maintain for devs.
| diegocg wrote:
| https://mobile.twitter.com/StanTwinB/status/1336890442768547...
|
| It's way more complex than it seems
| aidos wrote:
| From memory, when this came up before there was a lot more
| information about how they have to jam in a load of stuff to
| deal with international travel so that everything works as soon
| as you arrive in another country.
| lhorie wrote:
| This exact question was answered before here:
| https://news.ycombinator.com/item?id=25376346
| twobitshifter wrote:
| Even reading all of that I am still surprised to find that
| the Uber Rider app is a over 1M LOC per platform. 20% of the
| functionality used by most users takes how many lines? It
| seems that they're chasing the long tail of covering every
| global trip-taking use case. It might be fun to build, but
| how many of those features have a positive ROI?
| ransom1538 wrote:
| You land in Madrid. You hail an uber using the app. You
| stand there for 2 hours, hail multiple drivers. The uber
| never comes. For some reason the uber on the map drives in
| circles around the airport - then cancels. You eventually
| get in a cab. The driver says 1) uber cars can't be that
| close to the airport - and 2) that he doesn't take cards.
| You no longer use Uber in Spain. You tell the rest of your
| family to not use Uber when they get to Spain.
| throwaway3699 wrote:
| It's the ancient Microsoft Word vs. Google Docs debate.
| Without long tail features Uber would be less valuable to
| each customer.
| [deleted]
| jamil7 wrote:
| I asked this question to another iOS dev a while ago and they
| mentioned the way that uber is a single app that functions in
| all countries it's available, you never have to download a
| country specific version by design. I think for that reason it
| needs a whole lot of regional specific functionality and
| localisation, I believe a lot of work is done client side too.
| Someone from uber could chime in and correct me if I'm wrong.
|
| Edit: looks like it was also brought up in a previous thread.
| solarkraft wrote:
| Here's a pretty comprehensive video about it:
| https://youtu.be/zmeCYiD0hnE
|
| TL;DW (IIRC): Uber is in a _ton_ of markets and each one has
| significant differences, for example in payment providers - the
| app needs to support all of them and each comes with its own
| SDK.
| jeffrallen wrote:
| Shit like this is why Uber should have failed. Total disregard
| for humans. TK is a monster.
| DevKoala wrote:
| > Ariana Huffington joins Uber's board around the same time and
| releases her book on the importance of sleep right around this
| time, and piles of free copies are available in the office. Yes,
| sleep is important. So why am I at the office past midnight?
| Right: the deadline that can not be moved.
|
| This feels too familiar. "You have unlimited vacation, just take
| a week off, but we are not moving that deadline."
| junon wrote:
| I was at Uber during this time. Her book was stacked on tables
| all around the main headquarters offices, free to take.
|
| Took a looooong time for them to disappear and I'm still not
| convinced more than a handful of people cared.
|
| People didn't really like her internally. She was good friends
| with Travis apparently and that's about as far as the
| relationship to the company seemed to go.
|
| There was once a time where employees could request a ride with
| her directly from the app but to be honest the prospect sounded
| horrifying.
| jogjayr wrote:
| > There was once a time where employees could request a ride
| with her directly from the app
|
| You'd share an Uber with her? Or would she be driving the
| Uber? Sounds fascinating either way.
| president wrote:
| To be fair this happens because there are those who are willing
| to hurt themselves in order to get ahead in their career. We've
| all had coworkers that send late emails at 2am in the morning
| or don't use up their vacation days because they want to appear
| as hard working and as loyal to the company. Many of these
| people have no real skill and sacrificing their free time is
| the only real edge they have. This is fairly common and
| observable at your standard big enterprise software corp. Not
| trying to shift the blame from companies but it does take two
| to tango.
| freewilly1040 wrote:
| This perspective ignores the power dynamics. The tangoing duo
| are not equal partners, the company has the power to exert
| pressure on the employee to burn themselves out, and in fact
| did in this story.
| DevKoala wrote:
| Shouldn't management be able to identify that their team is
| comprised of these otherwise incompetent individuals? I see
| those 2am emails as a symptom rather than a catalyst.
| president wrote:
| The managers do it too. I've seen it all the way up to the
| VP level. There is just too much bloat in these large
| companies.
| Snowflame wrote:
| Xoogler here. Back at Google, there was an explicit policy
| that "heroic" efforts result in at most a bump of one level
| on the internal assessment scale - there was an idea that
| you should be doing productive work without having to
| overwork yourself, and too much "willing to work
| constantly" from reports was a danger sign for the
| manager's performance review. (Granted, this also could be
| super stressful for borderline employees, who are
| essentially told that even if they work hard, we'll still
| fire you.)
| benhurmarcel wrote:
| It just goes in the long list of things companies say but
| clearly don't mean.
|
| "Make sure you do the mandatory training called Safety First.
| On an unrelated note, we're pulling back on remote working and
| you're expected to be at your desk 3 days a week. Don't forget
| to wear your mask at all times."
| anotherv wrote:
| You have free lunch, catering onsite chef or order from
| practically any takeaway. Right: just don't think you're
| getting a lunch for anything other than eating
| harshaw wrote:
| There are truly great things that can be accomplished in life
| with crunch time. I'd consider good examples as the early days of
| SpaceX or some other game changing endeavor.
|
| A rewrite of a mobile app doesn't seem to fit the bill. I'm also
| reminded of the time that during an Uber ride the app took me
| into this secret screen where it attempted to get me to do some
| kind of coding assessment for recruiting. The engineering on this
| feature was absolutely nuts. But also completely over the top and
| laughably irrelevant for me and my geographic location.
| callesgg wrote:
| A hundred engineers to rebuild the Uber app....
|
| That is insane! To me it seams like something that could be done
| with 2-3 guys over 6 months with room to spare.
|
| But yeah I get it they have the money and everyone wants to work
| with their friends.
| Farbklex wrote:
| Highly focused development crunch time is "refreshing" from time
| to time. And since the involved engineers probably got pretty
| good salaries, it's hard to feel sorry for anyone.
|
| But as I always say: Don't let a management problem become an
| engineering problem. The deadline was artificial and put stress
| on everyone. Uber is not an agency which has to deliver a product
| to some client. They have the freedom to release apps "when
| they're done" but choose not to do so.
| sg47 wrote:
| While it's not an agency, Uber has to allocate resources for a
| finite amount of time for a particular task. If there are other
| projects predicated on this app rewrite, it's absolutely
| essential that the rewrite is completed by a certain date.
| awillen wrote:
| The only things predicated on the app release are marketing,
| etc. that they've decided to predicate on it. They also chose
| to take the launch on in the first place.
|
| Sometimes there are external deadlines (events whose schedule
| you don't control, things for partners, etc.), but this just
| clearly isn't one of them. They could've released the update
| after Christmas - that was 100% under their control.
| yashap wrote:
| Indeed. Sounds like TK was a major issue here, but so were the
| whole tree of managers under him that agreed to ridiculous
| timelines that would destroy their teams instead of just
| pushing back on the deadline. Then again ... wouldn't surprise
| me if TK very intentionally built such a management team.
|
| Regardless, to be safe, I'll definitely never work at Uber, or
| any other company TK has touched. This sounds like an absolute
| nightmare, and is so different from my experiences in ~a decade
| at other tech companies that it's crazy.
| JI00912 wrote:
| TK?
| mavelikara wrote:
| Travis Kalanick.
| foobarian wrote:
| The crazy thing is... his bold, risk-takey leadership worked.
| So who's to blame these executives for continuing to pull
| these kinds of stunts?
|
| Sure a bunch of people burn out and leave but does that
| actually move the needle on any metrics besides turnover that
| these people look at? Assuming that departures get filled I
| don't know that turnover has that much of an impact on other
| metrics.
| greazy wrote:
| I think you are focusing on the business aspect and
| ignoring the humanity or lack of it in this situation.
|
| I wonder what the health toll was on the all the hundred of
| people who worked on this? The reduction in life
| expectancy, health issues that arise due to 16 hour days...
|
| It's not necessary.
| edoceo wrote:
| I think their point was that TK&Co _do_ ignore the
| humanity for the "success" (ie: their money)
| pjerem wrote:
| > [it] worked.
|
| What worked ?
|
| What important thing have been achieved ? An application
| that worked fine and did the business well have been
| rewritten entirely. Nothing wrong would have happened if it
| was released 1 week or 1 month later.
|
| Even letting appart the human cost, I can't even imagine
| all the shitty << dirty but it somehow works >> code this
| kind of management create. The kind of technical debt
| maintenance teams will have to live with for years.
| edoceo wrote:
| Until the next bullshit-deadline-rewrite driven by
| single-KPI managers
| yashap wrote:
| Losing fully onboarded people certainly hurts velocity on
| projects after this one. And being known as a bad place to
| work is very damaging for recruiting efforts - I'd think a
| strong majority of devs would agree with either/both of
| "I'd never work for Uber" or "I'd never work for a Travis
| Kalanick owned company." For a B2C tech company, where the
| product is absolutely crucial to the business' success,
| having trouble hiring good people is a major downside.
|
| Personally, about a year ago, a recruiter reached out to me
| about an interesting sounding role at "CloudKitchens", that
| I was legitimately considering, but once I learned it was a
| Kalanick startup, I turned it down. The product/idea
| sounded interesting, but I've heard too many horror stories
| to work for the guy. I'm sure I'm far, FAR from the only
| one.
| rahoulb wrote:
| Did Uber struggle as a result of this?
|
| I know they struggled as a result of Travis in general,
| but do they have trouble hiring? Has their development
| pace really slowed down (Kotlin aside)? Is their product
| suffering as a result?
| yashap wrote:
| I'd imagine they do have trouble hiring, and this hurts
| them via lower quality hires that they have to pay more
| for. But it's a guess, I don't have any hard numbers,
| likely few people outside of Uber HR do.
| OskarS wrote:
| I'm sorry, but I don't buy the theory that burning out your
| most talented engineers until they quit in an effort to put
| out an app update 6 months early (or whatever) is a healthy
| long-term strategy.
| bcrosby95 wrote:
| He says lots of people burned out and quit afterwards. Doesn't
| sound very refreshing. Sounds like the exact opposite: they
| spent everything they could and the adrenaline of the situation
| kept them going, and when that stopped they collapsed.
| kungito wrote:
| Well they resigned from the company, they didn't die. They
| took the money and went elsewhere or on a break
| phtrivier wrote:
| I love the author's writing in general, and it obvious that he
| learned the lesson from this experience. What's insane is that
| their management has not learned that lesson, from, say, the last
| 30years of people writing about software project management.
|
| I guess it will take such a caricatural death march to kill a
| unicorn before people get a clue. (And even that would be
| surprising. Not the killing - the getting a clue. )
|
| Still, that's almost as great an advertisement for _not_ working
| at Uber as, say... everything else coming out of Uber.
| sdljfjafsd wrote:
| my question is, what was the business value of doing a full write
| rather than incremental shift? same question on the specific date
| - was their any business value delivered by specifically
| launching the new app on New Year's?
|
| If not, this rewrite basically cost huge amounts of burnout
| across the mobile and server teams for no business value. At best
| it seems like an intense learning experience that didn't actually
| unlock $$.
___________________________________________________________________
(page generated 2021-05-24 23:02 UTC)