[HN Gopher] The algorithms that make Instacart roll
___________________________________________________________________
The algorithms that make Instacart roll
Author : mbroncano
Score : 87 points
Date : 2021-03-25 21:17 UTC (1 days ago)
(HTM) web link (spectrum.ieee.org)
(TXT) w3m dump (spectrum.ieee.org)
| gigatexal wrote:
| Interesting that item availability (qty on hand?) is powered by a
| ML model and not a field in the DB.
| petercooper wrote:
| Though quite an interesting solution and an appropriate one for
| ML, I think, given the sporadic and temporally limited nature
| of the availability data they receive from the stores. After
| all, that a store has 100 cans of soda right now doesn't mean
| they'll have any when my order is being picked in 2 days time.
| 29athrowaway wrote:
| I stopped using Instacart when they were unable to deliver an
| order in the time windows I selected and then unilaterally
| changed the time to a time when I was not going to be at home. I
| complained and they insisted and eventually the food was left
| outside for hours until it went bad.
|
| A shortsighted decision on their part. Because right after they
| took that stupid decision I stopped using their services forever
| and have told everyone I know to do the same.
|
| Plus, sometimes you want to buy one thing and add extra stuff to
| your cart in order to meet the minimum order value, and they do
| not have the one thing you were interested in on stock and you
| end up wasting your time.
|
| Amazon is much better at estimating stock and much more reliable.
| Amazon customer support is better and prioritizes long term
| relationships with customers over short term revenue.
| drewg123 wrote:
| I've had similar issues. I made an instacart order on Sat.
| night that I selected a 9a->11a Sunday AM window for. I was
| unexpectedly kept up until 5am due to a personal emergency, and
| I was unable to adjust the window later, so I was woken up by
| the delivery at 9a sharp. (and actually by the incessant
| notifications at 8a sharp).
|
| I've also often wished for a "cancel the entire order if you
| don't have this item" button. Especially when shopping for a
| specific recipe where if one ingredient is missing, the rest
| are useless.
| sn_master wrote:
| Algorithms aside, I just found out that one of the victims in the
| Colorado shooting was an Instacart shopper.
|
| It hit me real hard thinking that the app user who made the
| Instacart order never knew why their order was late and may have
| even gotten upset and asked for a refund, not having the
| slightest idea about the tragedy that happened behind the scenes,
| or the risk to their own life had they gone shopping themselves
| that day instead of ordering online.
|
| "The mother of two was at the supermarket filling an Instacart
| order, something she did in retirement to help others, he told
| the newspaper."
|
| https://www.usatoday.com/story/news/nation/2021/03/23/boulde...
| gigatexal wrote:
| Yeah definitely tragic. Condolences to her family. I wish I had
| the answer to gun violence in America :(
| curiousgal wrote:
| 'No Way To Prevent This,' Says Only Nation Where This
| Regularly Happens
|
| https://en.m.wikipedia.org/wiki/%27No_Way_To_Prevent_This,%2.
| ..
| gigatexal wrote:
| I'm not making any value or policy judgments. Anyone who
| understands a bit of American politics knows how hard the
| pro-gun lobby fights anything related to curbing gun
| ownership. I was merely saying that as an American it
| saddens me that there are all these senseless killings and
| that I'm not all that hopeful that real change is possible.
| Maybe I'm a glass half empty kind of guy I dunno.
| zikzak wrote:
| It's time for the adults to take care of this. I
| understand the deeply embedded gun culture and
| stockpiles. But what would you do if you found out
| porcelain toilets were killing people at this rate? You'd
| ban them and start programs to subsidize replacements.
| No, no, actually you'd have a significant percentage of
| the country insisting you can have thier toilets when you
| pry them from thier cold, dead hands. I sincerely wish
| this wasn't the way it is. The problem is generational, I
| guess. We need to start today by making guns a subject of
| ridicule among young people by using stats and studies to
| show there is no net benefit.
| gigatexal wrote:
| I say let's try it! Do a gun buy back. Ban new sales of
| assault rifles and high capacity magazines; institute
| longer more invasive (effective?) background checks (have
| a buyer have to see a shrink before buying? Makes sense
| to me.)
|
| Before moving to Germany I owned some guns for sport: my
| friends and I would go into the mountains to designated
| shooting areas and do target practice. If the hoops to
| get a gun were increased 10 fold it wouldn't effect me at
| all.
| gigatexal wrote:
| HN curiosity: I post the most practical gun-control
| things I can think of that veer rather draconian yet get
| down-voted because they're not enough? If you disagree,
| perhaps state why?
| throwaway53453 wrote:
| The definition of assault rifle is very poorly defined.
| There are dangerous weapons on both sides of the
| definition.
| sn_master wrote:
| Yes. Each time shooting happens I see 'analysts' thinking
| everything is full-auto only because of how it looks, and
| that automatically discredits them.
| sn_master wrote:
| (I don't own a gun, not have intention of getting one
| myself)
|
| I get that for big cities with large police presence, but
| the US is very wide and has countless suburbs and small
| towns with very little law enforcement present. Please
| look up YouTube, you'll find many single moms and small
| families whose lives were saved because of guns, where it
| took law enforcement forever to reach them.
|
| I don't think seeing a 'shrink' would make much
| difference except for those with very obvious conditions
| than the gun shop would refuse selling to them anyway.
| Most psychiatrists are nothing like the movies portray
| them to be, and they can't deduce everything about you
| from a couple hours of talking unless you're in a real
| dire state of mind. Hell, most of the good ones even
| refuse to take new patients or have 6 months long waiting
| list, and only the new inexperienced ones are easy to get
| hold of. Source: Tried to find a psychiatrist myself
| couple years ago for depression.
| Rule35 wrote:
| > If the hoops to get a gun were increased 10 fold it
| wouldn't effect me at all.
|
| I imagine this line is annoying people. That you aren't
| bothered isn't the issue if others are. That you don't
| see the issue with a delay, or a background check, says
| more about your one-size-fits-all view than the
| complexity/simplicity of the problem. Personally I never
| have menstrual issues so I don't understand what all the
| fuss is about.
|
| I come from country people, where even today going to the
| city is an event. And it's almost a holiday to go to "the
| big city". So a purchase you might be able to go to walk
| to the store to start a waiting period for might be
| between two months-apart trips for my family and much
| more inconvenient.
|
| You also sound like you live in the city so your needs
| for protection are a lot different than in the country.
| In the city a pistol with frangible ammo, or a shotgun at
| home, are the max you'd need. In the country my driveway
| was 1.5km so different solutions are needed. What's a
| scary black gun in the city, when in the country, is a
| way to both keep coyotes from livestock at great distance
| and defend yourself from people if needed. Being 500m+
| from a problem means you need more bullets (larger
| magazine) than being at 3' from the bad guy in the city,
| etc.
|
| They're different worlds with different problems.
|
| Look at Canada where the issue recently become polarized
| but not across party lines. Some nutcase in a surplus
| police car with a replica uniform drove around a small
| community gunning people down while the police holed up
| in the city for safety. He preyed on country people many
| kilometers from anyone else, with a rifle he'd smuggled
| up from the USA. Now the city people want to ban scary
| guns despite that a ban wouldn't have interfered with
| this shooter, but people from the country want to buy
| exactly the same gun he used because it's the best-in-
| class for home defense in those same scenarios.
| jfengel wrote:
| Facts and figures don't have a great track record of
| convincing people of things that they don't want to
| believe.
| zikzak wrote:
| That may be true (ok, it is definitely true) but we have
| to start somewhere. I know a lot of people my age who
| grew up (like me!) in a house with guns and hunting that
| do not hunt or own guns. I don't even have anything
| against responsible gun owning and hunting. I am talking
| about getting people to understand that going beyond this
| (carrying a sidearm around all the time, keeping them in
| your car, thinking you can be Clint Eastwood when you get
| mugged, etc) all lead to people dying on purpose or
| accidentally at far higher rates than if we just didn't
| do this stuff. Gun OWNERSHIP is not the problem. Tons of
| households in countries all over the world have guns in
| them. You simply don't see the same level of gun death as
| in the US. It is because of their culture (e.g.
| compulsory military service - I'm not a fan but you will
| learn how to responsibly own and house a firearm and when
| you could reasonably expect it to be a useful tool vs
| something that will go off and kill your kid or be stolen
| and used in a crime later).
|
| Rant over, sorry, I get emotional when I think about this
| having been mugged at gun point with a pistol that was
| undoubtedly stolen from a car glove compartment or
| something during a routine car break in (assumptions, but
| you know what I mean). I survived the encounter because I
| didn't try to fumble in my bag for some gun I barely know
| how to use (in fact the first thing they did was grab my
| bag and frisk every inch of me). It was a crew, they were
| all business, and they got me. I walked away with my
| house keys and my empty wallet (note, they did NOT steal
| my ID).
|
| So yeah, that sort of thing has happened to me and I
| still, somehow, don't have fantasies about shooting a
| bunch of teenagers. Call me crazy.
| splithalf wrote:
| I'm with you and personally dislike guns to a degree that
| many find weird but I think the issue is that gun
| violence is illegal in the first place and so making laws
| seeking to prevent it is ineffective at best and really
| just smacks of legislative hubris. I can get an illegal
| firearm easily, today. I can get an illegal gun much
| easier than I can a legal one. Most gun violence is
| committed by criminals. These problems are often denied
| by advocates for reform, and that doesn't help their
| credibility with bipartisan independents wary of
| authoritarian government.
| sn_master wrote:
| Exactly. Outside the US, major gun holders are criminals
| who get them illegally. Even in the US, most criminals
| have their guns illegally because they can't get one
| legally after their first felony.
| YoungWeb wrote:
| This is very sad.
| iandanforth wrote:
| Let's talk about the hard problem here. Control and Flexibility.
| I'm a regular Instacart user and I don't think I've ever had a
| perfect order. There's always something wrong in terms of size,
| quality, or availability. Most of the time it's no big deal but
| it begs the question, can these problems be eliminated?
|
| Take the perspective of a 3 star chef. You've got two major
| strategies to work with 1. Control and 2. Flexibility.
|
| The control strategy means you have deep trust in reliable
| suppliers. You've spent years finding the people who can get you
| what you need when you need it. You build your menu in
| consultation with your suppliers and then you check, check, and
| recheck every single order to make sure the standard doesn't
| slip. It's incredibly hard.
|
| The flexible strategy means that you adapt your menu daily or
| nearly daily to the supply you select yourself. Your talent is
| taking what is available, finding the best, and creating surprise
| and delight in a clientele who like that kind of thing. This is
| also incredibly hard.
|
| Basically nowhere in the high-quality-fresh-food world can you
| 100% rely on supply. And that's the challenge that Instacart
| faces. It doesn't have any strong relationships with suppliers
| (it's all third party or arms-reach) and it has almost no ability
| to be flexible because the orders are set by people who are hard
| to reach and haven't explained their needs.
|
| There's _some_ light in this tunnel. For example plantoeat.com
| (recipe and meal planning service) now integrates with instacart,
| but it doesn 't work well at all. Even when it does populate the
| cart shoppers don't know the recipe associated with what they are
| buying and aren't incentivized to care.
|
| Throwing layers of ML on top of unpredictable systems isn't a
| terrible idea, but it's a bandaid. The problems as I've described
| above are 1. Lack of control and 2. An inability to be flexible
| because you don't understand the needs you're fulfilling.
|
| The best answer to this is vertical integration. Goodeggs.com is
| one vertically integrated grocery delivery service. You could
| also think of it as a net-first grocery store, ghost grocer,
| whatever. When I buy from them I'm _much_ more likely to get what
| I ask for. And in the cases where I am ordering a meal kit and
| they are out of some component, they know what can and cannot be
| substituted to fulfill the requirements of the recipe.
|
| I don't think Instacart is going anywhere, it does what it does
| pretty well and I think it will keep getting better. But Amazon
| was right to buy Whole Foods and until Instacart owns the
| entirety of the production and distribution chain they won't get
| to the kind of experience people have been used to when they plan
| a meal, visit a store, and select for themselves.
| cmckn wrote:
| I don't understand the purpose of the Item Availability Model. If
| you're getting inventory updates from a store at least once a
| day, can't you use a more simple approach to stock prediction
| than a machine learning model? I.e. they can calculate the
| historic sale velocity of an item and with the current stock
| number decide whether the item will be there when the shopper
| arrives? They give the example of seasonal items, but if a
| customer tried to order egg nog in July, wouldn't they have
| inventory data that the nog hasn't been in stock for months?
| Maybe I'm overestimating the fidelity or completeness of the data
| they receive from retailers, and I guess ML is a way to do all
| this math without expressing it in code.
| vasco wrote:
| There's a great aversion to coding up any heuristic or rule
| system when you can learn about ML and apply it. It's more
| interesting, flexible and pads resumes. And sometimes it works
| as well or better than the rule systems, though it's rare.
| rdw wrote:
| I asked a similar question of someone who worked on these
| systems at Instacart, and the answer was that story inventory
| is often innacurate or incomplete. The stores just haven't
| needed the degree of accuracy before.
|
| There's also a fun integration problem: food manufacturers
| essentially lease shelf space and then the manufacturer handles
| all the inventory/restocking, and the store itself doesn't know
| or care what's there. So now you have to contact a bunch of
| third parties to learn what's in the store.
| muskox2 wrote:
| Yep. The inventory drift from wrong shipments and damaged
| items varies from store to store, but in the store I work at
| inventory numbers are often off by 5-10%. And much higher
| numbers aren't unheard of. Most stores completely re-count
| every item they have quarterly.
|
| Inventory management is something that's easy in theory, and
| very very hard in practice.
| chasd00 wrote:
| Totally agree, I work on the COVID vaccine
| distribution/administration for one state in the US.
| Inventory tracking is very hard, not technically so much as
| operationally. For example, healthcare workers didn't go to
| school to learn how to maintain inventory levels, they went
| to school to deliver healthcare. Our systems have to
| constantly remind and nag the users to keep reported
| inventory at least close to reality.
| cmckn wrote:
| That makes total sense -- I guess that's one of the things
| the Amazon Go stores are trying to solve.
| slt2021 wrote:
| the store does know, or can figure out if they really wanted
| to.
|
| every vendor is assigned some shelf space and has some
| restocking day (like every morning for bread) when the
| shelves are expected to be 100% full. then they have
| purchases data from point of sale terminals/online orders.
|
| the problem becomes A minus B
| ToFundorNot wrote:
| We deal with a similar problem in construction materials,
| and it's not that simple. Situation:
|
| 1)Customer picks up product, other falls on the ground and
| becomes damaged, product -x Equation: a-b-x where you don't
| know how much product is damaged 2)Product arrives in a bad
| batch, x number is affected, and requires manual
| adjustment, this doesn't happen, or happens incorrectly
| Equation: a-b-x where you don't know how much product is
| damaged 3)Customer picks up X amount, however x-y was
| registered as a sale Equation: a-b-y where you don't know
| how much product is unaccounted for 4)Delivery is expected
| on x day, however due to traffic/sickness/equipment failure
| delivery is delayed Equation: a-0, stock isn't available as
| it didn't arrive, however the assumption was that product
| arrived (trivial to fix this one, but I'm laying out
| scenarios).
|
| You now have four scenarios that are guaranteed to happen
| around %10 of the time. Issues can be expanded to the
| manufacturer/border/trade agreements/ thousands of other
| potential scenarios that disrupt sourcing.
|
| In terms of taking stock, it's not a trivial task to take
| accurate inventory on a regular basis. It's a manual
| problem that can only be done in a reliable fashion in most
| cases through estimation (therefore inaccurate).
|
| The reason why they provide availability ratings is that it
| provides a clearer picture of what a customer can purchase,
| and in the event it has a low rating, prompt for potential
| replacements. It's not binary, it's a case of 'probably' or
| 'probably not'.
|
| I've seen stock that should have lasted a week disappear in
| a day, stock mis-allocated(multiple times for the same item
| from multiple vendors in the same day), large volumes sold
| incorrectly resulting in stock adjustments, wastage from
| random occurrences, etc.
|
| I hope this provides a level of insight into the
| complexities of
| subpixel wrote:
| In the same way that I find Mealpal infinitely more satisfying
| than any food delivery, I am over the moon about shopping for
| groceries online and picking them up in person.
|
| The interaction with the business I am paying is valuable to me.
| I can - and do - give direct feedback about the service, which
| improves it.
|
| And if I forgot something, I can also run into the store and grab
| it, COVID circumstances permitting.
|
| The _same guy_ puts groceries in my car every time. A guy with a
| job that has health insurance. That matters.
| persedes wrote:
| can't highlight this enough. We've tried instacart when the
| pandemic started, but ended up with our stores express lane
| service. The online order works pretty well and the grocery
| shoppers are less likely (by a lot) to forget something and are
| pretty good at replacing out of stock items. With instacart, it
| always felt like babystitting the shopper.
|
| And lastly yes, it helps, that I feel like the ones with the
| job are less exploited than the IC shoppers....
| paulcole wrote:
| Neat content but the "secret sauce" is really the
| willingness/ability to take advantages of the employees they call
| contractors. Throughout human history exploiting people has been
| one of the most effective business models imaginable and yet
| somehow Instacart (and others) have figured out a way to lose
| absolutely astounding amounts of money doing it.
| fblp wrote:
| This is a elegant writeup that explains how instacart works with
| detail that I feel like I've never seen from other companies that
| face similar challenges (uber, doordash). Thank you to the
| authors!
| an_opabinia wrote:
| I don't know. They're losing money every day, every delivery.
|
| > DoorDash, Shipt, and Uber Eats in the United States, and
| Buymie, Deliveroo, and Grofers, based elsewhere
|
| They all lose money.
|
| So how do we know any of this is actually working?
| rodonn wrote:
| This article suggests Instacart is making a small profit of
| $3 per order in 2020 https://www.forbes.com/sites/chloesorvin
| o/2021/01/27/instaca...
| mytailorisrich wrote:
| I think this can only be solved by taking humans out of the
| equation as much as possible in order to lower costs and
| boost productivity.
|
| To have one person shopping in store and then another one to
| deliver to your door is low productivity and almost
| guaranteed to cost more than most people are willing to pay
| for the service, unless this is in a developing country like
| India and China where you have both people with Western
| purchasing power and people that can be paid peanuts.
|
| Same for services like Deliveroo, a driver cannot make that
| many deliveries per hour in most locations. It has
| historically worked for things like pizzas because pizzas are
| very simple, quick, and cheap so you can have someone on
| minimum wage do deliveries and still be in the black but as
| soon as you move to things with more overhead and lower
| margins you hit the wall.
| sn_master wrote:
| How to take humans out of the equation? Use drones to make
| the deliveries?!
| mytailorisrich wrote:
| For picking it's already doable. It's possible to build
| automated warehouses that could pick and pack orders
| ready for delivery or "click and collect".
|
| Delivery is indeed the difficulty and the solution
| depends on the coming of age of self-driving cars (level
| 5). My experience of ordering online from supermarkets is
| that a guy drives a van around and takes your order to
| your front door. In suburban areas (i.e. houses not
| flats) we can imagine self-driving vans stopping in front
| of your houses and letting you collect your order like a
| vending machine. This is not possible now because there
| are no self-driving cars but the instant they become
| commercially available I bet this will quickly become the
| norm.
|
| Automated warehouses plus self-driving cars enable a
| fully automated process from order to delivery. I'm sure
| we'll get there.
|
| Drones are simpler to fly autonomously but I don't think
| they are practical for delivering 10s of kilos of stuff
| to residential areas.
| sn_master wrote:
| Self driving will take decades before it becomes viable
| in developing countries. The state of driving there is
| very different than what the algorithms are being trained
| for in the US and Germany. We might have to wait until
| quantum computing is a thing.
|
| https://www.youtube.com/watch?v=x05SFt9BN0w
|
| https://www.youtube.com/watch?v=mDLCd70iTeE
|
| https://www.youtube.com/watch?v=3y_NiOvvALc
| sn_master wrote:
| I am surprised Uber Eats is losing money given how expensive
| it is. Is there a source on that? Could it be due to
| expansion costs in new cities/countries rather than losing on
| per-order basis?
| disgruntledphd2 wrote:
| Uber Car still loses money, ten years after it's founding,
| so that doesn't surprise me at all.
|
| Also, the last mile is really, really expensive to service,
| as every startup in this space finds out sooner rather than
| later.
| sn_master wrote:
| I dread having to use Uber anytime I have to.
|
| The moment a competitor starts offering self-driving
| option for the general public, there won't be any reason
| to use Uber Car.
|
| A single 10 mile trip back and forth to downtown will
| easily cost you 100$+ without tips and 8 times out of 10
| it'll be in a filthy and smelly car with a driver who
| doesn't care about the speed limit or the volume or genre
| of music he's playing in the car.
| Scoundreller wrote:
| They're probably incredibly profitable in some places, and
| incredibly losing in others to kill off competition.
|
| Those promos for restaurants, couriers and consumers don't
| pay for themselves.
| vector_spaces wrote:
| I do technical consulting for the grocery business. The grocery
| industry is a lot of fun -- lots of extremely hard technical
| challenges and interesting CS problems, but also lots of low
| hanging fruit in the way of painful manual process. Couple this
| with the fact that the grocery marketplace is dynamic in the
| extreme. Prices and availability of items in categories like
| produce change on a daily if not hourly cadence. And it's _all_
| about relationships, which is something nearly every startup in
| this space misses, which is why nearly all of them fail faster
| and harder than they do in other industries. You won 't get far
| if you're thinking in terms of replacing people with APIs.
|
| Anyway, I'm excited to see what the next ten years will bring
| here.
| ollieglass wrote:
| > And it's all about relationships, which is something nearly
| every startup in this space misses, which is why nearly all of
| them fail faster and harder than they do in other industries.
|
| Could you expand on this? It's not clear from the outside how
| this would be such a central part of the industry.
| bruiseralmighty wrote:
| Not the original poster, but I also work in the grocery
| services industry with a startup that does door-to-door
| grocery delivery.
|
| My guess is this person meant relationships with the farmers
| and producers being critical. At least this is true for us.
|
| If you want the freshest produce and other local items, then
| you need to know the farmers and other vendors in your area.
|
| If you want to go mass market instead, then there are only a
| few wholesalers to choose from.
| jfengel wrote:
| Where are they getting "petabytes" of daily data from? Even if
| you took every single transaction anywhere near the industry, in
| all its details, and represented it in a verbose format like JSON
| or XML, that's still only kilobytes of data times billions of
| transactions = terabytes.
|
| If they were gathering tons of video, maybe. Or vast numbers of
| high-precision sensors generating data continuously. But I didn't
| see anything like that.
|
| It's just hard for any human activity to generate that much data.
| Is "petabytes" just some kind of hype, or is there something I'm
| missing?
| sn_master wrote:
| A lot of big data datasets are de-normalized to make queries
| faster. Joining is very expensive.
|
| Source: Worked in big datasets at Bing for a couple years.
| YoungWeb wrote:
| To me it seems to be more of a volume data point rather than
| storage. So, it might only be terabytes of data, but for
| instance in ML algorithms, you could be running that data
| through the algorithm many times every two minutes. So by the
| end of the day, you've processed "petabytes" of data. This is
| just my interpretation of it. I could be wrong to.
| TrainedMonkey wrote:
| Maybe data streams collected from phones of people ordering
| groceries and buyers going to stores could add up to petabytes.
| hnpobdo wrote:
| 1 user searches for a product, and then clicks on 1: although
| you estimate this transaction generates 1kb of data, in fact
| this is more on the order of 1kb x the number of candidates, so
| a few MB.
___________________________________________________________________
(page generated 2021-03-26 23:03 UTC)