[HN Gopher] When will M&S take online orders again?
___________________________________________________________________
When will M&S take online orders again?
Author : fredley
Score : 60 points
Date : 2025-05-30 16:11 UTC (6 hours ago)
(HTM) web link (moneyweek.com)
(TXT) w3m dump (moneyweek.com)
| nickdothutton wrote:
| I don't believe most (pre-internet) retailers should be building
| and operating their own sites. They already run core supply
| chain, distribution, and certain other apps (e.g. rostering and
| so on, accounting and payroll), but they probably shouldn't even
| be running some of those either.
| fredoralive wrote:
| M&S tried that, Amazon used to run the website:
|
| https://www.theguardian.com/technology/2005/apr/19/business....
|
| But they eventually took control back, so it clearly didn't
| work for them:
|
| https://www.theguardian.com/business/2014/feb/18/marks-spenc...
|
| M&S orders still use the same ###-#######-####### order number
| format as Amazon, so I'm not sure if it's still some sort of
| fork of whatever white-label Amazon technology they were using
| back then.
|
| I'm not sure if getting Amazon to run your own ecomerce website
| is really the greatest idea in the long term (Amazon kinda want
| your customers to use Amazon, not your website), but M&S using
| them isn't as mad as that bit in the early 2000's where
| Waterstone's website was just a subsection of Amazon.co.uk.
| spacebanana7 wrote:
| > I'm not sure if getting Amazon to run your own e-commerce
| website is really the greatest idea in the long term
|
| Amazon has a clear conflict of interest with anyone in
| e-commerce. Shopify is probably a better example.
| xp84 wrote:
| Famously, Borders and Target both allowed Amazon to run their
| ecommerce operations on this side of the pond, until they
| realized what a bad idea it is to partner with your
| competitor on something important. Target can be forgiven, I
| suppose, as in those days Amazon was mainly a store for
| books, CDs, and DVDs. Unclear how Borders didn't see it
| coming, though!
| fredoralive wrote:
| The Waterstone's example I gave is similar to Borders - if
| your not from the UK you might not know it's a bookshop,
| and they were basically just routing traffic to Amazon as a
| glorified affiliate link for a while.
| arp242 wrote:
| This incident has little to do with website or web store as
| such, and the only reason those are impacted is because pretty
| much all of M&S's IT systems have been impacted. Even if
| someone else would be running all of that, chances are that
| would still interface with the M&S computer systems to
| accurately get inventory information and the like.
| neepi wrote:
| Err they were breached most likely through Tata Consultancy's
| helpdesk apparently which is literally the people they
| outsourced it to.
|
| Their approach was to sell the UK operation to Tata in 2018 and
| piss everyone off until they leave and replace them with Indian
| staff to save costs over time.
|
| You get what you pay for. They're now paying for it.
| benjaminwootton wrote:
| They outsource as much as they can to the cheapest system
| integrators they can find, primarily TCS.
| ecshafer wrote:
| This is the core thesis of a company like Shopify. Shopify will
| run everything else about being an e-commerce company (website,
| inventory, shipping, returns, ads, sales channels, etc) and
| then the merchant can focus on selling their product. But this
| is part of the larger thesis about running a business you hear
| in business school classes, to focus on your specialization and
| outsource your non-core expertise. Buy Workday/ADP/Paychex
| don't do payroll or HR. Don't build a data center, buy
| AWS/Azure/GCP. Don't build a sales database or marketing get
| Hubspot or Salesforce. Does your company take in a lot of mail?
| Outsource to a company that specializes in processing mail.
| Outsource your Technical Helpdesk. Outsource your customer
| support. This is why componentization is accelerating.
| madeofpalk wrote:
| I guess the question is whether e-commerce should be a core
| competency of a business with a significant e-commerce
| business.
|
| I'm not sure what it's like in the US, but grocery delivery
| is a reasonably big deal in the UK.
| runako wrote:
| > to focus on your specialization and outsource your non-core
| expertise
|
| Most retailers will argue that connecting with their core
| customers and delivering delightful experiences to them is
| their core expertise.
|
| More practically, it will be tension between things like "our
| marketing department wants X on the site for summer" and
| "Shopify is planning on launching X in January." It will be
| less of a resistance to using a third-party provider and more
| that the third-party provider imposes constraints on the mode
| of contact with customers. That's a hard pill to swallow for
| a lot of consumer-focused companies.
| xp84 wrote:
| > Most retailers will argue that connecting with their core
| customers and delivering delightful experiences to them is
| their core expertise.
|
| Having worked in e-commerce for most of my career, for
| individual retailers, I can assure you that the perpetual
| tension you describe is real. The problem as I see it is,
| every little retailer thinks that their two-bit designers
| and product managers are so uniquely visionary in designing
| interactions that they rightfully should have full control
| over the product that is the ecommerce website. Shopify
| employs God-knows-how-many engineers to build and maintain
| this experience, and probably thousands of SREs to be there
| 24/7 making sure a random DDOS or slow query doesn't take
| your site out. "But we think we can build a better site
| than Shopify with 10 engineers and a couple of managers,"
| they say.
|
| They can build one that has the 3 cute whiz-bang features
| that their self-important product design staff thinks
| matter, but it will be unreliable, and they won't have
| sufficient expertise to get right the other 90% of what a
| "good" ecom site should have. And on top of it all, none of
| those gimmicks will likely improve conversion or order
| value enough to be worth doing.
|
| The smarter ones IMHO do use Shopify. It lacks so many
| things in its core that it's infuriating (decent search,
| any nontrivial filtering), but retailers who use it mostly
| patch over those flaws with plugins sold by third parties
| (which often introduce ghastly single points of failure
| that you have no visibility into, and you can't sue some
| random plugin vendor you pay $50 a month for your site
| going down on Black Friday).
|
| Ecommerce is hard tbh. But I do personally think that most
| of my previous employers probably should have done
| lightweight Shopify skins and made their core competence
| sourcing, merchandising, and advertising product rather
| than designing cute search filters, or their own product
| recommendations algorithm.
| softwaredoug wrote:
| At the same time we're talking about AI replacing developers we
| also see cases like this of organizational technical
| incompetency.
|
| How does one square those two realities?
| jacobsenscott wrote:
| AI replacing devs talk is about short term stock pumping and
| short term COGS reduction. The long tail is someone else's
| problem.
| umanwizard wrote:
| I'm pretty sure devs are not usually counted as part of COGS.
| imhoguy wrote:
| We just need one event of C*O of critical/big company bragging
| about firing engineering and replacing it with AI and then
| followed by huge cyberattack like that. Then see how AI balloon
| pops across news outlets.
| coliveira wrote:
| They'll respond saying they need to invest the entire revenue
| of the company on new data centers to fix the issue, and the
| stock will double in price.
| AnotherGoodName wrote:
| Well we need to fix the business leadership problem asap. From
| the bio of the current M&S CEO.
| https://en.wikipedia.org/wiki/Stuart_Machin
|
| >He resigned as managing director of Target in April 2016
| because of accounting irregularities that he was unaware of but
| "happened on [his] watch".[4] He then became the chief
| executive of Steinhoff International.[4] (which seemed to have
| a lot of issues too https://en.wikipedia.org/wiki/Steinhoff_Int
| ernational#Debt_p...)
|
| Foresight to mitigate potential major issues is exactly what
| CEOs are expected to do. I'm not sure how being unaware of
| major account irregularities is not seen as a career ending
| move here.
|
| AI replacing CEOs seems straightforward as well. Accounting is
| such a data driven environment i think spotting account
| irregularities early would be straightforward. Likewise AI has
| the potential to think past short term thinking that leads to
| IT outsourcing (to the extent the store is not coming back
| online anytime soon!).
| e2le wrote:
| >AI replacing CEOs seems straightforward as well.
|
| I'm not sure I want AI replacing all CEO's, ideally it would
| raise the bar for quality and performance forcing human CEO's
| to compete.
| paxys wrote:
| 99% of "AI" talk in the public is for the sole purpose of
| making wall street happy to boost stock price and/or pump
| private valuations of AI startups. The reality on the ground is
| very different. CEOs are bragging about replacing senior
| software engineers with AI meanwhile their recruiters and
| hiring managers are desperately advertising $300-500K/yr jobs
| for these same engineers while still not being able to hire
| enough of them because of high demand.
| bobthepanda wrote:
| also at least some of the businesses that were doing this are
| now being run into the ground, like Klarna.
| barbazoo wrote:
| I honestly doubt that there is any overlap between the
| "$300-500K/yr" jobs and the jobs being replaced by AI.
| bradly wrote:
| > How does one square those two realities?
|
| People eat terrible food because they are bombarded with
| messages to do so. People can use terrible software for the
| same reasons. It doesn't matter that the food tastes worse than
| it used to-food companies are having record profits.
| tonyhart7 wrote:
| wait until they release AI for security and system
| orchestration
| benjaminwootton wrote:
| How can it take 3-4 months to get an eCommerce site back online?
| I assume you could redeploy everything from scratch in less time
| if you have source code and release assets. With backups and
| failover sites I can't think of any world where this would
| happen?
| pavel_lishin wrote:
| > _with backups and failover sites_
|
| What a fun pair of assumptions!
| internetter wrote:
| There are no backups. There are no failovers. There is no git.
| There is no orchestration and deployment stratagies.
| Programmers ssh into the server and edit code there. Years and
| years of patchwork on top of patchwork with closely coupled
| code.
|
| Such is a taste of what needs to be done if you wish to have a
| service that takes months to set back up after any disruption.
| 98codes wrote:
| [citation needed]
| internetter wrote:
| Sorry if I phrased it poorly. I wasn't definitively saying
| that all these things are the case. But what always _is_
| the case is that when an attack takes down an organization
| for months, it was employing a tremendous number of
| horrendous practices. My list was supposed to be some.
|
| M&S isn't down for months because of something innocuous
| like a full security audit. As a public company losing tens
| of millions of dollars a week, their only priority is to
| stop the bleed, even if that means a hasty partial
| restoration. The fact they can't even do that suggests they
| did stuff terribly wrong. There's an infinite amount of
| things I didn't list that could also be the case. Like if
| Amazon gave them proprietary blobs they lost after the
| attack and Amazon won't provide again. But no matter what
| they are, things were wrong beyond belief. That is a given.
| pavel_lishin wrote:
| To be fair, I would be that nearly _every_ organization
| employs a tremendous number of horrendous practices. We
| only gasp at the ones who get taken down for some reason.
| internetter wrote:
| Horrendous practices exist on a spectrum. Every org has
| bad code that somebody will fix someday(tm). It is
| reasonable to expect that after a catostrophic event like
| this, a full recovery takes some time. But at a "good"
| org, these practices are isolated. Not every org is
| entirely held together with masking tape. For the entire
| thing to be down for so long, the bad practices need to
| be widespread, seeping into every corner of the product.
| Ubiquitous.
|
| For instance, when Cloudflare all went down a while ago
| due to a bad regex, it took less than a hour to rollback
| the changes. Undoubtably there were bad practices that
| lead to a regex having the _ability_ to take everything
| out, but the problem was isolatable and once adressed
| partial service was quickly restored, and shortly after
| preventative measures were employed. This bug didn 't
| destroy cloudflare for months.
|
| P.S. in anticipation of the "but cloudflare has SLAs!!"
| that isn't really a distinction worth making because M&S
| has an implicit SLA with their customers -- they are
| losing 40 million each week they can't offer service.
| Plenty of non-b2b companies that invest in quick recovery
| as well, like Netflix's monkey testing.
| PaulHoule wrote:
| No, best practice is that you have a checklist to bring
| up a copy of your system, better yet that checklist is
| "run a script". In the cloud age you ought to be able to
| bring a copy up in a new zone with a repeatable
| procedure.
|
| Makes a big difference in developer quality of life and
| improves productivity right away. If you onboard a new
| dev you give them a checklist and they are up and running
| that day.
|
| I had a coworker who taught me a lot about sysadmining,
| (social) networking, and vendor management. She told me
| that you'd better have your backup procedures tested. One
| time we were doing a software upgrade and I screwed up
| and dropped the Oracle database for a production system.
| She had a mirror in place so we had less than a minute of
| downtime.
| throwawaymgb123 wrote:
| This is a perfect description of how things work at one of
| the largest health care networks in the northeast US
| (speaking as someone who works there and keeps saying
| "where's the automation? where are the procedures?" and keeps
| being told to shut up, we don't have TIME for that sort of
| thing.
| aspenmayer wrote:
| If you don't have time to prepare for failure, then you'll
| have little time to invest in success, either, if/when
| failure strikes.
| internetter wrote:
| lol the healthcare industry was definitely in my mind as I
| wrote this. Never worked there but I read a lot of
| postmortems and it _shows_ whenever I use their digital
| products. Recent example is CVS.
|
| Somehow, at some point, they decided that my CVS pharmacy
| account should be linked to my Mom's extracare. Couldn't
| find any menu to fix it online. So the next time I went to
| the register I asked to update it. They read the linked
| phone number. It was mine. Ok, it is fixed, I think. But
| then the reciept prints out and it is my mom's Extracare
| card number. So the next time I press harder. I ask them to
| read me the card number they have linked from their screen.
| They read my card number. Ok, it is fixed, I think. But
| then the reciept prints out and the card number is
| different--it is my mom's. Then I know the system is
| incredibly fucked. Being an engineer, I think about how
| this could happen. I'm guessing there are a hundred
| database fields where the extracare number is stored, and
| only one is set to my mom's or something. I poke around the
| CVS website and find countless different portals made with
| clearly different frameworks and design practices. Then I
| know all of CVS's tech looks like this and a disaster is
| waiting to happen.
|
| Goes like this for a lot of finance as well.
|
| E.g. I can say with confidence that Equifax is still as
| scuffed as it was back in 2017 when it was hacked. That is
| a story for another time.
|
| Nobody bothers to keep things clean until it is too late.
| The features you deliver give promotions, not the
| _potential_ catastrophes you prevent. Humans have a
| tendency to be so short sighted, chasing endless earnings
| beats without anticipating future problems.
| squiffsquiff wrote:
| This is an ignorant position. Look at e.g. https://engineerin
| g.marksandspencer.com/mobile/2024/09/05/re...
| didroe wrote:
| How do you know it's safe to redeploy? If your entire operation
| may be compromised, how can you trust the code hasn't been
| modified, that some information the attackers have doesn't
| present a further threat, or that flaws that allowed the attack
| aren't still present in your services? It's a large company so
| likely has a mess of microservices and outsourced development
| where no-one really understands parts of it. Also, if they get
| compromised again it would be a PR disaster.
|
| They're probably having to audit everything, invest a lot of
| effort in additional hardening, and re-architect things to try
| and minimise the impact of any future attack. And via some
| bureaucratic organisational structure/outsourcing contract.
| chatmasta wrote:
| The Co-Op (grocery store chain) was hacked around the same time
| in likely the same incident. It took three weeks for them to
| get food back on the shelves at my local store. I don't
| understand how that's even possible... what happened to all the
| meat and vegetables in the supply chain? They just stopped
| flowing? They rotted? Why couldn't they use pen and paper? It's
| unbelievable to me that a business would go three weeks without
| stocking inventory.
| Henchman21 wrote:
| You forget we have entered the "Who the fuck cares?" era.
| When no one in the chain is incentivized to _care_ , things
| just fall apart.
| chatmasta wrote:
| Interestingly Co-Op is so-called because it's a cooperative
| business, which vaguely means it's owned by its employees,
| and technically means it's a "Registered Society" [0].
|
| If you check CompaniesHouse [1], which normally has all
| financial documents for UK corporations, it points you to a
| separate "Public Register" for the Co-Op [2].
|
| So, your comment has more basis in reality than simply
| being snark... the fact that "nobody is incentivized to
| care" is actually _by design._ That has some positive
| benefits but in this case we're seeing how it breaks down
| for the same reasons nobody in a crowd calls an ambulance
| for someone hurt... it's the bystander effect applied to
| corporate governance with diluted accountability.
|
| [0] https://www.gov.uk/hmrc-internal-manuals/company-
| taxation-ma...
|
| [1] https://find-and-update.company-
| information.service.gov.uk/c...
|
| [2] https://mutuals.fca.org.uk/Search/Society/7240
| Henchman21 wrote:
| I guess this is more snark, but honestly I am genuinely
| shocked when people care about anything anymore. Sad
| times.
| chatmasta wrote:
| There is a serious crisis of competence and caring all
| throughout society and it is indeed frightening. It's
| this nagging worry that never goes away, while little
| cracks keep appearing in the mechanisms we usually take
| for granted...
| bonaldi wrote:
| I'm not following your logic. The co-op is designed for
| everyone to care _more_ because they are part-owners and
| because the organisation is set up for a larger good than
| simple profit-making.
|
| In practice the distinction has long been lost both for
| employees and members (customers), but the intent of the
| organisational structure was not for nobody to care;
| quite the opposite
| tonyhart7 wrote:
| You can say this because ignorant, stock inventory is really
| hard especially huge warehouse where many items come and go
| 24/7
|
| they can "move" it of course but who can guarantee how many
| amount goes from where and who ????
|
| paper and pen where there are thousand items in single rack
| is nightmare, I can tell you that
| chatmasta wrote:
| well, apparently co-op couldn't answer those questions with
| their computers because they got locked out of them...
| coliveira wrote:
| When everything is done by computers, no human really knows
| what needs to be done even for a simple thing as buying
| vegetables.
| glenjamin wrote:
| I chatted to a staff member on the checkout of my local coop
| supermarket
|
| She said that every shelf item is ordered on a JIT basis as
| the store stock levels require them - there are no standing
| orders to a store
|
| Based on that, I presume they didn't really know what any
| store would need
|
| Even when they were struggling my local store still had a
| decent stock of lots of stuff - just some shelves were empty
| bobthepanda wrote:
| You could (and people did) run this in the pre-internet
| days with basically just phone calls and a desk to receive
| them. The problem is that by now this represents an
| incredible increase in manpower required overnight.
| chatmasta wrote:
| In my case all the perishable shelves were empty - no
| fruit, no vegetables, no meat, no dairy. I checked every
| few days for multiple weeks and it wasn't until three weeks
| after the incident I was able to buy chicken again.
|
| It's possible they were ordering some default level of
| stock and I just didn't go at the right time to see it, but
| it sure looked like they were missing the inventory... when
| I first asked the lady "is the food missing because of the
| bank holiday?" and she said "no because of the cyber
| attack" I thought she was joking! It reminded me of the
| March 2020 shelves.
| paxys wrote:
| It isn't surprising at all. There's a reason why tech companies
| have insanely large engineering teams even though it _feels_ to
| an outsider (and inept management) that nobody is doing
| anything. It takes a lot of manpower and hours to keep a
| complex system working and up to date. Who validates the
| backups? Who writes the wikis? Who trains new hires? Who staffs
| all the on-call rotations? Who organizes disaster recovery
| drills? Who runs red team exercises? After the company has had
| repeated layoffs and fired, outsourced or otherwise pushed out
| all this "overhead" eventually there's no one remaining who
| actually understands how the system works. One small outage
| later, this is exactly the situation you end up in.
| CobrastanJorji wrote:
| Yep. It takes way fewer people to operating a working system
| than to build a new one. And the nature of capitalism is that
| you will pare down your numbers until you have the absolute
| minimum staffing you need to keep the lights on. Then when
| everything explodes, you completely lack the know-how to fix
| it. Then the CEO yells as the tech executive who responds by
| demanding hourly updates from the two junior devs who operate
| the site, and nobody wants to admit that they aren't capable
| of fixing it, and nobody's gonna OK a really expensive "we're
| gonna spend a month emergency building a new thing" plan
| because nobody's okay with because a month is obviously way
| too much time you need to fix it right now, and then three
| months go by and here you are.
| spacebanana7 wrote:
| I get the opposite impression. Stale software organisations
| with steady operating products seem to use massive
| headcounts, whereas startups building new products often
| get by with relatively few people.
| esseph wrote:
| Startups don't have to run a software stack for decades,
| hardware refreshes or SKU updates and replatforms,
| dealing with multiple types of turnover and reogs,
| knowledge transfer, etc.
|
| Plus at least monthly if not daily, even hourly system
| patching.
|
| Planting a garden is one thing.
|
| Weeding it is another.
| coliveira wrote:
| Agreed, and that is a wonderful punishment to these
| companies.
| phatfish wrote:
| Yup, it turns out all those Indian contractors/outsourced
| staff don't really give a shit.
| wrs wrote:
| "If you have source code and release assets." And a build
| process that works from a clean code base. And a deploy process
| that works on fresh servers.
|
| All of which assumes you even know _what services exist_ ,
| which in any company of this age and size you probably don't.
| cjs_ac wrote:
| Your comment suggests that you're not familiar with the
| diversity in M&S' operation.
|
| Marks and Spencers started as a department store; they still
| have this operation. They sell clothes, beauty products,
| cookware, homeware and furniture. All these things are sold in
| physical shops and online. Most of this is straightforward for
| an e-commerce operation, but the furniture will involve
| separate warehousing and delivery systems.
|
| They also offer financial services (bank accounts, credit cards
| and insurance). These are white labelled products, but they are
| closely linked to their loyalty programme (the Sparks card).
|
| Finally, they have their food operation: M&S is also a high-end
| supermarket. You can't do your food shop on the M&S website
| (although their food products are available from online-only
| supermarket Ocado), but you can order _some_ food products
| (sandwich platters and party food) and fresh flowers from the
| website.
|
| So M&S is a mid-tier department store _and_ a high-end
| supermarket. These are very different styles of retail
| operation: supermarkets require a _lot_ of data processing to
| ensure the right things get to the right shops at the right
| time to ensure that food doesn 't go to waste but also shoppers
| aren't annoyed by the unavailability of staples like bread and
| milk.
|
| Finally, M&S is traditionally fairly strong in customer
| service; it's not exactly Harrod's or Fortnum and Mason's, but
| their bra-fitting service, for example, has a legendary
| reputation. The internet isn't their natural home.
|
| So all-in-all, you have a business doing complicated things
| online because they think they _have_ to, not because they
| _want_ to: a pretty clear recipe for disaster.
| neepi wrote:
| Their banking op is a fucking mess as well. Had no end of
| problems with their card services which were rebranded HSBC.
| dylan604 wrote:
| For this particular audience, it's one of those things that
| could be rewritten in Rust over a weekend and then deployed on
| the cheap via Hetzner. At least then it'll be memory safe!
| briffle wrote:
| of course, if you redeployed everything from the source code,
| you could very well still have the same vulnerabilities that
| caused the problem in the first place..
| woah wrote:
| Holy shit why don't they just set up a Shopify
| wavemode wrote:
| Bureaucracy is almost always the reason. They don't just need a
| website, they need -their- website back, because it was
| programmed with a million little business rules and pricing
| logic and regulatory requirements.
| xp84 wrote:
| You're surely not wrong that a lot of things would need to be
| done without, but I'd like to think that if I were 'king of
| M&S' I could have identified a subset of merchandise that
| could be loaded into a suitable interim solution like Shopify
| within say, 4 weeks, if the only other option was forgoing
| all online sales for 12 weeks +.
|
| That would also take a lot of the pressure off of the "full
| recovery team."
|
| Of course, the real situation must be 100x more complex than
| I'm imagining it so "I'd like to think" != "I am confident"
| wyager wrote:
| It's weird to me how it often seems like the US and China are the
| only countries capable of mega-scale tech infrastructure like
| this (and even then, only in some industries). Can you imagine
| Wal-mart's website going down for multiple months?
|
| I think a lot of companies (especially in Europe) have not
| internalized that, yes, you actually do need to expend apparently
| exorbitant amounts of money on highly-paid engineers if you want
| your tech to _actually be good_. Many countries, including the
| UK, are simply not wealthy enough to do it at scale. They produce
| plenty of engineers, but most of the ones capable of holding
| complicated stuff together probably end up working for US
| companies that can pay them market rates.
| tristor wrote:
| With the case of M&S, and in many other cases in UK tech
| history that have gone poorly, it's mostly examples of the
| failure of hiring outside consultancies in India to do
| everything. Business executives continuously fall afoul of the
| fungibility myth. They believe that engineers are fungible, and
| that they should therefore simply pay for the cheapest
| engineers possible that meet the "requirements" on paper,
| usually set by someone who is not an engineer (HR, project
| manager, or a lower ranked middle-manager).
|
| Time and time and time again we have seen major failures
| globally, and especially in the UK, that prove that there is no
| fungibility of engineers, and that outsourcing the critical
| technical infrastructure for your core systems and services is
| doomed to failure. They'd rather save a dollar today and lose
| ten million dollars tomorrow by damaging their national economy
| and sending more money to India. India's GDP is basically
| entirely propped up by tech services, and most of that is
| /failed service delivery/, hard to differentiate from frauds
| and scams at scale.
___________________________________________________________________
(page generated 2025-05-30 23:00 UTC)