[HN Gopher] New Zealand's $16B health dept managed finances with...
___________________________________________________________________
New Zealand's $16B health dept managed finances with single Excel
spreadsheet
Author : indy
Score : 139 points
Date : 2025-03-11 07:56 UTC (15 hours ago)
(HTM) web link (www.theregister.com)
(TXT) w3m dump (www.theregister.com)
| FirmwareBurner wrote:
| Plot twist: this actually serves as more positive advertising for
| Microsoft Excel.
| EdwardDiego wrote:
| Honestly, I'm surprised that they didn't pay a foreign IT
| company $100 million to for software less functional than a
| spreadsheet like our government departments usually do.
| tokyovigilante wrote:
| Don't worry, we have definitely also done that.
| ivan_gammel wrote:
| How so? Apparently, the conclusion is that Excel is the wrong
| tool for such tasks, so it should be used less, not more.
| FirmwareBurner wrote:
| _> How so?_
|
| The healthcare system of a country was run in Excel. Didn't
| you read the title?
|
| _> so it should be used less, not more_
|
| I never said it should be used more, I said the fact that it
| could be used for that, makes it an impresive achievement for
| the app.
| spacebanana7 wrote:
| I suppose it's kind of like a billion DAU app using a
| single SQLite file for their DB.
|
| Certainly the wrong tool for the job, but nevertheless
| impressive for both the technology and the operators if it
| somehow works.
| theonewolf wrote:
| If it creates long term value, is it truly the wrong
| tool?
|
| Perhaps we should re-evaluate how and when we use things
| when presented with evidence that others are getting
| massive value out of tools we typically would have been
| biased against.
| spacebanana7 wrote:
| I think so. Using SQLite for an application of that scale
| would require lots of additional work around caching and
| queuing at the very least. Much of that could be reduced
| by using Postgres or some other database designed for
| that workflow.
|
| Still, if many successful enterprises started SQLite for
| things of this scale, I should probably reach out to them
| to see whether there's anything missing from my analysis.
| ivan_gammel wrote:
| >The healthcare system of a country was run in Excel.
| Didn't you read the title?
|
| Obviously I have read not just the title, but also the
| article.
|
| >I said the fact that it could be used for that, makes it
| an impresive achievement for the app
|
| You can try to use a fork to dig a tunnel, but it is hardly
| an impressive achievement of the fork that it can be used
| this way, neither you could possibly dig anything big with
| it. It would be an impressive achievement for Excel, if it
| actually worked, but the article mentions that there were
| problems with accounting done this way.
|
| Excel is a powerful tool that can handle massive amounts of
| data, no doubt. However its feature set is not enough for
| this kind of tasks. As @zimpenfish said in another comment
| in this thread, the problem was both in Excel and in
| processes. Task-appropriate tools help to build good
| processes around them.
| zimpenfish wrote:
| > Excel is the wrong tool for such tasks
|
| From what little information there is in the article, it
| sounds like lack of process was equally to blame as using
| Excel.
|
| (Not that I would suggest using Excel, personally - I'd go
| with a small database and a simple web UI but then I'm not a
| Corporate Consultant looking to charge PSPSPSPS per day to
| faff around for 5 years and then abandon the project.)
| ivan_gammel wrote:
| >I'd go with a small database and a simple web UI
|
| There's nothing simple in running accounting in a
| healthcare system with 16B budget. A small database with
| simple web UI would have the same problems as Excel
| spreadsheet. The list of requirements for software
| automation of such thing would be on four digit number of
| pages scale. It does require a proper ERP.
| Hamuko wrote:
| This article proves that a) Excel is not the right tool for
| everything b) people will (over)use Excel even without being
| advertised to.
| bell-cot wrote:
| > The report found "monthly financial reporting usually took
| 12-15 days to consolidate and five days to analyse."
|
| > It gets worse: Health Minister Simeon Brown last week delivered
| a speech in which he said HNZ operates "an estimated 6,000
| applications and 100 digital networks" - or roughly one
| application for every 16 staff members.
|
| > He also said HNZ's Senior Leadership Team members "have only
| just begun weekly in-person meetings, and have continued to
| operate from different offices, despite the majority living in
| Auckland and the organization being two and a half years old."
| Most work in different buildings.
|
| > However the Minister doesn't have a plan to get off Excel, or
| fix HNZ's tech leadership woes.
|
| HNZ's Senior Leadership should be reassigned to clearing fatbergs
| from sewer mains, and some competent grownups brought in.
|
| And probably similar, further up in the NZ government. Two and a
| half years is how long it took for the 1940's USA to go from
| Pearl Harbor to D-Day.
| lostlogin wrote:
| It's not going well. But keep in mind that running down the
| public sector and privatising it is their aim. Simeon Brown is
| a terrible choice for minister and takes some pretty shitty
| positions for someone in his position.
|
| Also see 'the financial black hole' that ACC was supposedly in
| when John Key came along. It's the playbook being played.
|
| Don't look to the US for guidance though, that healthcare
| system and that political direction is going to be fascinating
| to look at in two and a half years.
| newsclues wrote:
| Good intentions without a good plan and leadership to execute?
| Or invent bureaucratic incompetence?
| EdwardDiego wrote:
| Don't accept anything Simeon Brown says without a healthy dose
| of skepticism, he's talking shit up for political reasons, like
| some people in the USA are doing right now.
|
| And if you do choose to accept it as unvarnished truth, welp,
| that's an ideological choice, not a rational choice.
|
| The current government would dearly like to bring in more
| private providers to our healthcare systems, so are following
| the classic neoliberal playbook.
|
| 1) Underfund public services
|
| 2) Point out how poorly they're performing
|
| 3) Bring in private companies to do it "more efficiently"
|
| 4) Pay them the money that you refused to pay the public
| service. And then some more.
|
| 5) Private companies make bank
|
| 6) Services further worsen.
|
| 7) But hey, shareholder value was great.
|
| If you want an example of this, just research the history of
| our railway system.
|
| Or, you know, the last National government when the Minister of
| Health directed funding for surgery from the public system to
| the private system.
|
| And then resigned to become the CEO of a private healthcare
| company.
|
| Which is why when NZ comes in the top 3 of "least corrupt"
| countries, I painfully point out that we're in the top 3 of
| _perceived to be_ least corrupt countries.
|
| Once again, The debacle that was NZRail -> TranzRail -> Toll ->
| KiwiRail is illustrative.
|
| Toll, an Australian company, ran down our rail network even
| harder than TranzRail did, and then forced the government to
| buy it off them at a massive premium by threatening to just
| shutter our entire rail system.
|
| And we gave fucking knighthoods to the two merchant bankers who
| advised the government to sell NZ Rail... _and then bought it_
| in a consortium with an American company which asset stripped
| it and then chose not to maintain the railway lines because it
| provided them a significant tax write-off in it the US on
| accounts of depreciation or similar.
|
| And of course, Sir Fay and Sir Richwhite now reside in overseas
| tax havens, although IIRC one of them still owns an entire
| island here.
|
| Oh, and the NZ government had to pay for the release of one of
| their children when they were detained driving through Iran for
| their Instagram account about how everyone is kind.
|
| Ah, neoliberalism.
|
| (As you can tell, I have some strong feelings on this. Because
| I lived through it. Was it all bad? No. It gave us one of the
| most efficient agricultural sectors in the world.
|
| Was it mostly bad? Well, inequality surged upwards in
| conjunction with neoliberal policies. So, yes.)
| alpaca128 wrote:
| > The spreadsheet-using agency is Health New Zealand (HNZ) which
| was established in 2022 to replace 20 district health boards in
| the expectation it would be more cost-effective and deliver more
| consistent services.
|
| Austria did the same with their national health insurance, to
| reduce costs and organisational overhead. It massively increased
| costs and organisational overhead.
| _ink_ wrote:
| Interesting, as a German annoyed by having 100+ insurances I
| always thought Austria is doing the right thing. Do you have a
| link for that? I am interested in understanding why the costs
| increased.
| alpaca128 wrote:
| https://www.derstandard.at/story/2000137202435/krankenversic.
| ..
|
| tl;dr: they promised savings of one billion Euros but it
| instead cost over 200 million.
| _ink_ wrote:
| Ok, the costs saving was promised by the populists Kurz and
| Strache. No further questions. Thanks for the link, tho!
|
| I am not saying Germany should merge all insurances into
| one, but I still think 100+ is one order of magnitude too
| much.
| FirmwareBurner wrote:
| Doesn't 100+ insurances mean more competition leading to
| better prices? OR so I'm told capitalism works.
| _ink_ wrote:
| I am only talking about the statutory health insurance
| providers not the private ones. For the private ones you
| might be correct. The statutory ones, they have a
| catalogue of services, which they are required to pay
| for. While there are some minor differences between the
| providers, they all offer the same benefits essentially.
| vasco wrote:
| Sometimes with big policy stuff I wonder how much is the effect
| of the policy vs the effect of the overall societal inertias
| and momentums there are. Like regardless of that policy, would
| costs have massively increased (inflation, financialization of
| healthcare)? Would organizational overhead have increased? Has
| it decreased in other industries? The policy might still have
| made it worse, of course.
| threatripper wrote:
| It's most likely that technical problems are turned into
| organization departements. The people making the decisions
| lack the knowledge and tools to solve them efficiently. They
| gain nothing from solving the problem itself but they gain
| from growing the organization in size. The solution is
| obvious while the path to get there can be very convoluted.
| FirmwareBurner wrote:
| Did the Austrian government or any other social western
| government ever manage do do anything that lead to a decrease
| in bureaucratic waste? From my PoV, no matter what they do, no
| matter their marketing messages, the costs always go up and
| efficiency goes down.
| alpaca128 wrote:
| Did any government, western or not? Without making other
| things significantly worse I mean.
| FirmwareBurner wrote:
| Maybe Denmark?
| rrr_oh_man wrote:
| I had a minor role in that project... A total shit show.
| aquir wrote:
| Not surprised...I am implementing ERP system for 20 years now and
| Excel is the most popular ERP system :(
|
| Related story when NHS England copied-pasted stuff for COVID
| stats but most of the data got lost because they are using an old
| version and there was a 65k line limit...
| ode wrote:
| https://www.bbc.co.uk/news/technology-54423988
| olavgg wrote:
| I've seen several multi billions companies running big processes
| in Excel. No automation, just put more human workers on the
| spreadsheets.
|
| I understand the decision though, because investing in a IT
| system is so friggin expensive, require 10x more people and there
| is a huge lack of talent to deliver something on time, that
| works, and actually saves money.
| consp wrote:
| I highly doubt the assessment that it would take 10x more
| people. If you let a consultancy do it, sure. Any normal medium
| sized IT company no. The problem with the assessment is
| invested time, since the invested manhours and failures over
| all the years of dodgy excel sheets is simply unmeasurable.
| semi-extrinsic wrote:
| The typical problem is that the organisation simply does not
| have the competence at the correct level to actually order
| the correct thing from the medium-sized IT company.
|
| In the past, the organisation has probably tried (maybe even
| several times) to replace the Excel mess with an IT system,
| only to receive something that costs them a lot of money and
| solves the wrong problem..
| netsharc wrote:
| Interesting gap in the market there, someone who can solve
| that can probably make a lot of money. With the help of AI,
| of course! /s
|
| I guess the solution is to have an intermediary that can be
| the in-between, but don't have them be from the IT systems
| supplier, because they'll bone you (including promising you
| everything you want without checking with their developers
| if any of it is possible; with a good 3rd party they'd be
| able to ask the devs if it's possible and to see a proof-
| of-concept).
| olavgg wrote:
| I have seen projects being implemented in SAP, by 20
| people, that a single junior software developer could do
| better solo in the same time frame.
| commandlinefan wrote:
| > deliver something on time
|
| Or ever.
| AnAfrican wrote:
| I'm not sure it's really a matter of initial investment.
|
| Excel allows managers/staff to iterate/change/adapt faster and
| independently while changing anything about an automated system
| requires much effort (in communication, in validation, in
| budget)
| datadeft wrote:
| The amount of issues I have seen in the last ~20 years caused by
| "off by one" type of errors in Excel is insane.
|
| Few examples:
|
| - incorrect schedule for the electricity grid for an entire
| country
|
| - incorrect assessment of an airport use for an airline (causing
| few millions USD loss in revenue)
|
| - incorrect financial position assessment for a mine (resulting
| incorrect deciosion to optimize the wrong business process, not
| sure about how much they lost)
|
| Making illegal states unrepresentable is a concept that benefits
| programming langues and business processes alike.
| jessekv wrote:
| IMO unit tests are roughly the software equivalent of double-
| entry bookkeeping.
| pjc50 wrote:
| Don't forget Reinhart-Rogoff, which ties into the other thread
| on academic fraud/nonreproduction, and arguably did huge
| amounts of economic damage:
| https://www.bbc.co.uk/news/magazine-22223190
| rco8786 wrote:
| Imagine how many errors occurred before we had digital
| spreadsheet technology.
|
| That said, I've always thought there was a product that sits
| somewhere in between Excel and full blown custom software that
| provides some of the controls we need while still being and
| build able by someone with low/average technical skills
| netsharc wrote:
| FileMaker seems to cover this (admittedly my interactions
| with it was in high school, which was last millennium), it's
| a database app but there's a lot of "make your own UI" parts
| in it, so you can create custom UIs (and wizards) for your
| use cases.
|
| I suppose MS Access offers this too, although FM feels more
| user-friendly, MS Access felt like Internet Explorer 4, where
| an error dialog would pop-up for every little JavaScript
| error (disclaimer: this opinion is from 25 years ago).
| rco8786 wrote:
| No you're right on the money in my opinion. I have very
| fond memories of spinning up apps in MS Access that were
| quick and dirty but extremely powerful, and able to be
| maintained/modified by folks that were not professionally
| trained software engineers. I think there's a missing area
| in the market for something exactly like that again.
| rrr_oh_man wrote:
| The hive mind really hated (hates?) MS Access and I don't
| know why.
|
| I built a mini ERP for my father's company when I was 16.
| Started out as an excel spreadsheet, but then added
| inventory, accounting, and printing contracts and
| reports.
|
| He kept using it until retirement and was very happy with
| it. I could learn & apply SQL. Win-win.
| masfuerte wrote:
| I've not used it, but I thought Airtable occupied this spot.
| It seems to have a few open-source clones if you want to run
| it locally. I don't know if they are any good.
| tomatocracy wrote:
| Microsoft also has Power BI which occupies this "database
| which looks like a spreadsheet" space. But it's not well
| known.
| dereg wrote:
| You're only looking at one side of it. Consider how much value
| that excel adds, and how many use cases it enables. It's
| incalculable. It's always easy to poke at excel's issues, but
| errors are almost always a skill issue.
|
| From a financial perspective, there are many ways you could
| enforce checks to ensure the model is balanced - it just takes
| time. Data entry can be an issue, but you can automate that
| too. The deloitte report is saying the health department should
| benefit from adopting a gigantic erp system but you could get
| 90% of the benefit by employing a couple people that really
| know what they're doing.
|
| You could say that excel should allow those things to happen
| but the flexibility is precisely what makes it so valuable.
|
| The people writing the reports are consultants. Consultants
| recommend things that benefit consultants. In their case, a
| multi year process and tens of millions of dollars trying to
| install ERP software is more a windfall for the vendors and not
| the companies.
| 7thaccount wrote:
| Yep. I was going to say the same thing. Installing some $10M+
| software costs the company more than just the install and
| ongoing licensing costs. You have to have an entire new IT
| team to manage it and staff to understand and so on. There
| are probably just as many chances of having a big issue.
| potato3732842 wrote:
| Say nothing of many crisis were avoided because somebody was
| able to pull up a local excel file despite a central service
| being down or go through their historical files and sniff out
| a discrepancy. Workflows like that are often impossible or
| require coordinating with a vendor in another time zone or a
| bunch of slow cross team communication and ticket submissions
| with the sort of minimum effort "we'll run a DB in the cloud
| and slap a JS front end built with a bunch of questionable
| but flashy libraries and ignore all the edge cases because
| I'll be working somewhere else in 2yr" solutions that the
| median software developer hive mind will implement if left to
| its own devices.
| ericd wrote:
| Absolutely, kudos to them for following KISS, despite the
| legions of people who have probably told them they're doing
| it wrong.
| threatofrain wrote:
| That assumes that Better Excel would somehow not look like
| Excel and would lose its benefits of being easy. My
| assumption is that Better Excel will still look like Excel,
| in the same way that Better iOS will also look like iOS.
| edude03 wrote:
| > but errors are almost always a skill issue
|
| Sure but to compare it to tech, this is why we build tooling
| like linters and typed languages so that we don't lose
| millions of dollars because we needed the average developer
| to implement a feature. I imagine in heath care it's even
| more skewed because a SW eng can see the compiler output and
| make changes but the person entering the data isn't (for
| example) the nurse that's using the needles to be accounted
| for
| tomatocracy wrote:
| There are indeed tools like this available for Excel. And
| there are also whole practices within the big accountancy
| firms, and independent firms which focus on providing
| "audit" of Excel files like this, which are used commonly
| where there are financial firms involved in a project with
| big money at stake (lenders, financial investors, etc). But
| in my experience these tools and services aren't often used
| for the spreadsheets which are used for day to day
| corporate-type management (I would count government as part
| of that).
| whiplash451 wrote:
| Do you have evidence that _any_ other software would not be
| prone to this kind of errors, taking into account the fact that
| at the scale of dozens of billions of dollars, the numbers you
| mentioned seem like rounding errors (except for the first one
| which would deserve a reference)?
| slovenskykanak wrote:
| They did not want to go with "premature abstraction"
| rubenvanwyk wrote:
| This is hilarious.
| kleiba wrote:
| Excerpt from the actual review document (Health NZ Financial
| Management Review, [0]):
|
| _Notably, one major issue was through a significant reliance on
| the use of an Excel file to manage the consolidated financials of
| the organisation. This spreadsheet was the primary data file used
| by HNZ to manage its financial performance. It consolidated files
| from each district into a single spreadsheet, and key reports,
| such as the monthly finance report, were produced from it. The
| use of an Excel spreadsheet file to track and report financial
| performance for a $28bn expenditure organisation raises
| significant concerns, particularly when other more appropriate
| systems are present on the IT landscape.
|
| This Excel file is flawed in that:
|
| * Financial information was often 'hard-coded,' making it
| difficult to trace to the source or have updated data flow
| through.
|
| * Errors such as incorrectly releasing accruals or double-up
| releases were not picked up until following periods.
|
| * Changes to prior periods and FTE errors in district financial
| reporting Excel submissions, would not flow through to
| consolidated file.
|
| * The spreadsheet can be easy to manipulate information as there
| is limited tracking to source information where information is
| not flowing directly from accounting systems.
|
| * It is highly prone to human error, such as accidental typing of
| a number or omission of a zero._
|
| --
|
| [0] https://www.tewhatuora.govt.nz/assets/Uploads/HNZ-
| Financial-...
| Springtime wrote:
| These same bullet points and half the first paragraph are
| quoted in the OP article fwiw.
| kleiba wrote:
| I think the important part that's not clearly portrayed in
| the OP article is that the "one Excel file" is actually not a
| single file used across the whole organisation, but rather a
| sort of master overview file that "consolidated files from
| each district into a single spreadsheet".
|
| I don't think that fact is made very clear in the OP article.
| tempaway64457 wrote:
| So actually top level planning/reporting is done by everything
| flowing into one big spreadsheet - as crappy as that is, thats
| pretty common. Thats not the same as "managed finances with
| single Excel spreadsheet". Like, they do have other systems.
| rco8786 wrote:
| Yea exactly my take. This seems like exactly what Excel is
| good at doing.
| nisegami wrote:
| Excel isn't 'good' at anything. It is passable at a great
| many things.
| potato3732842 wrote:
| Quantity has a quality all its own.
| whymeogod wrote:
| Sounds like a certain LLM I've read about
| TMWNN wrote:
| It really does.
|
| Just the massive, massive ecosystem of online tutorials
| and answers and YouTube videos for every conceivable
| question [1] is incomparable.
|
| [1] Except the one you want the answer to at a given
| moment, amirite fellas?
| sc68cal wrote:
| You are mistaking "being commonly used for this task" with
| "good at doing"
| tempaway64457 wrote:
| Replacing this excel sheet with a 'proper' system would
| cost between $5million and $20million probably, depending
| on what sort of consultancy delivered it
| datadrivenangel wrote:
| And would be inflexible, and maybe still wrong, and not
| necessarily more transparent... So it may mean several
| extra million dollars per year in ongoing maintenance if
| things turn out typically...
|
| Excel has many issues, but the cost of replacing it is
| surprisingly high.
| TMWNN wrote:
| >And would be inflexible, and maybe still wrong, and not
| necessarily more transparent...
|
| I'm going through this right now. Not anywhere in this
| scale, but just in terms of deciding whether to move an
| order-tracking/accounting system I've built in Excel to a
| "real" database.
|
| The database I am considering (Panorama X for Mac) is
| quite inexpensive, I've heard very good things about it,
| and has a spreadsheet-like UI. However, I've used Excel
| enough to know that I haven't tapped more than a small
| fraction of its ability, especially things like Power
| Query. As much as I loathe VBA, what if the cost of
| moving to a "real" database isn't the up-front cost, but
| the longer-term inflexibility of Panorama (and, pretty
| much, anything else in my price range) compared to the
| beast that is Excel?
| tokyovigilante wrote:
| Probably why the consultancy who produced the report
| referenced here was delighted to point it out.
| Retric wrote:
| Anything commonly used for a task it's good enough for
| that task even if "much better" options exist.
|
| Nail guns have a lot of downsides vs using screws, but
| being much faster and cheap offsets quite a lot.
| craigmcnamara wrote:
| Just bad enough
| fpoling wrote:
| 25 years ago a small Norwegian company in oil industri
| replaced a big Excel that it were using to manage its
| inventory with Access database and discovered that they have
| like extra 200K USD in equipment.
|
| The reason for the saving was enforcement of constrains in
| various Access tables so a subtle entry error in Excel was
| discovered.
|
| The transition to DB file was done by a single person in a
| month who had not worked previously with DB (however he was
| young and talented programmer I must admit).
|
| And the Access DB in a single file with its UI was quite
| similar to Excel so guys maintaining it quickly learned it
| and really liked it.
|
| So it is still puzzles me why such single file DB with nice
| UI are not more popular? Why it is always Excel?
| zimpenfish wrote:
| > why such single file DB with nice UI are not more
| popular?
|
| Because it needs skills that often aren't immediately
| available whereas a lot of people have enough Excel
| experience to lash up even a moderately complex
| spreadsheet.
| acdha wrote:
| At many places there's also some nasty politics around
| siloing where a database is an IT System so it has to be
| owned by the centralized IT organization, operated by a
| DBA team, have a budget, be audited by security, etc. and
| half of those groups look down on Access and want to
| architect for job security by picking some Oracle/SAP
| product where even if you have an unlimited budget you
| are looking at years of guaranteed delay before you can
| use it with a significant risk of failure.
|
| In a fair number of places that I've worked the finance
| and IT departments have a fairly substantial level of
| enmity which also discourages use of anything which
| involves them. Meanwhile, everyone still needs to do
| their jobs and there's a business analyst with decent
| Excel chops who will have something you can use
| "temporarily". A few years pass, then a few more, and now
| everything lives in Excel as the calcified scar tissue of
| broken official processes...
| Out_of_Characte wrote:
| Excel is really easy to use for smaller data collection and
| manipulation. There's no input validation, signing and
| other complicated design decisions needed to start an excel
| spreadsheet. There's alot of tools out there for easy
| integration. So its not something you need to switch away
| from once you have rather static requirements. Billions are
| just numbers that fit inside a cell just fine until someone
| complains.
| TylerE wrote:
| And then once you need actual calling Access lags far
| behind even MySQL.
| datadrivenangel wrote:
| Because Microsoft decided to kill/neglect access and Visual
| Basic... for reasons?
| codeulike wrote:
| Someone builds an access database and it works for a bit
| but then they leave and no-one else knows how it works so
| they start exporting the data to excel instead. Access is
| all about restrictions (the form has these fields; the data
| is structured like so; input will be validated by these
| rules) ... Restrictions deliver some value but are brittle
| - its about as hard to maintain as any other custom built
| app would be.
| MichaelZuo wrote:
| Access was much easier to get started with than any other
| similarly capable database product, last time I checked
| in 2017.
| RugnirViking wrote:
| as an anecdote, my own mother, who studied french in
| university and works in education, has built access DBs
| for places she's worked before. It seems its a very
| approachable and beginner friendly UI for non-
| programmers, somehow.
| layer8 wrote:
| To be fair, you can define value constraints in Excel as
| well.
| dzhiurgis wrote:
| > The transition to DB file was done by a single person in
| a month
|
| If Deloitte is going to do this it will take 3 years, take
| about 20 people and cost tens of millions.
| toasterlovin wrote:
| FWIW, having a human regularly count the equipment would
| have also uncovered the error. As well as other interesting
| reasons that discrepancies arise, such as theft. Always
| important remember that these systems are tools, not the
| truth, and have to be reconciled to the truth on a regular
| basis.
| CodeWriter23 wrote:
| > This spreadsheet was the primary data file
|
| >> - as crappy as that is, thats pretty common.
|
| No. This is not a common practice. An Accounting / ERP system
| with full audit trails and data entry compliance is.
| Exporting data from such a system into Excel for analysis is.
| But Excel as the [re-reads] _Primary_ data file is not a
| practice at all.
| campl3r wrote:
| I've worked on dozens of Excel to ERP migrations. Excel is
| way more common as the primary data source including
| financial data. If you only look at financial data by
| volume (and not count) maybe ERP win, but I wouldn't be
| sure.
| tempaway64457 wrote:
| > and was used for "consolidation, journals, business-
| critical reporting, and analysis."
|
| Not sure what 'journals' is supposed to mean here but
| "consolidation, business-critical reporting, and analysis."
| does not mean operations day to day use, it means high
| level management reporting
|
| My position is that the article (or the consultants who are
| trying to sell the ERP) are exaggerating the role of the
| spreadsheet.
| CodeWriter23 wrote:
| "primary", I hope as a commenter out here you understand
| what that means.
| bArray wrote:
| If the Excel file is just a snapshot of the financial situation
| and it doesn't feed back into the main system, I find it harder
| to see the issue. Essentially it sounds like a manager takes
| the Excel file, makes some small changes, writes some
| equations, produces tables/graphs and then writes a report.
|
| Maybe there is an error in the reporting, but it can always be
| corrected. The idea isn't to explain where $16.129626356 bn
| dollars went, it's to explain where ~$16 bn dollars went.
|
| I can think of many processes just off the top of my head where
| commercial software outputs something like a CSV file, I add my
| data to it, then generate a report. I don't think the process
| is insane at all. We catch errors by predicting the data to be
| entered and flagging if off by 10%, and then also looking at
| the distribution of the data entered (it should look like a
| skewed bell curve). We then have a stand-up meeting where we
| review the outputs and then have a process for later
| correction. Of course it's possible for errors to creep in, but
| the errors are very unlikely and have somewhat limited scope.
|
| My point is that the situation in NZ seems entirely resolvable,
| and they shouldn't just throw it out if it was somewhat
| working. The whole "Excel" thing seems like a nothing-burger.
| cherryteastain wrote:
| If you can't explain where NZD129m of spending went, you
| screwed up big time.
| potato3732842 wrote:
| There's a difference between "can't explain" and "isn't
| immediately visible on a high-ish level report"
| ceejayoz wrote:
| If you spend a billion and a couple years redoing the
| generally working system to (maybe!) increase the number of
| significant digits on a dashboard for no good reason, you
| may have screwed up even more.
| notorandit wrote:
| At page _42_. Not a random number.
| NoMoreNicksLeft wrote:
| Some of those defects seem as if they could be used to improve
| spreadsheet software more than they mean that spreadsheet
| software is intrinsically inappropriate.
| dzhiurgis wrote:
| Read this as Deloitte trying to sell some $1b worth of
| consulting to replace a spreadsheet.
| kleiba wrote:
| Just out of curiosity, what are some professional tools to handle
| finance information of that magnitude?
| ivan_gammel wrote:
| Bespoke ERPs implemented by external contractors on top of
| enterprise-grade database/cloud etc.
| franktankbank wrote:
| > enterprise-grade
|
| What is this?
| rubenvanwyk wrote:
| Good question! Feel this is just a vague way of referring
| to formal compliance requirements for software rather than
| technical functionality supporting large scale operations.
| ivan_gammel wrote:
| Sufficiently secure, scalable and reliable for a large
| organization. Usually implies that vendor can offer SLA for
| support and security updates, training etc. See for example
| the difference between RHEL and Fedora. In context of
| bespoke ERP it means that there are at least 2 tiers of
| vendors: top tier is responsible for building the
| integrated solution, underlying tiers are providing
| individual components for it.
| jpc0 wrote:
| I was looking for the \s in there...
|
| How does something like SAP solve this problem? I would
| have loved to just drive by but actually let's have a
| discussion about it. Even Odoo or something more "move
| fast". How would I sell the product to someone who runs
| their excel spreadsheets and is price concious so will
| actually balk at the pricetag to replace the thing that
| works fine and passes audits.
| ivan_gammel wrote:
| > replace the thing that works fine and passes audits.
|
| You surely don't mean Excel, do you? It did not pass the
| audit.
|
| The main selling point of any ERP is that it may be used
| to solve the problem in a cost-efficient way. And the
| problem here is not a single report or aggregation of
| data sources, it is an enormous list of requirements,
| implementing which with a manual or semi-automated
| processes based on Excel would be even more expensive or
| risky. Those guys in NZ failed even to understand what
| they actually need (i.e. produce that list of
| requirements).
| frontfor wrote:
| Perhaps this is telling us that the financial losses and
| inefficiencies incurred from these supposedly error-prone
| processes are small enough (relative speaking at least most of
| the time) that there's no real consequences to the stakeholders
| (politicians or bureaucrats in this instance) or the survival of
| the organisation (this org isn't for-profit after all). So from
| the stakeholders' POV it just doesn't justify putting the
| investment upfront into proper software systems. Without greater
| accountability and real stake in the org from the politicians and
| bureaucrats, headlines like this will continue to appear in the
| decades to come.
| jl6 wrote:
| It's all about risk. Maybe you can run Finance on a spreadsheet
| and get away with it, and maybe you can survive driving without
| wearing a seatbelt too.
| tibbetts wrote:
| Headlines like this are more pleasant than headlines about how
| the $100M project to implement SAP is five years behind
| schedule, how Deloitte has consumed tens of millions of dollars
| that could have gone to patient care, or that the new ERP
| system has lead to a total lack of accountability for patient
| outcomes.
| tgv wrote:
| Perhaps it means that the reorganization was unnecessary, and
| badly executed. Things seemed to go well before.
| jl6 wrote:
| Excel is an incredible swiss army knife that _can_ be used to do
| a lot, but it has a mean-time-to-catastrophe. You can keep
| spreadsheet risk (==cost) under control for a long time, which
| can make it seem like the risk is negligible, but it 's a ticking
| time bomb.
| ExoticPearTree wrote:
| The world runs on Excel. Contrary to popular belief that ERPs and
| what not is what companies use. Big companies use Excel to verify
| the ERP data and reports.
|
| There was an article shared here on HN sometime in the middle of
| last year that broke down how versatile Excel is and how
| different companies are trying to steal just a niche use of Excel
| (I remember reading about Airtable doing the automation bit much
| easier than in Excel, some company doing charts easier than Excel
| and so on). I wish I could find it again.
| rubenvanwyk wrote:
| Why do you think have we not been able to evolve beyond this in
| the enterprise world?
| ExoticPearTree wrote:
| In simple terms: because it is a tool everyone uses and knows
| to some degree. It has all the bells and whistles to do
| pretty much what you want with it. And, I guess one
| overlooked fact: the data is portable, everyone* in the world
| has Excel.
|
| My view is that if you switch companies, and at the last you
| used SAP and the new one you have to use Oracle, it is going
| to be a learning curve with very little added value for you
| (except the fact that now you know to proprietary systems).
| With Excel, well, it is going to be the same Excel, just
| different data.
| tomatocracy wrote:
| The fact that it's already installed on almost everyone's
| computer in any given corporate environment also has a lot
| to do with why it's so widely used for these types of
| things I think - noone needs to ask permission to get a new
| app installed, or jump through hoops to pay for another new
| piece of software.
| albrewer wrote:
| For the same reason Python is so popular. It's not the best
| at anything (besides being really easy to use), but it's top
| 5 at all the things that matter. When your use case outgrows
| Excel, then yeah, invest in a specialized system... when
| budget allows (which is never).
| nikanj wrote:
| The same reason we have not been able to evolve beyond a
| spoon. Once a tool matches a use-case really well, the best
| you can do is make it worse for the sake of being modern
| emeril wrote:
| as someone who works in the enterprise world as an accountant
| with some programming experience...
|
| excel is flexible and easy for reporting/analysis for data of
| varying sizes from small to not ginormous whereas doing
| anything in an ERP requires an, often, low paid SWE with a
| thick accent in a different time zone at, comparatively, high
| cost and big delay
|
| i.e., I can often solve problems in excel in an afternoon
| that would take months to solve with one of these SWE's
| crohr wrote:
| "The world runs on Excel" - I think it's true and we should
| bring AI tooling inside the spreadsheet, not build yet another
| SaaS silo from where your data will never escape.
|
| Also opens a lot of doors for integrating with private tools or
| data sources, especially with MCP tooling or such. I published
| a POC of what I mean at https://supersheets.ai/.
| rimbo789 wrote:
| Is there another budget tool other than excel? It's the only one
| I've seen working in forecasting and budgeting in private and
| public sector
| rubenvanwyk wrote:
| There are many, like mosaic.tech or causal.app
| oldjim69 wrote:
| But does anyone use them? Everyone org I've worked for and
| with - large government departments, publicly traded
| entities, small PE owned entities use excel.
|
| As others say in this thread, the world runs on Excel.
| rubenvanwyk wrote:
| Blows my mind that these things are still happening despite many
| large scale ERP implementations being the norm?
| priced_in wrote:
| _laughs in hedge fund manager voice_
| notorandit wrote:
| Interesting fact: Deloitte reports the use of the Excel file at
| page 42.
|
| Where all of the answers go.
| whalesalad wrote:
| Excel runs the world, literally.
| feverzsj wrote:
| Unbelievable. They should at least manage the excel in git.
| emeril wrote:
| I generally try to have staff time stamp their files with
| date/time and mark them read only - I suppose that's easier
| than git though not as good...
| vanderZwan wrote:
| And once again Felienne Hermans is proven right about the world
| running on spreadsheets and that we should take them a lot more
| seriously for that reason alone.
|
| [0] https://www.youtube.com/watch?v=0yKf8TrLUOw
| dwighttk wrote:
| Hopefully better than ol' SBF did
| lenerdenator wrote:
| If we simply implemented my benevolent dictatorship for life,
| we'd have a system under which installing Microsoft Excel became
| a capital crime.
| actionfromafar wrote:
| Computers with Microsoft Excel already installed on them before
| your edict would skyrocket in value. :)
| Spooky23 wrote:
| Yes, a benevolent dictator would want to make financial
| reporting as difficult as possible as they rob the treasury
| before the "aviator sunglasses" phase.
|
| I worked in a line of business leadership role for a multi-
| billion dollar organization. The ledger/financial ops stuff for
| super high-level reporting and payments were done by old
| mainframe apps. Everything else was done on a series of Excel
| spreadsheets hosted on a fileserver somewhere. All sorts of
| complex stuff was done in there. Finance guys are good at
| finance and frankly, stuff worked.
|
| They eventually transitioned to a modern at the time Peoplesoft
| system. The spreadsheet system was better in almost every way
| to the business units. The ERP was better for the financial ops
| teams. The shortfalls of the various ERP modules meant that the
| excel work was pushed down to the lines of business -- we
| probably had exponentially more spreadsheets, made by idiots
| who had no idea what they were doing. It also worked, but it
| was slower and certainly alot more expensive. The tools
| available for things like budgeting were primitive and
| ineffective - I literally used a P-Card to buy quickbooks and a
| built my own little finance shop.
| lenerdenator wrote:
| > Peoplesoft
|
| Yep, yep. There's yer trouble. Larry found another app to
| stack on his DB and he knows he's got the finance guys by the
| balls with vendor lock-in, so it doesn't matter how crap it
| is.
|
| Fortunately under my BDFL rule he's going to Devil's Island.
|
| > before the "aviator sunglasses" phase
|
| Before? Already there. Got the combination cap too. Been
| practicing the smile I'll use for the portrait mural that
| every large building has to have painted on the side.
| 1970-01-01 wrote:
| 1. Buy a state of the art supercomputer.
|
| 2. name it 'prod'
|
| 3. Move ALL infra onto it
|
| 4. ????
| commandlinefan wrote:
| Probably better than people with an "enterprise" tools did, too.
| forinti wrote:
| Sometimes those enterprise tools are used mostly as glorified
| spreadsheets. I see a lot of BI tools that are really being
| used to make big spreadsheets.
| commandlinefan wrote:
| The first feature I've ever seen anybody go looking for in
| any of them is the "export to Excel" (or CSV) function.
| jppope wrote:
| Sounds like the missing technology here is double entry
| accounting, not the persistence layer.
| akpa1 wrote:
| This makes me think of a system that just got replaced where I
| work - multiple hundreds of temporary workers being tracked
| through their setup and onboarding processes through one massive
| Excel spreadsheet, saved on a shared Teams group that everyone
| edited at once. It was held together with prayers, fickle
| filters, custom views and loose validation rules and I hated it
| the whole time it was active. We'd frequently have to make sure
| to remove rows from it and put them in an archive spreadsheet to
| keep the main one performant enough to function - but that split
| made it difficult to sift through data when someone had an issue.
| Not to mention that records routinely went missing or got
| overlooked.
|
| It's now a SharePoint database - I don't know what I think of
| that, but at least it was built for the job and has some solid
| integrations with other systems in it that make life easier.
| 4b11b4 wrote:
| This makes me feel better about a current situation, briefly.
| tokyovigilante wrote:
| Ah Excel, the second-best tool for everything.
___________________________________________________________________
(page generated 2025-03-11 23:02 UTC)