[HN Gopher] The Tyranny of Spreadsheets
___________________________________________________________________
The Tyranny of Spreadsheets
Author : shortleash
Score : 131 points
Date : 2021-07-22 20:38 UTC (2 hours ago)
(HTM) web link (timharford.com)
(TXT) w3m dump (timharford.com)
| markus_zhang wrote:
| In my last company we keep a spreadsheet that has tabs occupying
| almost the maximun number of columns allowed and in different
| headers. Updating that Excel file needs the full team to work for
| two full days each month. That was after I created a VBA script
| to scrape data from internal dashboards and put them directly in
| the spreadsheet.
|
| From my experience these monstrosities are usually created by two
| reasons: Push from business side and a data team too slow to
| react. Thus the analysts have to create the first version that
| seems to be OK and smart but eventually evolved into a
| monstrosity.
| rvba wrote:
| Maximum number of columns is 16 384 (in modern Excel).
| PostreSQL has a limit of 16 000.
|
| So Excel > PostrGRE? :)
|
| I wonder what kind of data did you use and why it had to be
| stored in so many columns. This approach would probably kill a
| "real" database too.
| swader999 wrote:
| Many businesses risk it all by building actual applications with
| Excel. Security, version control, automated testing are pretty
| much set aside. Case in point: https://insideparadeplatz.ch/wp-
| content/uploads/2013/01/ubs-...
| recursive wrote:
| I worked at a company that did product testing from
| spreadsheets. And not statistically modelling them. Like
| physical products. The spreadsheet logic was controlling power
| supplies on a physical test bench.
| LeifCarrotson wrote:
| It's not like they had a meeting where a consultant said "We
| reviewed all your requirements, and you could build this with
| normal software engineering practices, or we could build this
| in spreadsheets with associated deleterious effects on privacy,
| correctness, durability, extensibility, readability, and
| interoperability, likely resulting in an estimated $N to $NN
| million shortfall in 5 years. What do you want? Yeah, let's do
| the spreadsheets."
|
| It's a two-pronged attack of using tools that you have figured
| out as you go (not everyone is an educated, practiced,
| professional programmer, or even good at Excel) and a sunk cost
| fallacy/resistance to changing what "works".
| escot wrote:
| Somewhat related, this is a fantastic talk about the design of
| the first electronic spreadsheet VisiCalc by Dan Bricklin:
|
| https://www.youtube.com/watch?v=YDvbDiJZpy0
| anthk wrote:
| This is more shameful:
|
| Scientists rename human genes to stop Microsoft Excel from
| misreading them as dates
|
| https://www.theverge.com/2020/8/6/21355674/human-genes-renam...
|
| Seriously. If I was a data scientist I wouldn't touch that turn
| even from 300,000 kms away.
|
| This mediocrity kill lives, FFS.
| paulpauper wrote:
| what if i told you spreadsheets are matrices
| dkersten wrote:
| So... We should be using spreadsheets for machine learning and
| 3D graphics?
| FabHK wrote:
| Spreadsheets are DAGs.
| romwell wrote:
| DA*Gs
|
| *mostly, as TFA indicates
| spywaregorilla wrote:
| You'd probably come off as missing the point
| d--b wrote:
| Disclaimer: I am building a spreadsheet alternative for
| javascript hackers based on Observable https://www.jigdev.com
|
| For those interested it's all in the disclaimer ;-)
| nodejs_rulez_1 wrote:
| _> Here at Jig, we believe that BI should be done by
| developers._
|
| _> Jig apps are written in Javascript._
|
| Validate the market ASAP.
| ineedasername wrote:
| This seems less about spreadsheets in general and much more about
| Microsoft not wanting Excel to eat into sales of Access and SQL
| server.
|
| Even now the limit is ~1million rows, meaning that if you have
| even a very simple table but lots of rows then you have to pay
| more for Access.
| rossdavidh wrote:
| A nice post, brought low by the unfortunate fact that it misses
| the primary reason we were able to eradicate smallpox (and it
| wasn't data tracking):
|
| smallpox had no animal reservoirs
|
| Most viruses (and other diseases) have animal reservoirs, and
| that's why few diseases which ever get widespread, are ever
| eradicated, and it's nothing to do with spreadsheets.
|
| Now if he wanted to make a point about data, it would be good to
| mention that we have very little data on, say, how many species
| have which of these diseases, and how that's changing over time.
| But that wouldn't have much to do with spreadsheets either.
| xj9 wrote:
| i think spreadsheets could represent an interesting future
| programming paradigm, but excel is not the tool that'll do it.
| goatlover wrote:
| Notebooks are the next-gen spreadsheets, except they use
| programming languages instead of formulas.
| hobs wrote:
| And they require so much additional setup and processing
| power that they will never supplant spreadsheets.
| CJefferson wrote:
| Jupyter is a joke compared to spreadsheets, it doesn't auto-
| recalculate when you edit cells.
| spoonjim wrote:
| Excel has its flaws, as do spreadsheets in general, but the
| upside is that spreadsheets allow people to become programmers
| who never would grok programming in purely abstract terms.
|
| Spreadsheets allow the user to see each intermediate state in
| their application, as well as each iteration over the input data
| set, laid out in a 2-dimensional grid. There just isn't any
| substitute I've seen that is as accessible to the computing
| "layperson."
| asdff wrote:
| I really don't think excel is accessible. I don't know how to
| do anything in it. If I had to make a plot I'd have to search
| for an article that would probably be about the same word
| length as the one I'd find searching "how to make a plot in
| python." Imo excel isn't easier for the lay person, it's just
| entrenched because you have people now in the workforce using
| excel since windows 95.
|
| It allows people to become programmers by making them learn
| this clumsy tough gui that isn't readily apparent how anything
| works, and changes between versions. I don't think the learning
| curve is any easier than what it would take to make a person
| actually into a programmer with a language like python or R. A
| two hour tutorial in either language is probably all you need
| to do 90% of what people use excel for. Why not just skip the
| abstraction and learn how to actually program if you have to
| spend time learning how to faux program in excel anyway?
| CJefferson wrote:
| In excel, I can make a plot by typing in my numbers, then
| clicking the 'plot' button. Excel shows a selection of plots
| of my data and lets me choose the one I like the most.
|
| To do that in Python, I'd have to first choose a format to
| enter my numbers (I'd probably use Excel, then export CSV).
| Then I have to install Python, pip install a graphing
| library, import the CSV, then plot. I have to go through the
| different types of plots one by one.
|
| I've taught python and graphing, you need much more than a 2
| hour lesson before people can be left alone without help I'm
| afraid, if they have zero programming experience.
|
| I don't claim Excel is good programming practice, or
| scalable, but it is much easier than Python or R. The best
| future thing (I think) would be an excel like interface which
| would show you, and let you edit, Python that did what you'd
| just chosen in the GUI.
| spoonjim wrote:
| Because the layperson I'm talking about can't think in terms
| of abstractions, kind of like a math student who hasn't
| learned algebra yet. They can only verify the correctness of
| their algorithm by inspecting the intermediate values at
| every step, and scrolling down through the input set to
| inspect the corner cases. The problems with this approach are
| extremely obvious (which is why spreadsheets error out so
| often, or even worse, produce the wrong answer without any
| visible errors), but in many cases this is the only way for
| the people with the domain knowledge of what needs to be done
| to build an application to meet their need (they don't know
| how to code and they don't know how to write a functional
| spec).
| simonw wrote:
| This is a great read - I love the Italian accountancy history
| lesson at the start.
| version_five wrote:
| I actually stopped reading there and went to the comments. I
| dont mean any disrespect, just trying to give a counterpoint: I
| like to have some more upfront info about an article's thesis
| before I invest in reading about an Italian history lesson
| hoping it will be relevant. I see this style a lot for example
| on substack, as if people are always trying to weave in a
| relevant historical anecdote or clever fact, which is fine, but
| it sets a much higher bar if the article is going to delay the
| build up to a point without first telling me what that point
| will be. It's easy to get bored along the way and bail.
| simlan wrote:
| Well tastes are different. I liked it because of the
| interspersed backdrop. Granted i am reading this on a chill
| evening not on the go. Would be different if i had expected a
| to the point article.
| version_five wrote:
| Yeah I definitely don't find it odd to like that style, I
| was just reacting because I found it interesting that the
| thing someone highlighted as being a strong point was
| exactly the thing that had put me off. Different tastes as
| you say.
| gcthomas wrote:
| Anything from Tim Harford is going to be good. His "More or
| Less" Radio 4 series is a must listen podcast.
| giardini wrote:
| How about a book on "The Tyranny of SPAM" or perhaps "How I
| Market My Books Online by Rewriting Redundant News Stories
| Already Better Written by Others" ?
|
| See
|
| https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
| mtoddsmith wrote:
| Shouldn't Excel warn you if your data is going to be truncated?
| Or did they use Excel in a way that suppressed such warnings?
| hartator wrote:
| Yes, hijack my scrolling. I really don't like have a consistent
| feedback when I am scrolling pages.
| sahila wrote:
| Dunno why you're being downvoted but it's true. I noticed it
| pretty quickly when I tried going back to the previous page and
| instead it scrolled up.
| stevenpetryk wrote:
| They're being downvoted because it's against the rules to
| criticize the design of the site on which the content is
| hosted.
| CivBase wrote:
| I hate Excel, not because it's a bad program but because in my
| experience people misuse it more often than not.
|
| Excel excels at creating spreadsheets. Spreadsheets are pretty
| reports used to report tabular data. Excel is capable of so much
| more, but for eveything beyond spreadsheets it's medicore at best
| and its complexity often makes it a liability.
|
| It's a bad calculator. It's a bad database. It's a bad medium for
| transferring data. It's a bad front-end. It's a bad notetaking
| app. If your use case goes beyond making tabular data look
| pretty, Excel is probably bad at it.
|
| People use Excel for _everything_ because they 're comfortable
| with it. But other solutions are often much better and I'm so
| tired of cleaning up all the issues Excel has caused (and
| continues to cause) me.
| romwell wrote:
| Nah. Spreadsheets are poor man's MapReduce.
|
| Most Excel tables I've seen aren't just data and presentation.
| It's _computing_ the data _as well as_ presenting it.
|
| And how is the data computed?
|
| * Applying formula to a row/column range to get another range
|
| * Applying formula to a range to get a single-cell output out
| of it
|
| Obviously, I am not talking about clustering and performance
| here, but a computational paradigm.
| bartread wrote:
| I think this is a really unfair comment. Excel is a very
| powerful tool. It's great for financial modelling and
| prototyping, for example. It can be great for testing out
| different scenarios and investigating likely outcomes.
|
| It's also something that most people in most businesses have
| installed and as such, can be incredibly helpful for sharing
| data, models, etc. The collaborative editing in Office 365 adds
| a new dimension in that (finally) we don't have to email round
| documents and implement half-cocked versioning any more (though
| plenty of people still do), and multiple people can make
| changes at the same time.
|
| If you want to slag off any of the applications in Microsoft's
| Office suite then the correct answers (and I don't think this
| is particularly controversial) are Outlook and Teams. If you
| want real productivity destroyers and daily pain, look no
| further.
|
| Excel? It's great.
|
| Yes, it can be misused, but so can Python, Rust, C++,
| JavaScript, and every other tool, language, and platform at our
| disposal. So can a circular saw for that matter. That doesn't
| make any of them bad tools.
| CivBase wrote:
| I think you missed my point.
|
| Yes, Excel is a very powerful tool and I would never argue
| otherwise. I definitely wouldn't argue against using it
| altogether. But I have come to loathe it because of how often
| it is misused.
|
| Sure, any tool can be misused. But in my experience Excel
| gets misused more than anything else combined. Excel's
| accessibility and ubiquity is part of what makes it so
| powerful... and what compels people to use it where other
| tools are much better suited.
|
| Don't get me wrong. It's okay to use a suboptimal tool... to
| a point. Users should be aware of a tool's limitations and
| seek out better solutions when it's clear those limitations
| will cause problems.
| greazy wrote:
| I completely agree with your main point: excel is terrible
| because people use it for everything.
|
| I've noticed there different ways people use excel: as a
| dashboard, to store data, to modify data.
|
| IMO storing data is the worst use of excel, especially when
| it's done in a non 'tidy' fashion , with highlights and
| formatting to encode data.
|
| This is from my experience as an R and python user in science.
| hodgesrm wrote:
| We run our company finances off Google Sheets, which is
| essentially Excel, but shared. It's an outstanding tool for
| financial modeling as well modeling in many other domains. We
| would not be able to function without it. The fact that others
| may misuse it does not change things for me.
|
| p.s., Google Sheets is one of the more underrated cloud apps of
| all time.
| rvba wrote:
| The article is a nice read, but very inaccurate.
|
| Most companies do not keep their books in spreadsheets. Perhaps
| there are some poor souls who use Excel for bookkeeping, but this
| is generally impossible for any company that is audited (so
| basically everything bigger than some micro company). Auditors
| will demand to store all bookkeeping data in some ERP system -
| that generally allows double entry bookkeeping and many other
| things "from the box".
|
| There are hundreds of better or worse bookkeeping/ERP solutions
| around word - some are expensive (and can cost millions to
| implement and still suck), some are cheap and some are even free
| (OpenERP) -> they are the tools used to input and store
| bookkeeping data, not Excel. Those solutions often come with own
| fair share of problems, especially in implementation. There are
| various business and programming horror stories.
|
| So the article talks first about double-entry, which is most
| basic form of bookkeeping, then proceeds to call it accounting
| (which is like a "bigger" term - because it involves much more
| things, e.g. reporting, calculation of accruals, tax..).
|
| Then somehow we move from double-entry (not done to Excel), to
| the UK mishap, where some low level public office clerk used
| Excel to combine tables that show number of coronavirus cases.
| What does it have to do with bookkeeping or accounting? I dont
| know. I guess for some, keeping something on a list =
| "bookkeeping". What is just plain wrong terminology. If some
| programmer makes some table in a relational datbase - you
| probably dont call him a bookkeeper, or accountant. But using the
| logic of this article - they should be, what is obviously wrong.
|
| In the UK case, Excel was used as a database - and something went
| wrong when combing data from multiple sources, because the people
| responsible did not check at all what they were doing. There are
| multiple questions here: did they use the correct tool? Why didnt
| they check via some report if all data was uploaded? Why did they
| use XLS format, which is from 2003, so 18 years old? Also "not
| checking what are you doing" happens not only to public servants,
| for example in programming there is the very well known case of
| MongoDB, that used math.random to randomly log just 10% of
| issues.
|
| The whole "16k missing cases issue" was reported multiple times
| by multiple sources and the narrative changed few times: IMHO the
| most plausible explanation is that UK government wanted to cook
| the books, so they made a bad way of consolidating data (that
| reduced number of cases) and when someone detected that it does
| not add up, they blamed it "on the computer". Public will eat it
| up, since everyone made some mistake. What is confusing for me
| that programmers also buy this. If your program does something
| wrong, probably there is the issue with the program. If some
| Excel VBA macro did something wrong when consolidating data -
| this is somehow problem with Excel, not the macro, or the ones
| who used it and didnt check it if even works.
|
| In fact the first time they reported the problem, they said that
| they stored each data in a separate column. Excel 2007+ had a
| limit of 16384 columns, so one could argue that Excel is better
| than PostgreSQL, because PosgreSQL has a limit of only 16 000
| columns ;)
|
| So if some bad programmer also used a real database wrong, same
| issue could happen. And please dont act there arent many bad
| programmers in the wild - when a new guy comes usually you hear
| that the previous code is spaghetti garbage, what is often quite
| true. There are lots of programming horror stories. There are
| also lots of accounting horror stories. And lots of Excel horror
| stories. But the missing 16k cases has nothing related to double-
| entry, or accounting. You dont store cases on some income
| statement or balance sheet account...
|
| If someone confused a motorcycle with a car, they would probably
| receive a lot of shit. Here we have same case: collecting few
| tables to make 1 big list has nothing to do with accounting. Or
| double entry. As I said, it is nice read, but completely wrong
| terminology.
| grouphugs wrote:
| the tyranny of white supremacy and globalism and pointless labor
| gcthomas wrote:
| Spreadsheets are a hammer so if that's all you know then all
| problems look like the nail. Really, many power users of Excel
| ought to be looking at more capable, testable and readable
| solutions from professional data analysts and scientists. Python
| would be a good start, but there are many options better than
| Excel for critical systems.
| Closi wrote:
| > Really, many power users of Excel ought to be looking at more
| capable, testable and readable solutions from professional data
| analysts and scientists. Python would be a good start.
|
| I can write python, but I can get the answer out of excel in a
| fraction of the time for most use cases.
|
| As an added bonus, excel documents are far more ubiquitous and
| I can share them with clients, and they can see my workings.
| Python isn't very transparent and business users can't usually
| check the logic.
|
| Excel is also much more interactive and allows for much more
| "discovery" and playing with the data.
|
| What I would say though, is most power users don't know about
| functionality in excel like PowerQuery, relational data models
| and DAX that actually do turn it into a serious and repeatable
| tool.
|
| (I'm not arguing that Python isn't better for many use cases,
| and excel definitely isn't right for most critical
| applications, but I also think Python is much slower for ad-hoc
| data analysis so both have different places in the market.
| Anyone think python is faster? I'm down for a race!).
| Xavdidtheshadow wrote:
| Spreadsheets are great. I just wish folks would use a more
| appropriate tool for what they really want: a very light-touch
| database. Something like Airtable is perfect for structuring a
| bit of data into rows and columns.
|
| But, nothing beats the availability (both of sharing and
| editing) of an open Google Sheet.
| dan-robertson wrote:
| Many power users of excel _do_ use more capable, testable and
| readable solutions. But they also use excel. Programming sucks
| in many ways that excel does not. Excel is incredibly flexible
| and adaptable.
|
| If you need to treat one row differently as a special case that
| is easy in excel but can be a lot of work with programming,
| both for technical reasons (you need to handle the special case
| with special code) and for silly reasons (in excel you click
| the row which is special; in a program you must describe how to
| find the row.) Certainly there is an argument that one
| shouldn't have such special cases as they make maintainability
| a nightmare but sometimes it matters a lot more to have
| something now rather than something later.
|
| In excel, the data is the most important thing and in the real
| world it often is too. In a program, the data (especially the
| intermediate data) is quite invisible. Power users may spot
| errors because intermediate values look wrong, rather than by
| carefully reading the formulas. Though if the values look right
| or there are a lot of them, errors may not be spotted.
|
| Excel is very ad-hoc which is the problem people complain about
| but also it's power in dealing with the changing real world and
| anything exploratory. Once it is working and stable in excel,
| it is more wise to do it in python.
| chasd00 wrote:
| what would python be working on then? a .csv, textfile,
| database, something else? Or would the data be stored as part
| of the script itself?
| anthk wrote:
| TSV, you have several tools to parse it. Even AWK.
| asdff wrote:
| You would have the script take in the data as input from
| somewhere else, maybe your organizations shared drive. could
| be a .csv or whatever format. could be a database like
| sqlite. Much better than some big hulking macro heavy excel
| workbook.
| CJefferson wrote:
| Then how do I edit, extend and quickly filter and search
| that CSV? I'd probably do it with Excel to be honest.
|
| Lots of data in business isn't generated by another
| program, it's built and maintained by users.
| asdff wrote:
| Python or R have tons of ways to edit, extend, and
| quickly filter and search within stuff like a CSV. You
| could even do all of that with a bash script. It will
| probably run a whole lot faster on your computer than
| opening the excel workbook.
| CJefferson wrote:
| I think we might be talking at cross purposes. How, in
| Python, do I extend a CSV with new numbers, or edit
| existing values? Use 'input'? I could implement an excel-
| like interface, but then I already have Excel.
|
| I could just open it in a text editor I suppose, but then
| I have to deal with escaping, and moving around cells,
| and that sounds painful.
| asdff wrote:
| You would read in the csv, edit what you need or
| add/remove data, then write a new file or overwrite the
| old one. You don't need to implement any interface, there
| are tools that do all of this. You could probably write
| the code up for something simple like this in the time it
| takes for excel to open the file and you to make the
| change manually by finding the field, clicking away with
| the mouse, and typing it into the cell. Specifics beyond
| that can be searched for on the internet as there are a
| wide variety of methods you could tailor to your purpose.
| pfranz wrote:
| I think you raise a good point that often gets forgotten.
| When you switch from a general purpose tool to something
| specific, you lose visibility and introspection tools.
|
| I see people move from a filesystem based workflow to a
| database and while that may be faster and more efficient
| you lose those tools to look at and tweak data. You
| either need to write those or teach them how to navigate
| the database. People rightfully hate black-boxes as soon
| as stuff stops working as expected.
|
| Personally, I love the Linux command line and will pipe
| stuff in and out or stash things in flat files. I may
| move the data back into a spreadsheet to get more
| visibility or look for trends.
| pfranz wrote:
| It varies and how you're outgrowing it. I heavily use
| spreadsheets, but if I start to hit some limits or need to
| formalize a process I'll switch to something like Python. As
| a data store I may still query the spreadsheet, or use csv,
| sqlite, or something that scrapes the data source. I really
| like columnar data because I like mucking with it by piping
| data on the command line (which has its own, well known ways
| to bungle data).
|
| As for reasons you're outgrowing a spreadsheet; access and
| permissions, friendlier error messages or a better "wizard-
| like" process, formalizing input data, outgrowing data size,
| smarter caching for computing values, protection or clarity
| around modifying "constant" or "magic" numbers. Many of those
| you _could_ implement in a spreadsheet, but its often more
| complex than using something else. That something else
| probably varies with which problem you 're trying to solve.
| vxNsr wrote:
| Excel is more than just a hammer it's the fork that is in every
| single drawer of every single company/org in the world.
|
| It's not about every problem looking like a nail, it's that,
| nobody had to pay for this fork, somehow everyone has it and
| it's such a great fork that it can do 90% of the problems you
| throw at it.
|
| Yea sure some problems would be better solved with specialized
| tools, (eg knife, spoon, etc) but those require training and
| budget. No one has those. Everyone has a fork.
| gizmo686 wrote:
| > nobody had to pay for this fork
|
| Actually, everyone had to pay for that fork. Excel is a paid
| product, and isn't even included in the cost of Windows. It
| is, however, bundled with MS Office, which includes Word,
| which is even more of a must have at almost every single
| company/org.
| pault wrote:
| Cutting steak with a fork is a pretty good metaphor for
| working with a spreadsheet heavy enterprise team.
| [deleted]
| zwieback wrote:
| Spreadsheet is the greatest SW ever invented, that's why it's so
| dangerous.
| Jaepa wrote:
| This was adapted from Tim Harharford's podcast Cautionary Tales
| eps Wrong Tools Cost Lives
|
| https://timharford.com/2021/05/cautionary-tales-wrong-tools-...
|
| If you have not listened to the series I highly recommend it. His
| episode "LaLa Land: Galileo's Warning" is by far one of my
| favorite pieces of media. In brief it is why redundant tightly
| coupled fail safes will often lead to cascading failures.
|
| https://timharford.com/2019/11/cautionary-tales-ep-3-lala-la...
| vxNsr wrote:
| Oh man, it reads exactly like his podcast 50 things that made
| the modern economy. I was having flashbacks but couldn't
| pinpoint why. Thanks for sharing.
| john-tells-all wrote:
| As a DevOps consultant, I highly recommend this series. The
| amount of things that can go wrong in the real world is jaw-
| dropping, let along what can happen when computers get
| involved! My DevOps buddies _love_ this stuff.
|
| I also adore the specific episode mentioned by Jaepa. It
| clearly describes why "hey this bandaid helps us, so let's do
| more of it" can actually _cause_ problems, vs helping us be
| more safe.
| dwohnitmok wrote:
| To what extent is the Galileo effect described in that episode
| really just a kind of survivorship bias? If you've eliminated
| all the failures that can be caused by non-redundant systems
| with no failsafes, then the only remaining failures are those
| that can be caused by redundant systems with failures (or at
| least slipped through). The way to see whether those
| redundancies and failsafes are working is to see whether the
| overall failure rate is decreasing, not solely whether failures
| are being caused by those redundancies and failsafes.
| narush wrote:
| Fantastic blog post. I highly recommend reading it in full, and
| also checking out the work of the European Spreadsheet Risks
| Group, and Felienne Hermans specifically (referenced in the
| original post). I've been working on a spreadsheet startup [1]
| for the past 8 months or so and those folks have a large amount
| of absolutely upsetting-but-helpful research on spreadsheet
| usage/errors.
|
| After the past few years working with spreadsheet power users,
| here's how I like to think about them: 1. Most spreadsheets are
| just used a lists/trackers [2]. 2. Some spreadsheets are very
| calculation heavy, and are better understood as complex software
| projects (usually modeling something; s/o to those PE mfs) than
| as anything else. 3. Spreadsheets make the transition between
| (simple list/tracker) and (complex software project very fluid.
| This flexibility usually means that the (complex software
| project) that is created is buggy as hell.
|
| Spreadsheets give not-super-technical users an incredible
| visibility into their data. Spreadsheets give not-super-technical
| users a way to program data transformations in what I would argue
| is the most generally intuitive way that exists. Spreadsheets
| give not-super-technical users tools to build software without
| introducing them to proper software development methods. Ya know,
| like maybe a test. Or a code review. Or no global variables. Etc.
|
| If you want to see how we're attacking the spreadsheet problem,
| check us out. Feedback highly appreciated! [1]
|
| [1] https://trymito.io/hn [2]
| https://www.joelonsoftware.com/2012/01/06/how-trello-is-diff...
| arthurcolle wrote:
| I never thought I'd be someone to do this but 'mito' is slang
| for pathological liar in French, i.e. short for 'mythomane'
| haha, kind of funny for something that's supposed to be a
| persistent source of truth. Names don't matter at all, but just
| wanted to comment on this. Lol!
| asdff wrote:
| I don't know why we think the user is some child and try to
| abstract away any and all complexity from their job, which ends
| up leading to some proprietary solution with huge
| inefficiencies somewhere compared to a flat file and a script
| in python or R. Software like excel is often seen as a way to
| do stuff you could do in R but without having to write code.
|
| Imo that thinking is wrong because it makes this assumption
| that doing anything at all with a gui is easy and frictionless.
| I don't know how to do anything in excel. If I have to look up
| how to make a plot in excel, I will have to find some article
| on the internet of how to do that in my particular excel
| version. Excel is tough software to use that only becomes
| easier after you spend time with it (hint hint, just like
| programming). If I don't know how to make a plot in R, I might
| also have to find an article. It's no different, no harder,
| usually easier in fact, to just do the job in R. All these
| common tasks in R or excel are all 5 min articles worth of
| information you have to cram in your head.
|
| It's not hard to code, children do it. I'd love to see
| spreadsheet power users realize that they could be working with
| much simpler things like parsing plain text files with a few
| lines of code rather than trying to open some macro heavy
| spreadsheet on the shared network drive that takes 5 mins just
| to open on the work issued workstation. I'd guess it would be
| up to business programs and accounting programs and all these
| other college majors to actually teach classes in python and R,
| rather than what they do now which is teach classes in excel.
| Imo the entrenchment in excel is rooted in ignorance to other
| (often simpler) options available with python or R, than in any
| actual critical proprietary features offered by excel.
| Microsoft is probably pretty happy that their software is so
| baked into these different academic curriculums, keeping it
| alive in industry going forward for an entire working persons
| career after college.
| CJefferson wrote:
| People aren't learning excel because Universities are part of
| some Microsoft conspiracy. They are learning it themselves,
| or from each other.
|
| My mother self-learnt excel, first as a to-do list, then a
| money tracker, then she learned some simple equations to keep
| track of weekly spending.
|
| How in R would someone do that? With a nice graphical view? I
| agree Excel has many many issues, but people use it because,
| in my experience, it super easy to use and in particular
| let's you easily mix data and code.
| Zababa wrote:
| People are also learning excel because that's the only way
| to program in a big company without going through "official
| IT", which won't listen to what you have to say if you're
| not a $100 000 project they can offload to a third-party.
| rexreed wrote:
| * The Utility of Spreadsheets
|
| No single application has been as widely adopted by as wide a
| number of people for such a wide range of uses as the
| spreadsheet. Love them or hate them, but the spreadsheet metaphor
| is at once highly useful, highly adaptable, and highly usable
| requiring minimal support to extract needed value.
|
| The spreadsheet, with all its questionable glory, is here to
| stay. From the first Visicalc to Excel 2300 I don't think we'll
| see the end of spreadsheets for a long time coming.
| echelon wrote:
| You're correct, but this entirely misses the point of the
| article. It's not calling for the end of spreadsheets -- it's
| stating that there's an unspoken problem that demands our
| attention.
|
| Spreadsheets aren't 100% reliable for use cases where you need
| to collaborate and share immutable health records. Especially
| during a time of global emergency when tensions are heightened.
|
| Spreadsheets don't impose validation, schema correctness,
| constraints, etc. and can amplify human errors. They can also
| inject errors of their own (eg. turning March1 the gene [1]
| into a date) when they're simply trying to be helpful.
|
| How do multiple people manage a spreadsheet? How do you safely
| merge spreadsheets? How do you keep records from being
| duplicated or replaced? How do you do double bookkeeping? Can
| you atomically identify individual records?
|
| This article is saying that we have to realize spreadsheets can
| be a source of scientific error simply because of their design
| intentions and ergonomics.
|
| The closing remarks estimate that 1,500 people died as a result
| of spreadsheet error. That's remarkable.
|
| [1] https://www.proteinatlas.org/ENSG00000145416-MARCH1
| ttul wrote:
| Personally, I strongly prefer working with GSheets to Excel. It
| handles large sheets better and you never worry about the app
| crashing and taking your precious work along with it.
| arthurcolle wrote:
| Large sheets? Google Sheets has a much lower cell and row
| limit compared to Excel
| rtkwe wrote:
| I assume they mean performance wise. Excel could have
| larger hard limits but handle complex sheets worse.
| dan-robertson wrote:
| Instead one only needs to worry that one can't get one's work
| done.
|
| I don't think I've ever had a time when I wanted to use a
| spreadsheet for anything moderately complicated where gsheets
| was up to snuff. That's not to say that excel doesn't have
| problems or does things easily, but it doesn't lead to quite
| the same level of frustration and hair-pulling, and indeed
| it's usually possible to achieve things in excel when they
| are impossible in gsheets. Indeed that app even struggles at
| the 'easy' task of being able to fiddle around with table
| formatting. Its sole advantage is the collaboration. It can
| be used as a place to dump small amounts of varied tabular
| data to share but not really for calculation or analysis.
| joe_the_user wrote:
| Data caught in spreadsheets is kind of an instance of the general
| siloing of data.
|
| The situation is something of a general problem of a data-driven
| Society. One group of people want information available with one
| interface and spends resources only in putting it in a format
| suitable - to the detriment of others who/want need it. With
| spreadsheets in particular, naturally you have a data-integrity
| but that's still an instance of the broader problem.
| themodelplumber wrote:
| It's a general psychological problem, too. A huge part of
| storage organization psychology, whether in software or in a
| physical room, is subjective/proprietary perception and
| judgment. People who prefer and naturally warp any job role or
| position to use these tools, as all humans do with subjectively
| preferred processes and perspectives, will never stop creating
| them.
|
| The good part of this not-invented-here mindset is that it's
| also how technology moves forward. "I need this new system to
| be mine/fit my way of organizing" is also often another way of
| creating a "this never existed before" product or outcome.
|
| These storage and organization tools and silos are born as
| subjective (context-fitting) design processes. So they tend to
| get locked up behind depth-oriented (subjective) processes, and
| eventually frozen by stabilizer groups.
|
| Stabilizer groups follow up to use the tools and promote the
| idea of keeping the data siloed the way it is. They don't like
| change at work because change breaks their preferred
| psychological processes, causes them to have to re-build their
| perceptual frameworks, and because they're humans, this makes
| them do stupid things at work, like lash out or become passive-
| aggressive or detonate their new diet plan or become late for a
| baseball game. They will blame all of this on "open data" or
| whatever it is that caused their stable workflow to
| destabilize.
|
| There are lots of solutions to this, and Excel by itself can be
| seen as an attempt at a solution to the problem...also the
| problem can be moderated by the system of energy surrounding
| the data valuation and access to the data.
|
| And "just don't silo stuff, guys" seems to be the proposal with
| the worst track record so far.
| CJefferson wrote:
| So, what's your alternative?
|
| Spreadsheets have many advantages, they are easy to edit, copy
| around, and quickly filter and do simple calculations (like
| find how many students are in a class if size smaller than 10,
| for example). I can't think of a sensible alternative given
| current tools -- we should make better tools of course.
| Koshkin wrote:
| O, the tyranny of web page animations...
___________________________________________________________________
(page generated 2021-07-22 23:00 UTC)