[HN Gopher] Spreadsheets Are Hot-and Cranking Out Complex Code
___________________________________________________________________
Spreadsheets Are Hot-and Cranking Out Complex Code
Author : LukeEF
Score : 140 points
Date : 2022-04-08 12:13 UTC (1 days ago)
(HTM) web link (www.wired.com)
(TXT) w3m dump (www.wired.com)
| grumblepeet wrote:
| I've been doing this for customers for years now using Microsoft
| Lists (sometimes I use Excel) as the back end and PowerApps as
| the front end gui/data entry. Now the language from PowerApps
| (which is based on Excel) has been open sourced and Microsoft
| want to make it available in other apps too. That has
| questionable value but my solutions continue to be popular
| because the people I make them or can easily edit and make simple
| changes.
|
| Some of the solutions we make have been widely adopted too, with
| hundreds and in some cases thousands of users.
| wintermutestwin wrote:
| >Suddenly, the field has begun to bloom. A small cluster of
| startups have in the past year released spreadsheet products
|
| All of which are subscription based SAAS. Maybe I am showing my
| VisiCalc oldster roots, but I want my document related tools to
| be pay once / run locally.
|
| I long for a computing world where the free tier is run-in-a-
| browser-on-someone-else's-computer and the paid tier is run local
| and native.
| nojito wrote:
| The best startup ideas are creating one part of the O365 system
| (usually Excel) and monetizing it separately.
| wintermutestwin wrote:
| Ironically, O365 is $7/month with max rows = 1m+ and
| spreadsheet.com is $9month for max rows = 10k.
| kgodey wrote:
| We're building a similar tool that's self-hosted only
| (https://github.com/centerofci/mathesar).
|
| I'm glad there are more tools to make working with data easier,
| but I don't use any of them because putting my data into a
| proprietary service I have to keep paying for is a dealbreaker.
| InflexHQ wrote:
| My reactive workspace tool aka spreadsheet replacement (Inflex)
| is SaaS, but I've also considered offering a completely
| downloadable tool for a one off price. I think if ever get a
| substantial user base and people ask for that, it's on the
| table.
| toss1 wrote:
| Yup
|
| It is a horrible and unusable business model/architecture.
|
| 1) Requiring subscription to access when it is not technically
| required is onerous and very questionable value, especially for
| tools that I'll use intermittently.
|
| 2) Requiring online access instead of local & native apps makes
| the false assumption that I'm always connected. Often my most
| productive times are on 6-hour flights (& no, the onboard wifi
| is not reliable) or away at locations where there is no
| connectivity.
|
| 3) Performance will always suck compared to a fully local app -
| all the optimizations a developer could do to make it run
| acceptably could also be applied to a local app to make it
| really snappy
|
| 4) Any SAAS carries significant extra risk that does not exist
| local native app - that the software /service could disappear
| any time due to a host of business reasons. Sure, native app
| companies can also go out of biz or discontinue products, but I
| still have the software and can upgrade on MY schedule.
|
| 5) MOST CRITICALLY, entire classes of very attractive customers
| are LEGALLY locked out of using your apps. In my industry,
| there is an entire new class of information called CUI --
| Confidential Unclassified Information -- anything related to
| DOD projects that isn't quite classified, but there are very
| strong restrictions on what can be exposed in what way. My
| wife's industry is legal, and they have almost nothing that
| could be put on a SAAS, even billing information. And of course
| same with medical.
|
| Of course there are some apps where the online SAAS is close to
| essential, like real-time collaboration / conferencing.
|
| However anything else, and it looks strongly like the business
| model is extractive and rent-seeking, instead of providing
| value. (and this also applies even to IOT devices - I should be
| able to access those via the IP, not only through your
| service...)
| ceilingcorner wrote:
| Sadly it feels like single purchase software is on its way out.
| All the momentum is toward SAAS.
| wintermutestwin wrote:
| As a greybeard, I have seen plenty of old ways become new
| again. It's hard to predict when the next shift will be or
| where it will go, but it will shift.
| satyrnein wrote:
| Subscription based SaaS doesn't bother me in and of itself.
| Cloud based supporting multiple collaborators and multiple
| devices, tech support often built in, no managing upgrades or
| patches, etc.
|
| The problem is portability and lock-in. Which itself might be
| due to lack of standardization.
| BlueTemplar wrote:
| Also that it's pretty hard to make good spreadsheet software
| I guess ? (Even Excel has bugs sometimes, not to mention
| various quirks for backward compatibility...)
|
| AFAIK FramaSoft decided to stop working on FramaCalc (based
| on / hosting EtherCalc, based on SocialCalc), because it was
| just too hard for a small association like that (which
| already has to deal with managing lots of other services for
| free), but they seem to have decided to postpone the
| shutdown, I guess because of COVID ?
|
| https://framacalc.org/abc/en/
|
| (For instance it supports exporting ODS but not importing
| it.)
| bdcravens wrote:
| We are a small B2B, and during the pandemic our primary source of
| income disappeared. Our president built out a new service, one
| that is VERY analytics heavy. He built it in weeks using Excel,
| rather than the months it would have taken using "proper
| development". It's a beautiful monstrosity, and eventually needs
| to be ported to our codebase, but it saved the company.
| joejohnston wrote:
| Check out Budibase.
|
| It's the leading open source low code platform and perfect for
| building UIs on top of spreadsheets
|
| https://github.com/Budibase/budibase
| LukeEF wrote:
| Founder of excel collaboration and versioning startup [1] here so
| I am a believer.
|
| There are only about 30 million programmers. There are over 1
| billion Excel users. Excel is Turing complete. Excel is _by far_
| the most used programming language on the planet. It is easily 20
| times more popular than the next contender.
|
| The value of Excel is that it is presenting the data, with a set
| of formulae that let you keep derived data up-to-date. This
| inferred data provides sums and computations, sometimes simple,
| but sometimes exquisitely complex. And through this whole range
| of complexity, with a billion users, virtually nobody treats
| Excel seriously like a programming language.
|
| We have a programming language which is essentially acting as a
| declarative database, and yet we don't do unit tests, we don't
| keep track of changes, we collaborate with Excel by sending it to
| our colleagues in the mail and god-forbid we should doing any
| serious linting of what is in the thing.
|
| Anyone who has used Excel in anger realizes why it is so
| brilliant. Show me another declarative constraint based, data
| driven inference language that I can teach to my grandmother.
|
| The problem isn't Excel. The problem is that we are treating
| Excel like its a word processor, and not what it is: a
| programming language.
|
| [1] https://versionxl.com/
| pete_nic wrote:
| Excel, and spreadsheets, need a re-branding campaign. They are
| cloud-based, low-code, ELT tools; these terms are typically
| associated with buzzy enterprise tech companies, not technology
| developed in the 1980's.
| nunb wrote:
| All this and not one mention of DabbleDB !!
|
| https://www.youtube.com/watch?v=MCVj5RZOqwY
|
| https://news.ycombinator.com/item?id=102
| recuter wrote:
| Well it disappeared over a decade ago. Very interesting demo.
| Time is a flat circle I guess. Nothing actually comes to mind
| from the present that is quite like this, hmm.
|
| Hmm.
| motohagiography wrote:
| I've learned products designed to replace spreadsheets have a
| huge hurdle because the people who use spreadsheets treat
| operating the sheet as their job. Replacing them removes their
| autonomy and control over an information process, and subsumes
| the value they bring to their employers - so they will resist
| products that threaten that. Excel is a complete management
| subculture.
|
| The other advice I give is if you are generating analytics, have
| a PowerBI connector of some kind because the people who make
| decisions (managers, etc) make them based on PowerBI, and not
| from an interface their staff is a peer at using, and likely has
| control over. In enterprise, they want data in metrics their
| staff can't see, hence a separate tool.
|
| Spreadsheets will always be with us I think. The opportunity may
| be in creating one that is has sufficient work-alike features
| with legacy ones, with new power features (python, etc) where
| there is a connector between the high power open development
| environment, and the familiar Excel ones managers use. Key thing
| being not asking managers or sr. employees to change.
| rr808 wrote:
| > I've learned products designed to replace spreadsheets have a
| huge hurdle because the people who use spreadsheets treat
| operating the sheet as their job. Replacing them removes their
| autonomy and control over an information process, and subsumes
| the value they bring to their employers - so they will resist
| products that threaten that. Excel is a complete management
| subculture.
|
| Spreadsheets are better because as you say the owner is their
| job to maintain it. If you replace with an IT process the new
| "owner" is likely a below-average developer that probably is
| uninterested in the business. A few years down the road the
| usefulness of the replacement will suffer.
| fivea wrote:
| > Spreadsheets are better because as you say the owner is
| their job to maintain it. If you replace with an IT process
| the new "owner" is likely a below-average developer that
| probably is uninterested in the business.
|
| I disagree. Spreadsheets are incomparably worse because what
| you charitably described as "the owner is their job to
| maintain it" in real life it's reflected as having a single
| employee who abused a first-move advantage to monopolize and
| excerpt unduly control over, and even hijack, key operation
| areas.
|
| We all heard horror stories of how employees screwed over
| their former bosses because only they had control over things
| like key spreadsheets. Advocating for spreadsheets is
| advocating for these vulnerabilities.
| civilized wrote:
| > I've learned products designed to replace spreadsheets have a
| huge hurdle because the people who use spreadsheets treat
| operating the sheet as their job.
|
| In general? Not really. Just as frequently, the fancy tool
| promoters don't care to understand the subtleties of the job
| and when it requires flexibility or judgment that the
| spreadsheet accommodates better. They have their hammer --
| software formally engineered by software experts for
| disempowered "users" -- and everything looks like a nail.
|
| "It's faster and more reliable (when everything goes as
| planned)" isn't really the slam dunk these folks think it is.
|
| Give these users a more flexible tool like Alteryx, that
| actually lets them do their job, and I've seen that they'll
| happily migrate off of Excel.
| doctor_eval wrote:
| I agree. One of the great things about Excel is that it's
| massively flexible. It's almost the antithesis of what most
| development environments want to be. Programming is about
| considering all the code paths. Spreadsheets are about "what
| if".
|
| The flip side of this is that understanding the behaviour of
| a spreadsheet is generally a specialist job, which is why we
| have people whose job it is to "run" the spreadsheet. The
| spreadsheet has rules and boundaries and it will stop working
| if you just start plugging random values into formula cells.
| rbonvall wrote:
| In my experience, just offering the ability to export to and
| import from Excel workbooks goes a long way in winning the
| spreadsheet-loving crowd.
| syshum wrote:
| To replace spreadsheets, you have to eliminate the roles (jobs)
| that create the spreadsheets. Automation and proper BI can do
| both....
| cm2187 wrote:
| Another way to see the same problem is that the IT system
| replacing the spreadsheet will take a long time to build and
| replicate the process in the narrowest possible way. Then
| something changes, updating the system will be an uphill battle
| that will take years fighting for budget and prioritisation,
| and you have to revert to spreadsheets.
| technofiend wrote:
| Plus the gap between a spreadsheet and an application can be
| huge in a large organization. No one needs a permit-to-build
| process or ci/cd pipeline to start a new spreadsheet and make
| changes to it.
| [deleted]
| theptip wrote:
| I think there is scope to replace Excel, but it's hard. You're
| seeing a gradual spread of "data science" tools in finance, as
| the more technical analysts start to use Python over Excel. But
| as you say, the management layer still uses Excel to poke and
| prod a model.
|
| I think that there will be a generational shift here - you are
| not going to train an SVP to use Python, but the next
| generation of SVPs might have more exposure and be willing to
| use Numpy in a Jupyter notebook.
|
| And in the other direction - there is definitely scope to come
| up with an "Excel isomorphic" Python framework for data
| science. It's fairly easy to generate an Excel sheet from
| Python computations, but maintaining bi-directionality is Hard,
| and would require restrictions on the Excel side. I think with
| the right UI, you could do this though.
| civilized wrote:
| I have a colleague who is a PhD in Applied Math. Pretty
| bright guy, huge Python/Jupyter lover with many years of
| experience. Loves to use git, loves to write dozens of unit
| tests. A true Man of the Future, according to Excel haters.
|
| He wrote some Python to solve some mildly complex business
| problem. I told him to translate it into Excel for
| stakeholders. He did, and the answer came out completely
| different.
|
| It turned out he had made multiple catastrophic errors in the
| Python. This is not the first, second, or third time this has
| happened.
|
| Python, and tech-beyond-Excel in general, just isn't the
| silver bullet software types often seem to think it is. Even
| experts sometimes seem to do a worse job in it than in Excel.
| pwnna wrote:
| I love both spreadsheets and Python/Jupyter. But neither is
| Excel the silver bullet? The reason the error is caught
| (and the real lesson) is more because he tried to reproduce
| his work as opposed to a particular technology stack.
| catgary wrote:
| That is hardly special to Python->Excel translation, and
| happens any time someone reimplements a piece of software.
| civilized wrote:
| I think Excel has something to do with it. In Excel,
| you're usually forced to do computations in small steps
| and look at the intermediate results. In traditional
| coding, you don't see anything but the final result
| (unless you ask to see it). It's much easier to assume
| everything is working as intended when it's not.
|
| Excel has its problems too. Different tools for different
| jobs. Tech boosters need to understand this and not just
| cynically assume that spreadsheet lovers are old fogeys
| who are afraid of their jobs being automated away.
| catgary wrote:
| I think you're just describing poor coding practices
| dkersten wrote:
| Certainly if you do traditional TDD or something like
| REPL-driven development, you see the intermediate results
| and validate the correctness of your code as you go.
| civilized wrote:
| Those are great! But they are coding disciplines one must
| choose to follow, and continue following every step of
| the way. You aren't inherently following them because
| that's how the tool works. Sometimes the tool-enforced
| discipline has advantages.
| cpeterso wrote:
| How would one write tests for a complex, mission-critical
| Excel spreadsheet? Or use version control?
|
| Spreadsheets often mix the data and code/formulas and the
| formulas are hidden behind the sheet view and sprinkled
| across many cells. At least Python scripts separate the
| code from the data so you can write tests using known
| good or fuzzing data. And you can use version control to
| track and review code changes.
| civilized wrote:
| I agree that, in the hands of a hypothetical ideal
| person, Python should be better than Excel for almost
| everything. But as my story above shows, even people
| you'd expect to be experts, seemingly following best
| practice, don't end up being very close to this ideal
| person.
| _the_inflator wrote:
| > the people who use spreadsheets treat operating the sheet as
| their job
|
| Yes, observed the same. And "The new tool is faster and more
| reliable!" does not help either. They got their workflow and
| cope with it - for years.
|
| Only time people adopted new tools - banking - when they could
| deliver their assignments way faster to their superiors.
| Personal advantages must be spotted.
| rco8786 wrote:
| To be fair, I don't think it's coping.
|
| Spreadsheets are malleable by design. Being able to modify
| how it functions on the fly is an enormous power for these
| users. And the better/faster/smarter SaaS replacement is also
| extremely rigid and inflexible. So while it works to replace
| the current version of their god spreadsheet, it can't adjust
| on the fly as the user's needs change like the spreadsheet
| can.
|
| I'm convinced that there does exist a "better spreadsheet"
| that treats power users as exactly they and incorporates
| things from the software engineering works like version
| control, modularity, reusability, sharability, etc. that
| hasn't been built yet.
| zarzavat wrote:
| Truly Excel could be improved in a vast number of ways. But
| would anyone use it? Excel is a kind of product that is
| perfectly understandable to the average business person.
|
| The more complicated you make it, the more it becomes like
| real software development. And that is a skill that most
| people don't possess.
| nerdponx wrote:
| It's not about autonomy in an abstract sense. It's the fact
| that regular people can customize, hack, modify, automate, and
| otherwise program the spreadsheet. Data analysts and managers
| are more than trained monkeys; they actually need to do those
| kinds of things (at least sometimes) in order to do their jobs.
|
| I agree that the ideal world is one where you can connect the
| spreadsheets to other data sources, so you get the best of
| both.
| lupire wrote:
| ODBC is decades old
| syshum wrote:
| This is a double edge sword, because often time there is no
| one validing the results of the spread sheets...
|
| "The number looks ok" is not a good validation, and there has
| been some very public data errors as a result of bad
| spreadsheets.
|
| I often wondered if in the average business is even 10% of
| the spreadsheets where actually audited what would happen....
| I suspect the results would be rather shocking
| m_mueller wrote:
| Power Query / Power BI is pretty much that, treating
| spreadsheets as one source among others, with transformations
| and analytics on top.
| TAForObvReasons wrote:
| Excel can be replaced, but it won't be replaced by the current
| crop of VC-funded SaaS.
|
| > Airtable really ought to be killing Excel, but the SaaS model
| combined with a stupidly low artificial row count limit (over
| 50000 rows is listed as "contact us for pricing") means that it
| will never achieve penetration into weird and wonderful use
| cases like Excel has.
|
| Many Excel processes are 20+ years old. No SaaS could replace
| the stability and pricing.
|
| https://news.ycombinator.com/item?id=30871148
| doctor_eval wrote:
| Airtable is great for throwing together a quick prototype,
| but it has so many arbitrary-feeling limitations that I've
| never actually used it in anger.
|
| It's a shame, because I really want to like it - but it's
| less flexible than a spreadsheet, and less powerful than a
| database.
| motohagiography wrote:
| The ability to email a spreadsheet and have it just work on
| the other side (and become the recipients copy they can fork
| a version and "own") is huge. In a SaaS situation, you have
| to solve IAM and security vs. leveraging what users already
| have as a sunk cost in email and windows.
|
| When we think of document based workflows as a problem (vs.
| say just the data/info), we tend to think of them as
| inefficient and prone to duplication, forking, editing,
| versioning problems - but I'd argue these are valuable
| features because they create levers for managing. Maybe I've
| spent too much time staring into the enterprise abyss and
| this is the inner deadness of a consultant speaking, but what
| documents facilitate (e.g. MS Office) is flexibility of
| ownership, provenance, authenticity, sources of truth,
| authority, and other qualities.
|
| When you solve a problem, it becomes inert, there is nothing
| about it to manage anymore, which means someone can't extract
| value from it, and that's value destruction to them. SaaS
| problematizes these document features and then "solves,"
| them, which in fact just constrains managers by concretizing
| data and workflows instead of being a tool that provides
| _some_ data that ultimately supports a narrative conversation
| without being a forcing function on a dynamic of ongoing
| "problems" that is producing value for the business.
|
| I'd suggest this is the quiet part your SaaS prospect
| customers can't say out loud, because managing isn't solving
| problems, it's extracting value from them, and using tech to
| collapse dynamics that are producing value is anti-value from
| that perspective.
| wildrhythms wrote:
| Every day I think about how Microsoft Access allowed otherwise
| not-so-tech-savvy users to, with just a little training and
| practice, build a complete relational database for their entire
| business, supported by a relatively sane GUI and a way to build
| forms and reports with very little (if any) 'code'. I have no
| idea what small companies are using nowadays, but I think there
| has to be some untapped middle ground somewhere between Microsoft
| Access and a dumb spreadsheet.
| segmondy wrote:
| There are still lot's of folks using Access. For those of us in
| the Linux space, we can use Libreoffice Base and script it with
| python. I once built for a local business that needed to join
| the computer age using this. Then about a year+ the owner's
| relative came to the business and decided that it was too old
| school and to get it redone as a web app. Needless to say, the
| owner is not too happy with it, took too long, cost too much
| both to develop and run and has more errors. He lamented to me
| about it when I was in town... The issue is the world has moved
| on from local/native apps.
| mathgladiator wrote:
| I'm structuring my UI for Adama (https://www.adama-
| platform.com/) around the baby of Access and Excel, and the
| engine is available in early access now. I love Excel and
| Access, and I plan a future pivot into small business space.
| CPLX wrote:
| I did wonders with FileMaker Pro for years and my understanding
| is that it's still around as well.
| mackrevinack wrote:
| sounds like you are describing grist
|
| https://www.getgrist.com
| pid-1 wrote:
| My anecdotal experience says Power Query.
| mch82 wrote:
| Claris Works on Macs in the 1990s is the best database product
| I've ever used. It was amazingly user friendly. I used it to
| make form letters & reports. Claris Works prioritized the
| application layer of the database and hid a lot of the
| complexity at the data layer.
|
| DB Browser for SQLite is the coolest database product I've used
| recently. Similar to Mito, DB Browser generates SQL into a log
| as operations are performed in the GUI. Kind of a fun way to
| learn some basic SQL. I need to find a good GUI authoring layer
| for SQLite...
| kevin_thibedeau wrote:
| > I need to find a good GUI authoring layer for SQLite
|
| MS Access with an ODBC connector.
| wslh wrote:
| Does it exist a good Microsoft Access replacement?
| password4321 wrote:
| Appsmith was recommended last year.
|
| https://github.com/appsmithorg/appsmith
|
| https://news.ycombinator.com/item?id=26657803#26658546 Ask
| HN: Best low-/no-code solution for simple web-based database
| frontends
| andy81 wrote:
| For read-only workflows Power Query is the killer feature of
| the past decade.
|
| For writes, there's Airtable, Google Forms, PowerApps etc.
| buescher wrote:
| Microsoft Access is, of course, still around. It would be great
| if there were a solid way to develop a database, forms, and
| reports in Access and then deploy it to the web.
|
| If you've ever had to work with someone else's Access database,
| it is unusual to see a reasonably normalized relational
| database. Most people are much more comfortable with the single
| flat file of Sharepoint lists.
| maxerickson wrote:
| For stuff at the scale of a small Access database, does the
| normalization (or lack of) end up being a big deal?
|
| I get that there are real advantages, I'm just wondering out
| loud whether they are universally applicable/important to
| small scale databases.
| bliteben wrote:
| Probably matters as it relates to data quality mostly
| willhslade wrote:
| Speaking as someone with direct experience maintaining
| Access databases that should have been SQL Server from
| inception, yes it absolutely does. If you need to force
| feed 10+GB through Access artificial 2GB constraint and
| your programming language (VBA) is both single threaded and
| interpreted, if you aren't clever then you will run into
| performance problems, just as you would if you were doing
| something similar in Javascript or Python + SQLite.
| maxerickson wrote:
| I'm imagining a few hundred thousand small records when I
| say small, not 10 GB.
| buescher wrote:
| That's a good question. For an honest and complete answer,
| I'd have to remember the messes I've seen and it's been a
| couple decades now, so I don't. But simply, everything in
| setting up forms and reports is more convenient if you have
| real relations as opposed to a bunch of columns like
| "supplier1, supplier2, supplier3...".
| dvdsgl wrote:
| This is what we're building with https://www.glideapps.com
| beefield wrote:
| As much as I hate to admit it, it is going to be a _long_ time
| before we get rid of spreadsheets in de facto production use. And
| for that, I _really_ would like to see one thing:
|
| I'd like to have a separate worksheet type for "datasheets".
| Looks and behaves just like an ordinary worksheet, but:
|
| - plugging in a formula applies the formula on every cell of the
| column. No exceptions. - you can not have different data types in
| cells within one column. That is, if you have dates in a column,
| you can't have a string in one cell of thecolumn etc.
|
| Yes, I know about powerthings in excel. No, I do not want to mess
| with those. Just normal spreadsheet formulas and sheets.
|
| (Okay, a couple of other things: Get rid of vba and bring in the
| full python ecosystem instead. And if not yet available, version
| control. It is luckily a while I have worked with excel, so these
| may be outdated comments)
| leowoo91 wrote:
| Of course you can do many stuff, except access control and server
| side validation. Spreadsheets rather shine with quick planning /
| analytics.
| MetaMonk wrote:
| This is a ridiculous piece of native advertising.
| KarlKemp wrote:
| Excel is by far the most successful programming language and IDE.
| People love to hate it (and the people using it), which is
| somewhat misguided: there is simply no way to change people (and
| they keep making new ones), so telling them that what works for
| them is somehow wrong is both wrong _and_ doesn 't work.
|
| Instead, the spreadsheet paradigm has the promise of being far
| more powerful. Jupyter notebooks are one example of adapting it
| to a different realm, and it _also_ ended up being used
| everywhere and looked down upon by the snobs.
| sys_64738 wrote:
| People always want to reinvent the spreadsheet with their latest
| whizzbang solution without a problem. But Excel solves 80% of the
| common problems so nobody really cares.
| OzCrimson wrote:
| YUP! And too often a company claiming to have something better
| than Excel isn't a direct equivalent. It's a way to market
| their whizzbang solution.
|
| Domo was claiming it could get companies off of spreadsheets.
| But it turns out that they're an enterprise level reporting
| system that's a minimum of $30,000. That's not a viable option
| for a 5-person non-profit, but they'll get swept up in the
| conversation only to find out later that no, Domo and Excel are
| not equivalents.
| [deleted]
| datalopers wrote:
| Excel users tend to do a far better job of data storage,
| analysis, and forecasting than most of overpaid SaaS-reliant
| "modern data team" (Data Engineers, Data Analysts, Analytics
| Engineers, Data Scientists).
| bliteben wrote:
| It definitely is a scary business proposition for them to
| improve Excel and awaken the 800lb gorilla. Microsoft is
| certainly capable of improving Excel in ways that might even
| improve the productivity of the entire world. Seems like some
| of these startups are likely just giving training to future
| Microsoft employees.
| pete_nic wrote:
| Agree with this. When battling Microsoft product can matter
| less than things like contracts and licensing. I have to
| believe that businesses are spending so much on Azure that
| they get Office for free (as an analogy - at a previous
| employer GSuite was nearly free because our AdSense bill was
| so high). Businesses are unlikely to pay extra for a few more
| features when they already have Excel and have no plans to
| move off Azure.
| OzCrimson wrote:
| Excel is constantly being improved. Are you aware of Power
| Query? And how about the 14 news functions that were released
| to Insiders a few weeks ago? Excel Online is far behind the
| desktop version of Excel but it's making fast progress.
| croes wrote:
| I doubt that. Data hoarding, yes. Creating unsupportable
| spreadsheet solutions, yes.
| b-luu wrote:
| Meh, depends on the data teams (and their chosen suite of
| tools)... Or on the Excel users
|
| Both sides can either hack together horrible work-arounds (a
| matter of "when you have a hammer, everything looks like a
| nail...") as well as brilliantly thought through solutions.
|
| Each tool should be used for it's best use cases, but not bent
| into what it wasn't designed for!
|
| IMHO spreadsheets excel at intuitively manipulating the data ON
| the data itself. While "modern data" tools (especially dbt) try
| to convert date teams to use developer best practices... At the
| expense of less intuitive/direct manipulation of the data.
|
| That being said, I think there are also things we could explore
| in that space: how to make the modern data stack more
| intuitive?
| datalopers wrote:
| > how to make the modern data stack more intuitive?
|
| I'd start with getting people to learn relational data
| modeling and SQL [1][2] at a deep level. Stop reaching for
| python and pandas/spark for every basic data manipulation
| task or query. Stop adding in layers of
| Airflow/Dagster/Prefect when a simple cron would work. Stop
| adding in Kubernetes/GKE/Fargate to manage the
| aforementioned. Stop moving data between systems constantly
| (meltano, airbyte, fivetran) when you already have it in a
| perfectly good place. Stop with the toxic positivity that's
| completely overflowing the modern-data world and all these
| bullshit VC-infused startups who are convinced they need
| every single element and more.
|
| 99% of business needs can be satisfied by a single
| Postgres/Mysql installation and a halfway-competent person
| armed with SQL and an understanding of normalization. Reach
| for Excel when you need to do more "hands-on" analysis,
| business modeling, charting, basic forecasting, and
| presentation for non-technical users.
|
| [1] https://mode.com/sql-tutorial/
|
| [2] https://use-the-index-luke.com/
| throwaway787544 wrote:
| I just learned how to use spreadsheets recently, and _I love
| them._ As a programmer, spreadsheets are _exactly_ the kind of
| tech I want to use every day. User-friendly point and click, IDE
| features, complex functions mapping data across rows
| /columns/tables. All I want now is to hook it up to Postgres, and
| then maybe a new app to automatically design simple web pages
| that pull out and display data with the same functions. This
| would save me months of time per year versus manually coding the
| same things.
| kgodey wrote:
| We're building an (open source, self hosted) tool that does the
| "hook it up to Postgres and pull out and display data with the
| same functions" portions of what'd you like.
| https://github.com/centerofci/mathesar
| jhokanson wrote:
| > Spreadsheets.com, for example, lets users dump almost anything
| into a cell. Drop a photo or a PDF into a cell and the product
| will immediately create a thumbnail, which you can then expand,
| as if the spreadsheet were some sort of blog content-management
| system.
|
| Yes please! Can I get this for Google Sheets? :/
| phkx wrote:
| What do are you using spreadsheets for privately? I never dig
| deep into their capabilities, so I mostly use them to track
| expenses within some particular scope, e.g. healthcare. When I
| recently wanted to compare several loans and estimate our
| financial situation several years in the future, I wrote some
| Python code and used a Jupyter notebook to enter parameters and
| make plots. Has any of you done something similar using
| spreadsheets?
|
| On a side note: I didn't find a Python library for time series
| generation (not analysis). Something where you can build some
| models (e.g. loan, income, expenses) which depend on a common
| parameter (time) and then evaluate all your models for different
| values of the common parameter. Right now, I generate pandas
| series/dataframes and combine them afterwards, which also took
| some massaging of pandas (which I also usually don't use a lot).
| vinceguidry wrote:
| I use Google Sheets for personal finance with TillerHQ to
| automate downloading bank data. It is, as far as I know, the
| only workable solution in this space.
|
| http://www.tillerhq.com
| narush wrote:
| Warning: I'm a founder working in the spreadsheet space, so take
| the rest of my this comment with a (large) grain of salt. I've
| written before [1] (HN and elsewhere) about how I think
| spreadsheets are the most popular programming paradigm ever, we
| just don't talk about it much. As this article mentions, there
| are many ways we can push this forward.
|
| I personally think the most powerful low-code spreadsheet tools
| we can build are those that allow spreadsheet users to easily
| transition to full programming languages, if they want to. So
| rather than locking users into limited and proprietary product
| number #115 (some of them are mentioned in this article), IMO
| it's better if users can transition to a full programming
| language (like Python) very naturally. Som I've spent the past 2
| years building Mito [2].
|
| Mito is a spreadsheet extension to your JupyterLab environment.
| You can display any Pandas dataframe as a spreadsheet, and edit
| it in a very similar way to Excel. For each edit you make, it
| generates the corresponding Python code below for those edits.
| Practically, you can think about Mito as recording a macro, but
| instead of generating scummy-crummy VBA code, it generates
| Python.
|
| We're open core [3]. Feedback greatly appreciated!
|
| [1] https://naterush.io/blog/Spreadsheets-are-the-Ultimate-
| Progr... [2] https://trymito.io [3] https://github.com/mito-
| ds/monorepo
| throwanem wrote:
| Wait, so, you're trying to make programming languages
| accessible to spreadsheet users, by giving a new _spreadsheet_
| to _programmers?_
| bliteben wrote:
| Shouldn't your product be independent of Jupyter? Jupyter is
| amazing but still seems to have a beta feel to me as often old
| Jupyter notebooks will not work at all (for lots of reasons).
|
| I love python, hate the limits on google sheets apis, and don't
| really honestly think VBA is "scummy-crummy" it is just
| unsupported. Microsoft made a huge mistake discontinuing real
| Visual Basic, as it honestly could have been where Python is
| now, instead VB.net is basically dead, and the momentum
| Microsoft had with wysiwyg code editing is way behind where it
| was.
|
| Very cool that you are generating code for the Jupyter
| notebook. What are the practical row limits as I see that as
| one reason someone might use Pandas instead of Excel?
| narush wrote:
| The main benefit (to our users) of being in Jupyter is that
| we don't have to force them to switch up their whole
| workflow. If they want to layer on a spreadsheet, bring an
| extension mak it easy to do this. We don't want to lock
| people into our platform vs just provide the best spreadsheet
| experience we can!
|
| For us, it's nice because we don't have to reinvent the
| wheel(s) that Jupyter comes with :-)
| lph wrote:
| Mito looks very cool! Looking forward to trying it.
|
| The blending of spreadsheets and notebooks seems inevitable.
| One trend notebook-as-program. I've heard several variations of
| this: "The data scientists give us a notebook, and our
| automation runs the notebook to do <ML thing> on <our internal
| data sets>". It's clunky to use a format initially intended for
| interactive visual use for headless automation, but there's a
| practical wisdom to sticking with whatever format the data
| scientists prefer. Twenty years ago, I saw the same sort of
| thing with engineers and spreadsheets.
|
| The one thing notebooks lose, though, is data flow computing.
| That's a major strength of spreadsheets, and the imperative
| execution of notebook cells seems like a step backwards.
| Although I'm sure somebody has bolted some kind of inter-cell
| dependency execution onto Jupyter by this point.
| UncleOxidant wrote:
| Another direction here is reactive notebooks where when you
| change one field in the notebook all of the dependent fields
| are automatically updated. Pluto.jl for example [1]. It has the
| feeling of a cross between a Jupyter notebook and a spreadsheet
| without looking like a spreadsheet.
|
| [1] https://github.com/fonsp/Pluto.jl
| haolez wrote:
| You need the commercial license to disable telemetry? That's...
| new.
| narush wrote:
| Yeah that's fair feedback. Our Pro features are a WIP
| currently, so this might evolve in the future. It was
| important to us that there is a way to be totally telemetry-
| less if users prioritize that - vs most other cloud based
| sass data science tools where you pretty much have no hope of
| total privacy.
| segmondy wrote:
| Just curious, why as a Linux user shouldn't I use Libreoffice
| calc with python? Why bring in something so proprietary?
| narush wrote:
| Mito isn't just a spreadsheet that works with Python, but a
| spreadsheet that allows you to generate Python code when you
| edit!
|
| If your just looking to work with spreadsheets with Python,
| I'd also reccomend checking out XLWings - I haven't used it
| myself but some of our users do and love it!
| kzrdude wrote:
| Seemlessly integrating spreadsheets into jupyter sounds like
| the holy grail to me. I haven't tried it yet, though.
|
| I think their users are people like me - who work a lot in
| Jupyter and swear by it - in a Python data
| analysis/visualisation environment. To bring in the best of
| spreadsheets into that could be magical. To just work in
| libreoffice calc or Excel would be a nonstarter, it just
| doesn't match all the other python tools in the workflow.
| newbie2020 wrote:
| Been thinking about this problem for a long time. I use
| python/jupyterhub notebooks for our data analysis flows at work.
| I've become an expert at it and am a go-to person in our org...
| but even still when I have to try something new on "small"
| amounts of data, I _still_ go to excel first. It's barrier to get
| the results you want os extremely low - thus-far unparalleled by
| any other software I've used in the last 20 years... and I have a
| hard time seeing anything that would replace that (though maybe I
| am just set in my ways...)
| peterkelly wrote:
| http://www.paulgraham.com/submarine.html
| janci wrote:
| Excel is great and versatile, yet lacks some critical features.
|
| - It's easy to load data from database or API without VBA, yet
| impossible to write updates back without VBA. With VBA it's still
| messy string concatenation of SQL queries.
|
| - VBA is security nightmare
|
| - Version control is bad. Distributing spreadsheets in emails and
| collecting changes back is nightmare. Sharing a file on network
| drive lacks fine-grained permission control.
|
| - It's hard to maintain proper normalized relational data model.
| It's impossible to abstract the normalized model from the user
| (i.e. show labels in selectboxes but store ID's)
|
| - It's locale dependent. Date formatting, column separator
| (comma/semicolon separated), even function names. Unusable in
| international data exchange.
|
| - It mixes formatting, logic and data. Impossible to make
| reusable blocks. New lambda functions help somewhat.
|
| And so much more.
| ogogmad wrote:
| > Impossible to make reusable blocks.
|
| The click and drag down feature results in a kind-of code
| reuse. It bypasses the need to consciously name things. Is this
| generalisable?
|
| For small scale code, these concerns may be overkill.
| dfox wrote:
| Excel is locale dependent in a weird way. The internal model
| (which includes the function names) is completely locale
| independent and only the UI is internationalized (according to
| OS locale) and localized (according to Excel language version).
| This is not a bad construction, but breaks down spectacularly
| when you start to use Excel as any kind of real programming or
| data analysis environment, because then you get to the cracks
| in the user-visible "API surface" where the internal/external
| format distinction is somewhat smeared.
___________________________________________________________________
(page generated 2022-04-09 23:01 UTC)