[HN Gopher] Show HN: A simple lazy Python calculation engine, wi...
___________________________________________________________________
Show HN: A simple lazy Python calculation engine, with spreadsheet
demo
Author : bsdz
Score : 112 points
Date : 2021-07-08 05:54 UTC (17 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| yssrnjm wrote:
| This is really interesting because I was working recently on a
| way to have complete spreadsheets using only CSV files. The
| project is in my Github (https://github.com/yassirnajmaoui/runcsv
| and https://github.com/yassirnajmaoui/runcsv-gui) The only
| (major) thing is that the cells are run in top-bottom left-right
| instead of running a graph like this repo suggests.
| Tipewryter wrote:
| you can use pip install -r requirements.txt
|
| I don't see a requirements.txt file.
| bsdz wrote:
| Good spot. You can actually just run the following if you don't
| want to use poetry. pip install
| git+https://github.com/bsdz/calcengine.git#master
|
| Unfortunately, this will only install the core library without
| the demo spreadsheet files. Even if you were to add the extras
| modifier "[demo]", it would only install additional
| dependencies and not the actual demo code. To use the demo
| you'll need to clone and follow instructions in my other
| comment.
| Tipewryter wrote:
| This outputs many pages of debug information and then dies.
| The error seems to be this: This package
| requires Rust >=1.41.0.
| sillysaurusx wrote:
| Hold it! (Pretend I'm Ace Attorney:
| https://www.youtube.com/watch?v=2lQgZ52iONU&ab_channel=DHC)
|
| The solution here is, your python environment is borked.
| I've seen this many times.
|
| Step one, is to install a fresh Python. You can do that
| using pyenv: https://github.com/pyenv/pyenv#installation
| curl https://pyenv.run | bash
|
| Once you can do `pyenv versions`, you should see `system`
| is your only python.
|
| You can run `pyenv install 3.9` to get the latest.
|
| Now that you're running the latest 3.9, the goal is to
| install each of the dependencies of this project, one by
| one.
|
| Step one: try to run the project. It will fail with an
| error.
|
| Step two: install the library that it says is missing.
|
| Step three: Go to step one.
|
| I've followed this algorithm so many times that I can't
| count anymore how often it's saved me. Forget poetry, and
| forget a requirements.txt file. This is the only way I do
| it on my local laptop. (And I usually use the regular
| system python, but, yours seems to be in a suspect state.)
|
| If after all of this you still get errors, then I'm truly
| sorry; god be with you, you're not alone, but I'm not sure
| anyone can help. It's definitely not the fault of this
| project that pillow can't be installed. :)
| DemocracyFTW wrote:
| > your python environment is borked. I've seen this many
| times
|
| Your solution looks simple and easy to follow. There's a
| flow chart to simplify this process here:
| https://xkcd.com/1987/
| bsdz wrote:
| That's very strange. The core library uses pure python and
| demo uses pyqt5, pandas, matplotlib and pillow. I wasn't
| aware that any of those projects (or sub projects) depend
| on Rust. I'm running on Linux / Ubuntu 20.04 with Python
| 3.8. How about you?
|
| (The pip command I just gave should only install the core
| library and so only depends on base python).
| eesmith wrote:
| Does it depend on the cryptography library?
| https://github.com/pyca/cryptography/issues/5771
| bsdz wrote:
| It doesn't look like it: $ grep name
| poetry.lock | cut -d" " -f3 | tr -d '"' | tr '\n' ', '
| appdirs,attrs,black,click,cycler,flake8,kiwisolver,matplo
| tlib,mccabe, mypy,mypy-extensions,numpy,pandas,path
| spec,pillow,pycodestyle,pyflakes,
| pyparsing,pyqt5,pyqt5-sip,python-
| dateutil,pytz,regex,six,toml,typed-ast, typing-
| extensions
| Tipewryter wrote:
| Here is how to reproduce it in a fresh docker container,
| so we have the same environment: docker
| run -it --rm debian:10-slim /bin/bash apt
| update apt install -y git python3-pip --no-
| install-recommends pip3 install
| git+https://github.com/bsdz/calcengine.git#master
|
| Result: "This package requires Rust >=1.41.0."
| bsdz wrote:
| Yes you're right! There was something wrong with poetry
| build system. When I initialised the poetry project
| roughly a year ago it placed an unpinned dependency (for
| the build part) for cryptography which recently switched
| to rust. I'm not sure why poetry hadn't vendorized that
| package or, at least, pinned the version. In any case, I
| updated the repo's pyproject.toml to use poetry's new
| build settings. Then, to check the install works in
| docker I switched to python's base images and did the
| following: docker run -it --rm
| python:latest /bin/bash curl -sSL
| https://raw.githubusercontent.com/python-
| poetry/poetry/master/get-poetry.py | python - cd
| /tmp git clone
| https://github.com/bsdz/calcengine.git cd
| calcengine python3 -mvenv --prompt calceng .venv
| . ./.venv/bin/activate pip install "cppy>=1.1.0"
| ~/.poetry/bin/poetry install -E demo
|
| Note that I had to force cppy to install using pip. At
| some point I'll refresh the poetry lock file to fix these
| glitches that seem to appear as an untouched project
| ages.
| bradleybuda wrote:
| This is cool! Also gives me a great excuse to post one of my
| favorite papers, "Build Systems a la Carte", about how make and
| Excel are solving the same problem in different ways:
| https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
| O_H_E wrote:
| > make and Excel are solving the same problem
|
| Well, that's a sentence I didn't expect to read today. Count me
| intrigued.
| [deleted]
| duckmysick wrote:
| Looks very interesting.
|
| Can we plug numpy into this? Or is it not necessary with the
| assumptions that most steps will be cached and don't have to be
| recalculated.
| bsdz wrote:
| Yes, you can use any existing python modules. In the demo
| you'll see I am using pandas dataframes in individual cells.
| Pandas is built on numpy.
| inshadows wrote:
| Nice Python + Qt demo. Gonna learn from it.
| bsdz wrote:
| Thank you. Hope there's something useful in there for you :-)
| dev_snd wrote:
| This is pretty cool.
|
| Spreadsheets are so powerful and really great and producing one-
| off solutions, but when you want to automate them you always need
| to redo most of the work in a "serious" programming language and
| wrap all of it in an API etc.
|
| I always wondered how one could bridge the gap between creating a
| PoC computation in a spreadsheet and getting the same code to run
| in production without the rewrite. This could be a nice first
| step for a python server application designed in a spreadsheet.
| arthurcolle wrote:
| There is a tool called wings iirc that lets you pull out
| DataFrames from an xlsx and vice versa
| quag wrote:
| It's called xlwings. Not a bad tool.
| https://docs.xlwings.org/en/stable/quickstart.html
| bsdz wrote:
| This is something I built a little while ago. It's a proof-of-
| concept and inspired by what I was doing in my day job.
|
| To play with the demo spreadsheet you'll need to clone the repo
| and run the demo script, ie: git clone
| git@github.com:bsdz/calcengine.git cd calcengine
| python3 -mvenv --prompt calceng .venv .
| ./.venv/bin/activate poetry install -E demo python
| demo/spreadsheet/main.py
|
| There's an inline gif showing the spreadsheet in use.
|
| Hope it's interesting :)
| phkahler wrote:
| Why QT and not tkinter?
| maleldil wrote:
| Why create a venv and then run poetry install? Poetry already
| manages virtual environments.
| doovd wrote:
| This is a core principle in a lot of the dependency-graph-based
| frameworks seen in banks (secdb, Athena, quartz) etc. Inspired
| by those?
| bsdz wrote:
| Indeed. Actually with-in the demo samples folder you can find
| a simple QuantLib example (quantlib.json). Although you'll
| need to install the QuantLib package separately as I didn't
| include it in the dependencies.
| wirthjason wrote:
| The Dag/graph is one of the best parts of those tool
| chains. That is one library I wish had a good open source
| equivalent. Partly because it would be nice to use in any
| project / environment, but also because there would be a
| lot more documentation and examples. Take Pandas for
| example, if it stayed internal then it wouldn't be the
| powerhouse that it is today.
|
| Most dependency graph examples are based around
| calculations but it's a versatile pattern and not limited
| to just math. I find the JS community has really taken hold
| of the dependency tracking and "reactive framework" idea to
| build UIs.
| timkpaine wrote:
| You may also be interested in: https://github.com/timkpaine
| /tributary/blob/main/docs/exampl...
| bsdz wrote:
| Yes I am. Thanks for sharing.
___________________________________________________________________
(page generated 2021-07-08 23:02 UTC)