[HN Gopher] Spyder - a free and open source scientific environme...
___________________________________________________________________
Spyder - a free and open source scientific environment written in
Python
Author : jka
Score : 157 points
Date : 2021-02-05 19:30 UTC (3 hours ago)
(HTM) web link (www.spyder-ide.org)
(TXT) w3m dump (www.spyder-ide.org)
| zwaps wrote:
| The variable explorer is much better than the one PyCharm has,
| for example.
|
| Of course, Pycharm has many other tools and a much better
| interface (Spyders scaling is grgrgrgrgr).
|
| But for scientific data work? Spyder is great!
| pyromine wrote:
| Oh boy, I can't even get spyder to move between two of my
| monitors because it always complains about scaling and then I
| can't get the popup to consistently not show up
| ccordoba12 wrote:
| Would you prefer to hide it forever? We could add an option
| for that, but scaling could fail in some circumstances.
| dunefox wrote:
| Every link at the bottom is invalid (404)...
| ccordoba12 wrote:
| Fixed now, sorry for the inconvenience.
| zupreme wrote:
| I use the Anaconda-packaged version of Spyder on a daily basis.
| Great product, although I can never seem to get the IDE updates
| to consistently work within that environment.
|
| Nevertheless it is my favorite IDE for Python.
| ccordoba12 wrote:
| You could consider using our new installers, if you're on
| Windows or macOS, to avoid that: https://github.com/spyder-
| ide/spyder/releases/tag/v4.2.1
|
| Then you can connect them to your conda envs, as described
| here: http://docs.spyder-ide.org/current/faq.html#using-
| existing-e...
| zupreme wrote:
| Thanks!
| lytefm wrote:
| I'm not much of an Anaconda fan, but installing Spyder as a
| user package (pip install -U spyder) has always worked out well
| for me. After an upgrade, all I've ever needed to do was to
| upgrade to the spyder-kernels in my venv.
|
| Spyder is such a great tool and much more useful than Jupyter
| Notebooks for explorative DS work, imho. Thank a lot for
| maintaining it, Carlos. I don't mind the kite installer - but
| I've tried it out and found the completions pretty useless.
| nether wrote:
| Jupiter Notebook is Mathematica, Spyder is MATLAB.
| greatgoat420 wrote:
| I like spyder a lot for the variable explorer, which makes it
| feel more like Matlab. However the constant bothering about
| installing Kite got annoying. Unfortunately there isn't a way to
| use Kite and have it not constantly annoy you about upgrading to
| the enterprise edition. If Kite had a no ads plan for like ~ $10
| a year it would make a lot more sense to integrate it in spyder.
| generalizations wrote:
| Not to be trite, but there is a github repo and the license is
| MIT. I'd guess it's possible to find the relevant nag code
| (possibly in its own github commit), strip it out, and
| recompile.
|
| Edit. maybe this is what you're talking about?
| https://github.com/spyder-ide/spyder/commit/6b602af50f632cf6...
| [deleted]
| fock wrote:
| but why is that even there? I guess that's just another
| reason to stay with emacs...
| detaro wrote:
| Sponsoring/buying things in exchange for kite promotion
| seems to be their MO - there was quite a bit of noise about
| the more brazen instances of it a while back when the used
| existing editor extensions to push their product or
| telemetry code (see e.g.
| https://theoutline.com/post/1953/how-a-vc-funded-company-
| is-... and HN discussions about "kite")
| ccordoba12 wrote:
| ( _Spyder maintainer here_ ) Kite is completely optional for
| Spyder. If you don't like it, you can uninstall it and Spyder
| will continue working without any issues.
| kleiba wrote:
| Is there an option to not install Kite in the first place?
| buo wrote:
| I have installed Spyder both from its own repo, and using
| Anaconda, and I don't have Kite. I don't recall doing
| anything special during installation.
| ccordoba12 wrote:
| Thanks for the confirmation @buo. There's really nothing
| special about installing Spyder, i.e. we don't install
| Kite behind the scenes or anything like that.
| ccordoba12 wrote:
| I don't understand your question. Kite doesn't come pre-
| installed with Spyder. We only show a dialog once informing
| users that Spyder can use Kite to improve code completions
| in our editor.
|
| After that, if a completion can't be provided by Jedi but
| it could by Kite, we show another message, which can
| dismissed entirely.
|
| Both things don't force you to install Kite, but also show
| a button in case you want to do it within Spyder.
| greatgoat420 wrote:
| The issue isn't it installing Kite, it is the messages
| about installing Kite that bothered me.
| ccordoba12 wrote:
| As I said, it's a two times thing, but if it bothers you
| so much, we could a command line option to disable those
| messages completely. Would that be ok?
| lend000 wrote:
| Spyder is great. More minimalist than Pycharm, but it actually
| allows you to run multiple Python consoles in parallel, which is
| strangely absent from Pycharm.
| ehsankia wrote:
| I generally find it really hard to use full on IDEs like
| PyCharm for doing more investigative/research kinda work in
| Python, which is where Jupyter/Colab/Spyder are great at. It's
| just a very different flow, creating a large application versus
| doing analysis/research.
| alexchantavy wrote:
| I like to do a bit of a mix: write the library functions in
| Pycharm, then import and call them in Jupyter Lab. I used to
| try to do implementation too in Jupyter Lab but that got
| messy very quickly.
| jonnycomputer wrote:
| Pycharm Professional edition has a Scientific project type,
| which I was hoping someone has experience with.
| _coveredInBees wrote:
| Personally, I can't stand Jupyter for research because the
| introspection capabilities are so poor and limited compared
| to working in something like Spyder, Pycharm or VsCode.
| Moreover, I much prefer having interactive matplotlib plots
| for data exploration in a Qt/Gtk window rather than the
| inline matplotlib plots in Jupyter (though you can sometimes
| get it work to pop up windows for plots, but you still have a
| more interactive plotting experience in
| Pycharm/Spyder/VsCode)
|
| As someone who loves Spyder and used it a lot many years
| back, I've had no trouble doing all my scientific/algorithm
| prototyping in Pycharm while benefiting immensely from the
| astoundingly superior code intelligence/auto-complete/type-
| hinting in Pycharm. There is also a "Code Cell" plugin you
| can download for Pycharm that basically gets you to parity
| with Code-cells in Spyder/Matlab.
|
| For me, the excellent Python shell in Pycharm that supports
| multiline copy-paste, tab completion, and function-signature
| overlays makes prototyping algorithms and ideas much faster.
| That paired with the excellent history browser in Pycharm
| lets me easily grab relevant bits of code and promote them
| into a script or function and go from there. And then if you
| are trying to debug any algorithm, Pycharm is far superior on
| that front and I am far more efficient at debugging and
| fixing problems in Pycharm than in something like Vscode
| (though I do recall Spyder's debugger being pretty decent
| since it supports integration with IPython)
|
| There is a steep learning curve with Pycharm initially, but
| imo it works just fine (and for me at least, much better) in
| comparison to other things like Spyder/Jupyter.
|
| I would definitely recommend Spyder to most non-software
| engineer Python users, especially if they are in Data
| science. Nothing by <3 for Spyder's awesome data viewer
| widgets.
| lytefm wrote:
| > Personally, I can't stand Jupyter for research because
| the introspection capabilities are so poor and limited
| compared
|
| definetely agree. I prefer to work with code cells and
| prefer Spyder for being less bloated and more responsive
| than Pycharm, for having multiple iPython consoles and a
| nicer Plot window when doing DS work, though.
|
| To me, PyCharms only shines when refactoring and testing
| code to make it production ready.
| jonnycomputer wrote:
| In fact you can do this, at least in the Professional edition
| of Pycharm. How to do it isn't especially obvious though. On
| the left hand side of your python console with be a +. Click
| that and it will give you a new one.
| _coveredInBees wrote:
| I may be misunderstanding, but Pycharm has supported multiple
| Python consoles (and running multiple files) for a while now.
| tobmlt wrote:
| As someone who has used spyder everyday for years, it's really
| getting in my way lately.
|
| Important caveat: I've just recently upgraded from some really
| old versions to the latest spyder.
|
| Now it throws off my work flow by continuously tripping code
| hints or suggestions or whatever. When I try and click into the
| interpreter I find now I'd better click two or three times
| instead of one -- to subvert the type hint that is sure to push
| cursor focus back to the code. If I don't do that I am suddenly
| typing into the code window that which was intended for the
| interpreter. It sucks and I need to find out how to fix it.
| Surely there is something I can do in settings to chill the
| hinting/or-whatever out.
| lambda_obrien wrote:
| I've been having that issue with vscode; sometimes the tooltips
| get in the way and I have to click several times to get my
| cursor in the code. I think the tooltips should be click
| through.
| superbcarrot wrote:
| Can I get the following behavior from Spyder (last time I tried I
| couldn't find a way) - for some keyboard shortcut like Cmd/Ctrl +
| Enter:
|
| - the shortcut sends the selected code to the terminal (this is
| possible)
|
| - if I hit the shortcut when nothing is selected, the line where
| the cursor currently is gets sent to the terminal and the cursor
| moves to the next line of code
|
| - if the cursor is at the beginning of a block of code like an
| if-statement or a loop, the whole statement is sent to the
| terminal and the cursor moves to after the statement
|
| RStudio has this behaviour by default, I find it very convenient
| and I haven't been able to replicate it in other environments.
| navbaker wrote:
| You can define code cells in Spyder and execute them with a
| shortcut. I'm on an older version, but for me it's "# In[]" to
| start a code block, then shift-enter to execute. -Edit: to
| clarify, that string starts a code block, which continues until
| end of file or you designate another block.
|
| Executing the current line or the selected lines is just F9.
| omarhaneef wrote:
| As a sibling points out, I use F9.
|
| I prefer the control enter because I am used to enter to
| execute, but it is fine.
| kleiba wrote:
| _RStudio has this behaviour by default, I find it very
| convenient and I haven 't been able to replicate it in other
| environments._
|
| Certainly you can replicate that in Emacs?!
| superbcarrot wrote:
| I'm sure that it's possible but every time I've tried to set
| up Emacs I've given up fairly early in the process out of
| frustration.
| [deleted]
| dplavery92 wrote:
| That sort of workflow sounds a lot like what you would get from
| a notebook like Jupyter, albeit you would have explicit block
| of code ("cells") to be executed together.
|
| It looks like Spyder now features Jupyter Notebook integration
| as well: https://github.com/spyder-ide/spyder-notebook
| vsskanth wrote:
| I really like spyder for it's Matlab like IDE but I wish they
| handled virtual environments better. I also don't like that I
| have to install their kernel in the venv to use it.
| ccordoba12 wrote:
| Hey, thanks for the feedback! We plan to improve our support
| for virtual environments in the future. Also, it's really
| necessary to install spyder-kernels in every env to make the
| variable explorer and debugger work for it. However, that could
| be by us handling virtual envs and installing spyder-kernels on
| behalf of the users.
| vsskanth wrote:
| That would work too. Thank you
| gspr wrote:
| I'm sorry, I'm sure Spyder has lots of merits (and I'm sure the
| devs are wonderful people), but I've spent so much time fixing
| fellow academics' Python problems that were in the end rooted in
| "I don't know what I'm doing, so I installed this Spyder thing
| and it has now taken over every aspect of Python on my computer"
| that I feel a visceral negative reaction to the name in my body.
| Not Spyder's fault, of course, but nonetheless.
| lytefm wrote:
| Things can get very messy when installing Spyder via pip and
| then trying to upgrade stuff. Distutils tends to screw these
| updates up. I've nuked my local Python packages recently due to
| that mess and made a clean reinstall, but I guess that's not
| what you'd like to do if you have many Python packages other
| than Spyder installed.
| ccordoba12 wrote:
| Agreed, installing Spyder with pip can lead to a lot of
| headaches. That's why we don't recommend to use it and
| suggest Anaconda/conda instead. But that solution also became
| problematic during the last couple of years (conda is too
| slow or throws back a huge dependency errors).
|
| That's why we decided to create our own, self-contained
| installers for Windows and macOS: https://github.com/spyder-
| ide/spyder/releases/tag/v4.2.1
|
| Since they are completely isolated and don't come with pip,
| they can't be broken. And you can connect them to other
| Python environments by following these instructions:
| http://docs.spyder-ide.org/current/faq.html#using-
| existing-e...
| ccordoba12 wrote:
| ( _Spyder maintainer here_ ) I don't understand your comment.
| Spyder is like any other regular Python package, so it can't
| take over your computer. We have some customizations to run
| code, but that's to provide a better experience to users.
| ct0 wrote:
| I tried to use spyder on some projects, but the constant crashing
| without restoration of my work got old very quickly.
| ccordoba12 wrote:
| This was fixed in Spyder 4 by providing autosave functionality
| to our editor. You should give it a try again.
| lytefm wrote:
| I've been using Spyder for my whole DS master and rarely
| experiences crashes on Ubuntu, unless I've managed to go OOM.
| That being said, I've been using Spyder 4 via pip since the
| 4.0.0 release while Spyder 3 was still the default on conda.
| Autosave is definitely useful.
| civilized wrote:
| The last time I tried Spyder, I think around 2015-2016, it really
| struggled with large datasets. A lot of pausing and hanging that
| wouldn't happen at an IPython prompt. Is that better now?
| ccordoba12 wrote:
| Yes, we definitely improved on that front. Spyder should be
| more responsive now.
|
| You should give it a try with our new installers for Windows
| and macOS, which save you the trouble of installing Anaconda:
| https://github.com/spyder-ide/spyder/releases/tag/v4.2.1
| StreamBright wrote:
| Is this written in Python? What does it use for the display?
| ccordoba12 wrote:
| Yes, it's written in PyQt and we integrate with a lot of
| libraries in the Scientific Python ecosystem (Pandas, Sympy,
| IPython, Matplotlib, Scipy, Cython, etc).
| samuell wrote:
| Spyder is great. With all the complexity of python dev
| environments (virtual env managers, package managers and then the
| IDE), it is nice to at least be able to provide users with a well
| functioning self-contained IDE that just works out of the box.
| _coveredInBees wrote:
| I couldn't agree more. While I have moved on from Spyder onto
| Pycharm for many years now, Spyder will always have a soft spot
| in my heart for being a great first IDE that brought so many
| great quality-of-life features and replicated and surpassed the
| MATLAB experience of prototyping and developing code/algorithms
| in an interactive manner.
|
| Also, I know ccordoba is here on HN. If you read this, know
| that you have immense respect from me for tirelessly
| maintaining Spyder for all these years (I started using it in
| 2009)
| ccordoba12 wrote:
| Thanks a lot your kind words! I'm really glad you enjoyed
| using Spyder and have fond memories of it.
| VectorLock wrote:
| The link to the 'editor' page 404s. All the other stuff in here
| sounds pretty nice, but I'm curious how the editor experience is
| (or if I can use something like vim instead)
| lytefm wrote:
| A vim plugin exists, but it's very limited.
| ccordoba12 wrote:
| Right, we haven't had resources to invest on it, so it's been
| developed by the community.
|
| It'd be nice if we could use Neovim for text manipulations
| and show what comes back from it in our editor. If only days
| had 36 hours...
| tomrod wrote:
| It's okay. It's a lot like RStudio and Matlab. I consider a
| bridge between Notebook work and IDE work.
| ccordoba12 wrote:
| > The link to the 'editor' page 404s
|
| We just fixed this. Sorry for the inconvenience.
| jonnycomputer wrote:
| Has anybody used Pycharm's Scientific project type for data
| analysis?
| cheptsov wrote:
| Hey, a PM of PyCharm here.
|
| Justin case someone will find it useful and relevant. Our team
| is currently working on a brand new IDE (based on PyCharm) for
| data scientists.
|
| A big part of what we're doing is convenient support for
| Jupyter notebooks within the IDE. This new IDE is currently
| under development but already provides the support for
| notebooks similar to how the native web Jupyter notebooks work
| - cells and outputs under each cell. The IDE's functionality
| will include SQL, Jupyter, Python, and R support.
|
| If you'd like to try the early EAP builds of this new IDE with
| the new notebooks support, please fill out this short form:
| https://pages.jetbrains.com/pycharm-data-science-insiders
|
| Once you've confirmed your participation, you'll get a detailed
| email with instructions on how to download the early builds and
| details about how to share your feedback.
|
| Will be also happy to answer questions if any.
| porker wrote:
| As a daily Jetbrains user it sounds great!
|
| But - no Julia support?
| cheptsov wrote:
| Julia is on the radar. I'd love some time in the future to
| include it into scope.
| euler_angles wrote:
| Just submitted my application. I use PyCharm and Jupyter on a
| daily basis.
| cheptsov wrote:
| Cool, I see the request! Will send you all the info the
| next week already.
| jonnycomputer wrote:
| Interesting!
|
| Curious as to whether you'd say the approach you are taking
| is closer to Jupyter's binary cell approach or RStudio's
| Notebook's text-based approach.
|
| Thanks!
| cheptsov wrote:
| Our current approach to is very similar to the native
| Jupyter experience where notebook contains multiple
| separate cells. You can manipulate these cells with
| shortcuts or mouse. It looks very similar to Jupyter or
| JupyterLab. At the same time, if you open a Python script,
| you'll be able to use cell delimiters (aka `#%%`).
| lytefm wrote:
| Good to know, I'll always prefer working with cell
| delimiters like in Spyder over working with Jupyter
| Notebooks.
|
| I've also tried out PyCharm Professional, but the
| experience with cells and iPython was just worse for me
| than in Spyder - so I'm only using PyCharm for
| refactoring right now. I think the main problem was that
| errors/results from cmdstanpy would simply cause PyCharm
| to crash/hang instead of getting handled properly like in
| Spyder.
| ZeroCool2u wrote:
| As a professional data scientist and daily user of PyCharm
| Pro at work, super interested in this. Just signed up!
|
| For what it's worth, our team is generally not a huge fan of
| notebooks, but I get why people like them.
| cheptsov wrote:
| Thank you! Will send all the info within a couple of days.
| mslip wrote:
| Always enjoyed spyder for its variable explorer.
| x32n23nr wrote:
| The RStudio equivalent for Python. Last used this so many years
| ago, and from the screenshots little seems to have changed.
| RA_Fisher wrote:
| RStudio supports Python these days. I use it daily and it's
| really enjoyable.
|
| I think they ought to call it DataStudio bc it's multilanguage.
|
| https://rstudio.com/solutions/r-and-python/
| RA_Fisher wrote:
| This is a better link: https://rstudio.github.io/reticulate/
| munk-a wrote:
| Please use carefully if you have aracnopyhopia.
|
| The naming seems pretty poor for me given that I'd assume that
| spyder would be a webcrawler written in python.
| ian-g wrote:
| I understand why Spyder. But they could also have called it
| spIDEr
| omarhaneef wrote:
| pIDE PYper (just thought of that!)
| ccordoba12 wrote:
| Haha, the original project Spyder is based on was called
| Pydee
| bluesmoon wrote:
| It often causes a double-take for me when I see projects on HN
| with the same name as projects I once worked on.
|
| Way back in time, maybe around 1998, a friend an I worked on a
| steganography tool that we called Spyder. I don't remember where
| it was published or under what license (probably freeware), but
| the algorithm was incorporated into Hide4PGP (http://www.heinz-
| repp.onlinehome.de/Hide4PGP.htm), and we stopped work on it.
|
| This post just caused a bit of nostalgia. I can still remember my
| co-developer taking a break jamming on a bass guitar while we
| tried to figure out the striping problem (the solution was to
| compress the data to increase entropy first).
| myhf wrote:
| There's also a recent video game named Spyder
|
| https://www.spyderthegame.com/
| bfjckgkgnt wrote:
| If you use this, you are a junior dev - embarrassing
| rscho wrote:
| If you say things like this, you don't belong on HN -
| embarrassing
| dokem wrote:
| This is more for using python as a science calculator, not for
| writing software.
| ajford wrote:
| If you judge someone by the tools they use, you are a junior
| dev. Embarrassing.
|
| Stop and consider that there are different tools for different
| purposes. Exactly the same way Word, Publisher, Photoshop, and
| Sublime or Vim are different from each other, so are things
| like PyCharm, VS Code, Jupyter, and Spyder.
|
| A "full" IDE like PyCharm is designed for application
| development, including deep introspection of your code,
| deployment automation, tool integration for things like
| preprocessors (such as sass/less), and integrated support for
| VMs and Docker. This is overkill if you're writing a simple
| script, but helps for larger projects.
|
| Something like VS Code is closer to PyCharm, but might not be
| as deeply integrated for any given language. It does provide a
| more consistent environment if you tend to work in multiple
| languages, which can help you avoid some of the context
| switching between langs.
|
| Jupyter and Spyder serve a different purpose. They aren't the
| kind of thing you'd (usually) write a full application in, but
| it's exactly the kind of thing that would be useful to data
| scientist, or someone trying to explore a library. The re-
| executable cells in Jupyter are quite useful for quickly
| iterating on data cleaning code, since you can re-write and
| rerun a series of cells without rerunning expensive
| calculations or data loading.
|
| I haven't used spyder in years (2008/9?), but it was a great
| way to put my fledgling python skills to use back then. I was
| performing a lot of data analysis on radio astro data in
| Matlab, but we only had so many 'roaming' licenses for our lab.
| Using numpy and spyder allowed me to practice my python and get
| my data analysis done outside of the lab.
|
| So consider the fact that not all tools are suitable for the
| same thing, and that a 'senior' dev considers what tool is
| right for the job at hand.
| approxim8ion wrote:
| Some of us drop out of the womb as senior devs and tech leads I
| guess.
| mmmBacon wrote:
| Guess I'm junior then. I learned how to code via scientific
| coding so I love Spyder for its simplicity. When I want to work
| deeply on a difficult problem and want to quickly flesh out
| ideas it's my go to because it sandboxes really well and I can
| iterate much faster in it.
|
| I use more advanced IDE when integrating the code I developed
| in Spyder into a larger code base. I'm sure many will criticize
| my workflow but I've tried doing other ways and this works best
| for me and I write less buggy code this way.
| ajford wrote:
| Ignore anyone criticizing your workflow. Tools like Spyder
| and Jupyter can be great when working with datasets in a way
| that IDEs aren't quite suited for.
|
| Variable explorers and integrated charting/plotting can be
| fantastic for visualizing and understanding your data.
|
| Cells in Jupyter are great for quickly iterating on data
| cleaning code.
|
| When I was part of a scientific applications team at a past
| employer (state agency), we used Jupyter heavily while
| developing the core code for the data processing pipelines.
|
| We were reading and processing data from instruments based on
| (bad) docs and notes from a former employee who reverse-
| engineered the format, so there was a lot of edge-case
| finding and testing. The notebooks fleshed out with comments
| and notes in Markdown upon completion and were saved within
| our repo as interactive documentation, demonstrating the more
| complicated processing steps in a far more approachable
| fashion.
| andi999 wrote:
| Which advanced IDE? Eclipse?
| ssully wrote:
| This comment is embarrassing. The tools a person uses is not an
| indicator of their skill level and thinking so is only holding
| you back, not the person using them.
| kleiba wrote:
| Move right along, folks, don't feed the troll.
| mslip wrote:
| What wisdom! Thank you senior dev person
| munk-a wrote:
| Aww snap - guess I'll go back to programming everything in nano
| then.
| elefanten wrote:
| Sounds like a junior dev attitude.
| refactor_master wrote:
| Eh, Streamlit and PyCharm is a much more powerful combination
| IMO. I can't quite put my finger on it, but something about
| variable explorer always felt like training wheels to me.
| dr_kiszonka wrote:
| PyCharm also have a variable explorer; one which I prefer over
| Spyder's :)
|
| >Eh, Streamlit and PyCharm is a much more powerful combination
| IMO.
|
| What is you PyCharm + Streamlit use case and workflow? Just
| curious because I have just learned about Streamlit from your
| comment.
| greatgoat420 wrote:
| Because it is super manly to guess at what the 20th element of
| an eigenvector is? Or you like super long printouts of data
| where it is hard to figure out what the index is? Matlab has
| one for a reason, and it is because very often you want a quick
| look at your often gigantic data.
| ccordoba12 wrote:
| I respectfully disagree. The variable explorer allows to check
| what variables you have declared and how they change with
| changes in your code.
|
| This is critical for scientific computing because you're
| constantly interacting with data (i.e. reading, cleaning,
| plotting and analyzing it), so having a dedicated pane for it
| is a great advantage.
| auxym wrote:
| The variable explorer comes from Matlab I think. It is handy to
| quickly and interactively check out what is going on in a
| matrix or ndarray, especially with the color coding.
___________________________________________________________________
(page generated 2021-02-05 23:00 UTC)