[HN Gopher] Setup Anaconda, Jupyter, and Rust
___________________________________________________________________
Setup Anaconda, Jupyter, and Rust
Author : batterylow
Score : 65 points
Date : 2021-01-28 13:42 UTC (9 hours ago)
(HTM) web link (datacrayon.com)
(TXT) w3m dump (datacrayon.com)
| auxym wrote:
| Worth mentioning that commercial use of anaconda repositories
| (including via miniconda, to my understanding) is no longer free
| and requires a commercial license.
|
| https://www.anaconda.com/blog/anaconda-commercial-edition-fa...
|
| Beware if you are following this guide for your work.
| StreakyCobra wrote:
| > We clarified our definition of commercial usage in our Terms
| of Service in an update on Sept. 30, 2020. The new language
| states that use by individual hobbyists, students,
| universities, non-profit organizations, or businesses with less
| than 200 employees is allowed, and all other usage is
| considered commercial and thus requires a business relationship
| with Anaconda.
|
| Looks like small companies are exonerated from commercial
| license.
| batterylow wrote:
| > businesses with less than 200 employees is allowed
|
| This is great!
| bawana wrote:
| No businesses with >200 employees should be allowed to
| exist.
| anchpop wrote:
| why? seems like that would make any large scale business
| venture completely impossible and would cut us off from
| important economies of scale
| batterylow wrote:
| I don't know about that... I just meant that it's
| exceptionally reasonable
| [deleted]
| pratio wrote:
| Why Rust? Conda and Jupyter are python based. What does one hope
| to achieve with a setup like this?
| jamesmishra wrote:
| Rust is a great language for writing Python extensions, using a
| library like PyO3. Some of the most CPU-intensive parts of my
| company's feature engineering pipeline are now handled in Rust.
|
| I would enjoy writing Rust for Python even more if I could
| compile and run Rust straight from the same Jupyter notebooks
| that I prototype my Python code with.
|
| https://github.com/PyO3/pyo3
| pratio wrote:
| I wasn't aware of pyo3 thanks for that.
| RayDonnelly wrote:
| conda, although written in Python, is language (and somewhat
| OS) agnostic. We provide lots of C, C++, R and Rust packages as
| well as Python ones.
| 1MachineElf wrote:
| I was initially confused over the name conflict with the GUI
| Linux distro installer Anacoda:
| https://en.wikipedia.org/wiki/Anaconda_(installer)
|
| In the context of this post, Anaconda is a Python and R
| distribution using the conda package manager:
| https://en.wikipedia.org/wiki/Anaconda_(Python_distribution)
| whimsicalism wrote:
| The python and R distribution is much better known than the GUI
| Linux installer.
| ibotty wrote:
| Depends on the bubble you are in.
| aardvarkr wrote:
| The decision to shoehorn rust into a jupyter notebook is
| baffling, especially for something like data science. I like rust
| and all but come on. Just because it's possible doesn't mean you
| should.
| superbcarrot wrote:
| From the banner for "Data Analysis with Rust Notebooks" this
| looks someone realised that both data stuff and Rust are popular
| on their own and decided to put them together to sell an e-book.
| It's cool that this is at all possible but ultimately it forces
| Rust into a use case that it isn't well suited for. I can't see a
| good reason to pick it over Python/R/Julia.
| whimsicalism wrote:
| Speed, static types. Definitely not a general alternative.
| batterylow wrote:
| In the section before there is agreement with your comment:
|
| > Can we write and execute all our code in a Jupyter Notebook?
| Yes! Should we? Probably not. However, I enjoy the workflow,
| and making this an enjoyable process is important to me.
| kyllo wrote:
| Yeah I think Rust's place in this ecosystem is a C/C++
| replacement. All the Python and R data science packages call
| into native code to do the heavy lifting--the linear algebra,
| gradient descent etc. The next Tensorflow or PyTorch type
| framework could be implemented in Rust instead of C++. But it
| shouldn't matter to the end users--they would still use the
| Python or R bindings so they can have an interactive REPL
| environment to do their analysis in.
|
| Julia is interesting because it's performant enough to
| implement these things directly--Julia's machine learning
| frameworks are written in pure Julia (with the exception of the
| calls to CUDA libraries for GPU) and can achieve near-native
| performance, so solving the "two language problem" is part of
| its value proposition. I do wonder how valuable it is for the
| whole stack to be written in one language, and whether that
| will blur the lines between the software engineers who
| implement data science packages and the data scientists who use
| them.
| amkkma wrote:
| Julia can do GPU kernels (Intel, AMD, Nvidia) also, they just
| aren't all implemented in pure Julia (or tuned) yet, hence
| the CUDA lib calls.
|
| https://juliagpu.org/
| https://juliagpu.org/2020-09-28-gemmkernels/
| j-wags wrote:
| > Once Miniconda is installed, we need to create and configure
| our environment. If you added Miniconda to your PATH environment
| during the installation process, then you can run these commands
| directly from Terminal, Powershell, or CMD.
|
| AFAIK, the PATH modification route is no longer recommended.
| Instead one should use, eg `. miniconda3/etc/profile.d/conda.sh`
| to add a particular miniconda installaiton to the environment.
| kowlo wrote:
| It does say "if", but with MiniConda's CLI installer it's
| currently
|
| > Do you wish the installer to initialize Miniconda3 by running
| conda init [yes|no]:
|
| with the default as [no] as well
___________________________________________________________________
(page generated 2021-01-28 23:02 UTC)