[HN Gopher] Ruby in Jupyter Notebook
___________________________________________________________________
Ruby in Jupyter Notebook
Author : Alifatisk
Score : 166 points
Date : 2024-08-25 10:28 UTC (2 days ago)
(HTM) web link (nbviewer.org)
(TXT) w3m dump (nbviewer.org)
| DagsEoress wrote:
| Is any of this usable on hosted GPU notebooks (e.g. paperspace)?
|
| I'm currently diving into Machine Learning using Python + Scikit-
| learn, and I'd love to one day replace Python with Ruby. But
| looking at the current ML ecosystem I don't see that happening.
| Does anyone have experience building (Supervised / Unsupervised)
| models using something other than Python (including deployment)?
| qsort wrote:
| Nbviewer is just a service that lets you host notebooks, there
| is no computation there.
|
| > Does anyone have experience building (Supervised /
| Unsupervised) models using something other than Python
| (including deployment)?
|
| XGBoost/LightGBM have a C API and can be used from pretty much
| anything, deployment is not a problem. Practically building
| models is more about dealing with data, the ecosystem tends to
| revolve around Python and R for that reason.
| thibaut_barrere wrote:
| Since you wrote "using something other than Python" and not
| necessarily only Ruby, definitely look into Livebook and
| Elixir, and the whole ecosystem around it, including:
|
| - https://github.com/elixir-nx/axon Nx-powered Neural Networks
|
| - https://github.com/elixir-nx/nx Multi-dimensional arrays
| (tensors) and numerical definitions for Elixir
|
| - https://github.com/elixir-nx/scholar Traditional machine
| learning on top of Nx
|
| - https://github.com/elixir-nx/bumblebee Pre-trained Neural
| Network models in Axon (+ Models integration)
|
| - https://github.com/elixir-explorer/explorer Series (one-
| dimensional) and dataframes (two-dimensional) for fast and
| elegant data exploration in Elixir
|
| - https://fly.io/blog/rethinking-serverless-with-flame/ (for
| offloading large work to remote containers)
|
| - https://www.youtube.com/watch?v=RABXu7zqnT0 InstructorEx
|
| And of course Livebook (https://livebook.dev)
|
| Old but interesting video
| https://www.youtube.com/watch?v=g3oyh3g1AtQ (Bumblebee: GPT2,
| Stable Diffusion, and more in Elixir)
|
| A talk on the upcoming Elixir conference
| (https://2024.elixirconf.com/schedule/#schedules) is actually
| titled "Livebook in the cloud: GPUs and clustered workflows in
| seconds".
| DagsEoress wrote:
| Very interesting, thanks for sharing! I didn't know Elixir
| was so invested in Machine Learning. I have a background in
| Erlang so I'll definitely dive in. Do you know of any
| individuals / companies that have had success with Elixir+ML
| in production?
| thibaut_barrere wrote:
| You welcome. Afaik it has been a long-time bet done by Jose
| Valim (former Rubyist & author of e.g. Devise) and others,
| this is not something that is going away anytime.
|
| Each library has been building on top of the previous
| libraries & abstractions (including transpiling Elixir
| instructions into GPU code, see "defn" etc).
|
| Since you mention Erlang, there is even a Machine Learning
| Working Group at https://erlef.org/wg/machine-learning.
|
| The most iconic & advertised case I'm aware of is the work
| done at Amplified https://www.amplified.ai ; this has been
| the topic of a Keynote at ElixirConf EU this year, which
| you can find at
| https://www.youtube.com/watch?v=5FlZHkc4Mq4.
|
| I am also starting to use ML + Elixir in production and I'm
| aware of other individuals doing so.
|
| I do not have a registry of companies doing so, but we're
| seeing more and more experienced ML practitioners
| mentioning they are coming from Python and willing to try
| something different (e.g. https://elixirforum.com/t/data-
| science-and-machine-learning-... and other posts on Elixir
| Forum).
|
| Hope this helps!
| nextos wrote:
| I had looked into Elixir/Erlang + ML before, and I keep
| revisiting it for Python alternatives.
|
| How developed is Elixir/Erlang in this area? What are the
| key advantages?
| michaelmior wrote:
| It's certainly possible, but the provider would need to add
| support for the kernel. That is, I'm pretty sure you can't
| install a new kernel in these environments yourself.
| xipho wrote:
| Perhaps not quite what you're looking for but Ankane does (a
| lot) of great work, e.g. https://github.com/ankane/torch.rb.
| crabbone wrote:
| R and Julia have had integration with Jupyter for a long time.
|
| Even so, trying to avoid Python in the world of Jupyter will
| put you in a very tough spot. In general, doesn't matter how
| much you dislike it, there's no real way around it. You'll have
| to face it in some capacity whether you like it or not.
| isodev wrote:
| I love the idea. It's similar to what the Elixir folks have been
| working on with Livebook https://livebook.dev which seems
| somewhat more refined on the UI side + the benefits of
| distributed erlang/elixir (e.g. a livebook can interface with a
| live system and interact with the remote application/gpu etc).
| michaelmior wrote:
| Jupyter notebooks have been around for almost a decade now. The
| interesting piece here is adding Ruby as a language that can be
| used instead of the original Python. (There have been many
| other languages integrated with Jupyter as well.)
| lucasoshiro wrote:
| Jupyter means "Julia, Python, R", the three original
| languages that it was created for. Nowadays you can add
| support for many languages by using other kernels.
|
| If you want to see the available kernels for each language
| you can check it here:
| https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
| dpflan wrote:
| Ruby fits in here nicely then...
| fifilura wrote:
| JuPytR aaaah!
|
| TIL!
| michaelmior wrote:
| Didn't know that. I always thought of Jupyter as originally
| a Python thing. Thanks for sharing that :)
| jcheng wrote:
| Jupyter is a rebrand of iPython, so you were right the
| first time!
| physicsguy wrote:
| It wasn't 'created' for the three languages at all.
|
| Originally the whole project was 'IPython' and then
| 'IPython Notebook'. It came out of Fernando Perez's attempt
| to make interactive Python a better experience.
|
| They renamed it Jupyter to reflect that it was no longer a
| pure Python project as kernels were written in even before
| it rebranded as Jupyter. Those three were the most popular
| at the time but by no means the only ones.
|
| I used to work with one of the core team about the time
| they changed this!
| lucasoshiro wrote:
| Thanks for the correction!
| pbowyer wrote:
| The Livebook UI is lovely. I'd like to see Jupyter get some
| love in this area.
| nichochar wrote:
| It heavily inspired us for https://srcbook.com (a TypeScript
| notebook)
| BaculumMeumEst wrote:
| Does anyone have a way to use generative AI to generate Jupyter
| notebooks? I've tried with prompts but it chokes on the markup,
| and also I'm wondering if the markup wastes too much context for
| that to be a good idea anyways. Right now I just use Cursor or
| Claude, copy replies, and chop them up into blocks manually.
| prasoonds wrote:
| We've made an open source fork of Jupyter - kind of like Cursor
| but for Jupyter.
|
| See GH: https://github.com/pretzelai/pretzelai/
|
| You can install it with _pip install pretzelai_ (in a new
| environment preferably) - then run it with _pretzel lab_. You
| can bring your own keys or use the default free (for now) AI
| server.
|
| We also have a hosted version to make it easy to try it out:
| https://pretzelai.app
|
| Would love to get your feedback!
| BaculumMeumEst wrote:
| Can it reliably generate interspersed blocks of markup and
| code with a single prompt?
| prasoonds wrote:
| Hmm do you mean you want to create multiple cells from a
| single prompt - some code cells, then some markdown cells,
| then some code cells and so on?
|
| The sidebar can certainly produce code mixed with markdown
| but right now, we process the markdown and show visually.
|
| https://imgur.com/a/bpYu8yN
|
| The cell level Cmd + K shortcut only works on a given cell
| to create or edit code and fix errors. Just tested it and
| it generates markdown well (just start your prompt with
| "this is a markdown cell")
|
| https://imgur.com/VuDciQN
|
| In the sidebar/chat window, it should be trivial to not
| parse the markdown and just show it raw. I'll work on it.
| In the main notebook, it's a bit harder but we are planning
| to allow multi-cell insertions but it will probably take
| 2-3 weeks.
| BaculumMeumEst wrote:
| Yeah the golden goose for me personally is the ability to
| say "create a jupyter notebook about x topic" and have an
| LLM spit out interspersed markdown (w/ inline latex) and
| python cells. It would be really cool if the LLM was good
| at segmenting those chunks and drawing stuff/evaluating
| output at interesting points. Quick example to illustrate
| the idea:
|
| https://imgur.com/04FUp9s
|
| I find Cursor to be extremely good right up to that point
| - I can work with Jupyter via the VS code extension and
| quickly get mixed markdown like how you're describing now
| - but it cannot do the multi-cell output or intelligent
| segmenting described above. I currently split it apart
| myself from the big 'ol block of markdown output.
| prasoonds wrote:
| I see, interesting. Hadn't come upon this use-case before
| but makes sense.
|
| I've made a GitHub issue for this feature:
| https://github.com/pretzelai/pretzelai/issues/142
|
| If you'd like to be updated when we have this feature in,
| please leave a comment on the issue. Alternatively, my
| email is in my bio - feel free to email me so that when
| we have this feature, we can send you an update!
| marc-fabihq wrote:
| This is something we've experimented with and I know some
| other tools out there claim to do this, I've just found
| that there's a very simple issue with this: if the AI
| gets any step wrong, every subsequent step is wrong and
| then you have to review every bit of code/markdown bit by
| bit, and it ends up turning into more work than just
| doing the analysis step by step while guiding the AI. I'm
| optimistic that this will change over time as the AI gets
| better, but it's still quite fragile (although it demos
| really well...)
| BaculumMeumEst wrote:
| So if you had 3 markdown cells and 3 python cells, I
| would design the tool to pull all the content out of
| those cells and present it (sans all that ipynb markup,
| just contents, probably in markdown) to the model as the
| full context for every edit you want to make. So the tool
| would need to know how to transform a given notebook into
| a collection of markdown/python cells which it would
| present to the model to make edits. The model would need
| to return updated cells in the same format, and the tool
| would update the cells in the document (or just replace
| them entire with new cells from the response). I would be
| fine with this just blowing away all previous evaluation
| results.
|
| Do you think that approach would work? Not sure if I'm
| misunderstanding the issue you're describing and I
| recognize it is likely much messier than I imagine.
| engine_y wrote:
| Sorry for the negativity here, but why?
|
| Why would there be a NEED for someone to use anything else than
| python in the context of a notebook?
|
| My assumption is that anyone who's good enough to develop in
| Ruby/Elixir/Scala would find it very easy to write small snippets
| of python. Why go thru the hassle of another layer?
|
| It reminds me of Scala.js - which transpiles Scala into JS.
| Doesn't make sense to me but happy to hear otherwise from the
| community.
| ironmagma wrote:
| I was talking with a coworker recently who was asking the
| question of why Ruby wasn't the data science language of
| choice. I am still not sure the answer. Why not? Ruby seems
| like it would be great for math.
| engine_y wrote:
| Well maybe it should be. I was thinking the same about
| languages which are better suited for data wrangling. But to
| me, the reason python is the DS language shows how good
| evolution is and survival of the fittest. You and I would
| probably argue that a DS language should have strong static
| typing but in nature what prevailed was the opposite. A
| nimble, easy, no fuss, comedy oriented named language.
| short_sells_poo wrote:
| From my experience, python won out the research crowd
| exactly for the reasons you mentioned, but also because it
| is a bit of a swiss army knife platform. Python has plenty
| of libraries for doing pretty much anything these days.
| Computer Vision? Can do. ML? Can do. Dashboards? Can do.
| And the list goes on. This means that a researcher can do
| their research in python and then also do the
| implementation in the same language and platform they are
| familiar with.
|
| If someone told me to use rust for doing any sort of
| interactive/iterative numerical research, I'd send them for
| the hills. And I love Rust. But when I want to quickly
| learn about the properties of some data, or try to use
| dataset A to predict dataset B, I don't give a rats ass
| about types, or type safety, or proper programming idioms.
| I just want to look at my data.
|
| Then, if I convince myself that I have some sort of model
| that I want to use, I could theoretically reach for Rust,
| but generally I don't need to. I can just take the output
| of the research - more or less verbatim - and start using
| it in production. Often with minimal changes.
|
| Yes, sometimes circumstances require to use something more
| performant, but these days you also have Numba or Nuitka to
| optimize some hot loops.
|
| For the longest time I wanted Julia to become a success,
| because I felt that it combined in one language and
| platform the possibility to write easy no-fuss, scalable,
| performant and robust code. A language which could evolve a
| codebase from initial research analysis to strongly typed
| production code. Unfortunately it never seems to have
| gained traction, so we are left with the next best thing:
| python. It's not really good at anything (as a language),
| but it's the least bad out of everything out there and the
| libraries are excellent.
| skagenpilot wrote:
| Python was the first programming language whose primary focus
| was not scientific applications, i.e. not MATLAB,Fortran or
| R, to have support for numeric arrays (matrices) via ndarray
| and then Numpy, while still being fairly approachable for the
| scientific community. If Travis Oliphant had picked Ruby for
| implementing ndarray, maybe would be the defacto language for
| data science.
| frou_dh wrote:
| Zooming out, this is the case in a lot of the programming world
| in general. Massive, _massive_ duplication of effort so that
| each language community can do the same stuff as everyone else,
| but in their preferred language. There 's what, 20-30 different
| general-purpose languages out there with a sizeable library
| ecosystem?
|
| Now don't get me wrong, I love programming languages. It's all
| fun and interesting. But it's a bit absurd too.
| rgoulter wrote:
| Jupyter Notebooks can be useful for more than small snippets of
| code.
|
| A Ruby Jupyter kernel would be useful if you've got a Ruby
| codebase, and want to run a Notebook using some of that code.
| vidarh wrote:
| My only USE for a notebook would be to use with my Ruby code. I
| have zero use for Jupyter with Python.
| fifilura wrote:
| Jupyter is a decent REPL.
|
| I use it when I want to try a new language. Or when solving
| adventofcode.
|
| If I had the desire to learn Ruby, i would probably start by
| downloading a docker image of jupyter with a Ruby kernel.
| Kalanos wrote:
| i thought there already was a ruby kernel.
|
| are you the original developer or a new maintainer of iruby?
| nomilk wrote:
| _Rub_ yter notebook =)
| joshdavham wrote:
| This looks great! I'm already running Node as well as python in
| Jupyter notebooks so to see more languages being added is always
| great
| nichochar wrote:
| Do you like using Node in Jupyter? Are there any downsides /
| problems that you find with it?
|
| We're trying to build a TypeScript notebook and I'm very
| interested in what people's current tooling for this looks like
| today.
| joshdavham wrote:
| I like being able to use Node in Jupyter, but I can't say
| that how I'm using it feels as natural as it does with
| Python. I'm way stronger at python than node so it was pretty
| hard to get set up with it.
|
| There's also probably a way to do it, but I haven't figured
| out how to use ESModule imports in Jupyter, so that's been a
| bit annoying also. It also hasn't crossed my mind until
| literally now, that one could write typescript in jupyter
| notebooks given that typescript needs to be compiled.
|
| Do you have a link to your current work? I think I'd be
| interested
| nichochar wrote:
| Imports is the thing that makes it useless. You cannot
| import stuff normally in the kernel..
|
| Yes, my current project is
| https://github.com/srcbookdev/srcbook
| quackware wrote:
| Deno has support for Jupyter notebooks which is quite nice.
|
| https://docs.deno.com/runtime/manual/tools/jupyter/
| DwnVoteHoneyPot wrote:
| This is a dead project. Everything in the SciRuby/sciruby-
| notebooks Github repository hasn't been updated in 8,9 years.
| bunderbunder wrote:
| This page is talking about a project called iRuby, which I
| assume is unrelated or successor. The repository this page
| links had a 0.8.0 release exactly one month ago.
|
| The same org does also have a sciruby project that hasn't been
| touched in 8 or 9 years like you say.
| keithxm23 wrote:
| While the example-notebooks repo you've mentioned is not
| actively maintained it looks like the IRuby kernel used by
| Jupyter still is maintained: https://github.com/SciRuby/iruby
| bmacho wrote:
| Needs (2016)
| vidarh wrote:
| This renders better for me straight from Github than it does on
| nbviewer.org:
|
| https://github.com/SciRuby/sciruby-notebooks/blob/master/get...
| turriblegrapes wrote:
| nbviewer has nothing to do with ruby specifically, this example
| links to a notebook using existing iruby kernel
|
| > nbviewer is an open source project under the larger Project
| Jupyter initiative > along with other projects like Jupyter
| Notebook, JupyterLab, and JupyterHub.
|
| https://nbviewer.org/faq#what-is-nbviewer
| kinduff wrote:
| There is also jupyter_on_rails [0] which integrates both. Using
| it feels so good, I love how an app can suddently become a
| playground or a sandbox.
|
| [0]: https://github.com/Yuki-Inoue/jupyter_on_rails
| Alifatisk wrote:
| That's so bizzare yet cool!
___________________________________________________________________
(page generated 2024-08-27 23:01 UTC)