[HN Gopher] Keras 3.0
___________________________________________________________________
Keras 3.0
Author : bratao
Score : 195 points
Date : 2023-11-28 14:53 UTC (8 hours ago)
(HTM) web link (keras.io)
(TXT) w3m dump (keras.io)
| eigenvalue wrote:
| Seems like they missed the boat and the whole world has moved on.
| Maybe if this came out 2 years ago it would be more relevant.
| pennaMan wrote:
| What is relevant now?
| arketyp wrote:
| PyTorch has the nicest debugging experience. JAX is supposed
| to be powerful too, but I'm skeptical of its static graph
| which is what made TensorFlow such a chore. I'm whishful
| about Mojo, in theory it sounds great.
| techwizrd wrote:
| On the contrary, that's why I find Keras 3 interesting
| (although I don't see myself switching anytime soon). It
| supports both PyTorch and JAX, and Keras Core models/layers
| can be used interchangeably with PyTorch modules:
| https://twitter.com/fchollet/status/1697020631752298982
| elforce002 wrote:
| I use keras for CV but for DL at work, we started moving
| towards pytorch and closely following modular.
| andy99 wrote:
| Competition is always good. That said, Keras is a tough sell. I
| think the ML framework market, particularly for gen AI is moving
| either away from abstraction to lower level frameworks (ggml or
| custom, etc) or consolidating around HF. It would be cool to see
| Keras emerge as a meaningful HF transformers competitor, but I'm
| not sure what it would take to get there. As it is, I don't see a
| big incentive to use it, or for people getting started to learn
| it (especially to the exclusion of the underlying frameworks)
| which is too bad.
| pantulis wrote:
| > consolidating around HF
|
| What is HF? Is it HF as in RLHF?
| kblissett wrote:
| Hugging Face: https://huggingface.co
| imachine1980_ wrote:
| I disagree because the majority of data science roles focus on
| optimizing production processes or customer
| acquisition/retention. In practical terms, while we often
| reference tools like OpenAI, the real-world application
| revolves around machine learning-based data analytics and
| unstructured business intelligence. Keras emerges as a valuable
| tool for businesses due to its combination of machine learning
| capabilities beaing easy,allow data experimentation, for the
| business knowledge. In this context, the emphasis on machine
| learning isn't as pronounced as it is in companies dedicated
| solely to ML, such as OpenAI or Google ML
| nabla9 wrote:
| I did not expect anything interesting, but this is actually cool.
|
| > A full implementation of the NumPy API. Not something "NumPy-
| like" -- just literally the NumPy API, with the same functions
| and the same arguments.
|
| I suppose it's like https://cupy.dev/
| m_ke wrote:
| If you like that you'll love this
| https://pytorch.org/blog/compiling-numpy-code/
| imjonse wrote:
| "Keras 3 is a full rewrite of Keras" and "It's also a battle-
| tested framework"
|
| In case of a full rewrite how can we talk about battle-tested? I
| can understand the API is, but not the implementation.
| kerasteam wrote:
| This means that the API, the abstractions, the workflows are
| battle-tested.
|
| The codebase itself went through 2 months of private beta and 5
| months of public beta. It is already used in production by
| several companies. It's not as battle tested as older
| frameworks, but it's fairly reliable.
| imjonse wrote:
| It also reads a bit like an official promotion for JAX (well-
| deserved btw) by mentioning its advantages separately and putting
| it first in the list of main frameworks.
| ipsum2 wrote:
| It's an open secret that TensorFlow is a deprecated framework
| internally at Google, and present/future model development
| should be done on JAX. Since Keras is a Google-owned product,
| it makes sense.
| jauntywundrkind wrote:
| I'm not in the field but as a techie this attempt to be a
| universal solvent, to play well with everyone, looks extremely
| smart & extremely sharp. This is a highly compelling sell for me,
| and greatly boosts the chance that when I do start to dabble,
| I'll look at Keras.
| andy99 wrote:
| See also https://github.com/unifyai/ivy which I have not tried
| but seems along the lines of what you are describing, working
| with all the major frameworks
| thornewolf wrote:
| I've been out of the ML game for a little while now (2+ years)
| but previously was under the impression everyone loved Keras.
| This thread says otherwise.
|
| The only mentioned alternative so far is `ggml`, which I have
| heard mentions of but am not familiar with. What are other
| alternatives & their corresponding pros and cons? Now, I am
| familiar with:
|
| - Tensorflow (complex API right?)
|
| - Keras (simple API?)
| andy99 wrote:
| Huggingface I would consider a high level framework, that uses
| pytorch as a backend (they may now be using rust/candle as
| well?)
|
| Mojo is another ML specific python dialect.
|
| I think most people including me associate Keras with being
| very simple and high level, which isn't really correct now
| based on the article.
|
| ggml is somewhat specialized, I was giving it as an example of
| the kind of framework I think we'll see more of, that is
| specialized, dependency free, written in a compiled language
| and directly optimized for speed. As use cases consolidate
| around specific architecture like LLMs, a faster and simpler
| low lever framework becomes preferable to a swiss army knife
| with abstraction upon abstraction like pytorch.
| uoaei wrote:
| Depends what kind of ML. If you were just spinning up models to
| shove data through, sure. If you wanted to craft a model to
| your particular needs, you will have a much better time with
| Torch, even 5 years ago when online tutorials were filled with
| TF/Keras.
|
| I think Keras more or less died when the TF 2.0 transition was
| botched so badly. Many moved to Torch and never looked back.
|
| FWIW I actually prefer JAX but my work is atypical compared to
| the usual "predict churn" or whatever application.
| the__alchemist wrote:
| My guess; perspective from a noob who's been using TF (Which I
| think absorbed Keras's API??) and OpenVINO: The Data
| Science/Python crew prefer PyTorch, and it's become self-
| perpetuating by WOM. So, people new are being recommended to
| use Torch, and it spreads that way.
| iskander wrote:
| Keras started out great but grew clumsily, making custom use-
| cases cumbersome and error prone and also became inextricably
| linked with TensorFlow (which was a big headache compared with
| PyTorch).
|
| It seems like this rewrite cleans up the meandering mess of
| Keras's middle period, or at least I'd be willing to give it a
| try.
| jasonjmcghee wrote:
| It's been a few years since I did Keras dev full time, but
| debugging was a nightmare. PyTorch- as long as you don't
| compile and you run on CPU, it's pretty much like debugging
| standard code. Debugging CUDA is harder.
|
| Edit: Seems like `run_eagerly` solves a lot of the pain of
| debugging Keras code.
| rychco wrote:
| I have only had positive experiences while using Keras in the
| past, but have not used it in a few years.
| FredPret wrote:
| What's the go-to ML framework nowadays?
|
| One day, I set off to create an ML project in Tensorflow; but I
| soon realized that getting the data in ready would be a huge
| challenge and also a massive value-add. It's been three years and
| I'm still working on the data!
| isoprophlex wrote:
| Pytorch itself, or abstractions built on top such as
| Huggingface
| kerasteam wrote:
| According to PyPI downloads and user surveys (like the yearly
| StackOverflow survey) the two main frameworks are TensorFlow
| and PyTorch for Deep Learning, and Scikit-Learn for classical
| ML.
|
| See: https://survey.stackoverflow.co/2023/#technology
|
| * TensorFlow: 9.53% usage among all devs
|
| * Scikit Learn: 9.43%
|
| * PyTorch: 8.75%
| gerash wrote:
| I have used both Keras and pytorch Lightning. They have a lot of
| similarities and I'm fine with both. That said the abstraction
| does not really cover the input processing part: ie. pytorch
| dataloader or tf.data part
| kerasteam wrote:
| Francois from the Keras team here -- happy to answer questions!
| daturkel wrote:
| Hey Francois, congrats to you and the team on the launch! I've
| generally chosen Pytorch over Tensorflow for my day to day, but
| now that Keras is framework agnostic I'm excited to revisit it.
|
| One thing I'm wondering about is if it's possible (or
| necessary?) to use Keras in concert with Pytorch Lightning. In
| some ways, Lightning evolved to be "Keras for Pytorch," so what
| is the path forward in a world where both exist as options for
| Pytorch users--do they interoperate or are they
| competitors/alternatives to each other?
| kerasteam wrote:
| Both Keras models/layers (with the PyTorch backend) and
| Lightning Modules are PyTorch Modules, so they should be able
| to interoperate with each other in a PyTorch workflow. We
| have not tried this with Lightning, but we've had a good
| experience with custom PyTorch Modules.
|
| More broadly, it's feasible to use Keras components with any
| framework built on PyTorch or JAX in the sense that it's
| always possible to write "adapter layers" that wrap a Keras
| layer and make it usable by another framework, or the other
| way around. We have folks doing this to use Flax components
| (from JAX) as Keras layers, and inversely, to use Keras
| layers as Flax Modules.
| minihat wrote:
| Any chance a keras.linalg suite of ops is in the works?
|
| Cross-platform differences between the behavior of tf.linalg
| and torch.linalg have cost me a lot of time over the years.
| kerasteam wrote:
| We don't have a separate `ops.linalg` package, but we do
| include `numpy.linalg` ops as part of `keras.ops`. For now
| only 2 ops are supported: `qr` and `solve`. We're open to
| adding any `numpy.linalg` op that turns out to be useful (or
| you could open a PR for any op you need).
| srvmshr wrote:
| Hi Francois, will there be breaking changes between 2.0 & 3.0
| API
| kerasteam wrote:
| All breaking changes are listed here:
| https://github.com/keras-team/keras/issues/18467
|
| You can use this migration guide to identify and fix each of
| these issues (and further, making your code run on JAX or
| PyTorch): https://keras.io/guides/migrating_to_keras_3/
| revskill wrote:
| You're my favorite Python developer. Very intuitive API design.
| dave_sullivan wrote:
| Congrats on the release!
| danaugrs wrote:
| Awesome! I'm a big fan of Keras and this is an exciting update.
| hedgehog wrote:
| This looks really cool, and it's good to see Keras come full
| circle back to being an ergonomic interface with support for
| multiple frameworks. I remember being pretty disappointed when
| pluggable back ends went away. I appreciate that in the NLP
| module there is the most humane and concise LORA implementation
| I'm aware of. The ability to embed Keras components into models
| using other frameworks is pretty key.
|
| Question: What's the model export story? If I want to export for
| CoreML or use the new PyTorch model compilation is there a
| straight forward way to do that?
| kerasteam wrote:
| Thanks! Hope you'll find the new Keras useful!
|
| So far the export story focuses on SavedModel and the services
| that consume that format, e.g. TFLite, TFjs and TFServing. You
| can just do `model.export(path)`, and you also have access to
| the `ExportArchive` class for fine-grained configuration.
|
| We have not tried CoreML export yet.
|
| PyTorch `.compile()` works with Keras models. It may not
| necessarily result in a speedup however.
| ofou wrote:
| How does it compare to Mojo?
| pineapple_sauce wrote:
| Mojo is a programming language, this is a library for Python
| that provides an abstraction over three Python-specific ML
| frameworks. Apples and oranges.
|
| You could use Keras inside of Mojo since Mojo is Python-
| compatible/embeds a Python interpreter.
| alok-g wrote:
| Love this.
|
| Am wondering how well this will work on Windows.
|
| Likewise for the pretrained models on my personal laptop (i7,
| 32GB RAM, RTX 2060 6GB).
| ipsum2 wrote:
| Keras is meant for training, not inference.
| wg0 wrote:
| ML Noob here. Went though the tutorial[0] and the readability of
| the API is impressive. Like composing a whole model layer by
| layer.
|
| Stupid question - can this also be used for composing transformer
| based LLMs?
|
| [0].
| https://keras.io/getting_started/intro_to_keras_for_engineer...
| kerasteam wrote:
| Yes, Keras can be used to build LLMs. In fact this is one of
| the main use cases.
|
| There are some tutorials about how to do it "from scratch",
| like this:
| https://keras.io/examples/nlp/neural_machine_translation_wit...
|
| Otherwise, if you want to reuse an existing LLM (or just see
| how a large one would be implemented in practice) you can check
| out the models from KerasNLP. For instance, this is BERT,
| basically just a stack of TransformerEncoders.
| https://github.com/keras-team/keras-nlp/blob/master/keras_nl...
___________________________________________________________________
(page generated 2023-11-28 23:01 UTC)