[HN Gopher] Validating Data in Elixir: Using Ecto and NimbleOptions
___________________________________________________________________
Validating Data in Elixir: Using Ecto and NimbleOptions
Author : amalinovic
Score : 101 points
Date : 2023-11-07 10:57 UTC (12 hours ago)
(HTM) web link (blog.appsignal.com)
(TXT) w3m dump (blog.appsignal.com)
| CodeCompost wrote:
| The only place I hear about Elixir is here on HN.
| Whiteshadow12 wrote:
| It's talked about a bit on Twitter with the hashtag
| #MyElixirStatus. Discord makes use of it as well. Same with
| Supabase.
|
| The problem is the languages one would hear about, fall into
| the OOP category, functional languages tend to not be as
| fashionable when you do get into one, you become almost a
| zealot.
|
| It's hard to fight the imprinting a language has with its first
| fans, Elixir because of its Erlang origins tends to have the
| identity of telephonic-based use cases. Similar to how Rust
| gets its reputation as a specialist language (cli,kernal, etc)
| even though you could use it for web apps as well.
|
| The last reason is, it's really hard to fight the gravity pool
| of NextJS and React.
|
| Personally my circle and company and country are solidly on the
| side of C#/Java/PHP/Python and React/Angular, the average
| developer who fits into that grouping might not have even heard
| of Svelt or SolidJs when they do hear about it, they
| immediately make some snide comment about JS fatigue, even
| though that hasn't been a thing for a while.
| freedomben wrote:
| > _it 's really hard to fight the gravity pool of NextJS and
| React._
|
| Indeed. I love Elixir and I even struggle with choice when I
| don't need much or any server-side functionality.
|
| There was a Jekyll-style project for Elixir some years ago
| but it fell into bitrot and deprecation. I'm still hoping
| somebody resurrects it or builds an equivalent, because using
| eex to build a static site is a really great experience.
| jkhdigital wrote:
| Back in 2018 I took a graduate level distributed systems course
| from a professor who spent most of his lecture time rambling on
| about how we'd be better off reading HN than our textbook.
| There were no exams, only four projects which involved
| implementing various classic distributed algorithms (gossip,
| chord, etc.) in Elixir.
|
| For what it's worth, Elixir is now my preferred language for
| personal projects, although I still haven't had a chance to use
| it professionally.
| dasil003 wrote:
| Interesting, are the syllabus + assignments published online
| somewhere?
| lycos wrote:
| Wish it was different but realistically I feel like you are
| right (outside of Elixir specific communities).
|
| I love working with Elixir and have done so successfully for
| the past 4 years at various companies, however I have been
| trying to find a new client for a while now and I'm getting
| close to the point of looking for other types of work because
| things are looking bleak. From what I can tell there are also
| way more people looking for Elixir work than available jobs,
| unfortunately.
|
| In my 15+ years of software development in various languages
| Elixir has been the most enjoyable developer experience for me.
| So I guess I'll just have to think of a side project where I
| can apply it and do less fun stuff to earn bread.
| freedomben wrote:
| Yeah it's tough right now. One of the saddest things for me
| is the vicious cycle of "we can't use elixir because nobody
| uses elixir, so we can't use elixir because nobody uses
| elixir" ad infinitum. It's such a wonderful and productive
| stack, but people are just insanely comfortable with their
| typescript (or laravel, or rails, etc).
| sph wrote:
| Sounds like to work in a boring monoculture, then. Thank pg you
| have Hacker News to keep you up to date with the rest of the
| programming world.
| ecmascript wrote:
| The main reason is probably because Elixir USP is the stability
| of the apps written in it and how easy it is to build
| distributable software with it imo. You have to be an
| experienced engineer in order to appreciate those two things
| and have specific problems which most don't have.
|
| However I think with Phoenix Liveview Elixir gives regular
| developers a productivity boost that could be it's own USP.
|
| I have only written a small elixir app but I was mind blown
| over Liveview and still is convinced that most people would be
| better of using it over a SPA.
| thibaut_barrere wrote:
| It's a very good point. I consider it is a language of
| "connoisseurs" at the moment. I've been exchanging with
| computer science schools here in France to see how to increase
| adoption.
|
| Students, presented with the capabilities of the language
| (scripting / machine learning / scalable web apps) are quite
| interested actually!
| karmajunkie wrote:
| there's a pretty well established community on mastodon as well
| as xitter. not sure where you hang out online obvs but it's not
| exactly niche anymore, maybe branch out a bit?
| rozap wrote:
| It doesn't have a major corporate sponsor, which is probably
| why. It's a very active community, though.
| ch4s3 wrote:
| For what it's worth there are at any one time 50-75 people
| writing Elixir at my company, and we have projects ranging in
| age from 6 years to some things started this year.
| mjhay wrote:
| I tried out Elixir a couple months back, but had no end of
| problems getting the tooling set up in VSCode.
| selykg wrote:
| My experience, admittedly is not one where I am a full time
| developer just a hobbyist and sort of dev adjacent for work, is
| that getting _any_ tooling setup in VSCode is a path with no
| end of problems. So, at least when someone says that type of
| thing here I go "ah, so the norm." I think saying things like
| that is more an indictment against VSCode.
|
| Don't get me wrong, I think getting vim/neovim setup to do the
| same types of things is possibly worse, but I haven't really
| had much of a better experience in VSCode either.
| matthewsinclair wrote:
| I have to say that my experience could not have been more
| different.
|
| So far, everything I've done within the Elixir ecosystem has
| "just worked" (with the exception of some stuff with
| supervisors, the config and intuition for which are currently
| eluding me).
|
| For example, I managed to code a full image processing pipeline
| (classification, labelling, and description) using Elixir, Nx,
| Bumbkebee, etc on the train on the way into work the other day.
| Which is about a 35 min trip.
|
| If we're talking about tooling, the last time I tried to just
| print "Hello, world!" in either Python or JavaScript, it took
| me down a hell dungeon of configuration and dependency
| nightmares. To be fair to both, tho, that was some time ago so
| I can only hope that situation has cleared up more recently.
| mjhay wrote:
| I was mainly interested in working with Nx and friends, which
| look pretty nice. Good to hear about your experience with
| that.
|
| The ElixirLS server always crashed for me on init. The
| problem is assuredly on my end, but I was never able to fix
| it.
|
| https://elixirforum.com/t/elixirls-not-working-runtime-
| termi...
| atonse wrote:
| I just discovered elixir-tools which has an LSP and it
| works instantly, and seems to have a more robust
| architecture than ElixirLS. I used ElixirLS for a couple
| years now but it was the best option for a while, but
| elixir-tools (which I literally installed 30 mins ago) has
| been a delight to work with.
|
| The instant feedback actually feels like how it always
| should've been.
| karmajunkie wrote:
| i tried it out but wasn't really happy with it in emacs.
| ended up going back to elixir-ls until elixir-tools is a
| little further along.
| mhanberg wrote:
| Next LS author here! I would love to hear about how your
| emacs experience went and what your setup/config is. My
| emacs skills are not the best (zero), so I've not been
| able to really work through any of the kinks.
|
| I have recently gotten it working tho with vanilla emacs
| with eglot, as well as doom emacs (again, with eglot).
|
| No pressure!
| karmajunkie wrote:
| happy to provide feedback when i get a minute! i'm also
| utterly useless in elisp so not sure how useful it'll be
| but ill try and make it helpful :)
| roblh wrote:
| I had this problem too when I started, for me it was an
| incorrect Erlang version. Using asdf to install both of
| them fixed it for me completely, and was pretty much a good
| enough experience all around that I use it to manage Nodejs
| installs too now.
| freedomben wrote:
| Yes, protip: Use asdf to manage erlang and elixir
| versions. The only exception might be if you're on Fedora
| the packages tend to stay pretty new, but anywhere else
| (and even there if you need a specific version) asdf is a
| godsend. I use it to manage go, node, python, and ruby as
| well.
| freedomben wrote:
| Getting set up a dev environment for me was painless, with
| ElixirLS being a big exception. For me the wrong version of
| erlang/OTP was causing crashes that weren't getting
| surfaced well. Fortunately I have enough Elixir experience
| that once I got the error message surfaced I was able to
| figure it out pretty readily, but yeah that experience can
| definitely be improved.
| thibaut_barrere wrote:
| I am a long time Elixir user and I also find the VSCode
| experience a bit frustrating ; sometimes the completion did
| not work, for hard-to-tell reasons (I'm switching computers
| often).
|
| I have seen a couple of issues that have been worked-out.
|
| In particular, it's nice to have completion in ".exs"
| files!
| epiccoleman wrote:
| One thing that's been bugging me recently is completions
| in eex/heex template files. It seems like a tough problem
| to solve, since you need reasonable completion for html,
| heex syntax, and arguably also completion on things like
| schema fields for forms and then components that the
| project defines.
|
| But man, if I could have a smart editor for those
| templates I would be like... three times as productive on
| my little side projects.
| rob wrote:
| > the last time I tried to just print "Hello, world!" in
| either Python or JavaScript, it took me down a hell dungeon
| of configuration and dependency nightmares.
|
| On Mac: brew install node node -e
| "console.log('hello, world');"
|
| Seems pretty easy?
| thibaut_barrere wrote:
| You can also do "brew install elixir".
|
| But for any serious work I will more likely use asdf or
| nvm/rbenv/rvm.
| karmajunkie wrote:
| i basically never recommend the brew version. on the
| occasion it gets installed i've had no end of problems
| from mismatched erlang version requirements and path
| management. asdf is just easier.
| matthewsinclair wrote:
| You left out what I said next in the comment you pull-
| quoted. :)
| pawelduda wrote:
| Had no problem with neovim but I think it uses the same LSP
| under the hood
| freedomben wrote:
| It does. That's what I use. If you have trouble, check that
| the erlang version you're using is compatible.
| pawelduda wrote:
| That's a good point which also applies to tooling such as
| observer, etc.
| sph wrote:
| That's an unrelated tangent better served by asking on
| elixirforum.com instead of polluting a thread with a drive by
| complaint.
| lawik wrote:
| The in-editor tooling definitely has rough edges. I've done a
| lot of Elixir and the ElixirLS stuff falls over a lot for me.
| There is a new effort with NextLS that seems very promising.
| mjhay wrote:
| Thanks, I'll check that out.
| mhanberg wrote:
| Next LS author here! Feel free to reach out with any
| questions!
|
| My contact links should be discoverable on the website:
| https://www.elixir-tools.dev
| sergiotapia wrote:
| Elixirls is finnicky and works 50% of the time in my
| experience. Don't know why.
|
| Try using Zed it comes with Elixir support out of the box, and
| uses nextls, which works always for me. Snappy as hell! Like
| Sublime Text 4.0 https://github.com/elixir-tools/next-ls
| mhanberg wrote:
| Next LS author here, thanks for the kind words!
| aeturnum wrote:
| NextLS also has an extension for VSCode![1] I've found the
| ElixirLS works best with the projects I use at work, but
| there's a lot of exciting language server development in the
| space.
|
| [1] https://marketplace.visualstudio.com/items?itemName=elixi
| r-t...
| sergiotapia wrote:
| This did not work for me. I installed the extension on
| Linux Mint, VS Code Insiders, and intellisense/autocomplete
| never worked no matter how many times I restarted the
| editor.
| itronitron wrote:
| That is why I don't use VSCode. While it's is a nice text
| editor, VSCode is lacking features that were standard in IDEs
| over twenty years ago.
| tokamak-teapot wrote:
| What kind of features? With Elixir, I use the 'click in stack
| trace to go to error' feature, do some debugging, use code
| completion, test running - and not much else.
|
| I'd be interested to know what other kinds of features there
| could or should be, as I'd like to be as productive as
| possible and I don't feel like I really am.
| lycos wrote:
| Hm, I hadn't heard of NimbleOptions before and always used Ecto's
| embedded_schema for these types of things (as also mentioned in
| the article).
|
| Looking at NimbleOptions based on this example I think I would
| prefer to stick to Ecto because I don't like the idea of putting
| the schema in a giant keyword list - but maybe soon I'll have a
| better play around with it myself, read the docs and see if there
| are better ways to use it.
| sph wrote:
| While I'm no strong typing zealot and find Elixir is decent
| enough on that aspect, I feel NimbleOptions is just runtime type
| checking of keyword params, and we wouldn't need much of it with
| a better type system.
| conradfr wrote:
| This seems interesting.
|
| I've been trying Goal[0] to validate API data at the controller
| edge, which uses changesets itself.
|
| [0] https://hexdocs.pm/goal/Goal.html
| ch4s3 wrote:
| That looks nice, but not a lot better than just embedded
| schemas. If it had more to offer for validating dates, I'd be
| interested.
| conradfr wrote:
| Maybe suggest it in the Elixir forum thread
| https://elixirforum.com/t/goal-a-parameter-validation-
| librar...
| ch4s3 wrote:
| Yeah, I could do that but I know how much of a tar-pit it
| really is so I'd be hesitant to inflict that on anyone else
| :)
| second_brekkie wrote:
| This is a nice article. I've used Ecto embedded schemas and
| nimble options separately but never thought to use them together.
|
| Faced with this kind of issue, I'd use Ash with the `Simple` data
| layer. You can get all the power of an embedded Ecto schema and
| nimble options together in a single dsl.
|
| If you need to validate with arbitrary logic you can do that with
| a 'validation'
|
| If you need to transform the input data before you return the
| struct you can do it in a change set.
|
| It's all there!
| freedomben wrote:
| A lot of people don't think of Ecto when doing data validation
| outside of the database, but that's a mistake! It's a really
| powerful design pattern and implementation, and I've found it to
| be immenseley useful. It's clear from inception that they
| recognized it's general-purpose usefullness for data validation,
| so you don't have to hack it to make it work. Very useful tool in
| the toolbox
| karmajunkie wrote:
| same here, so much so that i usually encourage teams to change
| any defstructs to embedded schemas.
| bicx wrote:
| Definitely. I've used it on general input validation for many
| endpoints over the years.
| derefr wrote:
| For sure, Ecto.Changeset "client-side" validations are a
| powerful tool even without any Ecto.Repo to insert the
| validated changeset into. You can just take the validated
| Ecto.Changeset, and "bake it down" to either the new/modified
| data, or the delta, and use that. And you get proper per-DDD-
| use-case type-casting "for free", through Ecto.Changeset.cast
| invoking Ecto.Type.cast on all your schema fields.
|
| Though! I _would_ say that you might just as well be making a
| mistake by not at least considering the question, when you
| reach for "raw" Ecto.Changeset validation, of whether whatever
| subsystem this is that has all these database-like needs that
| you're plumbing your way into, is just a Greenspunning _of_ a
| database -- something you could potentially rip out and replace
| _with_ a database, and in so doing, end up with an easier-to-
| understand, easier-to-maintain system. (It 's "easier-to-
| understand" not in the sense that your system is now _less
| complex_ ; but rather in that the complexity is now
| _constrained by a formal abstraction_ -- one that allows
| reasoning about the behavior of that component on a higher
| level. It 's the same reason you'd turn a generic server "with
| state", into a finite-state machine; or turn a generic
| recursive-descent parser into a PEG grammar.)
|
| The Erlang runtime is nice, in that it has in-memory database-
| like features (e.g. ETS), that let you get pretty far in
| writing internal state-management logic, without using any kind
| of database. But when you find yourself writing code that
| relies on 1. taking complex user-submitted data, 2. validating
| it and casting it into domain types, 3. inserting it record-
| wise into an ETS table, 4. and then manually indexing that
| table across multiple fields... maybe it's time for a database!
|
| (It doesn't have to be a _big_ database. Or even one that
| involves any external dependencies. There 's an Mnesia adapter
| for Ecto, which is a combination I've always found fascinating
| -- especially in the context of a single-node, memory-only
| Mnesia setup. It's an ORM over transactionally-joined ETS
| tables! But of course, if you have control over your deploy-
| time environment and so can suffer a few native deps, then
| SQLite works great as your "internal database" too.)
___________________________________________________________________
(page generated 2023-11-07 23:01 UTC)