[HN Gopher] From Ruby to Node: Overhauling Shopify's CLI for a b...
___________________________________________________________________
From Ruby to Node: Overhauling Shopify's CLI for a better developer
experience
Author : mooreds
Score : 69 points
Date : 2023-01-12 18:53 UTC (4 hours ago)
(HTM) web link (shopify.engineering)
(TXT) w3m dump (shopify.engineering)
| tiffanyh wrote:
| I always got the vibe Shopify developers didn't want to rock-the-
| boat to much with Tobias (founder/ceo) on tech stack, given his
| past role as a core Rails developer.
| eatwater123 wrote:
| Cool. But god, the Shopify CLI / developer experience has
| degraded so much in the past few years. With the newest
| iteration, they want a full app re-structure just to work with
| the CLI properly. It is absolutely brutal.
|
| Sure, before they had basically no tooling, but then they had
| some basic ones that everyone was basically happy with, and then
| they just decided to iterate too much.
|
| (saying this as a ~ 8 year long Partner / Expert / App Developer)
|
| I've spoken to some Shopify employees and they all agreed that
| the developer teams are a bit of a mess. It seems like nobody
| managing the developer products are actually using the end
| results, otherwise they'd realize the hoops needed to work
| through (unless you are starting a brand new project right now
| --- and hoping that they don't decide to change it all, like they
| have 5 times in the past 3 years)
| primax wrote:
| This is where I'm at right now. We have 40000 items on the
| store and I'm writing an updater for it, and the entire
| official ecosystem is a dumpster fire.
|
| I've found some ok golang api clients which seem to do the
| trick so far.
| btown wrote:
| One of the things I find fascinating is that in the days where
| it was Liquid templates, and Vercel etc. were barely a twinkle
| in people's eyes, Shopify had a huge value add - that you or an
| app development consultancy could iterate on customizing
| interfaces here and there, and deploy changes from the CLI or
| directly on the Shopify platform, without needing to worry
| about deployment, payment processing, etc.
|
| But so much has changed now. Deploying a customized frontend
| has a negligible cost, and the API surface that Shopify powers
| could be powered by a myriad of tools, with Stripe etc. doing
| large amounts of heavy lifting on the administrative side.
|
| Were contractual obligations, and not wanting to bite the hand
| that feeds, the only thing preventing longstanding Shopify app
| developers from joining together and creating an industry-
| standard alternative API surface that could be deployed in
| different ways?
| threeseed wrote:
| My partner runs a medium sized ecommerce store on Shopify and
| her priority is having a system that is simple and stress-
| free.
|
| And whilst she appreciates all of the Shopify Apps they are
| secondary to the fundamental process of taking, processing
| and shipping orders.
|
| The key to the whole ecosystem are non-technical users like
| her not the app developers.
| Bombthecat wrote:
| And people who think they will be the next big store :)
| drchiu wrote:
| A lot of Shopify Partners I've talked to feel the same way. The
| tooling and API has become harder to work with over the past
| couple of years.
|
| The previous designs (which worked very well for a long time)
| seemed to have been all thrown out/deprecated to such a large
| degree it has caused a lot of long time devs to be scratching
| their heads.
| seeekr wrote:
| How does something like this happen? Shopify should be
| absolutely on top of its game here, seeing as user experience
| seems to be one of their main priorities?
|
| Nowadays it's not that hard to get UX/DX right if you are
| actually committed to it.
| [deleted]
| lelandfe wrote:
| They sure do like to start over, huh? What's funny is that
| Slate was canned while still in beta:
| https://github.com/Shopify/slate#-slate---end-of-support-jan...
| DwnVoteHoneyPot wrote:
| My read on this is they just wanted to cut out Ruby and do
| everything in Node. That's fine as a decision, but rest of the
| article feels like just trying to justify it after the fact.
| Like someone at the top decided, now lets pretend it's a good
| decision.
|
| As you mentioned, if they're not OK with CLI, they could
| refactor in Ruby. The whole "embracing functional programming"
| and MVC architecture (i think Rails when i hear MVC) as reasons
| to go to Node, and not Ruby, is nuts.
|
| If they were worried about dependencies, they'd use Rust or Go
| as they mentioned.
| masklinn wrote:
| > Like someone at the top decided, now lets pretend it's a
| good decision.
|
| It might not even be a top thing, if they migrated internal
| eng from Ruby to node internally I would not be surprised if
| this was dev-driven either
| reillyse wrote:
| That's how I read the article too. We wanted to move from
| ruby to node. They even admit that the most well know
| language is ruby in their org. Seems like a pretty serious
| waste of resources to me.
| Mystery-Machine wrote:
| Agreed. Every year or two Shopify reinvents the wheel. Ruby =>
| Go => Rust => Node => Wasm?
|
| It will happen again. I believe they have terrible tech leads /
| managers who push/approve wrong things. They also grew
| exponentially during the pandemic, so that grew their manager's
| incompetence exponentially as well.
| jumpyjumps wrote:
| I'm begging you, just use a language that compiles to a single
| binary. It can self update in place, be easily installed on
| developer machines and CI pipelines without requiring a whole
| additional toolchain in every container. Google, Heroku, AWS,
| Shopify all do and it's an awful developer experience.
| LAC-Tech wrote:
| Huh? To run a node application, you need node installed on the
| container, as well as the JS files. Is that a big deal?
| eddsh1994 wrote:
| Yes, because now I need to install node, probably run npm
| install, etc...
| LAC-Tech wrote:
| there's no npm install and there's no etc.
|
| All the files needed to run a node application are in
| node_modules. If node is on the container, you can just
| rsync the scripts.
|
| I think you're comparing distributing stand alone binaries,
| vs distributing the source code for a node app, and then
| installing its dependencies, and maybe doing some
| transpiling. That's not a fair comparison.
| eatonphil wrote:
| You don't even need that: https://github.com/vercel/pkg.
|
| I think there are equivalents in the Python ecosystem.
| GOATS- wrote:
| I have used PyInstaller for this before, which has the
| major caveat of it not being a cross-platform solution. You
| can only build binaries for the platform you're currently
| on. However, you can work around this by configuring GitHub
| Actions to build binaries for you for Windows, macOS (x86)
| and Linux.
| seeekr wrote:
| Rust is amazing for CLIs (Go potentially as well, too little
| experience with that personally!), but if you're already quite
| comfortable with JS & have existing investment in it for CLI or
| related code, then 'npx <tool> ...` while asking the user to
| install Node (or Bun) is an acceptable tradeoff in most cases.
| Updating via npm is also easy enough for users, though it's not
| self-updating of course, but basically no CLI will do that
| fully automatically for you. Not even Cloud SDK CLIs. And for
| good reason.
| maccard wrote:
| > I'm begging you, just use a language that compiles to a
| single binary. It can self update in place
|
| Is there a library that does self updating in place for
| binaries? Go is the langage that comes to mind when talking
| about single binary, so maybe a go library?
| eatonphil wrote:
| What's the mechanism for self-updating in place? Something more
| intelligent than just downloading and copying the file over
| itself? A running program itself is a copy of the file on disk,
| I think. So that sounds fine. But this still depends on getting
| permissions right though I think. Also, does a binary always
| know where it lives (across Win/Mac/Linux)?
| cbarrick wrote:
| Download-and-copy is the way. A single binary CLI can just
| copy over itself and exec into the new version. You can
| preserve the permissions of the original file.
|
| With scripting-language CLIs, you either have to deal with
| the module install process or use an external packaging
| solution.
| silasb wrote:
| This works pretty good, but we've ran into issues where the
| downloaded file isn't code signed and then this becomes a
| big issue. AFAIK, if this is a public CLI then homebrew
| will codesign for you, if you are distributing a private
| CLI then you need to solve the code signing issue yourself.
| Not many blog posts on how to handle this...
| [deleted]
| rickette wrote:
| Exactly, I get that they initially used Ruby given the whole
| company is build on it. But overhauling the CLI and NOT going
| for a single binary is beyond me.
| cmoski wrote:
| Seems odd on the face of it. They say it is because the CLI
| already depended on Node and they couldn't escape it. So,
| they moved from depending on Ruby and Node to just Node I
| guess.
| jacobsenscott wrote:
| Good luck! I've found any cli with any kind of runtime dependency
| other than maybe sh eventually becomes difficult to run. I think
| they all tend to end up getting packaged with an isolated copy
| (node/ruby/python) of whatever runtime they need.
| ruby_duby_doo wrote:
| PSA: The singleton context configuration is probably the most
| disgusting thing I've seen in Ruby culture. Please do not bring
| this practice to node -- I've already witnessed this crime in
| shopify SDKs and stripe.
| cezart wrote:
| Honest question: Why is it a bad pattern? And what are some
| better alternatives from other ecosystems?
| latchkey wrote:
| The singleton pattern generally tends to be hard to write
| tests for. Singletons are basically a global object with
| static methods.
|
| The dependency injection pattern fits better. DI is basically
| passing around objects which conform to an interface. This
| way you can mock out those objects more easily.
| [deleted]
| ericb wrote:
| Regarding one of the problems they cited--If anyone is interested
| in deploying a _stand-alone_ ruby CLI executable using a modern
| Ruby (3.1), the fork here can do that, but you need to compile
| the rubyc for each platform first (not used the linked ones in
| the readme).
|
| https://github.com/ericbeland/ruby-packer
|
| Also, there was an issue with the most recent X-Code, so I had to
| actually downgrade my local X-Code to compile rubyc.
|
| I wonder if Shopify had known they could build the ruby CLI into
| stand-alone binaries (no Ruby install or Node needed) if they
| would have still gone the Node route? Not that I blame them for
| not knowing--I had to fork the motor-admin fork, _and_ update a
| few things to get it working. The original ruby-packer only works
| with ancient Rubies.
| phaedryx wrote:
| Interesting. TIL about the Open CLI framework that they all seem
| to be moving to: https://oclif.io/
| meagher wrote:
| CAC also quite good: https://github.com/cacjs/cac. Used by Vite
| and many others.
| xavdid wrote:
| It's from Salesforce/Heroku. The documentation is definitely
| lacking, but it's battle-tested and under fairly active
| development.
|
| I moved the Zapier CLI to oclif a few years ago over a bespoke
| version, since it handles a lot of the nitty-gritty of
| displaying text in terminals. It's a pretty nice setup and
| allowed us to focus on the important things.
| skipants wrote:
| It's great too -- spiked out a CLI using it in my last company.
| It was really easy to set up and program in.
| swyx wrote:
| its from heroku! we even did a conference once!
| https://www.heroku.com/events/2019/oclifconf (and i gave one of
| my most random talks ever
| https://www.youtube.com/watch?v=1_w1YWCHXFg)
| notkurt wrote:
| Nice. But there is so much more work to be done. From a customer
| perspective, it's great. However, really feel as the entire
| developer experience is in need of an overhaul (And it seems
| pretty clear that they know that too). In saying that, even if
| things don't change, it's still the best option for a lot of
| merchants.
| hit8run wrote:
| Questionable decision making. Why not use Go when minimizing
| runtime dependencies is first priority? I'd probably just have
| cleaned up the Ruby code and used something like Ruby-Packer to
| create binaries. When typing is so important why not use rbs?
| Makes no sense to me.
| gregors wrote:
| Agreed. Go is the obvious clear winner here for a CLI
| executable.
| wiseowise wrote:
| Rbs doesn't hold a candle to TS.
| dorianmariefr wrote:
| Heroku did the same
| mtmail wrote:
| more background info
|
| https://blog.herokuapp.com/evolution-of-heroku-cli-2008-2017
| Their version 6, started with Ruby, then go/node, then pure
| node.
|
| https://github.com/heroku/cli "This is the next generation
| Node-based Heroku CLI. The goals of this project were to make
| plugins more flexible, remove Ruby as a runtime dependency, and
| make the CLI faster. [...] It has identical functionality to
| the old Ruby CLI."
| revskill wrote:
| The original article has no relation to Heroku, that's why
| the downvote.
| swyx wrote:
| Oclif is from Heroku so in a way Shopify's CLI is
| downstream of them
| berkle4455 wrote:
| If only Heroku had any customers remaining to use it
| hit8run wrote:
| The free tier shutdown was definitely not the best marketing.
| Their prices are steep in a docker/kubernetes world. They
| could have had at least left the free tier for Ruby projects
| as it's their origin.
| goatlover wrote:
| I don't understand going from Ruby to JS. Go or Rust sure, but
| Node for CLI? What is gained by doing that lateral move?
| wiseowise wrote:
| Tooling is much better, for starters. And they plan to use TS,
| not JS, did you even read the article?
| aliqot wrote:
| Developers who'll work right out of school, is my guess. Rails
| is great for the people who used it during that generation, and
| still is, I'm told. JS offers cheaper and more plentiful
| talent, and doesn't require as big of a mind shift as rust. If
| they become more JS centric, given the overlap in JS and Rust
| ideology, I could absolutely see them involving Rust soon
| after.
| pastor_bob wrote:
| I just find working with Node to be gross.
|
| Look at this:
|
| https://github.com/Shopify/cli/blob/main/package.json
|
| but wait...this too:
|
| https://github.com/Shopify/cli/blob/main/packages/app/packag...
|
| https://github.com/Shopify/cli/blob/main/packages/cli-hydrog...
|
| https://github.com/Shopify/cli/blob/main/packages/cli/packag...
|
| https://github.com/Shopify/cli/blob/main/packages/create-app...
|
| etc, etc, etc
|
| VS
|
| https://github.com/Shopify/shopify-cli/blob/main/Gemfile
|
| What am i missing? Why do you have to rely on all that junk to
| run a CLI? It feels like people choose Node because creating a
| Rube Goldberg machine creates job security.
| lacasito25 wrote:
| The project looks extremely messy, so many packages inside
| packages. It feels like java, but less performant
| bretthopper wrote:
| You are missing a few things about the Ruby based CLI:
|
| 1. dependencies for Gems are specified in the gemspec file and
| not the Gemfile. See https://github.com/Shopify/shopify-
| cli/blob/main/shopify-cli... for example. There's a few non-
| development dependencies.
|
| 2. since it's difficult to package up a Ruby gem for
| distribution, maybe dependencies were vendored directly in the
| codebase: https://github.com/Shopify/shopify-
| cli/tree/main/vendor
|
| This isn't meant to be a comparison of the number of
| dependencies or anything. Just pointing out a few nuances to
| how the Ruby dependencies were handled.
| [deleted]
| DwnVoteHoneyPot wrote:
| They think of it as a benefit: "Node's module system allows
| having multiple versions of the same transitive package without
| conflicting with each other"
| jacobsenscott wrote:
| One reason is because ruby has a large standard library. Also
| people just pick what they know - there are more js devs than
| ruby devs. Also neither ruby nor node are a good choice for a
| cli, so it is kind of a moo point.
| mccolin wrote:
| It's like a cow's opinion.
| joshmn wrote:
| > there are more js devs than ruby devs.
|
| Not at Shopify there ain't.
| olso wrote:
| The CLI is nice, but developing Shopify apps+themes is still a
| wild west.
| aliqot wrote:
| So who's left, basically Github?
| wiseowise wrote:
| Left with what?
| croes wrote:
| Developer experience seldom correlates with user experience.
| golondon wrote:
| It increasingly feels like the decisions taken and the work done
| at Shopify API / Libraries / CLIs / Docs etc are done for promos,
| rather than thinking about DevEx of people who try to build
| reliable product on their platform.
|
| I notice this pattern of shoving nodejs / js, especially react
| through people's throat for the apps developed on the platform.
| Polaris, Session tokens and all the boilerplate apps the CLI
| generates are great examples of that.
|
| I would rather having Polaris as CSS framework + some JS bits in
| vanilla-JS, Auth + Session tokens as just as a small JS library.
| But instead I am forced to use React. I know Polaris is also
| available as CSS only but that's almost impossible to use as it's
| not designed for human use really. Or Bridge exists as a library
| but for many cases it's just way too bloated and documentation
| for non-react version of those things are not so good.
___________________________________________________________________
(page generated 2023-01-12 23:00 UTC)