[HN Gopher] You Don't Need to Rebuild Your Development Docker Im...
___________________________________________________________________
You Don't Need to Rebuild Your Development Docker Image on Every
Code Change
Author : one2three4
Score : 88 points
Date : 2021-05-31 13:39 UTC (9 hours ago)
(HTM) web link (vsupalov.com)
(TXT) w3m dump (vsupalov.com)
| tlarkworthy wrote:
| I have done this but if possible I just run my code locally and
| reserve docker for infra (e.g. DB). Java, nodejs, go are all very
| easily run locally.
|
| Maintaining a complex Dockerfile is just more crap that makes
| programming tedious.
| nucleardog wrote:
| Sure, nodejs is easy to run locally. nodejs+npm of the exact
| version that a specific project needs, which is different and
| incompatible with the version some other project needs starts
| to get more annoying.
|
| Basically all of our projects have some level of external
| dependencies in the form of libraries or binary tools. Some
| projects overlap dependencies, but on incompatible versions.
| This gets really hairy.
|
| Trying to do this without containers (from experience, this is
| what the devs were doing before) turns into a massive
| documentation project and a nightmare of trying to get all
| these incompatible things to exist on a single system where we
| don't even have a consistent target because everyone's local
| dev environment is different enough to make it a nightmare. And
| then continually hounding the devs to keep the documentation
| up-to-date. But even that doesn't solve the problem because
| "apt-get install nodejs" today is not necessarily the same as
| "apt-get install nodejs" in six months or a year and nobody's
| thinking about this so it's just "worked for me last time!".
| Onboarding people at one point was a _days_ long process and a
| group effort.
|
| Instead we just re-use the same 30-odd line Dockerfile we use
| for deploying out to the infra for local dev. Instead of pages
| upon pages of out-of-date documentation. a bunch of tribal
| knowledge, and continually running into new and interesting
| bugs, we now have a short and simple source of truth which
| _has_ to be up-to-date because otherwise the environment the
| project is deployed to is broken. Onboarding people takes 5
| minutes.
|
| If you're a single person working on a single project, yeah,
| containerizing stuff is maybe some annoying overhead. Even then
| I still use it because once you've gotten past the learning
| curve, it's actually a really effective way to document and
| define your project's environment and ensure it stays
| consistent.
| majkinetor wrote:
| If only life was such a beauty.
|
| Now you have bunch of new and interesting problems with
| docker ecosystem.
|
| Newer tools are much much better in this cross-dependency
| problem as they evolved too, not only docker. I don't
| remember dependency hell nowdays while it was more or less
| common 15 years ago.
|
| I had the same project with and without docker and I totally
| prefer without docker. There are few quirks here and there
| but problems are mostly solvable within minutes unlike
| basically anything you can get with docker.
| forty wrote:
| My own fix for the node version issue is that we use a single
| version of nodejs (currently latest v12), and when we update,
| we update everything (which happens every two years, and is
| generally not too painful in my experience), so we just have
| to have this version locally.
| tqkxzugoaupvwqr wrote:
| nvm (node version manager) is your friend. Allows you to
| use different node versions for different projects.
| scns wrote:
| https://volta.sh too
| Fredej wrote:
| Honestly my experience is basically the opposite: Maintaining a
| coherent development environment is what makes programming
| tedious.
|
| I'd much prefer to just specify in a dockerfile that I need
| this compiler, these tools and these env-variables, and then I
| never have to worry about that stuff again.
| [deleted]
| tlarkworthy wrote:
| It's hotcode reload and debugging where I find docker starts
| to annoy me. (Also resource consumption and performance to an
| extent)
| jansommer wrote:
| I have the same experience. If cross platform loading of
| environment variables and shell scripting is needed one can use
| Powershell. Together with Windows Terminal and tmux for Linux
| and Mac, you can start every microservice in different panes
| and tabs. You won't get the full Docker experience where you
| know for a certainty that your code will run the same way in
| production, but leave that to ci/cd, perhaps a local kubernetes
| cluster you can use for production-like testing.
| brundolf wrote:
| It's my impression that there are lots of platforms that ask
| very little of their environment, and therefore don't benefit
| much from Docker, and therefore can be developed with minimal
| friction in a local environment without using it. Node, Java,
| often Python, even Rust because of its static dependencies, etc
| often fall into this category. And of course front-ends,
| especially.
|
| In practice you'll still use Docker to deploy because it's a
| nearly-unavoidable deployment protocol at this point. But I see
| very little reason to have it house your in-progress code in
| these cases.
| iofiiiiiiiii wrote:
| A little known feature is that the Docker integration in Visual
| Studio gives you this automatically.
| metaltyphoon wrote:
| Container fast run or something like that right ? If you
| observe the commands it does, it simply mounts everything it
| needs on the container and uses the first stage of the docker
| file (which is usually a base image) to run your stuff.
| robflynn wrote:
| We use this where I work and love it. It has the added benefit
| of letting you package preferred vscode extensions with the
| devcontainer as well.
| Noumenon72 wrote:
| I think PyCharm Professional's docker-compose remote
| interpreter[0] does too. It was a lot of setup but it's the
| only way I know to have your code run inside containers and
| actually be able to stop at breakpoints inside them. (I don't
| know much about Docker.)
|
| 0: https://www.jetbrains.com/help/pycharm/using-docker-
| compose-...
| gmaster1440 wrote:
| https://code.visualstudio.com/docs/remote/containers
| [deleted]
| snapetom wrote:
| I have to scratch my head every time I see a blog post about
| creating a dev environment that doesn't use volumes like this.
| IMO, this is the way to go and what Docker was meant to be as a
| dev environment.
|
| Maybe it's because I'm never confident in the changes I make, but
| my style is to write a few lines at a time, save, re-run or
| refresh. I can't imagine having to do a docker build every code
| change. If you do the later, it's just nominally better than
| developing on a remote server, which brings its own challenges.
| qbasic_forever wrote:
| Using host volumes can have big performance problems on OSX and
| Windows. Node or frontend development with huge node_modules
| folders in particular can run into problems like flakey file
| change notifications or just general slowness accessing files.
|
| Linux users aren't immune from trouble either. Host volume
| permissions are a big source of pain--if your mounted files
| don't have the same UID/GUID as the user baked into the
| dockerfile then you'll hose access to the files on your local
| machine. Most simple dockerfiles run as root (especially
| dockerfiles created by windows or mac developers who never deal
| with this problem) and all the created files inside them show
| up as root owned on your localhost. You have to take special
| care to craft your dockerfile to not run as a root user, launch
| with explicit user UID/GUID that matches your host machine, and
| use a tool like fixuid as an entrypoint to fix up mismatches.
|
| A good workaround though is to use docker's internal volumes
| instead of host mounts. This fixes all the Linux permission
| issues (the internal volumes live in their own root/docker
| managed location), and fixes the slowness on other platforms
| (no VM folder mount slowdown). But you give up the freedom of
| seeing your files locally and need to go all in with something
| like VS code remote containers, or running your IDE directly in
| the container.
| globular-toast wrote:
| Honestly I'm quite blown away to know there are people who
| weren't doing it this way. I think it's due to a fundamental
| misunderstanding of what docker is for and that is caused by
| using a solution without knowing the problem.
| krono wrote:
| How awesome would it be if we could exclude specific dirs or
| files from bind mounts. Think of cache directories, log files,
| and, of course, node_modules.
|
| The common workaround to this, is to create an empty volume and
| map whatever you don't want mounted to it. You'll end up with
| an empty directory in your host system, but its contents are
| available from within the container.
|
| It has many problems though, mostly permission and ownership
| related, but it somehow makes Docker for Mac even slower and
| crash even more often than usual.
|
| Example minimal docker-compose.yaml file for a custom
| "devcontainer" we use in VSCode that showcases the workaround:
| version: '3.8' services: project:
| image: devcontainer:dev container_name: devcontainer
| build: context: . dockerfile:
| Dockerfile args: ALPINE_VERSION:
| '3.13' NODE_VERSION: '14.16.1' ports:
| - 3000:3000 volumes: - ..:/project
| - node_modules:/project/node_modules:cached volumes:
| node_modules:
|
| This feature gets asked for quite often. Here are two issues on
| GitHub I was quickly able to find:
| https://github.com/docker/compose/issues/6470
| https://github.com/docker/compose/issues/6997
| sm4rk0 wrote:
| @bdcravens mentioned this as a solution in another comment:
|
| http://docker-sync.io/
| roofwellhams wrote:
| Did you try typescript? Now I can write for hours without
| trying out the app. And when I try... It just works
| redleather wrote:
| This is such a weird comment.. nothing about the OC suggests
| they were talking about front end dev; not to mention that
| writing for hours before actually running it is a pretty
| terrible way to code. Typescript is also not a workaround for
| volume binding, which can accommodate any stack.
| dehrmann wrote:
| > nothing about the OC suggests they were talking about
| front end dev
|
| >> I'm never confident in the changes I make, but my style
| is to write a few lines at a time, save, re-run or refresh.
|
| This is a pretty typical frontend development pattern,
| especially seeing "refresh."
| dividedbyzero wrote:
| I read that as "re-run or refresh, as applicable", i.e.
| re-run a Go app or refresh a Node.js webapp. Besides, it
| absolutely can be a big issue with non-frontend
| development, so the point still stands.
| snapetom wrote:
| Yes, that is why I specifically mentioned "re-run." About
| 75% of my work is actually API and backend data
| engineering work (Python Go, Node in that order). I do
| "save and re-run" less in that type of work, but
| definitely not enough to make frequent docker builds a
| hassle.
| pgwhalen wrote:
| As another commenter mentioned, "refresh" tipped them off
| as a front end dev.
|
| As a backend dev working in statically typed languages, I
| will sometimes code for hours without running, and I
| wouldn't say anything about it is terrible. I haven't
| worked with Typescript, but it wouldn't surprise me if it
| enabled a similar development process.
|
| I wouldn't necessarily infrequent running as a technique
| worth emulating, but in certain situations it works pretty
| well.
| lanstin wrote:
| Even there it is beneficial to run sooner. E.g, for
| servers as soon as the listen is up and dispatching, run
| it and see that the handler runs. Often times the only
| times my error handling code runs is during this initial
| code writing time. (For certain annoying to test for
| types of error). Fast iterations around a known good
| baseline.
| snapetom wrote:
| The majority of my work is actually in Python and Go on
| the backend. I do save/re-run less than when I'm doing
| front end, but docker builds are still a hassle. Flask
| will automatically reload on file changes, taking
| advantage of a docker volume. With Go, I'm doing go runs
| in development anyway up until deployment with go build.
|
| Maybe I'm doing things wrong, but docker volumes are
| essential in how I like to dev.
| archsurface wrote:
| A star fruit is also known as a carambola.
| gprasanth wrote:
| Another thing I've recently found helpful was docker layer
| caching inside github actions. Pretty easy to integrate, saves a
| lot of build minutes.
| pastage wrote:
| Works remotely on kubernetes, for us the game changer was running
| inotify_wait on cygwin for our windows users, changes are up in
| about 600ms.
|
| 1. Configure a service 2. Use start command sleep infinity 3.
| Install inotify_wait on windows 4. Do a loop like this
|
| Look for changes, Rsync changes to pod, Pkill java, Run start.sh
|
| We do a grep on changed files and only kill java if jar/classes
| and other deps has changed, that makes it possible to edit html
| in pod and get fast updates.
| eysi wrote:
| Garden[1] is a tool we built (yes, I'm affiliated :)) that has
| a lot of this functionality built-in. Might fit your use case.
|
| We recently re-wrote the hot reload functionality to use
| Mutagen[2] under the hood and it's insanely fast (<200ms
| anecdotally). It also does two way sync which can be useful.
| The old implementation used rsync but a lot of our Windows
| users struggled with that. So I figured I'd share in case that
| sounds familiar.
|
| What happens after a sync event depends on the stack, but we've
| had pretty good success with Entr[3]. We often have it watch a
| single file so that multiple watchers in a shared dev cluster
| don't eat up all the node's resources.
|
| 1: https://github.com/garden-io/garden 2: https://mutagen.io/
| 3: http://eradman.com/entrproject/
| mehphp wrote:
| Have you seen https://skaffold.dev/ ?
| emj wrote:
| When you have a team that already know the out and ins of
| Kubernetes it's easier to just use something that has all the
| integrations you need locally, certs, end points, firewall.
| Especially when you are not allowed to install things on your
| local machine.
| orlovs wrote:
| I have never understood kubernetes development in local
| development workflow unless kubernetes functionality being
| developed or some yaml magic. It's super overhead IMHO. If
| needed to test api's as unit test you can always port-forward
| traffic of desired enpoint or use something like telepresence.
| Less kubernetes is always better
| erulabs wrote:
| It's cpu/memory overhead yes, but between matching prod and
| dev, extremely simple config (redis is always located at
| "redis:6379"), zero setup instructions (skaffold dev),
| remote/local hybrid development, and fostering prod-ready
| skills for developers (still working on that devops koolaid),
| I find it -extremely- worthwhile. I've yet to meet someone
| who doesn't hate it to start (because kubernetes) and winds
| up loving it and swearing by it.
|
| It's essentially a vastly superior "vagrant up" on anabolic
| steroids
| orlovs wrote:
| I agree, for e2e/integration testing, excellent
| [deleted]
| gravypod wrote:
| While this is great for people with a fundamental understanding
| of containers and your prod environment this will usually lead to
| some issues with developers that don't need to, or want to, have
| context in these areas.
|
| In the past, to make a very similar workflow possible, I've built
| tools that automatically watch your source files and rebuild &
| restart only what is needed [0]. This was built for bazel +
| docker-compose but there isn't a reason one couldn't watch the
| "build:" contexts for what files are important.
|
| At a previous company one of our engineers was a huge fan of this
| volume mount approach and every single time something broke
| (which was very frequent due to some prod/dev env magic we had) I
| had to assist quite a few more junior devs figure out what was
| wrong with their machine. For those with scripting languages, was
| it their system's newline endings? For compiled languages, was
| their system SDK different then what was in the container? For
| prod bugs, did they forget to rebuild & test the container before
| opening their PR (we had no automated integration testing)?
|
| In my opinion, if you can make your build system in charge of
| building/packaging things you'll have a much happier time.
|
| [0] - https://github.com/CaperAi/bazel_compose
| vsupalov wrote:
| Hi HN! First time I see one of my articles on here. What a nice
| surprise.
|
| If the above link caught your attention, you might also enjoy the
| following ones:
|
| * For the quickest ROI: https://vsupalov.com/improve-your-docker-
| images/
|
| * Stuff I WISH I knew: https://vsupalov.com/12-docker-facts/
|
| * If your image builds are slow: https://vsupalov.com/5-tips-to-
| speed-up-docker-build/
|
| Looking forward to join the discussion later!
| SatvikBeri wrote:
| I've been doing this for about 6 months now, it saves a lot of
| time. Especially if you need to upload changed docker images to
| AWS.
| wiredfool wrote:
| This is great, until you need to watch a tree of files in macOS.
| Then docker takes a core to do the fsnotify/watch.
|
| There are caching settings to make it better, but there are race
| conditions between the watch and when file contents change, so
| sometimes the JS stack will compile an inconsistent file.
|
| OTOH, on linux, it's grand.
| bdcravens wrote:
| docker-sync fixes this by using an intermediate container that
| handles sync (I believe there are other options which work
| essentially the same way). It supports ignoring files you don't
| need on your main machine, like tmp folders and the like, which
| can improve performance further.
|
| http://docker-sync.io/
| richardwhiuk wrote:
| We wrote a tool to make using development Docker images easy -
| https://metaswitch.github.io/floki/
| robsalasco wrote:
| Looks awesome!
| alpb wrote:
| Skaffold is a great tool for this and employs hot reloading
| techniques without having to rebuild the image.
| https://skaffold.dev/
| spiddy wrote:
| I find it very interesting the idea of separating development
| image from production one and skaffold promotes it in a way
| ("skaffold dev" vs "skaffold run")
|
| Same as in frontend for example you don't "npm build" your way
| through development, instead you want hot-reload and other
| similar features.
| aszen wrote:
| Docker truthfully told is unusable for Dev envs that need to
| change constantly. It was never built to handle this use case. I
| haven't seen a good docker compose file that can be reliably used
| as a development environment in multiple oses with good
| performance. There's all sorts of edge cases where volumes don't
| work. In my opinion docker is very useful for thing like dbs,
| queues and other processes where the underlying code doesn't
| change. But for everyday frontends and backends it's not worth
| it. Nowadays I write a shell.nix file which contains all the
| dependencies the project needs, it works but is definitely not as
| easy to learn as a docker file
| okamiueru wrote:
| Out of curiosity, what OS are you on?
| efrecon wrote:
| In the past few weeks, I have spent some time and released dew
| [0]. It helps encapsulating this kind of setups in configuration
| and minimising typing. dew is still evolving, but it has served
| me well.
|
| [0]: https://github.com/efrecon/dew
| sebyx07 wrote:
| For any rails dev out these
|
| volumes: - .build/.bundle-cache-
| dir/app:/usr/local/bundle/
| iooi wrote:
| For devs on macOS, keep in mind that your filesystem is case
| insensitive!
|
| If you're using a linux image that expects a directory like
| `UpperCase` and you named it `uppercase` locally it would work..
| until you bake the source for your production release and you get
| errors around that directory not existing.
|
| "But it works locally!"
___________________________________________________________________
(page generated 2021-05-31 23:02 UTC)