https://dockerpull.com

Examples

Changing dependenciesUpdating OllamaMoving filesSquashing an image
Altering metadata

Docker pulls more than it needs to

There oughtta be a way to make repeated docker pulls smarter. Adding
a single byte to a single layer invalidates it and every following
layer, even if every other file in those layers is byte-for-byte
identical. For anyone with many dependencies (looking at you, ROS!),
it could mean gigabytes of unrelated files dragged along with your
small change.

And you'd better hope you're not pulling on a robot out on the farm,
or a device at a client's warehouse at dial-up speed. We can do
better.

The image I have[basisrobotics/demo-c]
The image I'm pulling[basisrobotics/demo-c]
Show me the waste

Adding apt install vim shouldn't cause every other dependency to get
pulled again. If your copy of Docker knew about files rather than
just layers, it could grab only the changes. (Did you know that
builds with clang are pretty reproducible nowadays? That's two
separate builds of Doom, byte for byte identical, shared)

Yes, robotics companies frequently "use Docker wrong". A lot of
Docker tooling isn't happy with images being so large. While it's
nice having your ML models bundled directly with your compiled code,
it can cause some difficulties, too. But rather than try to teach
every roboticist all of the two dozen little Dockerfile tricks to get
this stuff "correct" (if such a thing is possible with a hundred
dependencies), why not teach the tool, instead?

There are a lot of benefits to running a registry with this
capability as well. How many copies of python3.10 are floating around
on DockerHub's servers, unable to be coalesced? As a user, why are
you paying for storage of those binaries, when the registry already
has that data, locked in a tarball?

Interested?

We're building a smarter docker pull. If you're pushing large images
to robots, edge AI, IoT, datacenters in space, or anywhere bandwidth
is painful, we'd love to talk.

Get in touch