PSA - Run "docker image prune" once in a while. reddit / selfhosted ---------------------------------------------------------------------- smbell · 2023-12-08 00:53 I just cleared up 142GB of space by doing so. ---------------------------------------------------------------------- xantioss · 2023-12-08 00:59 Solid advice! ---------------------------------------------------------------------- TheoR700 · 2023-12-08 01:03 To add to this, you might also find `docker system prune` to be useful. Just make sure you are fully aware of what it is pruning. ---------------------------------------------------------------------- nerdyviking88 · 2023-12-08 01:15 Cronjob to do this weekly will save your life. ---------------------------------------------------------------------- adamshand · 2023-12-08 01:26 Or even `docker system prune -a` ... ---------------------------------------------------------------------- SillyLilBear · 2023-12-08 01:45 You can also use watch tower to update your contianers and auto remove old ones. ---------------------------------------------------------------------- cakee_ru · 2023-12-08 02:14 Why not just `shred /dev/sda` to remove all the bloat? ---------------------------------------------------------------------- ButCaptainThatsMYRum · 2023-12-08 02:15 After every update, before removing snapshot/checkpoint... ---------------------------------------------------------------------- tf9623 · 2023-12-08 02:30 Thanks! I had wondered if something like that was around and I didn't RTFM. ---------------------------------------------------------------------- dan4223 · 2023-12-08 03:36 How do I make watchtower remove old ones? ---------------------------------------------------------------------- zfa · 2023-12-08 03:50 As someone who knows just enough about Docker to be able to scratch together a docker-compose.yml every now and again only yo promptly forget all the commands I need to ever maintain them going forward, I'm **eternally** grateful that [dockcheck.sh](https://github.com/mag37/dockcheck) prompts me to do this as a final step whenever I run it. ---------------------------------------------------------------------- SillyLilBear · 2023-12-08 03:51 WATCHTOWER_CLEANUP: true ---------------------------------------------------------------------- amcco1 · 2023-12-08 04:03 Or you could just use portainer and easily delete your unused images with the gui ---------------------------------------------------------------------- z-lf · 2023-12-08 04:23 docker system prune -a --volumes To be more fun. ---------------------------------------------------------------------- thekrautboy · 2023-12-08 04:37 Yay for dockcheck. I use `dockcheck.sh -a -p -s` as an alias to check and update all containers including stopped and autoprune dangling. ---------------------------------------------------------------------- thekrautboy · 2023-12-08 04:39 Not everyone *needs* another thirdparty program with shiny buttons to click when Docker itself can already do it with a basic command... ---------------------------------------------------------------------- makanimike · 2023-12-08 04:44 Yep. I tried portainer for a month or two and hated it. CLI is so much cleaner and - shockingly - more intuitive. ---------------------------------------------------------------------- nunavailabul · 2023-12-08 04:51 Nerd trolls really are savage XD ---------------------------------------------------------------------- DubDubz · 2023-12-08 05:04 The thing I appreciate most about portainer is being able to quickly manage multiple compose files. I’ve got like 10 different composes to keep my containers logically separated and the idea of handling that in command line just sounds exhausting. I’m sure it would be better if I was more skilled. But just not worth it to me. ---------------------------------------------------------------------- leknarf52 · 2023-12-08 05:20 Yeah you’re on Linux. Windows users have a harder time because you also have to shrink the virtual disk file after running docker image prune otherwise running the prune command has no effect on disk space. It’s frustrating. ---------------------------------------------------------------------- smbell · 2023-12-08 05:34 I have a folder that is DockerCompose. In that folder I have one folder per docker compose file. In each folder is the same update script that just does `docker-compose pull && docker-compose up -d`. I have a script in root that just loops through the folders in DockerCompose and runs the update script. I run it manually because some things (like immich) occasionally have breaking changes I need to manage. ---------------------------------------------------------------------- smbell · 2023-12-08 05:35 I was unaware of dockcheck.sh. That looks really cool. I'll have to check it out. Thanks! ---------------------------------------------------------------------- makanimike · 2023-12-08 06:08 When I do it manually I simply keep every container (and compose file) in its own folder. I had huge issues with permissions when I tried portainer. Some stacks would not work because (I think) different components of it could not communicate with each other. And I also couldn't get a grip of the network connections. Some stacks would just not be accessible. Trying to figure out Macvlans was a headache. In CLI I just pick a port that isn't already taken and I'm done. ---------------------------------------------------------------------- Jaymoon · 2023-12-08 06:27 What I ended up resorting to was creating a "npm" network in Portainer (since I use nginx-proxy-manager, then adding the following to the end of every stack: networks: default: external: true name: npm Then all stacks with that added are by default part of the npm network, and can talk to each other by container name. ---------------------------------------------------------------------- samaritan1331_ · 2023-12-08 07:20 Just flushed out 25GB of images haha. ---------------------------------------------------------------------- boggie26 · 2023-12-08 07:26 This is the way ---------------------------------------------------------------------- sulylunat · 2023-12-08 07:28 Ok I put my Pi in the shredder. Now what? ---------------------------------------------------------------------- woah_m8 · 2023-12-08 08:15 … why are you on windows tho ---------------------------------------------------------------------- zfa · 2023-12-08 08:28 Did you know you can create a compose file which 'includes' others? Because I was like you and then decided to have a single `docker- compose.yml` (default name so I don't have to specify it all the time on my command lines) which 'includes' all the `docker-compose.yml` files I have in the *one-subdir-per-service* directories underneath it. So by sitting in that dir and executing whatever docker cmd I need it sees that default file, grabs all the nicely segregated included files and works on them. Upshot is organised folders, segregated compose files, yet still super-simple docker commands. ---------------------------------------------------------------------- TheFumingatzor · 2023-12-08 08:28 I don't have that problem with Watchtower removing old shite. ---------------------------------------------------------------------- Lanten101 · 2023-12-08 08:29 Watchtower has an option you can enable, to clean up an image after updating the container. It's an Environment variable WATCHTOWER_CLEANUP: true That save me a lot of Weekley clean up. ---------------------------------------------------------------------- usrdef · 2023-12-08 08:41 I was running this for awhile, but eventually moved over to Portainer, which has been a nice addition. Tried Docker Desktop some time ago before this script and that app was a damn disaster. ---------------------------------------------------------------------- Significant-Neat7754 · 2023-12-08 09:06 I wish this worked for bind mounts 😔 ---------------------------------------------------------------------- wrath_of_reaper · 2023-12-08 09:07 This. I mean, Docker on Windows? Why? ---------------------------------------------------------------------- Le_Vagabond · 2023-12-08 09:16 eyh at least that'll teach you not to store persistent volumes with important data in docker's overlayfs. ---------------------------------------------------------------------- D_F_D · 2023-12-08 09:50 Last time I tried watchtower it entered a loop updating one of my containers that used all the same I had on the system. I don't remember why or which container was affected but it was enough to keep me away. ---------------------------------------------------------------------- xCharg · 2023-12-08 09:57 Removing french language pack is also a good practice `rm -fr` ---------------------------------------------------------------------- rursache · 2023-12-08 10:31 added that line to my crontab, its running daily after watchtower ---------------------------------------------------------------------- rursache · 2023-12-08 10:32 watchtower is working fine, probably user error ---------------------------------------------------------------------- prone-to-drift · 2023-12-08 11:07 If anyone is a beginner with docker, do NOT prune the volumes!!! Several tools' default docker-compose files store persistent data in unnamed volumes. Biggest example that comes to mind immediately is Immich, and you'd hate to lose all the albums and tags and faces metadata you've created. ---------------------------------------------------------------------- peschmae · 2023-12-08 11:10 That's really good advice! Here an example to do it using systemd timers, to run it daily (but weekly should be fine as well) `docker-prune.service` [Unit] Description=Docker system prune [Service] Type=oneshot ExecStart=/usr/bin/docker system prune -af --volumes `docker-prune.timer` [Unit] Description=Daily docker system prune [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.target ---------------------------------------------------------------------- GolemancerVekk · 2023-12-08 12:37 This will remove all stopped containers, regardless of the reason why they're stopped. Please make sure that's what you want. ---------------------------------------------------------------------- Riptide999 · 2023-12-08 12:47 Or let docuum do the job for you, based on storage threshold. https://github.com/stepchowfun/docuum ---------------------------------------------------------------------- leknarf52 · 2023-12-08 13:02 Work ---------------------------------------------------------------------- woah_m8 · 2023-12-08 13:28 I mean windows for development is common but deploIng to a windows container... :( ---------------------------------------------------------------------- leknarf52 · 2023-12-08 13:39 I don’t know how to do windows containers. ---------------------------------------------------------------------- Evantaur · 2023-12-08 13:59 There's also this if you find it helpful: https://containrrr.dev/watchtower/ ---------------------------------------------------------------------- Leonichol · 2023-12-08 14:08 Ah there's nothing more fun than watching a container upgrade shitting over itself because you've ran out of space. Leaving the database I'm a half updated state. Good Times. ---------------------------------------------------------------------- AnApexBread · 2023-12-08 14:17 Just run a sudo rm -rf / just to be safe ---------------------------------------------------------------------- RockG · 2023-12-08 14:38 Guaranteed to remove any bloat ---------------------------------------------------------------------- carolina_balam · 2023-12-08 14:40 Docker system prune -a - -volumes ---------------------------------------------------------------------- ProbablePenguin · 2023-12-08 14:58 Mine always says 0B reclaimed, I think because watchtower is automatically cleaning up after itself. ---------------------------------------------------------------------- trafficnab · 2023-12-08 15:19 Automatic updates sounds like a nightmare, I don't want to be a beta tester ---------------------------------------------------------------------- SillyLilBear · 2023-12-08 15:20 99.999% of the time it is fine, the 0.001% of the time, you can easily roll back. ---------------------------------------------------------------------- jameson71 · 2023-12-08 15:34 Unless I am misunderstanding something, if you shrink the virtual disk, it is just going to need to re-expand it next time it needs the space possible leading to fragmentation. Pruning alone will still free up space for future images. ---------------------------------------------------------------------- ag237 · 2023-12-08 15:40 docker rm -f $(docker ps -qa) ; docker system prune -a --volumes ---------------------------------------------------------------------- leknarf52 · 2023-12-08 15:56 Try it yourself. It don’t work. Try it with windirstat. You’ll literally be able to see the disk space not free itself. I don’t really know how it could lead to fragmentation or what that means really. Anyway, every few months I docker system prune and shrink the virtual disk. After I shrink, windirstat tells me the disk space is back. ---------------------------------------------------------------------- Le_Vagabond · 2023-12-08 16:07 > Several tools' default docker-compose files store persistent data in unnamed volumes. the important part here is that ***you*** (yes you) can choose to store that somewhere else, like a local folder that is not a docker volume. that way your important data is safe and you can even move it to another server easily. ---------------------------------------------------------------------- ModischFabrications · 2023-12-08 16:31 Be careful using it. For me the "graceful shutdown" it promotes wasn't as clean as it should have been and services didn't restart property. Might work often enough, but check once in a while if everything is still where it should be. ---------------------------------------------------------------------- Evantaur · 2023-12-08 16:50 It probably has something to do with the containers themselves (flexget for an example shits itself every time it's restarted because it doesn't remove it's own lockfile) ---------------------------------------------------------------------- ThetaDev256 · 2023-12-08 17:25 Docuum is great if you accumulate unused container images on your system (like on a work/testing machine or a CI system) and not just old/dangling versions. For a regular server docker-prune does the job. ---------------------------------------------------------------------- BCIT_Richard · 2023-12-08 18:12 I chuckled at this, Thanks. ---------------------------------------------------------------------- Riptide999 · 2023-12-08 18:25 Well, on a selfhosted server there are probably a lot of images that get continuously upgraded to newer versions. docuum handles that use case as well. But sure, a cron job with a docker image prune works as well. Just wanted to give options to the crowd. ---------------------------------------------------------------------- OverAster · 2023-12-08 19:04 He answered this question 4 hours before you commented this... Some people don't have control of their environments. ---------------------------------------------------------------------- PaleMongo · 2023-12-08 19:36 To get a sense of what prune can remove, run `docker system df` ---------------------------------------------------------------------- zfa · 2023-12-08 19:57 Thanks, I used to use it. I moved to using [diun](https://github.com/crazy-max/diun) to just notify of updates but not apply them though. My reasoning was if I was going to check stuff works after upgrade notifications from watchtower anyway, I might as well just run the upgrades via dockcheck.sh myself when I'm next at my pc so I can fix anything that doesn't start. Just removes the risk of downtime if upgrades happen whilst I'm not able to get on and fix stuff. ---------------------------------------------------------------------- Trash-Alt-Account · 2023-12-08 20:14 IMPORTANT NOTE TO ANYONE BEFORE RUNNING THAT: > If anyone is a beginner with docker, do NOT prune the volumes!!! > Several tools' default docker-compose files store persistent data in unnamed volumes. Biggest example that comes to mind immediately is Immich, and you'd hate to lose all the albums and tags and faces metadata you've created. source: https://www.reddit.com/r/selfhosted/comments/18daji7/psa_run_d ocker_image_prune_once_in_a_while/kchjlg9/ ---------------------------------------------------------------------- imnotabotareyou · 2023-12-08 20:48 Nah ---------------------------------------------------------------------- beelaxter · 2023-12-08 20:58 I have a similar setup with notifications via webhooks to home assistant. I've managed to send an entire message of the updates, but would rather have individual messages for each image updated I'm a noob for most of this, just curious how you're moving the notifications to home assistant. ---------------------------------------------------------------------- Evantaur · 2023-12-08 21:59 I get a single notification with multiple entries too like Updated: \- One thing \- Two thing ---------------------------------------------------------------------- Sp8198 · 2023-12-08 22:14 I did something similar once by accident. I felt so dumb. ---------------------------------------------------------------------- shadowonyx23 · 2023-12-08 22:36 You can also choose when to run the updates by manually starting the watchtower container. ---------------------------------------------------------------------- Global_Shock_1511 · 2023-12-08 22:47 Or better yet, `rm -rf /` !! ---------------------------------------------------------------------- Fishamatician · 2023-12-09 00:23 I'm in the same boat and hadn't heard of this so cheers friend. ---------------------------------------------------------------------- Fishamatician · 2023-12-09 00:27 Oh fsm I hadn't even thought of defragging a hard drive in years, thanks for the blast from the past. ---------------------------------------------------------------------- freebullets · 2023-12-09 02:04 Make sure to tell it to also clear the french language pack for the root user too: `rm -fr --no-preserve-root` ---------------------------------------------------------------------- Thefaccio · 2023-12-09 02:05 How to do it in a docker compose file? Like immich's one. I cannot understand it ---------------------------------------------------------------------- beelaxter · 2023-12-09 04:43 Oh okay, do you use the webhook method too? ---------------------------------------------------------------------- Evantaur · 2023-12-09 07:17 Yes ---------------------------------------------------------------------- Le_Vagabond · 2023-12-09 15:47 https://immich.app/docs/FAQ#how-can-i-purge-data-from-immich > Metadata stored in a postgres database, persisted via the pg_data volume Files (originals, thumbs, profile, etc.), stored in the UPLOAD_LOCATION folder. https://github.com/immich-app/immich/blob/main/docker/docker- compose.yml > volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload https://github.com/immich-app/immich/blob/main/docker/example.env > # The location where your uploaded files are stored UPLOAD_LOCATION=./library and for the postgreSQL db: > volumes: - pgdata:/var/lib/postgresql/data your files are safe in the library folder next to your docker- compose.yml, but your pgsql db isn't because it's living in docker's overlayfs (it only has a volume name, not a location) so you need to change it from that to > volumes: - ./pgdata:/var/lib/postgresql/data to store it in a pgdata folder in the same location as your docker- compose.yml and the library folder. ---------------------------------------------------------------------- fjixdla · 2023-12-10 17:03 What's the worst thing that could happen if I accidentally remove a container? From my understanding, containers aren't meant to keep any state (that's what volumes are for), so it should be fairly simple to recreate a container. ---------------------------------------------------------------------- GolemancerVekk · 2023-12-10 17:22 Since the container is gone docker will completely forget about that service until you notice and re-provision it. If you have lots of containers you may not notice right away. What's the worst that can happen depends on what was in it. ---------------------------------------------------------------------- magicaldelicious · 2023-12-10 18:50 For most people it's probably easier to leverage something like Watchtower and use the cleanup [0] facilities. Saves you from having to manually earn things up if you're auto-tracking / upgrading a specific version train. You can also use Watchtower to track your changes and look for updates but handle all hr logistics for you if you don't trust automatic updates. [0] https://containrrr.dev/watchtower/arguments/#cleanup ---------------------------------------------------------------------- cakee_ru · 2023-12-10 18:57 Yes. Also auto-updates are bad, at least for servers. For clients it's kinda okay if can be easily rolled back. ---------------------------------------------------------------------- magicaldelicious · 2023-12-11 01:30 That's an overly broad statement that's not true when considerations are taken. As I said you can configure Watchtower to apply updates when configurations are applied that lock down auto updates altogether or only allow a minor train of software update, for example. You *can* schedule updates with Watchtower as was implied you couldn't do with your comment. Ultimately Watchtower can be very powerful. Simply letting it auto- update will eventually yield a broken deployment. That's not what I'm suggesting at all. ---------------------------------------------------------------------- cakee_ru · 2023-12-11 08:04 How can you be sure that you'll be able to fix issues every week or whatever is configured? ---------------------------------------------------------------------- faceproton · 2023-12-12 18:50 lol i completely forgot about this and I had 100 GB of unused images ---------------------------------------------------------------------- Mag37 · 2023-12-14 20:17 Thank you for mentioning [dockcheck](https://github.com/mag37/dockcheck)! So glad to see that people are using it. I've not updated it much but recently got a few suggestions that I've slowly worked on. Today I merged to v.0.3.0 with some new additions and fixes. ---------------------------------------------------------------------- zfa · 2023-12-14 21:55 It's awesome mate, I literally couldn't live without it. I mention it every chance I get. Thanks for your continued work on it.