[HN Gopher] Syncthing: Syncing All the Things
___________________________________________________________________
Syncthing: Syncing All the Things
Author : sohkamyung
Score : 370 points
Date : 2021-07-23 10:40 UTC (12 hours ago)
(HTM) web link (lwn.net)
(TXT) w3m dump (lwn.net)
| incanus77 wrote:
| Over a year of use here, with about 75GB of my most critical
| document data across two Mac laptops and an old FreeBSD server.
| No hiccups. I use Documents.app on my iOS devices to access any
| of those computers via LAN versions or port-forwarded, out-of-
| the-house versions, and everything is great. Highly recommend.
| NelsonMinar wrote:
| SyncThing is an absolute treasure. I mean I have 20 years of
| rsync scripts, I've used Unison for real. But SyncThing is next
| level. The key part is once you have it set up it just runs and
| runs reliably without any attention. Also it works great on
| Windows too, including handling differences in filesystem
| semantics with Linux. Really well done.
|
| I mostly use it to keep Windows' Document folders in sync across
| various machines (through a Linux server). Windows apps still put
| files in lots of random places but most of the important stuff is
| in Documents these days. I also use it to sync GPS traces from
| GPSLogger on an Android phone to my server. The Android app works
| great!
|
| The setup of a new node is more complicated than it should be,
| that's the one product weakness.
| sedawk wrote:
| I'm one of those who's in the RSYNC school and take snapshot-
| style backups of my data. I wonder, how a folder under
| _syncthing_ that has RSYNC snapshot-style backup data work,
| which is, how does SYNCTHING deal with hardlinks (I suppose it
| ignores them and treats them as individual /new files?). I know
| that's not what syncthing is for, but I'm just curious.
| papaf wrote:
| Syncthing has a good approach to telemetry. It optionally
| collects stats and makes them publicly available.
|
| https://data.syncthing.net
| creativeembassy wrote:
| Muse Group could have taken a lesson from this with their
| (attempted?) addition of telemetry to Audacity. If the
| community feels a sense of ownership of the data that's being
| collected from said community, you'll probably receive less
| push back.
| srinathkrishna wrote:
| I've been using Syncthing for the past few days and it has been
| working flawlessly. One thing I haven't tried is syncing between
| Windows and UNIXes (macOS/Linux) - does anyone have experience
| around this? How do permissions work?
| qmmmur wrote:
| I'm not sure what your concern is. Differences in how user
| space permissions are handled? You give synctjing read write
| access and it does its thing.
| Spivak wrote:
| Either perfectly or not at all depending on what you want to
| do. You get file contents, modification time, mode bits on
| *nix's, and the read only bit on Windows.
|
| So for you-to-you it's seamless but if you want to sync shared-
| user directories then you lose most info.
| Macha wrote:
| I have Windows, Android, and Linux devices in my shares, I've
| yet to encounter issues.
|
| Someone indicated they did have issues if you do a case only
| rename between a case sensitive (Linux) system and a case
| insensitive (Windows/Mac) one. I've never done this myself, but
| I guess that's one to run into. Also the usual caveats with any
| program, like that you won't be able to sync con.py to a
| Windows system, regardless of which program you use.
| dangerface wrote:
| I use syncthing to keep keepass and my private keys in sync
| across my devices, it's great for that works perfect with easy
| setup and its cross platfrom.
|
| I tried to use it to sync SASS and webpack between developers and
| it didn't work, it seems to have an issue with node_modules has
| too many files for it to deal with and it thinks the directory is
| hundreds of gb and just stalls with a million years eta.
|
| The versioning is cool too but I found it a bit flakey.
|
| If you need reliable versioning and syncing of lots of files use
| GIT, if you need easy cross platform syncing syncthing is the way
| to go.
| mleo wrote:
| My rule of thumb is that anything in gitignore should be in
| syncthing's list to ignore. There is some work on both sides
| when doing simultaneous development, but it keeps local files
| local.
| dang wrote:
| Past threads:
|
| _Syncthing: Untrusted (Encrypted) Devices_ -
| https://news.ycombinator.com/item?id=27702785 - July 2021 (5
| comments)
|
| _Open Source Continuous File Synchronization_ -
| https://news.ycombinator.com/item?id=27149002 - May 2021 (146
| comments)
|
| _Syncthing Untrusted Device Encryption_ -
| https://news.ycombinator.com/item?id=26424096 - March 2021 (1
| comment)
|
| _Syncthing is everything I used to love about computers_ -
| https://news.ycombinator.com/item?id=23537243 - June 2020 (159
| comments)
|
| _Do not use Syncthing (2019)_ -
| https://news.ycombinator.com/item?id=23116462 - May 2020 (1
| comment)
|
| _Emacs ' Org-Mode and Syncthing = Perfect (2017)_ -
| https://news.ycombinator.com/item?id=23058358 - May 2020 (77
| comments)
|
| _Syncthing: An open source Dropbox replacement_ -
| https://news.ycombinator.com/item?id=20466469 - July 2019 (41
| comments)
|
| _Syncthing graduation day_ -
| https://news.ycombinator.com/item?id=18832517 - Jan 2019 (114
| comments)
|
| _Syncthing Usage Data_ -
| https://news.ycombinator.com/item?id=13856552 - March 2017 (119
| comments)
|
| _Syncthing_ - https://news.ycombinator.com/item?id=10331031 -
| Oct 2015 (1 comment)
|
| _Syncthing: Open Source Dropbox and BitTorrent Sync Replacement_
| - https://news.ycombinator.com/item?id=7734114 - May 2014 (184
| comments)
| tailspin2019 wrote:
| Anyone using Syncthing to sync SSH config/keys and (eg)
| bash/shell profiles between machines?
|
| I've gone through various attempts at keeping things in sync
| across a couple of Macs but always seem to hit issues.. (the
| issues not being specific to Syncthing)
| moviuro wrote:
| Sounds like a job for SSH certificates.
|
| See e.g. https://news.ycombinator.com/item?id=20955465 ;
| https://goteleport.com/blog/ssh-certificates ; and the usual
| man pages
| remram wrote:
| Are certificates supported by any cloud provider or Git
| hosting platform? They usually only let you add public keys
| to your account.
| pritambaral wrote:
| > by any cloud provider
|
| I don't care much for cloud providers adding unnecessary
| restrictions, so I manage my own sshd_config on VMs. I'd
| like to rent a flat, not a hotel room, thank you very much.
|
| > Git hosting platform
|
| It appears GitLab, one of my platforms of choice, does
| support this in self-hosted setups[1]. Even if they didn't,
| it'd be trivial to extend their sshd_config by myself
| anyway.
|
| 1: https://docs.gitlab.com/ee/administration/operations/ssh
| _cer...
| remram wrote:
| It is true that you are free to change the config on a VM
| manually. I included it because it's the only other
| service I could think of where you might set up SSH
| authentication.
|
| This is interesting about GitLab, thanks for the pointer.
| As you mentioned it only appears to be configurable
| instance-wide, there is no way for a user or organization
| to set it up for themselves (for example for your
| gitlab.com organization). That makes it a bad replacement
| for SyncThing'd private keys as moviuro suggested.
| stuaxo wrote:
| I like syncthing, but when I was trying it on just 3 laptops on
| the same wifi network, the speed seemed much slower than what I
| expected the maximum to be.
|
| (Anecdata I know).
| csdreamer7 wrote:
| Make sure your firewall is not blocking syncthing ports. You
| might be using one of the internet proxies instead of your full
| lan. An easy way to test is to unplug the wifi network from the
| internet.
| TonyTrapp wrote:
| Has anyone here switched between (from/to) ownCloud/NextCloud and
| Syncthing? I'm currently using ownCloud on a BananaPro and I'm
| really not happy with the performance, but also with some bugs
| that appear to be a result of this low performance. I have
| considered Syncthing before but I wonder if anyone else has made
| the same transition. I'm mostly syncing Windows desktops.
| Macha wrote:
| Syncthing is a lot narrower in scope than owncloud. It just
| syncs. If you're using owncloud just as a dropbox replacement,
| the one thing you're likely to run into is syncthing does
| nothing to facilitate sharing with untrusted devices, like
| giving a share link to your friend or something.
| TonyTrapp wrote:
| Thanks. I mostly use it for synchronizing my various machines
| and as a backup. Sharing with other people is absolutely
| secondary and something I can do without.
| genpfault wrote:
| They support arbitrary locations on external SD cards on Android
| yet?
| celsoazevedo wrote:
| SD cards work on Android 11, although I think it's because of
| the "scooped storage" change Google made, not because Syncthing
| fixed it. It works as if it was internal storage.
|
| On Android 10 and older, Syncthing can only "read" form SD
| cards (read/write works if you're rooted).
|
| https://forum.syncthing.net/t/android-11-and-syncthing-what-...
| Multicomp wrote:
| So to be clear, in /old/ versions of Android before they
| started making apps get permissions per directory, Syncthing
| can write to an arbitrary SD card folder just fine.
|
| In the newest versions of Android you give it Scoped Storage.
|
| In the intermediate, if you want 2-way sync on your phone,
| your data has to live in
| sdcard/android/media/com.nutomic.syncthingandroid/<sync
| folder1 >, <sync folder 2> which it does not need root to
| write to.
|
| But for just grabbing your pictures, Android 10 has the
| stupid limitations on external file system access that
| Syncthing has to abide by or root over.
| DanTheManPR wrote:
| Yes, _if_ you have root on your phone.
| celsoazevedo wrote:
| Not needed on Android 11+
| Archelaos wrote:
| Recently, there had been an article in a prominent German IT
| magazin about using Synthing to create backups from any type of
| machine on a Raspberry Pi. So last week I tried it out and
| quickly run into a problem that could have been dangerous. What I
| tried out was to backup my Firefox and Thunderbird profils with
| synthing. I switched on simple file versioning to be on the save
| side even in case that something is accidentally deleted. The
| problem was that, probably due to interrupted synchronisation
| sessions, Synthing could not decide whether the file version on
| the orginal or on the backup machine is the most recent one. In
| this case it transfered an old file version with some additional
| intermediate filename extension from the backup to the original
| machine. So I ended up having doublicates of my email inbox and
| other folders in my Thunderbird profil that Thunderbird
| nevertheless recognized as valid inboxes, etc. The problem was
| easily solved by searching for the intermediate filename
| extension and deleting all those files manually. This experience
| suggests to me that it is perhaps not a good idea to use Synthing
| for anything else than the deliberate exchange of manually
| managed files in folders only and specifically dedicated to
| synchronisation.
| delcaran wrote:
| You should check out the "sender only" and "receiver only"
| options: https://docs.syncthing.net/users/foldertypes.html
|
| EDIT: added link
| Archelaos wrote:
| Thank you for the hint! This looks like as if it should solve
| the issue.
| dpedu wrote:
| Syncthing is pretty awesome. It's versatile, I use it to sync
| photos from my phone to more stable storage but also on
| webservers to keep files in sync across a couple boxes.
| jchw wrote:
| Syncthing can be a little confusing if you are coming from
| something like Dropbox or WebDAV. It's not really quite like
| anything else I've used.
|
| And with that having been said, it's great. It's maybe more
| powerful than you'd expect. My NAS for example has a one-way
| synced folder from a different box for backup, while having a
| default shared folder with all of my PCs. It's also great because
| not all machines have to always be online; it can gracefully
| handle deferring stuff. For me, since my NAS is connected all the
| time, I can use it a bit like a central service, syncing between
| machines that have no overlap in uptime. Simultaneously, if the
| NAS needs to be down, it doesn't interrupt syncing between other
| nodes. And unlike centralized solutions like Dropbox, I'm not
| limited by my internet connection, since it's all local... unless
| I leave the house. In which case Syncthing appears to continue to
| stay connected, which is really handy.
| stavros wrote:
| Syncthing has worked flawlessly for me for years. Syncthing-fork
| on Android also works very well:
|
| https://play.google.com/store/apps/details?id=com.github.cat...
| wyuyva wrote:
| What's does this offer above and beyond the official version?
| NelsonMinar wrote:
| In the old days the Syncthing on Android had a pretty clumsy
| UI; Fork fixed that. But when I evaluated this in June 2021
| the main app had improved enough the fork didn't seem
| necessary any more.
| NickHoff wrote:
| Syncthing is great, I've used it for years to sync all my stuff
| and it has always been solid. I even use it on my kindle fire
| tablet (there's a Syncthing app in F-droid).
|
| My only annoyance with Syncthing is when I reinstall an OS on one
| of my machines. Let's say I'm syncing files between my main
| workstation, two laptops, a phone, and a cloud backup. Now I want
| to reinstall the OS on one of the laptops. When I install
| syncthing on that machine, it gets a new ID. I can make it join
| the sync swarm but all the other members will think it's a new
| machine and won't trust it so I have to go to each of the other
| machines and manually remove the "old" laptop and bring in the
| "new" one.
| canistel wrote:
| I keep a backup of Syncthing configs and then re-use it.
| Config.xml can even be manually edited to fix directory paths.
|
| ~/.config/syncthing in Linux, the pem files and config.xml.
| Similarly in Windows too (from
| C:\Users\<username>\AppData\Local\Syncthing). Even in Android,
| you can export and back-up. When you are switching to new
| device, copy from the back-up and then import.
| danhor wrote:
| There's an option to automagically add devices from other,
| trusted devices.
|
| I simply save the syncthing config and remove the database, so
| that the device stays the same, but doesn't retain any sync
| data
| Macha wrote:
| Yeah, my primary workstation and home server are configured
| as "introducer" devices, so they'll propagate devices to
| anything else.
| dolni wrote:
| A long time ago, I used CrashPlan for backups. They had support
| for Linux, and I had a Linux server at the time that ran ownCloud
| and some other services that needed backups. I pointed CrashPlan
| at the appropriate directories and let it roll.
|
| Fast forward several years, I needed to restore something (non-
| critical) and it was a giant pain in the ass. I was able to get
| it done, but it wasn't immediately clear that I would be able to.
| Something about CrashPlan not retaining deleted files and some of
| my paths changing or something.
|
| Anyway, I was so frustrated by the whole experience that I dumped
| CrashPlan.
|
| I run Syncthing to provide file sync services. A Raspberry Pi
| acts as the "central server" on my home network. Once a night, an
| EC2 instance launches, syncs the data up, and then performs an
| offline backup to S3. I get a report e-mailed to me every night
| so I can keep tabs on stuff.
|
| I did have to spend time building it out, but it costs something
| like $5 per month and it's been bulletproof.
|
| Lots of love for Syncthing over here. It's been a champ.
| gsich wrote:
| Had to stop using it because the devs couldn't fix the SD card
| issue on Android.
| notyourday wrote:
| Could you elaborate on the issue?
| snerbles wrote:
| Android filesystem permissions.
|
| https://github.com/syncthing/syncthing-
| android/wiki/Frequent...
|
| https://github.com/syncthing/syncthing-android/issues/1008
|
| https://github.com/syncthing/syncthing-android/issues/1366
|
| https://old.reddit.com/r/Syncthing/comments/g9es5d/alternati.
| ..
| kwhitefoot wrote:
| Could you expand on that a little?
| celsoazevedo wrote:
| In 2014 I tried to reduce my "cloud" usage and went from Dropbox
| - BitTorrent Sync (now Resilio Sync) - Syncthing. I have it
| running on all computers, tablets[0], and phones[0] I
| have/manage. In total, it keeps ~70K files and ~9TB in sync
| across devices running Android, macOS, Windows and
| Linux/Ubuntu/Raspberry Pi OS.
|
| I don't remember when was the last time I had to use a flash
| drive or a cable to move files around.
|
| ---
|
| [0] Android. iOS is too restrictive to do this.
| [deleted]
| Jamie9912 wrote:
| I was disappointed with the slow sync speed, as in sync
| 'realization' compared to say Dropbox, but it's great otherwise
| squarefoot wrote:
| It could be due to the time between filesystem checks, which
| can't be set too low to avoid excessive load. It is set by
| default to 10 seconds however, so there's room for
| improvements. The option should be called "fs watcher delay".
| sneak wrote:
| Note well that in the default config, syncthing grants remote
| code execution on your machine to the syncthing developers in the
| form of Solarwinds-style no interaction autoupdate.
|
| A compromise of the centralized release process could steal all
| of your (and everyone else's!) files by updating to a malicious
| version automatically with zero interaction.
|
| Set the syncthing binary's file ownership as root and run it as a
| normal user so it can't get overwritten, and set STNOUPGRADE=1 in
| the environment to disable this dangerous default behavior.
| bennyp101 wrote:
| Yep, I know that if I chuck stuff into my synced folders it goes
| around my machines, onto my NAS, and then onto Backblaze.
|
| Check it out every now and then to make sure, but it just kinda
| works
| qmmmur wrote:
| Is that to a B2 instance? Any chance you could let me know how
| you did that?
| leipert wrote:
| e.g. Synology supports syncing to Backblaze B2 out of the
| box.
| notyourday wrote:
| I use rclone
| hank_z wrote:
| I wonder if it can sync files from a machine in China to a
| machine in US which doesn't have great firewall implemented.
| Anyone tried this?
| thepete2 wrote:
| I know there's Syncthing lite, but is there a way to "browse"
| your synced folders (without actually syncing) and then mark them
| for syncing to the phone? I'd love it if I could do that because
| my phone's storage is obviously limited.
| spoid wrote:
| shoutout to DecSync (https://github.com/39aldo39/DecSync) to use
| Syncthing with your CalDAV/CardDAV contacts/calendars/tasks and
| sync local RSS feeds across devices. I don't really use nextcloud
| anymore for stuff that is only for myself and does not need to be
| shared
| pphysch wrote:
| Deployed Syncthing for the first time recently with Obsidian
| notes, setup was seamless on both Android mobile and Linux
| desktop (systemd), works flawlessly so far. _Enormous_ upgrade
| over (lazily) using Google Keep for synced notes.
| DanTheManPR wrote:
| Syncthing has re-framed how I think about the file systems of my
| computers. I used to occasionally plug in my phone to sync
| pictures and documents to my desktop. Now I don't even think
| about the distinction between them - the files simply exist on
| all of my computing devices, and any changes I make automatically
| propagate to the other devices.
|
| I normally do all my work on my laptop, but I went ahead and set
| up sync between my work folders and my personal desktop. Syncing
| ~1tb of work data has been no problem.
|
| I turn off my computers when I'm not home, and my phone runs
| syncthing-fork on a limited schedule, so I did at one point have
| some issues with stuff not syncing because devices weren't turned
| on. So I set up syncthing on a raspberry pi with a big external
| hard drive plugged in, and it sits next to the router just
| quietly synchronizing everything in the background.
|
| It's all very set-it-and-forget-it. Just a great piece of
| software.
| brnt wrote:
| How's the sharing situation nowadays? Tried Syncthing a few years
| back and while excellent and encountering zero problems, the UX
| for setting a share with someone else was a bit of a hassle
| compared to Resilio (which has been rock solid for the last
| decade for me as well). Sharing a single string and be good to go
| is UX you can't beat. Do you still have to go through the accept-
| peer-rigmarole with Syncthing, or have they added 'light weight'
| shares, for, you know, sharing?
| frosted-flakes wrote:
| That's not really what SyncThing is for. It's not a full
| replacement for cloud storage like Dropbox.
| remram wrote:
| I do miss the simplicity of Bittorrent Sync. I can see how this
| anonymous file sharing could be awkward to add though, as the
| app would immediately be used for illegal file downloads at
| scale.
| rayrag wrote:
| Does it support syncing local folders? For example a game put
| saves in Documents folder and I would to sync it to my Dropbox
| folder, is such thing possible?
| joombaga wrote:
| Assuming you're on Windows, you could create a hard link
| between the files or a junction between directories.
|
| https://docs.microsoft.com/en-us/windows/win32/fileio/hard-l...
| rayrag wrote:
| Thanks for pointers. I looked how to create a junction and I
| found this small app: https://schinagl.priv.at/nt/hardlinkshe
| llext/linkshellextens..., it's a GUI for creating symbolic
| links/junctions/etc. Which for me is great as I'm not an avid
| user of cli.
| delcaran wrote:
| Not by default. I did that running two instances of the
| executable and pointing them to different "home" folders.
|
| It works, but it's an hack...
| rayrag wrote:
| Check this: https://schinagl.priv.at/nt/hardlinkshellext/link
| shellextens...
| knaik94 wrote:
| I looked into syncthing and have set up a folder pair as well but
| does anyone run a separate additional server to do remote access
| of a specific file they might want to download? I am wondering if
| there's any clashes to let's say having something like nextcloud
| folders overlap with the folders used in syncthing?
|
| I am looking for an alternative that helps bridge the gap that is
| Google File Stream where even in windows it mounts as a drive but
| streams everything in the background as needed. Plex seems like a
| clunky way to go about it but I am open to any suggestions. The
| file stream feature was mainly used to stream videos in an
| uncompressed way.
|
| I mainly do local sync between android and windows and a rpi. I
| would love to have file streaming type support for the android
| and maybe an ios device.
|
| Android has a way to do file streaming built into some of the
| cloud manager apps but a diy solution might not have remote
| access.
|
| I am open to any suggestions, I understand that syncthing is
| about p2p syncing and the model is mainly for whole file copy.
| jerry1979 wrote:
| I sync my android phone to my laptop using syncthing. Works
| great, and the ignore patterns below prevent annoying access
| warnings.
|
| Folder Type: Send Only
|
| Ignore Patterns: Syncthing/**
| Pictures/.thumbnails/** Android/data Android/data/**
| Android/obb Android/obb/**
| Arkanosis wrote:
| I've been using Syncthing for years now, between ~10 Linux
| servers, Linux desktops, Linux and Windows laptops and Android
| smartphones, over LAN, wired and wireless Internet.
|
| I'm synchronizing less than 100 MiB of data, but it's changing
| all the time. I've not yet had a single issue with it. I've a few
| conflicts every week when the same file is changed in different
| locations at the same time, but Syncthing keeps all versions of a
| file until I resolve the conflicts; I can't see how it could
| handle that better.
|
| Long story short: I highly recommend this tool.
| ahnick wrote:
| What's the process for resolving conflicts?
| DanTheManPR wrote:
| You may have to manually step through the conflicts menu to
| resolve them, choosing which device's version to keep. But
| typically that won't happen unless both device's versions of
| a file change at the same time.
| metalliqaz wrote:
| Same. I sync my working docs between a Windows PC, a Windows
| laptop, and a FreeBSD NAS. It just works and that's awesome.
|
| On Windows, SyncTrayzor is the way to go.
| Andrew_nenakhov wrote:
| I like Syncthing, but it is significantly held back by Google's
| restrictions on running background apps on Android. Every working
| app has to display a persistent notification, and if you run xmpp
| client, vpn and Syncthing, on a phone with a notch, you are
| effectively down to ONE notification icon - it could be two but
| if you have more than that, they all collapse into an
| ininformative dot.
|
| Working without a vpn and xmpp is a no-go for me, so I have to
| sacrifice Syncthing, running it only from time to time.
| [deleted]
| rustyminnow wrote:
| idk if this works for all phones (I hope so!) but on my Pixel,
| you can long-press on a notification to open settings for that
| notification type. Changing it to "silent" will move it to the
| bottom of the list and no longer shows an icon in the status
| bar. You could also "hide" that notification channel entirely.
| I'm fairly sure this doesn't affect the ability to run
| background processes, but I could be mistaken
|
| EDIT: This is a more recent feature so you might need the last
| one or two versions of Android, but if you have a phone with a
| notch I bet you're good :)
| Andrew_nenakhov wrote:
| notifications silenced this way are not guaranteed to be kept
| in memory, so this roughly equals just stopping the app -
| especially on crapphones by Samsung and Xiaomi, which are the
| worst offenders of Android's background functionality.
| howaboutnope wrote:
| SyncThing and FreeFileSync make computing so much nicer for me, I
| would not ever want to go back.
|
| After several attempts, I ended up dividing the SyncThing folders
| by device storage and level of trust. Simply put, there are
| things I don't want on my tablet because it has very little
| storage space, but I still want small things like my todo list
| and my keypass database there. Then there is stuff I don't mind
| having on my work laptop, that's mostly stuff like music or
| personal things that are not sensitive, like desktop wallpapers
| (so no private stuff I would minder others at work seeing, and no
| work stuff I would mind being stolen from my other devices). Then
| there are things I want on my laptops and desktops, but not on my
| phone, because they're either too big or only useful on a
| computer, like downloaded software. That's about it, and this way
| I only have to think about what to put in those folders, but I
| don't really add new synced folders a whole lot, and when I get a
| new device, it's self-evident what folders "should" be on it.
|
| FreeFileSync is for stuff I sync manually, e.g. Firefox and
| Thunderbird profiles, htdocs folder and MySQL database. The only
| reason is that if I forget to stop Firefox, Thunderbird, or my
| webserver before starting it elsewhere -- which I should never
| do, but it can still happen -- and try to sync, I get a much
| nicer graphical interface to resolve conflicts en masse.
| dmos62 wrote:
| > my keypass database
|
| Aren't you worried that a corrupted keepass database will
| propagate across your devices and that way you'll lose it? I'm
| having trouble finding an alternative for git in these cases.
| howaboutnope wrote:
| Well, I still make regular one-way backups of everything, so
| I'm happy to cross that bridge should I encounter it. Unless
| something really freaky happens, SyncThing would probably
| just complain about the conflict until I resolve it. At least
| I don't have to worry about silent corruption, i.e. either
| the file is fine or there is something wrong and keepass
| won't open it.
| stef25 wrote:
| It's never resulted in a corrupted file but I do have plenty
| of "conflict" versions of my Keepass DB which is kind of
| annoying.
|
| And like the other guy said, Syncthing isn't for backups.
| MikusR wrote:
| You can set syncthing to keep versions of changed files
| josalhor wrote:
| > FreeFileSync
|
| I discovered it yesterday and migrated my backup system to
| FreeFileSync (well, actually I made a double backup with the
| old and new system). It is _AWESOME_. I finally feel in control
| of my backups.
| throw7 wrote:
| It's great. Big thanks to those that run relays.
|
| It's not a backup solution though... you really want to use
| actual backup software.
| notyourday wrote:
| I syncthing all devices/systems onto staging server and just
| rclone it via encrypt target onto S3-compatible remotes. It is
| about 4TB. After the initial "long" backup, the hourly runs
| take minutes
| contravariant wrote:
| The one thing I miss would be a somewhat more user friendly way
| to have file versioning along with a UI (doesn't have to be
| terribly user-friendly, my baseline is dropbox's interface)
| Multicomp wrote:
| TL;DR: give syncthing a try. first time setup is odd, then bam,
| your files magically sync anywhere with internet.
|
| I carried around my 1GB Data Traveler flash drive with great
| conviction. Then, a massive 4G SanDisk drive I would 'never fill
| up'.
|
| Then, when my 16GB Nexus 4 filled up, one day while I was MTPing
| my old pictures to my computer, someone told me about Dropbox and
| how it would sync my pictures for me. After living out of Dropbox
| for all my non-cold-storage files...I hit the free limit. But I
| was a student and didn't have money to pay per month.
|
| Skydrive gave me 25 GB. And 3 successive phones ate that like
| candy. So start paying up Mister. ugh.
|
| But then, Syncthing came along here on HN. I installed it onto my
| phone and pc for camera download. then another, and another, and
| a separate folder for sharing files with a colleague, then my
| work SAN got laggy and lost files, so syncthing to keep my work
| files local and synced between work devices to the rescue.
|
| And now, I don't pay for storage. or I do, but it just happens to
| come with word 365, but I don't bother using it.
|
| And the flash drives? I'm up to a 256GB Sandisk Pro of some kind,
| but only for 'just in case' scenarios. Syncthing handles all of
| my LAN files access needs.
| creativeembassy wrote:
| I used to use Dropbox as well, but it started freaking out when
| I had folders with over 500,000 files in them. I started
| writing scripts to zip up folders first, moved some things off
| Dropbox, but it would just take up several CPU cores when going
| through everything. I tried Syncthing, and it had zero issues
| with the number of files I had. Everything just worked. I was
| amazed that the free solution was so much better than the
| commercial solution for which I was paying $10+ each month.
| e3bc54b2 wrote:
| Over long enough timeline, free software almost always gets
| better, acheiving parity and eventually surpassing the
| proprietory service. Free software doesn't need to be
| reinvented and can keep on steadily fixing issues. It
| happened for gcc, Linux, GNU, Blender, OBS, VLC, Calibre and
| obviously Syncthing. As long as scope is clear and target is
| not moving too much, FLOSS _will_ catch it.
| tapoxi wrote:
| I've been thinking of buying a home NAS and running Syncthing as
| a replacement for Dropbox/Google Drive. Does anyone have any
| recommendations? Synology comes up a lot, I might also just use a
| Raspberry Pi.
| motiejus wrote:
| I've been doing this with an rpi+ usb3-attached ssd for years
| (kept upgrading rpis). Syncing across my laptop, phone and
| server (server backing it up nightly). Worked well for years.
|
| Now moved to a more powerful machine, but the concept is still
| the same.
|
| This setup allowed me to conveniently stop using google drive.
| nicolaslem wrote:
| You can reuse old PC parts and install TrueNAS on outdated
| hardware, it will likely be more stable than anything built
| with Raspberry Pis.
|
| I used to run Syncthing on an old mac mini with two 2.5" HDD
| running FreeNAS (previous name of TrueNAS).
| trulyme wrote:
| Not sure about mac mini, but some old machines might draw
| considerably more power than rpi. Rpi has a problem with sd
| cards, but you can make it use them in the read-only mode, in
| which case rpi becomes a very reliable computing device.
| chrisdhal wrote:
| I use unRaid and Nextcloud.
| NortySpock wrote:
| Currently running Syncthing on an RPi 4 NAS, in a docker
| container. Works great between RPi, laptop and desktop.
|
| Haven't tried it while out-and-about, though that's not really
| my use case.
|
| *RPi4 NAS has one external thumb drive for flash storage and
| two spinning rust drives, one for storage, one for backup; all
| connected via USB. Occasionally I have to plug-unplug the
| spinning rust drives if the system reboots, as the drives go to
| sleep.
| jvanderbot wrote:
| Syncthing for desktops / servers "Just works". I've used it
| between an AWS instance and my home network for years. Highly
| recommended.
|
| I finally got rid of it because of the awful mobile experience,
| but frankly I shouldn't have been counting on that anyway.
| adrusi wrote:
| On Android it works acceptably. On iOS (at least iPad) you
| have the proprietary Mobius sync port which is a bit annoying
| in that it requires a regular push notification to perform
| any syncing, but with iOS 15 focus modes you can hide it
| without breaking the app. Only a few months left if you don't
| want to run the betas.
|
| What problems were you experiencing on mobile?
| qmmmur wrote:
| Do you encrypt the data going up to Amazon or just raw dog
| it?
| Majestic121 wrote:
| Syncthing is a great tool : I've been using it for years for
| backups, picture synchronization between phone and computer, and
| as a way to share a KeePass DB. In all those years, not a single
| issue arise : you connect a device, decide what to share or not,
| and then just forget about it
| creativeembassy wrote:
| I love Syncthing. I've found some unconventional uses for it in
| syncing program settings. My fonts folder is now synced between
| my Mac, Windows and Linux computers and that works surprisingly
| well. My projects folder does a one-way archive (like rsync) for
| backup to my home NAS and an offsite VPS. I have a Streamdeck
| that I use between my Mac and Windows work machines, and I sync
| the settings on that so it operates the same way regardless of
| which machine I plug it into. I love it.
| gregmac wrote:
| One of my unconventional uses is to sync my HomeAssistant
| configuration to my desktop, where I have it in a git
| repository. I can edit manually from my desktop, and/or manage
| from the HomeAssistant UI, and either way, commit discrete,
| working changes to git once I'm ready.
| tailspin2019 wrote:
| Interesting.
|
| I've gone through so many iterations of trying to work out
| how to do something like this for Home Assistant config.
|
| Do you set it up as realtime 2-way sync? And do you just
| exclude the .git folder from the sync to HA?
| torarnv wrote:
| This. I'd like to sync my ~/development directory,
| including all the git directories, so I don't need to push
| and pull branches between machines. Is this at all
| feasible?
| tailspin2019 wrote:
| Yeah been trying to work that one out for years.
|
| I've read of other HN'ers doing this successfully
| (syncing git repos between machines) but I just seem to
| get a lot of issues - git repos often broken temporarily,
| conflicts etc.
| torarnv wrote:
| Don't know how Syncthing works, of it it would even work,
| but what if Syncthing would detect that it recurses into
| a .git folder, and then use git push to push all refs to
| the remote .git repo? That should give a transparent
| "sync" of the entire git repo and workdir for the user,
| but via the safety of git's own sync mechanism.
| frosted-flakes wrote:
| It works fine. I've been doing it for ages with no
| problems. Note that I added node_modules, etc. to the
| ignore file so that it's not always syncing thousands of
| tiny files, which always tend to take a long time for
| some reason.
| tailspin2019 wrote:
| Out of interest, what OS are you using? I'm wondering if
| some of the issues I've had in the past may have been
| macOS specific.
|
| That said, I can't recall if I tried excluding
| node_modules when I last gave this a go. Maybe I'll try
| it again.
| Fronzie wrote:
| My NAS does btrfs snapshotting of the Synchthing folder. So I
| have an up-to-date backup of my phone pictures, laptop files
| and can go back in time if I accidentally delete something.
| yobert wrote:
| I've used syncthing without problems for years, but then recently
| had an issue where it seemed to duplicate an entire folder of
| files and all their contents and whenever I would delete the
| duplicate it would bring it back.
|
| I was really annoyed with it until I realized, it's actually
| OSX's fault, because I renamed a folder from uppercase to
| lowercase, and there's no good way for syncthing to handle that
| when you have case sensitive servers syncing with insensitive
| ones. (Linux and OSX in my case.)
|
| Moral of the story is, don't rename a file only changing it's
| case-- add a dumb character or something so it can sync to more
| civilized servers properly.
| reidjs wrote:
| Had this issue with git more times than I can count. every time
| someone commits a file with the wrong casing I just want to
| quit my job and live in the woods somewhere.
| Tomte wrote:
| For iPhone and iPad: MobiusSync (which is a compatible, but non-
| free implemenation). Works great, just as SyncThing does.
| codetrotter wrote:
| Does it sync photos from the iOS camera roll? I assume not
| since the screenshots in App Store don't look like it. But if
| it does, and keeps the albums, then it would be useful to me.
| Currently I transfer my photos from my phone to an external
| drive using the iMazing app on macOS with the phone connected.
| nicolaslem wrote:
| Apple makes it surprisingly difficult to synchronize photos
| outside of their iCloud offering and I'm willing to bet that
| this is no accident.
| codetrotter wrote:
| Also worth mentioning if it makes a difference, I only need
| to be able to transfer photos off of the phone (along with
| info about which albums they belong to as mentioned), and
| don't need to sync in the other direction onto the phone.
| notyourday wrote:
| I'm about to pull the trigger of getting a mac mini just to
| get it all into the syncthing and autobackups using the
| following workflow:
|
| * iphone -> icloud
|
| * icloud -> mac
|
| * mac -> syncthing to the backup server
|
| Standard backup policies from that point on.
| mosselman wrote:
| I use the app PhotoSync with great success. I was about
| to setup my old MacBook like you described, but now I
| don't have to anymore.
| codetrotter wrote:
| In the end this might be the most simple and workable
| solution. Still feels kind of bad having to roundtrip all
| the data through the cloud when trying to get it from one
| local device to another.
| wjnc wrote:
| I am paying about EUR 20 / month for my somewhat paranoid
| backup system that includes Apple, Dropbox, Synology and
| Backblaze (and excludes depreciation). Worthwhile all things
| considering, but if Mobius Sync could sync the iOS Photo Stream
| ("further down the roadmap") I'd pay EUR 100 or more for that.
| It would make the backups in my house a lot more manageable.
| thinkling wrote:
| Dropbox is able to back up the photo roll on an iOS device,
| isn't it? Not sure if it can do it without you manually
| opening the Dropbox app, though.
| wjnc wrote:
| That's why Dropbox is an integral part of the backup scheme
| ;) but it's pricey if it's only for photosync!
| gingerlime wrote:
| Have you checked PhotoSync? does a pretty good job, although
| not sure it preserves things like slow mo videos, live photos
| or metadata...
|
| edit: seems to support live photos and exif metadata... I
| have a feeling _some_ formats or metadata might be lost
| though, but not entirely sure...
| shaan7 wrote:
| +1 PhotoSync is pretty neat, have been using it over an
| year now (to sync with Nextcloud over WebDAV).
| m-p-3 wrote:
| On of my favorite program, and it keeps getting better.
|
| You can even set an encryption key per share, and if you want to
| use an untrusted device in your nodes, you can simply not provide
| it with the encryption key and it will simply synchronize the
| encrypted payload across nodes.
|
| So if you have a shady/cheapo VPS, you can use its storage
| without worrying about the plaintext data being stored there.
| e3bc54b2 wrote:
| Syncthing is one of the "fire and forget" kind of programs. It
| takes some time to setup when doing for first time, but then you
| pretty much forget its there, and when you need it, you realise
| it has been chugging along doing its job.
|
| Two things I like about it in particular, are
|
| 1. It can and favors syncing over local network. This has a huge
| cost savings in developing world. Even when you do have massive
| bandwidth, local network sync still has more throughput and lower
| latency. My music is shared between devices and if I add a track,
| it takes less than a second to appear on my other devices.
|
| 2. You can set conditions, such that delets can be ignored. Eg. I
| have a WhatsApp message+media backup going back almost 7 years
| now, ~65GiB. But I don't need all that on my phone. So I just
| disable syncing delets on my storage. Now my phone can get away
| with ~3 GiB of whatsapp data (only because I don't delete very
| often) and I still have complete backup in case I need it.
|
| 3. Absolute rock solid stable. At $DAYJOB we use syncthing to
| sync multi dozen GiBs of new data every day, total file count in
| 10s of thousands in about ~300 directories. And in past 5 years
| we have had about 3 instances where we had something that needed
| attention, out of which 2 were not syncthing's fault.
|
| I've been using Syncthing for years and have absolutely zero
| complaints against it. Everyone with 2 or more devices should
| give it a try, just to see there are better options out there.
| divbzero wrote:
| What are the "3 instances where we had something that needed
| attention, out of which 2 were not syncthing's fault"? I'm
| curious what rare exceptions there are to watch for.
| 45ure wrote:
| >2. You can set conditions, such that delets can be ignored.
| Eg. I have a WhatsApp message+media backup going back almost 7
| years now, ~65GiB. But I don't need all that on my phone. So I
| just disable syncing delets on my storage. Now my phone can get
| away with ~3 GiB of whatsapp data (only because I don't delete
| very often) and I still have complete backup in case I need it.
|
| I would like to know, how are you using Syncthing to create
| local backups for WhatsApp database/msgstore (crypt14) and
| media files. You also mention 65GiB as your complete backup -
| have you tested it via a restore? If so, how?
| swiley wrote:
| Android used to let you administrate your devices and it was
| easy to do stuff like this.
| cesarb wrote:
| > I would like to know, how are you using Syncthing to create
| local backups for WhatsApp database/msgstore (crypt14) and
| media files.
|
| Not the parent poster, but I also do this; I just configured
| Syncthing to share the /storage/emulated/0/WhatsApp (aka
| /sdcard/WhatsApp) directory. WhatsApp stores its daily local
| backup on the Backups subdirectory of that directory, and the
| media files are all in the Media subdirectory of that
| directory.
|
| > have you tested it via a restore? If so, how?
|
| I have actually used it to move all the WhatsApp and Signal
| data from an old phone to a new phone. Just have Syncthing
| synchronize these directories (with the same path) on the new
| phone _before installing WhatsApp and Signal_ , then install
| and launch WhatsApp and Signal. When first launched, if that
| directory already contains a backup, both WhatsApp and Signal
| ask if you want to restore from that backup. Signal then asks
| you to type a long backup encryption key you should have
| written down somewhere, while WhatsApp asks its servers for
| the backup encryption key.
| srinathkrishna wrote:
| Excellent idea to sync Whatsapp. Do you use any means to read
| your Whatsapp chat history on other devices - say your laptop?
| e3bc54b2 wrote:
| There are a few applications that can read WhatsApp database.
| I used one couple years ago to run some analytics on my chat,
| but don't remember now. I'm fairly certain it is still around
| and works, though.
| Royi wrote:
| Regarding WhatsApp, How do you merge different database files
| into one coherent big database?
| NortySpock wrote:
| > 2. You can set conditions, such that delet[e]s can be
| ignored.
|
| I was evaluating Syncthing for 1-way, append-only phone photo
| backup, and my Google searches warned me away from this,
| talking about "unsupported" and "database corruption."
|
| I assume you have not experienced this, but I guess I should
| have just read the docs and trusted them, instead of forum
| posts.
|
| https://docs.syncthing.net/advanced/folder-ignoredelete.html
| egeozcan wrote:
| That scared me too. I just move what I want to delete from
| the original sync folder over to another folder, causing the
| phone data to be deleted, I even wrote a script that
| "partitions" that old data, also syncing to some random
| server after being fully encrypted (that server ofc doesn't
| have decryption keys).
| XorNot wrote:
| In practice you could just turn on Trash File Versioning
| with infinite retention too.
| stewbrew wrote:
| i had some of these problems with earlier versions. removing
| the database (after making sure everything is on sync)
| usually solved the issue. but that's years ago.
| e3bc54b2 wrote:
| I have been using that feature for, I don't even remember how
| long, but definitely since it was marked beta or something. I
| am not a heavyweight like some other in this thread, but my
| current local state is 70GB and ~30k files, global state is
| ~50GB (I need to prune WhatsApp on phone again) and ~10k
| files. I am no biggie, but its not a slouch to keep all this
| working in 7 devices, with overlapping directories shared
| between them, across 2 timezones, over local and public
| network. Zero complaints is exactly my experience. Syncthing
| Just Works(tm)
| egeozcan wrote:
| Great, because even 1 complaint there would be too many,
| wouldn't it?
| e3bc54b2 wrote:
| I agree. And that's my gripe with Dropbox, Drive,
| OneDrive (the big 3 that I was able to try, no apple
| devices). Every one of them has shit the bed on me, on
| more than one occasion. That's after burning my bandwidth
| and taking away my privacy. Syncthing just wins out every
| way (as long as problem scope is file sync, not cloud
| storage or sharing).
| orblivion wrote:
| I can come up with a couple issues related to Syncthing
| (whether or not it's about Syncthing per se).
|
| 1) I like to minimize the number of APT repositories I use.
| Debian's and Ubuntu's are well behind on Syncthing versions.
| Every so often (much more rarely than before), they have
| breaking changes. As such, my phone can no longer sync with my
| computers. Perhaps the answer is that Debian should just not be
| packaging it, because they have such a conservative policy. I'm
| considering downgrading my phone's version, or building the
| latest version from source. Worst case I may add their APT repo
| but I'm not eager.
|
| 2) At least with the version I've been using, opting for local-
| only sync isn't so straightforward. It's per-instance, not per-
| folder. This means that I have to have my home server running
| it as local-only, even for small things that I wouldn't mind
| getting synced over the wire. It means that if I want to have a
| local-only directory shared between my phone and laptop, it has
| to go through my home server first. Unless there's a better way
| I don't know about.
|
| Otherwise Syncthing has been pretty great. I like your no-
| delete thing, I did not consider that. I could definitely use
| it for photos.
| yunohn wrote:
| > Worst case I may add their APT repo but I'm not eager.
|
| What's wrong with adding a first party apt repo for syncthing
| (or other software)?
| joshu wrote:
| supply chain attacks
| jedahan wrote:
| Just checked the debian issue tracker. Not sure if there is
| supporting work that could move it along, but you are not
| alone in wanting the latest release
| https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983500
| api wrote:
| Tangent but: I sometimes feel like programs that just work like
| this don't get traction because they don't provoke as much
| discussion, search volume, etc. A complex mess that needs to be
| constantly babysat creates a cottage industry around it and
| gets discussed a lot, but something that works is like the
| quiet kid who sits in the corner and gets A's and that everyone
| ignores
|
| Kubernetes is today's poster child for this. The Hashicorp
| stack can do just about all the same things, but it just works
| so there's no cottage industry of consultants and no market for
| as-a-service implementations. Why promote something like that?
| aborsy wrote:
| Any third party app that can sync a Synchting folder on iOS (with
| other Synchting instances on desktop and android)?
___________________________________________________________________
(page generated 2021-07-23 23:02 UTC)