[HN Gopher] Open Source Continuous File Synchronization
___________________________________________________________________
Open Source Continuous File Synchronization
Author : quyleanh
Score : 400 points
Date : 2021-05-13 23:55 UTC (23 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jeltz wrote:
| Syncthing seems way too manual for my taste. I want something
| which i can easily provision with Ansible and which does not run
| a management ui. I want everything to be in config files which
| can be provisioned. It might be possible to run Syncthing like
| this but last time I checked it was poorly documented.
|
| I have also tried Unison, which is closer to my needs, but it has
| its own issues like having to be compiled with the same version
| of Ocaml on all servers.
| mab122 wrote:
| Well, everything is in config file in syncthing. and you can
| disable ui in that config file.
|
| https://docs.syncthing.net/users/config.html
| incanus77 wrote:
| Another vote for Syncthing here. Been using it about a year on
| two Macs (and occasionally a FreeBSD server / vintage machine,
| when I turn it on, for backup). Don't have to think about it and
| it keeps about 75GB of data in my ~/Sync up to date without
| effort.
|
| I started trying it last year as part of an effort to reduce my
| third-party & cloud services & effectively got rid of Dropbox.
|
| The one trickier piece of the puzzle for me was mobile device
| support. I use Documents.app on iPad & iPhone combined with SFTP
| to my machines. Two versions per machine -- LAN IP access or
| port-forwarded WAN access, depending on where I am. All setup
| with SSH keys so it's really just one tap to get into whichever
| home machine I choose.
| aborsy wrote:
| Great product. I only stopped using it because it has no iOS app.
|
| It was surprisingly able to punch through firewalls in some cases
| where Tailscale couldn't (even though both use UPnP and similar
| stuff).
| thefz wrote:
| > It was surprisingly able to punch through firewalls in some
| cases
|
| It uses external, community-run relays for that.
| BitPirate wrote:
| And hole punching
| escalt wrote:
| The cool thing about Syncthing is their public network of
| community provided relay servers https://relays.syncthing.net/
|
| When two devices can't find each other on the local network
| they look for each other on public discovery servers and then
| pick a community provided relay to connect. The data is end-to-
| end encrypted between both devices, so the relay doesn't see
| the file contents. This works out of the box on almost all
| networks. Unfortunately IPv6 doesn't seem to work with their
| public relay network though...
| BitPirate wrote:
| The use case of IPv6 relays isn't that strong to be honest.
| You don't have to deal with NAT and Syncthing can use UDP
| hole punching if the firewall doesn't completely block
| outgoing UDP traffic.
| 1vuio0pswjnm7 wrote:
| Once the devices have discovered each other is it necessary
| for the traffic to pass through syncthing servers. Once each
| end has discovered the address:port of the other, then the
| devices can connect directly. I use a different peer-to-peer
| solution, older than syncthing and open source, and it does
| not send traffic through a supernode by default, it only uses
| the supernode for discovery.
| 1over137 wrote:
| "Works on Mac OS X, Windows, Linux, FreeBSD, Solaris and
| OpenBSD."
|
| But notably missing: iOS.
| OneLeggedCat wrote:
| Please see my other comment below in this thread. But briefly,
| https://apps.apple.com/us/app/m%C3%B6bius-sync/id1539203216
| jchw wrote:
| It is for syncing files, not transferring or serving them. I
| don't think it's really possible to make an iOS app for
| Syncthing. It's not like there's a Dropbox client for iOS that
| does actual folder sync.
|
| If you are looking to fully replace Dropbox, you need a server
| to serve files off of that you can access from the Internet.
| You could do this using a NAS running Syncthing for example.
| (There's at least a couple different ways to get it working on
| a Synology NAS.) That way, you could have a CIFS or WebDAV
| share to actually browse from mobile devices.
| wereHamster wrote:
| Resilio Sync does have an iOS app. And I've used it a couple
| times, when I needed access to my documents while on the go.
| frosted-flakes wrote:
| It works great on Android. It's completely replaced my use of
| Google Photos and OneDrive (except for the rare use of
| sharing photos), and I keep local copies of everything.
| Phones these days have oodles of storage, so why not? (my
| phone has 256 GB internal storage, plus a microSD card slot
| which can more than double that).
| AshamedCaptain wrote:
| You can store "files" on iOS. Just store them on the app's
| private directory, then offer a launcher service or "file
| browser" of some sort.
|
| And apparently these days they do offer a shared storage
| space, anyway.
| OneLeggedCat wrote:
| No one seems to know about it, but there is now a 3rd party
| app: https://apps.apple.com/us/app/m%C3%B6bius-
| sync/id1539203216
|
| The developer made it with a little help from the syncthing
| team. There are comments there:
| https://forum.syncthing.net/t/mobius-sync-ios-client-now-
| in-...
|
| I've been using it for some months, and it works fine, when
| it works. I sometimes have to start it manually because iOS
| killed it in the background.
| darkwater wrote:
| Can you sync automatically photos/videos taken from the
| camera? That's my #1 for Syncthing on Android
| OneLeggedCat wrote:
| Unfortunately I don't think so. Apple walls off those
| photos very effectively. I gave up trying to sync those,
| and instead use an app called Copy That to get them off
| the phone and onto a SAMBA share on my network.
| vinw wrote:
| I'm just trying it out and it seems not, iOS doesn't
| expose the photos folder to the app.
|
| My workaround for this has been to install iCloud on a
| Windows machine which then downloads all the photos from
| my phone, and Syncthing on Windows monitors the iCloud
| folder and shares it with a Linux machine that does my
| backups. A bit convoluted but it works.
| darkwater wrote:
| You can probably use icloudpd [1] to consolidate that on
| just a Linux machine, as I'm (more or less) doing now.
| I'm saying more or less because we ran out of iCloud
| space, I still don't want to delte forever old photos
| from iCloud but I should probably do it.
|
| [1] https://github.com/icloud-photos-
| downloader/icloud_photos_do...
| vinw wrote:
| Thanks, looks like I've already starred this project on
| github, but I can't remember why I'm not using it... I
| think maybe the auth token expires every few weeks and
| you need some manual intervention to get it working
| again.
| karlicoss wrote:
| I've never had an iOS device, but does it even have a concept
| of a 'file system' exposed to the user? Presumably if the app
| existed, you could only view the files inside the 'syncthing'
| app, but other apps won't be able to read its data?
| vineyardmike wrote:
| iOS does in-fact have a sort-of file system. It's heavily
| tied to iCloud and I don't think it's really meant for sync
| thing style files. But it's not like drop box is much
| different in this regard.
| lostapathy wrote:
| How do you thinks Dropbox gets by on iOS?
| dustyharddrive wrote:
| If Syncthing ever supported iOS, it would likely be as a
| "file provider"
| (https://developer.apple.com/documentation/fileprovider).
| tomcooks wrote:
| I'm sure that with funding they'll have resources to focus on
| that
| christiansakai wrote:
| I've been thinking to create a desktop app for family uses. Like,
| family photo albums, notes, precious documents, and more
| ambitious projects, maybe passwords? My relatively big families
| are scattered throughout a few different countries and homes.
|
| I am a fullstack web dev so creating web app or electron app is
| right up my alley, but not in distributed databases or peer to
| peer stuffs.
|
| I was looking at OrbitDB the other day, but I'm curious can it be
| programmed to just connect peer to peer to my family computers ip
| addresses?
|
| Also, as far as I know, I can't upload blob data to OrbitDB, and
| has to use IPFS instead. Can this syncthing solves this problem?
|
| Or more generally, is there any OSS software that already solves
| this problem? I'd love to learn how to build it.
|
| The whole system doesn't need to be high performance, it is just
| for family uses.
| rakoo wrote:
| It sounds like you want Nextcloud
| (https://nextcloud.com/athome/) or cozycloud
| (https://cozy.io/en/features/). Both are servers you can self-
| host, and mobile/desktop apps to make it more integrated.
| elmimmo wrote:
| It does not support [0] (i.e. loses) extended attributes even
| between systems that share support for them, which in the case of
| macOS means no support for Finder tags, custom icons, legacy Mac
| fonts and countless other Mac features, maybe niche to most, but
| still important to those that make use of them. AFAIK it is not
| even in the roadmap.
|
| [0] https://docs.syncthing.net/users/faq.html#what-things-are-
| sy...
| Fiahil wrote:
| I'd love to use Syncthing, but they don't have an iOS app for
| iPad.
| rspoerri wrote:
| Try mobius sync
| meonkeys wrote:
| Syncthing's most awesome features are speed, decentralization,
| security, reliability and easy firewall traversal.
|
| I've heard several mentions of it being used in place of services
| like Dropbox, OneDrive, Drive, Box. For the features above, I
| agree.
|
| What Syncthing is missing is usability for folks that use the
| proprietary services because they are easy to set up and maintain
| and share files from. For that I recommend Nextcloud. It is
| centralized, so perhaps a different use case. But certainly a
| more complete replacement if you (or people you support) are used
| to Dropbox.
|
| Also, just curious, but why is this on hacker news today? Was
| there a new announcement or release for Syncthing?
| frosted-flakes wrote:
| It was mentioned in a comment in another thread yesterday,
| which may be why.
| juskrey wrote:
| After testing for a while, moved to commercial Resilio Sync (they
| offer one-time payment lifetime licenses for personal/family
| use).
|
| More tightly integrated, much faster, lighter and polished app
| overall, has good iOS client.
| srijan4 wrote:
| I've been using syncthing for more than 3 years, and it's been
| just amazing. It replaced dropbox for me completely, and provides
| a better experience in all aspects - speed, stability, low power
| usage on mobile.
|
| One of the few projects I'm sponsoring on Github.
| srijan4 wrote:
| Ah, I never considered or used Dropbox as a backup tool, only
| for synchronization. Backup is separate.
|
| For simple undo operations (recover a recently deleted file),
| Syncthing's version history is good enough.
| cowmix wrote:
| Ditto. Sometimes I get some CPU spikes but overall it has been
| amazing.
| 6ak74rfy wrote:
| Syncthing is more of a synchronization tool and less of a
| backup tool. For e.g., if I screwed up a file on one machine,
| that will automatically be replicated to other machines.
| Dropbox, on the other hand, "can" be used as a backup tool
| because I can go and recover an older version if I want.
|
| I do know that Syncthing has a way to keep older file versions
| around for awhile but I find that a bit cumbersome to use,
| especially during restore.
|
| So, I am curious to understand more on how you use Syncthing as
| a replacement to Dropbox. (I'd love to do the same but don't
| for reasons I wrote above.)
| Macha wrote:
| Turn on staggered file versioning.
|
| If you need to go back to an older version, pick it in
| .stversions .
|
| If anything, this seems simpler than dropbox for me. Does
| Dropbox still make you use the web UI to restore old
| versions?
| kuroguro wrote:
| Yep, still have to use the web UI. Just switched to
| syncthing from dropbox a month ago, been awesome so far!
| js8 wrote:
| I don't see why should Syncthing be a complete replacement
| for Dropbox (Unix principles blah blah blah). It's probably
| not hard to set up a periodic backup (for example, Borg) on
| one of the replicated machines. At home, I replicate
| everything on my phone and netbook to a (bigger) desktop
| machine, so I could just add a periodic backup job there.
| gnufied wrote:
| I love syncthing, but one thing which I think it does not quite
| do right is - on my Windows desktop each time desktop is resumed
| from sleep or reboot, it performs full sync by scanning
| everything. It takes all the CPU for awhile until syncthing
| settles down. I am not sure if this problem is Windows specific
| though.
| ElCapitanMarkla wrote:
| I had this same issue, ended up uninstalling it all because I
| couldn't figure how to address it.
| knz_ wrote:
| Even worse if you're trying to sync a file that might be
| actively changed while syncing - syncthing will reread the
| entire file as soon as only a few kilobytes are changed. On a
| file >10GiB this quickly leads to a death spiral of IO and CPU
| starvation.
| mekster wrote:
| That's why I run a custom backup script every 5 minutes to
| simply rsync my data over instead of realtime syncing.
| karlicoss wrote:
| There is a restartOnWakeup which is true by default, seems like
| it's the one?
|
| There is also a setLowPriority setting which I believe should
| prevent it from taking all CPU, but not sure since never had
| such problem on Linux.
|
| https://docs.syncthing.net/users/config.html
| ComputerGuru wrote:
| Low priority stops it from consuming all cpu when there is
| contention for the cpu, but not when the pc is otherwise
| idle.
| TuringTest wrote:
| If the PC is otherwise idle, why is it a problem that a
| process takes all CPU if it needs it?
| TeMPOraL wrote:
| 'gnufied mentioned a Windows desktop so the biggest
| problem would be noise, but what about portable Windows
| devices?
|
| My secondary computer is a Windows 2-in-1. It's
| frequently running on battery power, and during that
| time, I often wake it up, check something, and put it
| back to sleep - all in a minute or less, repeated a dozen
| times an hour. If each time I turn the screen on
| Syncthing is going to run at full throttle for a minute
| or more, it'll drastically cut the effective work-on-
| battery time.
|
| (This article popping up on HN reminded me that I wanted
| to check Syncthing out. But what 'gnufied described is a
| total deal breaker for me, so I'm not going to bother for
| now.)
| ComputerGuru wrote:
| Battery consumption, fan noise, heat, etc.
| Sophistifunk wrote:
| My guess: Sleep/resume breaks continuity across file change
| watchers, and it's a requirement to scan everything in case
| it's been modified.
| sir_eliah wrote:
| For anyone interested in simple, zero-configuration alternative
| for local networks, check dragit [0].
|
| [0] https://github.com/sireliah/dragit
|
| * Note *: author here
| wazoox wrote:
| Can syncthing scale up to a couple petabytes? So far I'm using
| rsync, that scales, but is sloooow.
| rpdillon wrote:
| 100th percentile shows about 325TB[0], so while it may be able
| to and users may be doing that, they don't seem to be reporting
| statistics publicly.
|
| [0]: https://data.syncthing.net/
| eximius wrote:
| I think my only complaint about syncthing is really a nixos
| problem where the config keeps getting deleted and my devices
| disconnect.
| rubin55 wrote:
| Absolutely LOVE syncthing!
|
| User for about 3 years, after I left Dropbox (they incorrectly
| tagged a shared file as copyright-infringing - it was a bit too
| invasive for comfort).
|
| I sync about 8TiB between 10+ devices, ranging from tablets and
| TV to desktops and laptops.
|
| Since relatively recently, they also solved case (in)sensitive
| file names, which used to be a pita when changing file name
| casing only, but that's all history now. Big fan!
| alpaca128 wrote:
| > they incorrectly tagged a shared file as copyright-infringing
| - it was a bit too invasive for comfort
|
| OneDrive also opens all uploaded pictures and automatically
| tags them based on the visible content. As long as an online
| service doesn't offer end-to-end encryption it's a safe bet
| that they'll read all your files (famously leading to the
| Google Drive-based DDOS attack scheme[0]).
|
| [0] http://chr13.com/2014/03/10/using-google-to-ddos-any-
| website...
| codeflo wrote:
| It's great that many HNers like Syncthing, but personally, I
| seriously tried it maybe two years ago and didn't have a great
| experience.
|
| I wanted to sync my personal files between three computers, two
| desktops and a notebook, which don't always see each other. At
| first, this worked great. Obviously, there were conflicts when I
| accidentally edited something in two places without syncing up.
| What I didn't figure out was how to permanently resolve those. I
| ran into endless cycles were I had a conflict between two
| systems, resolved that, but the third one hadn't seen the
| conflict resolution, which produced another conflict, which I
| resolved, which the first system hadn't seen yet, ad infinitum.
| At least, that's what it looked like. Among other errors, files
| long since deleted on _all_ systems would still show up as
| conflicts.
|
| To be fair, I didn't actually lose any data, and it's very
| possible I did something wrong that would be obvious to other
| people. But to me, it wasn't at all clear what and why, which
| scared me enough to move away from this setup. I don't want to
| take a lot of chances with my data.
| [deleted]
| Siira wrote:
| What do you use now?
| js8 wrote:
| Not really sure if this is required for Syncthing but I would
| (cautiously) set it up so that there are no cycles between the
| shares.
|
| So if you have 3 machines that you need to sync, designate one
| as a hub which syncs with the other two, and the other two do
| not see each other. If you absolutely need to transfer data
| between two others as well, set up another completely
| independent share between them (a different directory on both
| machines).
|
| I am not sure but fully automated replication setup with cycles
| might be impossible due to
| https://en.wikipedia.org/wiki/CAP_theorem.
|
| (Addendum: I think actually what is needed is even stronger -
| you should make sure before editing files that the designated
| "central hub" node is available and has been synced.)
| austincheney wrote:
| If you would like to help with an alternative I plan to add
| syncing as a new feature to my networked file system
| application soon. The application is already cross OS and
| feature a familiar GUI and fully recursive directory hashing
| with ignore lists (ignore lists are only available using
| terminal commands at this time).
|
| It seems like synchronization is the next logic step and I
| could really use feedback and testing from interested users.
| The application is available at
| https://github.com/prettydiff/share-file-systems
|
| I would love for syncing to occur upon changes in real time as
| opposed to timed intervals like a cron job.
| frosted-flakes wrote:
| I love Syncthing. I have abysmally slow home internet, so any
| cloud storage service simply takes too long to sync, and trying
| to do the first sync of 150 GB takes many weeks. Besides, none of
| these services (dropbox, onedrive) support ignore files, so
| they're _constantly_ syncing node_modules folders, etc.
|
| Instead, I have Syncthing set to sync a half-dozen folders
| (Pictures, desktop, documents, audiobooks, etc.) between my
| phone, laptop, and desktop over the local network. In total, I
| sync more than 150 GB of files and it works almost instantly and
| seamlessly. I use the SyncTrayzor GUI for Windows and the
| Syncthing app for Android. The Android app occasionally needs to
| be restarted manually, but that's the only problem I've had with
| it.
|
| Importantly, I keep local copies of everything on all of my
| devices, including my phone (256 GB storage plus a microSD card
| slot which I have yet to need). Lack of support for Syncthing on
| iPhones is one major reason I stick with Android, even though
| there's a lot to like about the platform. Once it gets too big
| I'll move a bunch of old stuff to backup/archive drives, but I
| don't expect that to happen for a while.
|
| Now, I use Google Photos to selectively share photos with others,
| and that's it.
| calt wrote:
| I use mobiussync on iOS. It's not actually FOSS, but given the
| absence of something like F-Droid that's the way the iOS
| ecosystem works.
| Tomte wrote:
| It works great. And what I really like: when you have already
| bought the pro version and go to the buying dialog again,
| they tell you that paying them once is enough and to maybe
| donate to the SyncThing project itself.
| rakshazi wrote:
| Try syncthing-fork [0] on android, it has more features and
| automated scheduling of sync
|
| [0]
| https://f-droid.org/packages/com.github.catfriend1.syncthing...
| e3bc54b2 wrote:
| I have observed that Syncthing-fork improvements eventually
| end up in official repo. Sometimes it takes few months, but
| it happens.
| relyks wrote:
| Dropbox has the ability to ignore files/folders:
| https://help.dropbox.com/files-folders/restore-delete/ignore...
| It's not exactly the most user-friendly method, and you have to
| run a script to do that consistently for generated
| files/folders like those from npm
| frosted-flakes wrote:
| That sounds quite cumbersome and inflexible. Why can't they
| just have a .dropboxignore file?
|
| Even so, Dropbox/OneDrive have other problems, such as only
| syncing a single folder and not syncing over the local
| network. Plus it's expensive.
| rrdharan wrote:
| Dropbox syncs over the local network.
| figassis wrote:
| So they advertise, but I'm a long time dropbox user and
| every time I needed to sync to a new device in my local
| network, with all firewalls disabled and lan sync
| enabled...crickets.
| frosted-flakes wrote:
| That's good to hear. I guess I assumed it didn't because
| I'm pretty sure OneDrive doesn't, and that's mostly what
| I used.
| cmckn wrote:
| > Why can't they just have a .dropboxignore file?
|
| Seriously though, this can't possibly be THAT much more
| difficult than fragile attributes on a filesystem node. If
| someone at Dropbox is reading: please, I beg of you.
| dmacvicar wrote:
| Synthing Lite provides Android with a remote Folder
| implementation that syncs on demand, and is very useful for
| things like password files (KeePass).
|
| https://github.com/syncthing/syncthing-lite
|
| Unfortunately, it is not maintained, even if it works and I use
| it daily.
| FrontAid wrote:
| > Besides, none of these services (dropbox, onedrive) support
| ignore files, so they're constantly syncing node_modules
| folders, etc.
|
| Tresorit has that feature and it works great. Aside from data
| encryption, it was _the_ main reason why I switched from
| Dropbox.
|
| https://support.tresorit.com/hc/en-us/articles/217103697-Exc...
| claytongulick wrote:
| For anyone interested in doing this programmatically in
| javascript, I wrote bit-sync [1], a js implementationof rsync.
|
| It's transport agnostic, so WebRTC or websockets or whatever is
| fine.
|
| Works in browser or Node.
|
| https://github.com/claytongulick/bit-sync
| orena wrote:
| Amazing software, great development team. First I replaced
| Dropbox with resilio and a year ago replaced resilio with
| synchthing
| rcthompson wrote:
| The best thing I can say about Syncthing is that after having set
| it up, I almost never have to think about it. It just works, and
| my files are always where I need them to be.
| renonce wrote:
| I wish I don't have to add devices and authorize every folder for
| each pair of devices manually. I like it but Resilio Sync is just
| much easier to use.
| Macha wrote:
| You can set a device as an introducer and it will push its list
| of devices to the new device.
| tut-urut-utut wrote:
| I love syncthing. The best sync tool so far. I use it to sync
| between my phone, laptop and a VM. However, there are some
| annoying things:
|
| - Frequently I get sync conflicts, although I always edit file
| only on one host.
|
| - Sync is not triggered automatically, but based on the schedule
| interval. If I need synced files immediately, I need to open the
| control UI and force rescan of files.
| thefz wrote:
| > Sync is not triggered automatically, but based on the
| schedule interval
|
| Pretty sure there's a "Watch for changes" toggle in the UI.
| asicsp wrote:
| From https://tonsky.me/blog/syncthing/:
|
| > _It's amazing how great computer products can be when they
| don't need to deal with corporate bullshit, don't have to promote
| a brand or to sell its users._
| tok1 wrote:
| Absolutely love syncthing, and migrated to it after using Seafile
| for several years. I also have quite some personal/business
| experience with iCloud and OneDrive, and syncthing is (mostly)
| superior.
|
| My wishlist for syncthing though, in prioritized order:
|
| 1. support for symlinks, to be able to sync files/folders that
| live outside of syncthing dirs. Some files I'd like to sync are
| in fixed locations that I cannot or don't want to change, and
| Seafile allowed me to simply symlink them into an already synced
| dir. The counter-argument in GitHub discussions is having to
| distinguish between syncing symlink destinations vs. the symlink
| ,,as symlink". For my situation that would be overthinking (as
| I'd never have the second use case) but I understand the
| hesitation, sadly.
|
| 2. provide a smart versioning/rollback solution leveraging ZFS
| snapshots, as my file server as participating node already
| creates rolling snapshots
|
| 3. (nice to have) mark certain directories as ,,sparse", to sync
| data only when really accessed. Similar to what OneDrive provides
| mikro2nd wrote:
| I'd respectfully suggest that HN is not the best forum for
| expressing those wishes, but using Syncthing's issue-tracker
| and/or fora/mailing lists (whatever form of community/dev
| discussion they use - haven't looked for it) would be more
| likely to get a response.
| Multicomp wrote:
| The recently release 1.16.1 supports the first beta of untrusted
| nodes, aka nodes that will sync data but don't know what it is,
| as it is encrypted with a passphrase you set.
|
| so drop off an old machine at your parents / buddies / exs house,
| give it internet connection and cron some updates + reboots +
| power on after an outage and boom, if your house and all your
| devices burn down, you can get your files via decrypting with
| that password you saved to your email / password manager.
|
| I love Syncthing and it's replaced Dropbox and Microsoft 365 for
| me. ok i keep the latter for online dictation when I'm not near a
| pc with dragon installed on it, but still, I don't need to trust
| MS with my data in their cloud.
| Bilal_io wrote:
| Ex's house? I'll go do that.
| remram wrote:
| Reminder that sync is not backup. If you delete a file, all
| your other copies will happily delete it too, giving you no way
| to recover.
| pokive wrote:
| > give it internet connection and cron some updates + reboots +
| power on after an outage and boom
|
| wtf does this even mean?
| tapland wrote:
| Cron jobs are time scheduled tasks.
|
| So schedule auto updating, automatic restart and login so
| that it always get back to syncing so that you don't have to
| actively maintain anything.
| pokive wrote:
| Thanks for explaining
| kuroguro wrote:
| They finally released it? Been waiting for this! Gonna hook it
| up to one of my VPSes.
| lamontcg wrote:
| I used Resilio fairly unhappily for years and their Mac client
| would incessantly lock up and they've never fixed it.
|
| Switched to Syncthing something like 2 years ago and it has been
| solid for me across Mac and Linux so far.
|
| And I do moderately crazy things like using it to sync git repos
| to create replicas across multiple machines instead of manually
| using git across 5 machines and a hundred or so repos.
|
| Only complaint is that initial synchronization is really slow.
| diarrhea wrote:
| > And I do moderately crazy things like using it to sync git
| repos to create replicas across multiple machines instead of
| manually using git across 5 machines and a hundred or so repos.
|
| You could set up your own git remote for this, Gitea is all the
| rage I've heard. But git remotes can be implemented many ways,
| no need for a dedicated server running.
| karlicoss wrote:
| It's much easier in syncthing, you don't have to remember to
| commit/push etc. Works really well as long as you don't
| change the repos too often so index conflicts don't happen
| (but even if they do, worst case you just lose index state)
| lamontcg wrote:
| I push everything to remotes as well, either GitHub, GitLab
| or just over ssh to a private server.
| p0d wrote:
| How do you monitor your syncthing? I am also a fan. It occured to
| me however if my headless server in the roofspace running
| syncthing goes away I don't know about it. Is there a way of
| doing notifications in syncthing which doesn't require an
| external monitoring system?
| killingtime74 wrote:
| Do you mean like metrics like Prometheus?
| https://github.com/f100024/syncthing_exporter. The http
| interface can be exposed on the network.
| p0d wrote:
| After posting I realised I was just being lazy. Yes, I need
| to get some sort of monitoring/graphing system going on. I
| have used Prometheus etc at work.
| solarkraft wrote:
| I love the idea of Syncthing (p2p, hell yes), but besides wishing
| for a "streaming"/"on demand" mode (only downloading a file from
| a node when you need it, basically Syncthing lite but in the
| official project, eventually evolving into a mixed mode with
| local and remote files besides each other) it just ... doesn't
| work that well for me. Initially it kind of works but doesn't
| saturate the network connection, then after a while it just
| doesn't do anything anymore. The web interface doesn't provide
| much information about what's happening.
|
| I tried it with multiple different devices and know someone who
| has the same problem. At the same time I know many people it
| works flawlessly for. Has anyone else had a similar experience?
| TuringTest wrote:
| I've had the same problem with Syncthing between Android mobile
| and desktop. Either devices don't see each other, or if I
| manage to sync one folder it synchronizes once and it never
| does anything again.
|
| I've switched to KDE Connect for manually transferring files
| from the phone (it can't be scheduled on Windows as far as I'm
| aware, but at least it works every time).
| quyleanh wrote:
| The most thing I love Syncthing is its selected folder sync
| function, which others services don't offer.
|
| Other advantages are opensource, privacy, lightweight, fast.
| Happy with it for months now.
| thefz wrote:
| Been a user for years, I also run a Discovery and a Relay server
| on two small VPS.
|
| Syncthing is great and has improved a lot over the years. It only
| hiccups when it detects changes in VeraCrypt containers which are
| still mounted and thus inaccessible to read.
| podiki wrote:
| Been really liking Syncthing, mostly moved off of Dropbox to it.
| I've noticed recently that Dropbox is very slow at resyncing when
| waking from sleep (on Linux, at least), taking easily 5-10
| minutes or something much longer before it realizing it is out of
| sync. On the other hand Syncthing is about as quick as anything
| once network connection is restored. It is syncing on a local
| network, but Dropbox used to be as quick, too. Another reason I'm
| glad I've switched.
| playpause wrote:
| I really wanted to like Syncthing, but it can't handle git repos
| inside it - the moment you commit anything or switch branch,
| Syncthing irresolvably borks the .git folder on all machines.
| Dropbox handles git repos inside it perfectly, syncing all
| uncommitted changes as well as commit/branch state without any
| issues, ever.
| grandchild wrote:
| Interesting, I sync many Git repos, and they all work just
| fine. I even shared some with other people (to co-author). It
| all worked splendidly.
| playpause wrote:
| Well that is weird. When googling for solutions, all I found
| was people saying it would never work, and smartass comments
| about using GitHub instead. So I gave up. Maybe I'll give it
| another try. It was a very simple setup though, I don't know
| what I could have been doing wrong.
| willemmerson wrote:
| If only there was some kind of service that could sync git
| repos properly, some kind of 'hub' that could also deal with
| conflicts
| playpause wrote:
| Yes very good. But GitHub doesn't magically sync uncommitted
| working tree state between your computers.
| JohnBerea wrote:
| My projects folder has a dozen or so git repos in it, and I've
| been syncing it for over a year without issues. Between two
| other computers.
|
| It doesn't matter which one I push/pull/commit/whatever from,
| the others mirror the operation.
| karlicoss wrote:
| Hmm, I'm using git in syncthing folders all the time, and it
| works great -- how does borking manifest?
| em-bee wrote:
| i love syncthing. but it is difficult to make backups from my
| phone where i need the ability to delete files that take up to
| much space, but that i want to keep on the backup.
|
| there is a function to ignore deletes, but it is hidden in the
| advanced settings which are difficult to access (some of those
| advanced settings should be hidden, but ignore delete is not one
| of them i think)
|
| another thing i miss is an easy ability to see if all files have
| been sent. currently the status only shows if all files from the
| remote device are received, but i care more if all local files
| are sent (and thus backed up)
| thunderbong wrote:
| When you set up a folder to sync, you can choose the 'Folder
| Type' under the 'Advanced' tab. This sets up that folder to do
| bi-directional sync or not.
| em-bee wrote:
| that does not matter, backup should sync files from the phone
| to the backup-device. it should not sync deletes from the
| phone to the backup-device. but uni-directional still syncs
| both, only ignoring delete on the backup-device prevents
| syncing deletes.
|
| the point is that it's complicated and the 'ignore delete' is
| easily missed. also because it is not shown in the status. so
| every time i have to wonder, did i set this up correctly?
|
| and i have to check the status on both the phone and the
| backup-device to make sure all is right.
|
| when i want to delete something on the phone i have to check
| on the backup-device to make sure that the file is backed up.
| because the phones status does not tell me that. it only
| tells me whether it downloaded everything from other devices
| which is less usefull, i can see which files are on the phone
| anyways. i can't see what's been uploaded to the backup-
| device unless i go look.
|
| just the other day i realized that for a new folder i had not
| set up ignore-delete. luckily, because i am paranoid, i made
| a hard-link duplicate of the backup before starting to delete
| files on the phone.
| seniorivn wrote:
| there is an alternative way, make the sync regular
| bidirectional but on server have a Cron job that
| archives(moves to a different directory) backup files that
| are no longer needed on local devices
| karlicoss wrote:
| Yeah, I think a cron job to move/rsync is the most robust
| method.
| em-bee wrote:
| doesn't work for me, because the cron job doesn't know
| when i no longer need the file on the phone.
|
| if i sync deletes from the backup device to the phone
| then a cron job to move files would potentially cause
| files to be removed from the phone before i am done with
| them.
|
| rsync would cause the files to be duplicated (although i
| could still hardlink using rsync)
| karlicoss wrote:
| Hmm, what if you use a special directory where you could
| move the files on your phone once you don't need them?
| And then the cron job would move stuff from that
| directory
|
| ah for rsync I meant with --remove-source-files flag --
| it's just sometimes rsync is easier to use than move,
| depending on the hierarchy of your files
| em-bee wrote:
| from the phone i want the files synced to the backup as
| soon as they are created. if i only sync those that i
| don't need then the once i do need never get synced.
|
| unles you mean to sync first, then move files i want to
| keep on the backup only out of the way and rsync
| --remove-source-files those files again.
|
| the problem with that is that when i do remove files on
| the phone, then it's because i need the space. at which
| point i don't want to wait.
|
| syncthing does what i want. that is not the problem. them
| problem is only that it is easy to forget to set up
| ignore-delete, and that it's hard to monitor the sync
| status of the backup from the phone.
|
| i appreciate the suggestions, but they don't really do
| much to help my case. what is really needed is an
| improvement of the syncthing UI and in particular a
| redesign of the way the status is handled.
|
| for my backup maybe i'll set up an rsync with hard-links
| to create a second version of all files which protects
| them from accidental deletion.
| pcnix wrote:
| How I do this: * Create a unidirectional push only sync
| between the phone and a remote * Remote has a cron that
| syncs to backblaze regularly, and deletes from the server
| (this does not affect the phone) * Delete from the phone
| whenever I want
| JeremyNT wrote:
| I think your complaint is that this is not the simplest use
| case for syncthing? These complaints seem subjective to me
|
| It has all the features you desire, but the UX/UI is
| streamlined for synchronization, not archiving.
| unhammer wrote:
| Hm, I just move them on my laptop from ~/Phone/DCIM/ to
| ~/Pictures/fromPhone/ (where only the first dir is synced with
| the phone). I can't go through stuff on my phone and move from
| there, but I find it faster to go through pictures and videos
| on my laptop anyway ...
| 2Gkashmiri wrote:
| I used bittorent sync right from the day it launched. Then
| stopped after they moved to paid tier with forced obsolescence of
| free version features.
|
| I was pointed to syncthing and never have had any serious
| complaints. Good job
| stuaxo wrote:
| I like it, but have no idea why it is so slow.
|
| I was syncing files on 3 laptops on the same WIFI and the speed
| was just abysmal.
|
| That was about 2 or 3 years ago so maybe it's better now.
| karlicoss wrote:
| It's been working very well for me, I completely replaced
| Dropbox, and on the VPS/phones [0] as well.
|
| The only slow bit I know of is when it's initially syncing too
| many files (e.g. tens of thousands). Even if they are tiny, it
| takes a while. I suspect it has something to do with the index
| serialization, but also there is a maxConcurrentWrites setting
| which is 2 by default -- perhaps on an SSD this is an
| unreasonably low default? Not sure.
|
| Is it possible that you didn't have inotify enabled
| (fsWatcherEnabled)? There is also fsWatcherDelayS [0] setting
| (10 by default), which groups changes to avoid excessive sync
| operations, so it might be percieved as latency.
|
| [0] https://github.com/Catfriend1/syncthing-android#readme -- I
| recommend the 'fork' over the original app, it's got some nice
| extras. Although seems that Google doesn't want it on play
| store https://github.com/Catfriend1/syncthing-
| android/issues/704
|
| [1]
| https://docs.syncthing.net/users/config.html#fswatcherdelays
| qbasic_forever wrote:
| Three laptops on the same wifi router all talking to each other
| at once can be very problematic, especially if it's an
| inexpensive consumer router with one radio. Only one device can
| shout into the wifi ether at once and the protocol has backoffs
| to make sure multiple devices don't clobber each other by
| talking at the same time. Add on top of that other backoff
| strategies in higher level protocols like in TCP and you can
| have abysmally slow connections for anything.
| mkl wrote:
| I set up Syncthing for my dad a few weeks ago, between his PC
| and his laptop. At first the sync was incredibly slow, and it
| turned out Windows on both machines though the local network
| was a public one, so they weren't talking directly to each
| other but through a relay out on the internet. I fixed that and
| it got 15-20 times faster. Maybe you have the same problem?
___________________________________________________________________
(page generated 2021-05-14 23:02 UTC)