Edit: Thanks for all the suggestions. I’m going to try sticking with syncthing and try the fork of the UI and see if that keeps everything working.
–
I want to sync files between my linux PC and Android phones (mostly for Obsidian notes).
Can anyone recommend a good real-time sync?
I’ve been trying syncthing, but despite turning off battery optimization for the app, it rarely sees the phone as connected. I don’t want to have to remember to check syncthing every time I edit a note.
I use resilio for syncing between PCs but it looks like it has a high battery usage on the phone, as if it is frequently polling for changes.
I use FolderSync for occasional scheduled syncs (e.g. updating my MP3s from the server to my phone), but a scheduled sync either is frequent enough to affect battery or it risks sync conflicts.
Cloud services such as OneDrive, Dropbox and Google Drive don’t show up as big battery drains, so I assume that they use change notifications from the OS instead.
Are there any real-time 2-way sync apps for phone that don’t have big battery drain and are not for cloud providers?
Do you use it on a phone too? I did find it tricky to set up (more options than I really need, and the phone app settings don’t really work unless you select “Web UI”, which is really strange), but I didn’t mind the setup if I could then leave it alone and it works. Ideally I want to set this up on other family phones, so I can update notes and they appear everywhere.
For me that can be more specific. If I select the offending notification (either in the settings for notifications history or directly in the notification area) I can disable that specific type of notification. For instance deals from Amazon or eBay or something like connection established from tailscale.
Are you sure that there are different type of notification for Syncthing? Have you disable the persistent one and still received a failed sync one (or something else)?
Good to know. If I can get it working reliably, it will be worth sticking with. Someone suggested it might just not be auto-starting on reboots. I’m trying the fork of the UI on f-droid to see if it helps.
I prefer Syncthing-fork for some more straightforward configuration. Mainly the three button options equating to “follow the run conditions, damnit”, “run damnit”, and “stop damnit”
I have a different app for photos, but SyncThing on my phone, and on my desktop, and again on one of my home servers, do most of the download and data syncing.
Occasionally I’ll have to manually run SyncThing; I’m not certain that Android is reliably starting it after reboots, but for the most part it just does it’s thing really reliably. There is a lag; it can take a few minutes for changes to sync - it’s not immediate. For me, this isn’t a problem, and I’d rather that than a battery suck, so I haven’t messed with it.
I could live with a few minutes, but it’s showig as offline for days. Maybe it is failing after a reboot. At least that would be a known situation to watch for.
In my experience syncthing is always a bit like that using the default discovery settings.
I use a hub & spoke set up now. Instead of A, B, and C all connecting to each other directly, they only connect to D. I also input the address for D specifically instead of using discovery servers.
Honestly, I think Android is fucked for debugging stuff like this. I installed a program on mine and my wife’s phones - different makes & models - and configured them exactly the same, including the app settings in the OS. Mine works perfectly and barely shows up in battery use, near the bottom. Her’s drains her battery even when she’s not using it, regularly running at 50% of total battery consumption.
With Android YMMV is the rule, rather than the exception. There’s just too many variables.
Yeah, phone to laptop, and I recently synced all backups and files from an old phone to a new one, too. Once you have the computer setup, you can basically connect phones by reading its QR code.
If the official Syncthing Android app is giving you a hard time, maybe try Syncthing-fork? IIRC that’s only the daemon and web GUI wrapped as an app. But I’ve used the main app only for the past few years.
Perhaps your phone has extra aggressive battery saving settings that kill the background process? The official Syncthing has a setting to run as a persistent service, which always helped me.
Otherwise see if you can make system exceptions for the app to run in the background, and allow it to auto-sync. It’s been a while since I used the forked app, buy it did help me out on a device where the official didn’t work for me.
NP! It’s a great app, the dev updates it really frequently and I’ve never had any functional issue with it. I keep meaning to drop onto their git issues board and make a couple of small quality of life suggestions for the UI/UX as I use it dozens of times per day for work (there are some processes that currently take 5 clicks/per that could be reduced to 1 or 2 max) but that is a very small and nice problem to have.
Oh yeah sorry, I misunderstood. I think what you’re looking for is local (network) versioning which I’ve had trouble finding in the past as well. I had hoped SyncThing would do it but it doesn’t. Versioning is something a service like git does perfectly (i.e. notifies of and/or resolves conflicts in text files on the fly, seamlessly). When I was doing a lot of writing from different devices I set up a private repo on Github (and later Gitlab) and got my text editor to auto-sync-on-save to the repo (from any device) and it worked great. There are very likely self-hosted solutions that wouldn’t rely on the cloud for that, but for me it worked fine as private repos because nobody but me would ever see those drafts (in a perfect world… we all know microsoft has almost certainly trained their shitty A.I. on my terrible writing versions over those years on Github because they own that platform).
I know there are ways to get Git working locally, probably for this purpose, but I don’t know of any simple ones to suggest.
Well, it’s not versioning I need (I have an rsync backup that makes incremental copies). I need a 2-way sync that happens when files changes and doesn’t randomly stop working:
I want to edit a note on my phone - it copies to the server. Edit on the server, it updates to the phone. Without having to manually run any separate syncs first. I only mention sync conflicts because right now, syncthing hadn’t updated with my phone for over 2 days, plenty of time for me to update a note elsewhere and then edit the same note on the phone.
Resilio does it, but it looks like it’s draining the battery. Syncthing doesn’t drain the battery much, but that’s because it has become inactive on two different phones for long periods of time repeatedly.
Cloud provider apps usually work instantly with little drain, they must trigger from OS notifications, but the apps that sync to local servers just don’t seem to work that way!
Git is a great solution just for versioning. After I messed up a big note file I had, I set up emacs to hook git into the save function. I just created a repo in that directory, then backed up the whole directory including .git, so the versioning was there with the backup. No need to even use a separate repo, git just gave me a version history for the local files.
That was the same issue I had with SyncThing, it just seemed to conk out at weird times and I gave up on it (for that purpose). It’s great for centralizing a directory of files from one machine to another but I didn’t love it for keeping a single file up-to-date with changes coming from more than one point on the network.
Looks like you already settled on this, but I’m doing exactly this (syncing obsidian, as well as photos/videos from camera reel), to desktop and NAS, using syncthing-fork. Let me know if you want some pointers.
Interesting, I’d not heard of that. But does it auto-sync files? It mentions the clipboard, sharing links and browsing remote directories, but I don’t see a file sync mentioned.
I’d like to use resilio, I even bought a license to support it as I use it for all my pc syncing. But it’s currently showing 41% battery use for today on my phone for 2 minutes screen time 11.5 hours background.
Lenny Voyager shows 7% for 1.5 hour screen time. So something is not good with the phone app. Maybe the Android battery info display is misleading somehow (it confuses me because it shows a percentage of the time-interval you’re viewing, not a percentage of the total battery drain(.
That’s a good call out. My user case allows me to terminate the phone app after it syncs, and battery doesn’t seem to be hit hard even if I forget. But it’s important to look at, and it might not work for you.
Pixel is clean, from a battery saver perspective, so that’s probably not your issue.
Not sure what to do next. I’ve used it for about 10 years now, and keep gobs of stuff in sync with it.
I do recommend Syncthing-Fork for Android, it moves the sync conditions into the individual since folders, so you get finer control.
Do you get any errors on the desktop console? On Android, if you launch the web client you get much more info and configuration capability (Menu - Web GUI). Once there, click the gear at the top right, and open Logs. Maybe there’s something there that can help.
I tend to just use FolderSync myself. To avoid battery issues, I have a schedule for most folders; but my DCIM/Pictures folders sync immediately upon changes.
I then have a widget on my homepage that triggers a ‘sync all’. Anytime I need files synced immediately, it’s easy enough to click that button.
I’ve been using syncthing for a while now, on different devices, and the only unreliability I’ve run into is with android killing syncthing to save battery life, which is kinda hilarious, considering all the vendor- and google-provided crap they happily waste battery on (I don’t use it, but for what I’ve heard iOS is even worse in this regard).
Specifically, I have a samsung tablet where, no matter how much I tinkered with system settings, synchthing would only run if I manually launched the app or while the tablet was charging (BTW I still use that same tablet, but it now runs LineageOS and syncthing works flawlessly).
All this is to say, you should probably look into system settings and research ways to convince your OS to do what it’s supposed to rather than tinkering with syncthing itself.
If you’re using Obsidian for free then maybe try the built-in link which you’ll find in the built-in options I think. It’s a cost option but cheap. I think it eliminates the problems I’m having (below). I’m stubborn.
I’m not having problem with Syncthing, bar dealing with the stupid attempts to deal with deleted files that Android leaves laying around. I have .stignore files with .trashed-* and .trash/ entries on the Linux machine. Still having problems with _ed directories though and Syncthing conflict files when the sync isn’t fast enough when I switch between the two.
Sometimes it takes Syncthing a while to work out the best route between the two nodes. Sometimes days. It used to send my packets to the internet before letting them back into the local network. Eventually it found a more direct route between them. I’m not sure but I think it has something to do with local IPv6; I’m talking out of my ass though.
I’m not affiliated to Syncthing or Obsidian besides being a happy user.
I have decent battery life on my Pixel 7 Pro. I have the respect battery save setting on so syncing stops at 20% or so I think.
If you don’t mind going full back-to-basics, you can do this with standard command-line tools and no cloud server. Contrary to popular wisdom, a server is not necessary to sync files between a computer and a mobile device.
I use ssh and unison over wifi hotspot, no cable required. Works fine though it does require a button to be pressed. It doesn’t sync constantly in the background. Personally, after many years of doing that, I decided that it was an anti-feature.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
I’d really recommend giving Syncthing a second chance, twist a few knobs in the settings until it works. I’ve used it for years with barely a hitch.
Do you use it on a phone too? I did find it tricky to set up (more options than I really need, and the phone app settings don’t really work unless you select “Web UI”, which is really strange), but I didn’t mind the setup if I could then leave it alone and it works. Ideally I want to set this up on other family phones, so I can update notes and they appear everywhere.
I keep multiple phones in sync, with hundreds of gigs synced every day.
Have you found a way to get rid of the notification and keep Syncthing running in background?
Just hide it in your OS notification settings
But in this way I don’t get any notification if something goes wrong
I think any issues would show in “other”
Daaaaamn, was that so easy to just disable “Syncthing active”?
Let’s try! Thanks
For me that can be more specific. If I select the offending notification (either in the settings for notifications history or directly in the notification area) I can disable that specific type of notification. For instance deals from Amazon or eBay or something like connection established from tailscale.
Are you sure that there are different type of notification for Syncthing? Have you disable the persistent one and still received a failed sync one (or something else)?
Good to know. If I can get it working reliably, it will be worth sticking with. Someone suggested it might just not be auto-starting on reboots. I’m trying the fork of the UI on f-droid to see if it helps.
I prefer Syncthing-fork for some more straightforward configuration. Mainly the three button options equating to “follow the run conditions, damnit”, “run damnit”, and “stop damnit”
Thanks. I’m giving that a try!
It’s the main way I sync my phone.
I have a different app for photos, but SyncThing on my phone, and on my desktop, and again on one of my home servers, do most of the download and data syncing.
Occasionally I’ll have to manually run SyncThing; I’m not certain that Android is reliably starting it after reboots, but for the most part it just does it’s thing really reliably. There is a lag; it can take a few minutes for changes to sync - it’s not immediate. For me, this isn’t a problem, and I’d rather that than a battery suck, so I haven’t messed with it.
I could live with a few minutes, but it’s showig as offline for days. Maybe it is failing after a reboot. At least that would be a known situation to watch for.
In my experience syncthing is always a bit like that using the default discovery settings.
I use a hub & spoke set up now. Instead of A, B, and C all connecting to each other directly, they only connect to D. I also input the address for D specifically instead of using discovery servers.
With this set up I’ve never had any drama.
Honestly, I think Android is fucked for debugging stuff like this. I installed a program on mine and my wife’s phones - different makes & models - and configured them exactly the same, including the app settings in the OS. Mine works perfectly and barely shows up in battery use, near the bottom. Her’s drains her battery even when she’s not using it, regularly running at 50% of total battery consumption.
With Android YMMV is the rule, rather than the exception. There’s just too many variables.
Yeah, phone to laptop, and I recently synced all backups and files from an old phone to a new one, too. Once you have the computer setup, you can basically connect phones by reading its QR code.
If the official Syncthing Android app is giving you a hard time, maybe try Syncthing-fork? IIRC that’s only the daemon and web GUI wrapped as an app. But I’ve used the main app only for the past few years.
I have the phones connected, but the app just decides to disconnect and stay that way until I check it. I’ll give that fork a try, thanks!
Perhaps your phone has extra aggressive battery saving settings that kill the background process? The official Syncthing has a setting to run as a persistent service, which always helped me.
Otherwise see if you can make system exceptions for the app to run in the background, and allow it to auto-sync. It’s been a while since I used the forked app, buy it did help me out on a device where the official didn’t work for me.
Hope this helps.
Fork works much better on Android, largely because it moves sync conditions into the individual sync jobs (what ST calls folders).
I use LocalSend between all my devices (work, personal, etc). Mac, Linux, Android, Winblows. All. It’s fast, effective, lightweight. FOSS.
I was recently looking for something like this and couldn’t find what i was looking for, so thanks!
NP! It’s a great app, the dev updates it really frequently and I’ve never had any functional issue with it. I keep meaning to drop onto their git issues board and make a couple of small quality of life suggestions for the UI/UX as I use it dozens of times per day for work (there are some processes that currently take 5 clicks/per that could be reduced to 1 or 2 max) but that is a very small and nice problem to have.
Thanks, but I really need a seamless automatic sync. I think that’s a manually-triggered file send?
Oh yeah sorry, I misunderstood. I think what you’re looking for is local (network) versioning which I’ve had trouble finding in the past as well. I had hoped SyncThing would do it but it doesn’t. Versioning is something a service like git does perfectly (i.e. notifies of and/or resolves conflicts in text files on the fly, seamlessly). When I was doing a lot of writing from different devices I set up a private repo on Github (and later Gitlab) and got my text editor to auto-sync-on-save to the repo (from any device) and it worked great. There are very likely self-hosted solutions that wouldn’t rely on the cloud for that, but for me it worked fine as private repos because nobody but me would ever see those drafts (in a perfect world… we all know microsoft has almost certainly trained their shitty A.I. on my terrible writing versions over those years on Github because they own that platform).
I know there are ways to get Git working locally, probably for this purpose, but I don’t know of any simple ones to suggest.
Well, it’s not versioning I need (I have an rsync backup that makes incremental copies). I need a 2-way sync that happens when files changes and doesn’t randomly stop working: I want to edit a note on my phone - it copies to the server. Edit on the server, it updates to the phone. Without having to manually run any separate syncs first. I only mention sync conflicts because right now, syncthing hadn’t updated with my phone for over 2 days, plenty of time for me to update a note elsewhere and then edit the same note on the phone.
Resilio does it, but it looks like it’s draining the battery. Syncthing doesn’t drain the battery much, but that’s because it has become inactive on two different phones for long periods of time repeatedly.
Cloud provider apps usually work instantly with little drain, they must trigger from OS notifications, but the apps that sync to local servers just don’t seem to work that way!
Git is a great solution just for versioning. After I messed up a big note file I had, I set up emacs to hook git into the save function. I just created a repo in that directory, then backed up the whole directory including .git, so the versioning was there with the backup. No need to even use a separate repo, git just gave me a version history for the local files.
That was the same issue I had with SyncThing, it just seemed to conk out at weird times and I gave up on it (for that purpose). It’s great for centralizing a directory of files from one machine to another but I didn’t love it for keeping a single file up-to-date with changes coming from more than one point on the network.
Looks like you already settled on this, but I’m doing exactly this (syncing obsidian, as well as photos/videos from camera reel), to desktop and NAS, using syncthing-fork. Let me know if you want some pointers.
KDE connect!
Interesting, I’d not heard of that. But does it auto-sync files? It mentions the clipboard, sharing links and browsing remote directories, but I don’t see a file sync mentioned.
I didn’t see the file sync part.for that I use next cloud.which works great but my he overkill if that’s all your going to use it for
Resilio sync is outstanding
It’s free for personal use, but the license process is annoying (email)
I’d like to use resilio, I even bought a license to support it as I use it for all my pc syncing. But it’s currently showing 41% battery use for today on my phone for 2 minutes screen time 11.5 hours background. Lenny Voyager shows 7% for 1.5 hour screen time. So something is not good with the phone app. Maybe the Android battery info display is misleading somehow (it confuses me because it shows a percentage of the time-interval you’re viewing, not a percentage of the total battery drain(.
That’s a good call out. My user case allows me to terminate the phone app after it syncs, and battery doesn’t seem to be hit hard even if I forget. But it’s important to look at, and it might not work for you.
Yea, I like Resilio, but it’s a battery and ram hog.
Syncthing should work fine for you.
Does your phone have some custom battery management? Some vendors do that, and it causes issues.
The only setting I see is “allow background usage”, which is on (I’m using it on a Pixel 7 and 8).
Pixel is clean, from a battery saver perspective, so that’s probably not your issue.
Not sure what to do next. I’ve used it for about 10 years now, and keep gobs of stuff in sync with it.
I do recommend Syncthing-Fork for Android, it moves the sync conditions into the individual since folders, so you get finer control.
Do you get any errors on the desktop console? On Android, if you launch the web client you get much more info and configuration capability (Menu - Web GUI). Once there, click the gear at the top right, and open Logs. Maybe there’s something there that can help.
I’m trying the fork now, thanks. So far, it’s behaving. Thanks for the pointer to the logs, I’ll take a look if it happens again.
I tend to just use FolderSync myself. To avoid battery issues, I have a schedule for most folders; but my DCIM/Pictures folders sync immediately upon changes. I then have a widget on my homepage that triggers a ‘sync all’. Anytime I need files synced immediately, it’s easy enough to click that button.
I’ve used Sycthing but it was somewhat finicky.
2 more cents :)
I’ve been using syncthing for a while now, on different devices, and the only unreliability I’ve run into is with android killing syncthing to save battery life, which is kinda hilarious, considering all the vendor- and google-provided crap they happily waste battery on (I don’t use it, but for what I’ve heard iOS is even worse in this regard).
Specifically, I have a samsung tablet where, no matter how much I tinkered with system settings, synchthing would only run if I manually launched the app or while the tablet was charging (BTW I still use that same tablet, but it now runs LineageOS and syncthing works flawlessly).
All this is to say, you should probably look into system settings and research ways to convince your OS to do what it’s supposed to rather than tinkering with syncthing itself.
It’s what i use but on the phone i only turn it on when i want to sync, otherwise it might drain battery. Always running on the PC.
Mega also works.
I use Nextcloud. I run it in Yunohost.
If you’re using Obsidian for free then maybe try the built-in link which you’ll find in the built-in options I think. It’s a cost option but cheap. I think it eliminates the problems I’m having (below). I’m stubborn.
I’m not having problem with Syncthing, bar dealing with the stupid attempts to deal with deleted files that Android leaves laying around. I have
.stignorefiles with.trashed-*and.trash/entries on the Linux machine. Still having problems with_ed directories though and Syncthing conflict files when the sync isn’t fast enough when I switch between the two.Sometimes it takes Syncthing a while to work out the best route between the two nodes. Sometimes days. It used to send my packets to the internet before letting them back into the local network. Eventually it found a more direct route between them. I’m not sure but I think it has something to do with local IPv6; I’m talking out of my ass though.
I’m not affiliated to Syncthing or Obsidian besides being a happy user.
I have decent battery life on my Pixel 7 Pro. I have the respect battery save setting on so syncing stops at 20% or so I think.
If you don’t mind going full back-to-basics, you can do this with standard command-line tools and no cloud server. Contrary to popular wisdom, a server is not necessary to sync files between a computer and a mobile device.
I use
sshandunisonover wifi hotspot, no cable required. Works fine though it does require a button to be pressed. It doesn’t sync constantly in the background. Personally, after many years of doing that, I decided that it was an anti-feature.If this is mostly for Obsidian, use the community plugin and a self hosted CouchDB.
https://github.com/vrtmrz/obsidian-livesync
I’ve found it quite good.