I’ve seen a lot of recommends for Immich on here, so I have an idea what the answer here is going to be, but I’m looking for some comparisons between it and Photoprism I’m currently using Synology Photos, and I think my biggest issue is it’s lack of metadata management. I’ve gotten around that with MetaImage and NeoFinder. I’m considering moving to something not tied to the Synology environment.
With any serious number of photos Nextcloud starts loading for minutes
What is this serious number you have in mind? I’m the first one thinking that some of the design choices behind nextcloud are laughable, and that their attitude towards code quality and best practices is inadequate, but even then I don’t think this threshold is easily reached.
Well. It just stopped working after the 2k items mark for me. Had to increase php memory and all for almost 6GB to make it work. Still sluggish AF. It’ll be just my file bucket for now on.
EDIT: I will like to add, Immich is now with 32k assets for me, working flawlessly and only using 300MB when active of memory.
I am about 3 orders of magnitude beyond you in terms of content, and I host that on 2GB RAM and a CPU that scores 440 on CPU benchmark.net . You might want to check your configuration, perhaps starting from the database (I use PG), then server (php-fpm). I don’t even use redis for caching, just the basic APC.
Great to know! Something is really odd on my instance. The DB is definitely working well on its own, all the queries are returned quite fast from what I gather on the monitoring side. Probably something odd on the server side. I’m using redis for caching, it helps a bit as it comes down to a halt without redis.
Interesting! It’s hard to continue without more details, and I’m not an expert anyway, but something definitely should be off with your configuration :)
Some people have luck like this and guess they’re the ones advocating. I first had a hand-installed instance with all the recommended optimizations like Redis, and then I started fresh with Docker + Redis. In both cases, after 10-15k files it was extremely slow. Both on server hardware with plenty of RAM.
To be totally fair, in my long history with it, I’ve been complaining more about the bad performance of nextcloud than the opposite (I still do), but things improved quite significantly after I moved (a long time ago) to postgresql+pgbouncer, which made it acceptable. Tuning did the rest.
There can be several reasons why your experience with nextcloud is not optimal. For instance, if you have a slow mechanical drive and a weak 2-cores CPU like I do, it’s enough to hit a large folder with many pictures for the first time to have it grind to a halt: the server will become IO and CPU starved while php-fpm fork bombs (on a default config) way too many imagemagick processes to render miniatures of the photos, which, when those are large, will make sure to eat the little amount of RAM you had left, further pushing you into swap and memory-compression territory, making things even worse IO & CPU-wise. This is easily mitigated by rendering the miniatures asynchronously via a cronjob and making sure that there is a reasonable limit to php-fpm’s parallelism to keep your app, navigation and overall system responsive.
Honestly, this is basic sysadmin/tuning stuff, that only you (with the knowledge of your hardware) can do right (and should do, the reward is immense). And it might very well be that nextcloud is (much) more demanding than other apps due to its inherently bad design & stack. It does the job, though, and isn’t afraid of taking hundreds of thousands of files.
Fair. Like I wrote previously (either here or in another thread) it’s likely that most self-hosters would at some point consider nextcloud given how versatile it is, and then it pays-off to tune for your config. If you do without, good for ya!
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
What is this serious number you have in mind? I’m the first one thinking that some of the design choices behind nextcloud are laughable, and that their attitude towards code quality and best practices is inadequate, but even then I don’t think this threshold is easily reached.
Well. It just stopped working after the 2k items mark for me. Had to increase php memory and all for almost 6GB to make it work. Still sluggish AF. It’ll be just my file bucket for now on. EDIT: I will like to add, Immich is now with 32k assets for me, working flawlessly and only using 300MB when active of memory.
I am about 3 orders of magnitude beyond you in terms of content, and I host that on 2GB RAM and a CPU that scores 440 on CPU benchmark.net . You might want to check your configuration, perhaps starting from the database (I use PG), then server (php-fpm). I don’t even use redis for caching, just the basic APC.
Great to know! Something is really odd on my instance. The DB is definitely working well on its own, all the queries are returned quite fast from what I gather on the monitoring side. Probably something odd on the server side. I’m using redis for caching, it helps a bit as it comes down to a halt without redis.
Nothing reported in the admin screen listing out config tips and missing indexes?
Nope, all I have is the default phone region not set.
Interesting! It’s hard to continue without more details, and I’m not an expert anyway, but something definitely should be off with your configuration :)
Thanks for trying though! Appreciate it. I’m happy to use nextcloud just for regular files for now anyways.
Some people have luck like this and guess they’re the ones advocating. I first had a hand-installed instance with all the recommended optimizations like Redis, and then I started fresh with Docker + Redis. In both cases, after 10-15k files it was extremely slow. Both on server hardware with plenty of RAM.
To be totally fair, in my long history with it, I’ve been complaining more about the bad performance of nextcloud than the opposite (I still do), but things improved quite significantly after I moved (a long time ago) to postgresql+pgbouncer, which made it acceptable. Tuning did the rest.
There can be several reasons why your experience with nextcloud is not optimal. For instance, if you have a slow mechanical drive and a weak 2-cores CPU like I do, it’s enough to hit a large folder with many pictures for the first time to have it grind to a halt: the server will become IO and CPU starved while php-fpm fork bombs (on a default config) way too many imagemagick processes to render miniatures of the photos, which, when those are large, will make sure to eat the little amount of RAM you had left, further pushing you into swap and memory-compression territory, making things even worse IO & CPU-wise. This is easily mitigated by rendering the miniatures asynchronously via a cronjob and making sure that there is a reasonable limit to php-fpm’s parallelism to keep your app, navigation and overall system responsive.
Honestly, this is basic sysadmin/tuning stuff, that only you (with the knowledge of your hardware) can do right (and should do, the reward is immense). And it might very well be that nextcloud is (much) more demanding than other apps due to its inherently bad design & stack. It does the job, though, and isn’t afraid of taking hundreds of thousands of files.
I’ve got the cron job and used Postgres in my previous instance. Limiting fpm concurrency is good insight, Ihavee not done that.
But do you know what works without all that and loads instantly? Photoprism.
Fair. Like I wrote previously (either here or in another thread) it’s likely that most self-hosters would at some point consider nextcloud given how versatile it is, and then it pays-off to tune for your config. If you do without, good for ya!