What are the things that makes a selfhostable app/project project good? Maybe another way to phrase this question is, what are the things that makes a project easier to self-host?
I have been developing an application that focuses on being easy to selfhost. I have been looking around for existing and already good project such as paperless-ngx, Immich, etc.
From what I gather the most important thing are:
Good docs, this is probably the most important. The developer must document how to self-host
Less runtime dependency–I’m not sure about this one, but the less it depends on other services the better
Optional OIDC–I’m even less sure about this one, and I’m also not sure about implementing this feature on my own app as it’s difficult to develop. It seems that after reading this subreddit/community, I concluded that lots of people here prefer to separate identity/user pool and app service. This means running a separate service for authentication and authorization.
What do you think? Another question is, are there any more good project that can be used as a good example of selfhostable app?
Thank you
Some redditors responded on the post:
easy to install, try, and configure with sane defaults
availabiity of image on dockerhub
screenshots
good GUI
I also came across this comment from Hacker News lately, and I think about it a lot
This is what self-hosted software should be. An app, self-contained, (essentially) a single file with minimal dependencies.
Not something so complex that it requires docker. Not something that requires you to install a separate database. Not something that depends on redis and other external services.
I’ve turned down many self-hosted options due to the complexity of the setup and maintenance.
What helps a lot for apps with multiple config files:
if you tell the user to “add code xy to the config file” : tell me which file. is it the main config file? the one of the reverse proxy etc.?
provide a sensible example library of the config structure. For example: duting the implementation of an importer for beancount I was struggling with what goes where. The example structure was really, really helpful.
also, if you have configurations which allow different options: TELL ME THE OPTIONS! If I get an error during startup, that for config.foo the value “bar” is not allowed, I need a list of options somehwere, so many hours lost to find out what I can write to config.foo
Yep, documentation and a good base level default installation configuration/guide with minimal friction.
I’m perfectly willing to play around once I know at the basic level that the core flow is going to work for me. If it takes me digging through a stack of documentation (especially if it’s bad) to even get something to experiment with on my own system? I won’t bother.
Ease of installation/use, I think, is the main big one, and one of the biggest obstacles.
People who want to give self-hosting a try aren’t going to be particularly fond of having to jump through a whole bunch of different configs, and manually set everything up.
They want something that they can just set up and go, without having to deal with server hosting, services, and all of that. Something you can just run on your computer, leave it be, and use it with relatively little fuss.
Second to that, would definitely be a case of better documentation/screenshots. A lot of self-hosted things, like Lemmy, didn’t provide much documentation of what the actual user side of it does, only what you need to do to set it up, which isn’t going to make me want to use the software, if I have no idea what it’s supposed to do, and how it compares to other things that do the same.
Not something so complex that it requires a docker
Docker is the thing that sandboxes your services from the host OS. I’d rather use Podman because of the true non-root mode, but Docker is still based. Plus, you can use Docker Swarm if you don’t want to switch to Kubernetes (though you don’t have easy storage integration for persistence).
The problem is when Docker is used to gift wrap a mess. Then there are rotting dependencies in the containers. The nice thing about Debian packaged things is the maintainer is forced to do things properly. Even more so if they get it into the repos.
My preference is Debian Stable in LXC or even KVM for services. I only go for Docker if that is the recommended option. There is stuff out there where the recommend way is their VM image which is full of their soup of Dockers.
Docker is in my pile of technologies I don’t really like or approve of, but don’t have the energy to really fight.
Not a single app, not minimal dependencies. It’s a file that gets processed and creates many gigabites of leftovers, with an enormous runtime and piles of abstractions
I disagree. Docker makes things a lot easier and I’m going to use it regardless.
My rule is pretty simple: not PHP. PHP requires configuring a web server, so either that’s embedded in the docker image, (violates the “do one thing” rule of docker) or it’s pushed onto the user. This falls under the dependencies part, but I uniquely hate dealing with standalone web servers and I don’t mind configuring databases, so I called it out.
I actually tried switching to OCIS from Nextcloud specifically to avoid PHP, but OCIS is even more complex so I bailed.
Give me an example configuration that works out of the box and detailed documentation about options and I’ll be happy. Don’t make me configure a web server any particular way, and do let me handle TLS myself. If you do that, I’ll probably check it out.
IMO a lot of what makes nice self-hostable software is clean and sane software in general. A lot of stuff tend to end up trying to be too easy and you can’t scale up, or stuff so unbelievably complicated you can’t scale it down. Don’t make me install an email server and API keys to services needed by features I won’t even use.
I don’t particularly mind needing a database and Redis and the likes, but if you need MySQL and PostgreSQL and Redis and memcached and an ElasticSearch cluster and some of it is Go, some of it is Ruby and some of it is Java with a sprinkle of someone’s erlang phase, … no, just no, screw that.
What really sucks is when Docker is used as a bandaid to hide all that insanity under the guise of easy self-hosting. It works but it’s still a pain to maintain and debug, and it often uses way more resources than it really need. Well written software is flexible and sane.
My stuff at work runs equally fine locally in under a gig of RAM and barely any CPU at idle, and yet spans dozens of servers and microservices in production. That’s sane software.
I can’t even count how many self-hosted or open source projects I’ve wanted to check out, and the project page is just text.
If I don’t know exactly what I’m getting into in the first 10 seconds, I’m onto something else, especially when it’s something heavily based on UI/UX with frequent interaction.
EDIT: Also, I’m a fan of docker apps to run off my Synology NAS, but it better come with step-by-step instructions, or I won’t bother. There are some good resources for detailed instructions for various self-hosted/NAS/docker related content, but it’s nice when a project actually has this in their documentation.
Because guessing what the hell your thing looks like and behaves like is going to get me to bounce pretty much immediately because you’ve now made it where I have to figure out how to deploy your shit if I want to know. And, uh, generally, if you have no screenshots, you have no good documentation and thus it’s going to suuuuck.
Before even getting to documentation, I see so many projects that don’t have a short summary of what they do (and maybe what to not expect them to do).
As an example, Home Assistant. I can tell that it involves home automation, so can I replace Google Home with it? It seems like it doesn’t do voice recognition without add-ons and it can work with Google Assistant. Do I still need accounts with the providers of smart appliances, or can it control my bulbs directly?
None of that is very clear from the website.
I’ve seen plenty of other projects where it’s assumed there’s no need to explain it’s overall purpose.
A docker image is available. Not some sort of make or build script which make gods know what changes to my system, even if the end result is a docker image. Just have a docker image out on Dockerhub or a Dockerfile as part of the project. A docker-compose.yaml file is a nice bonus.
Two factor auth. I understand this is hard, but if you are actually building something you want people to seriously use, it needs to be seriously secured. Bonus points for working with my YubiKey.
Good authentication logging. I may be an outlier on this one, but I actually look at the audit logs for my services. Having a log of authentication activity (successes and failures) is important to me. I use both fail2ban to block off IPs which get up to any fuckery and I manually blackhole entire ASNs when it seems they are sourcing a lot of attacks. Give me timestamps (in ISO8601 format, all other formats are wrong), IP address, username, success or failure (as a independent field, not buried in a message or other string) and any client information you can (e.g. User-Agent strings).
Good error logging. Look, I kinda suck, I’m gonna break stuff. When I do, it’s nice to have solid logging giving me an idea of what I broke and to provide a standardized error code to search on. It also means that, when I give up and post it as an issue to your github page, I can provide you with some useful context.
As for that hackernews response, I’d categorically disagree with most of it.
An app, self-contained, (essentially) a single file with minimal dependencies.
Ya…no. Complex stuff is complex. And a lot of good stuff is complex. My main, self-hosted app is NextCloud. Trying to run that as some monolithic app would be brain-dead stupid. Just for the sake of maintainability, it is going to need to be a fairly sprawling list of files and folders. And it’s going to be dependent on some sort of web server software. And that is a very good place to NOT roll your own. Good web server software is hard, secure web server software is damn near impossible. Let the large projects (Apache/Nginx) handle that bit for you.
Not something so complex that it requires docker.
“Requires docker” may be a bit much. But, there is a reason people like to containerize stuff, it avoids a lot of problems. And supporting whatever random setup people have just sucks. I can understand just putting a project out as a container and telling people to fuck off with their magical snowflake setup. There is a reason flatpak is gaining popularity.
Honestly, I see docker as a way to reduce complexity in my setup. I don’t have to worry about dependencies or having the right version of some library on my OS. I don’t worry about different apps needing different versions of the same library. I don’t need to maintain different virtual python environments for different apps. The containers “just work”. Hell, I regularly dockerize dedicated game servers just for my wife and I to play on.
Not something that requires you to install a separate database.
Oh goodie, let’s all create our own database formats and re-learn the lessons of the '90s about how hard databases actually are! No really, fuck off with that noise. If your app needs a small database backend, maybe try SQLite. But, some things just need a real database. And as with web servers, rolling your own is usually a bad plan.
Not something that depends on redis and other external services.
Again, sometimes you just need to have certain functionality and there is no point re-inventing the wheel every time. Breaking those discrete things out into other microservices can make sense. Sure, this means you are now beholden to everything that other service does; but, your app will never be an island. You are always going to be using libraries that other people wrote. Just try to avoid too much sprawl. Every dependency you spin up means your users are now maintaining an extra application. And you should probably build a bit of checking into your app to ensure that those dependencies are in sync. It really sucks to upgrade a service and have it fail, only to discover that one of it’s dependencies needed to be upgraded manually first, and now the whole thing is corrupt and needs to be restored from backup. Yes, users should read the release notes, they never do.
The corollary here is to be careful about setting your users up for a supply chain attack. Every dependency or external library you add is one more place for your application to be attacked. And just because the actual vulnerability is in SomeCoolLib.js, it’s still your app getting hacked. You chose that library, you’re now beholden to everything it gets wrong.
At the end of it all, I’d say the best app to write is the one you are interested in writing. The internet is littered with lots of good intentions and interesting starts. There is a lot less software which is actually feature complete and useful. If you lose interest, because you are so busy trying to please a whole bunch of idiots on the other side of the internet, you will never actually release anything. You do you, and fuck all the haters. If what you put out is interesting and useful, us users will show up and figure out how to use it. We’ll also bitch and moan, no matter how great your app is. It’s what users do. Do listen, feedback is useful. But, also remember that opinions are like assholes: everyone has one, and most of them stink.
Yes, at least for hobby use. If it really needs something more complex than SQLite and an embedded HTTP server, it’s probably going to turn into a second job to keep it working properly.
I’ve turned down many self-hosted options due to the complexity of >the setup and maintenance.
Do you agree with this?
No.
I like my services and stack.
From the *arrs and Jellyfin, HortusFox, Unifi Network Application (though it should run with just a SQL-Lite DB) over many things else.
Yes, databases are annoying but if the service that wants it is sane I have no problem doing it.
What does grind my gears is when services have many breaking changes (e.g. Immich). If it wasnt for that I would be more open to finally start with that and maybe install good and working immich service.
The things redditors mentioned are very good already. Primarily screenshots.
Please, please always add screenshots to let me have a general idea of the UI.
At the very least a demo instance if you can’t be bothered to add screenshots (yes, I have seen many services that would rather share a demo instance than screenshots…)
The things redditors mentioned are very good already. Primarily screenshots.
Please, please always add screenshots to let me have a general idea of the UI.
I’ve read this mentioned many times. Is it really that bad XD
Not something so complex that it requires docker. Not something that requires you to install a separate database. Not something that depends on redis and other external services.
This comment is a bit silly. Databases just make sense for many services, although many could just use sqlite which would be fine (and many do). Redis etc is usually optional and might increase performance in some cases.
I wouldn’t be a fan of something requiring docker, but it’s often just the easiest way to deploy these types of services (and the easiest way to install it as a user).
Anyway, I’ll echo that clear, up-to-date documentation is nice. I shouldn’t have to search through actual code or the bug/issues section to find current information (but I get this is very challenging). And I’d rather projects didn’t make Discord a source of documentation (especially not the primary one).
I’ll add that having a NixOS module is a big plus for me, but I don’t expect the developers themselves to maintain this.
Unless you are a business with millions of users, I would be really skeptical of redis improving performance. At the end of the day, any simple KV store can replace it. It’s not like postgres where having sqlite in it’s place means not only a different performance profile, but also different semantics, sql dialect, rpc protocol, etc
One thing that makes a project good is knowing what it does, I’ve seen quite a few projects where they talk about all the features and technology and how to configure it but not a word about what it actually does, what problems it solves and so on.
I won’t self host your program if you don’t even tell me what it does, don’t make me search and clue together large parts of the documentation just to find if I want it. A simple explanation is enough but somehow I’ve seen quite a few programs that don’t have it.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
@hono4kami To me, good documentation is the number one thing that makes a selfhostable application good.
Second would be “is it dockerized ?”
I agree. If you don’t mind: what are your qualifications for good documentation? Do you have some good examples of good docs?
What helps a lot for apps with multiple config files:
@hono4kami
One of the best documentation I’ve encountered so far:
https://borgbackup.readthedocs.io/en/stable/
including examples for everything in the docs is the best way to explain imo
Yep, documentation and a good base level default installation configuration/guide with minimal friction.
I’m perfectly willing to play around once I know at the basic level that the core flow is going to work for me. If it takes me digging through a stack of documentation (especially if it’s bad) to even get something to experiment with on my own system? I won’t bother.
Has docker compose file for deployment
Can be hosted on sub-path and not only subdomain
Can easily be integrated into SSO lime authelia
Ease of installation/use, I think, is the main big one, and one of the biggest obstacles.
People who want to give self-hosting a try aren’t going to be particularly fond of having to jump through a whole bunch of different configs, and manually set everything up.
They want something that they can just set up and go, without having to deal with server hosting, services, and all of that. Something you can just run on your computer, leave it be, and use it with relatively little fuss.
Second to that, would definitely be a case of better documentation/screenshots. A lot of self-hosted things, like Lemmy, didn’t provide much documentation of what the actual user side of it does, only what you need to do to set it up, which isn’t going to make me want to use the software, if I have no idea what it’s supposed to do, and how it compares to other things that do the same.
Docker is the thing that sandboxes your services from the host OS. I’d rather use Podman because of the true non-root mode, but Docker is still based. Plus, you can use Docker Swarm if you don’t want to switch to Kubernetes (though you don’t have easy storage integration for persistence).
The problem is when Docker is used to gift wrap a mess. Then there are rotting dependencies in the containers. The nice thing about Debian packaged things is the maintainer is forced to do things properly. Even more so if they get it into the repos.
My preference is Debian Stable in LXC or even KVM for services. I only go for Docker if that is the recommended option. There is stuff out there where the recommend way is their VM image which is full of their soup of Dockers.
Docker is in my pile of technologies I don’t really like or approve of, but don’t have the energy to really fight.
Docker is also the thing that allows the distribution of the app as “single file with minimal dependencies”.
Not a single app, not minimal dependencies. It’s a file that gets processed and creates many gigabites of leftovers, with an enormous runtime and piles of abstractions
I disagree. Docker makes things a lot easier and I’m going to use it regardless.
My rule is pretty simple: not PHP. PHP requires configuring a web server, so either that’s embedded in the docker image, (violates the “do one thing” rule of docker) or it’s pushed onto the user. This falls under the dependencies part, but I uniquely hate dealing with standalone web servers and I don’t mind configuring databases, so I called it out.
I actually tried switching to OCIS from Nextcloud specifically to avoid PHP, but OCIS is even more complex so I bailed.
Give me an example configuration that works out of the box and detailed documentation about options and I’ll be happy. Don’t make me configure a web server any particular way, and do let me handle TLS myself. If you do that, I’ll probably check it out.
IMO a lot of what makes nice self-hostable software is clean and sane software in general. A lot of stuff tend to end up trying to be too easy and you can’t scale up, or stuff so unbelievably complicated you can’t scale it down. Don’t make me install an email server and API keys to services needed by features I won’t even use.
I don’t particularly mind needing a database and Redis and the likes, but if you need MySQL and PostgreSQL and Redis and memcached and an ElasticSearch cluster and some of it is Go, some of it is Ruby and some of it is Java with a sprinkle of someone’s erlang phase, … no, just no, screw that.
What really sucks is when Docker is used as a bandaid to hide all that insanity under the guise of easy self-hosting. It works but it’s still a pain to maintain and debug, and it often uses way more resources than it really need. Well written software is flexible and sane.
My stuff at work runs equally fine locally in under a gig of RAM and barely any CPU at idle, and yet spans dozens of servers and microservices in production. That’s sane software.
I see, it’s probably good to have some balance between those. Noted
For me, it’s screenshots.
I can’t even count how many self-hosted or open source projects I’ve wanted to check out, and the project page is just text.
If I don’t know exactly what I’m getting into in the first 10 seconds, I’m onto something else, especially when it’s something heavily based on UI/UX with frequent interaction.
EDIT: Also, I’m a fan of docker apps to run off my Synology NAS, but it better come with step-by-step instructions, or I won’t bother. There are some good resources for detailed instructions for various self-hosted/NAS/docker related content, but it’s nice when a project actually has this in their documentation.
10000% this.
Tell me what it does, and SHOW me what it does.
Because guessing what the hell your thing looks like and behaves like is going to get me to bounce pretty much immediately because you’ve now made it where I have to figure out how to deploy your shit if I want to know. And, uh, generally, if you have no screenshots, you have no good documentation and thus it’s going to suuuuck.
Before even getting to documentation, I see so many projects that don’t have a short summary of what they do (and maybe what to not expect them to do).
As an example, Home Assistant. I can tell that it involves home automation, so can I replace Google Home with it? It seems like it doesn’t do voice recognition without add-ons and it can work with Google Assistant. Do I still need accounts with the providers of smart appliances, or can it control my bulbs directly?
None of that is very clear from the website.
I’ve seen plenty of other projects where it’s assumed there’s no need to explain it’s overall purpose.
My list of items I look for:
As for that hackernews response, I’d categorically disagree with most of it.
Ya…no. Complex stuff is complex. And a lot of good stuff is complex. My main, self-hosted app is NextCloud. Trying to run that as some monolithic app would be brain-dead stupid. Just for the sake of maintainability, it is going to need to be a fairly sprawling list of files and folders. And it’s going to be dependent on some sort of web server software. And that is a very good place to NOT roll your own. Good web server software is hard, secure web server software is damn near impossible. Let the large projects (Apache/Nginx) handle that bit for you.
“Requires docker” may be a bit much. But, there is a reason people like to containerize stuff, it avoids a lot of problems. And supporting whatever random setup people have just sucks. I can understand just putting a project out as a container and telling people to fuck off with their magical snowflake setup. There is a reason flatpak is gaining popularity.
Honestly, I see docker as a way to reduce complexity in my setup. I don’t have to worry about dependencies or having the right version of some library on my OS. I don’t worry about different apps needing different versions of the same library. I don’t need to maintain different virtual python environments for different apps. The containers “just work”. Hell, I regularly dockerize dedicated game servers just for my wife and I to play on.
Oh goodie, let’s all create our own database formats and re-learn the lessons of the '90s about how hard databases actually are! No really, fuck off with that noise. If your app needs a small database backend, maybe try SQLite. But, some things just need a real database. And as with web servers, rolling your own is usually a bad plan.
Again, sometimes you just need to have certain functionality and there is no point re-inventing the wheel every time. Breaking those discrete things out into other microservices can make sense. Sure, this means you are now beholden to everything that other service does; but, your app will never be an island. You are always going to be using libraries that other people wrote. Just try to avoid too much sprawl. Every dependency you spin up means your users are now maintaining an extra application. And you should probably build a bit of checking into your app to ensure that those dependencies are in sync. It really sucks to upgrade a service and have it fail, only to discover that one of it’s dependencies needed to be upgraded manually first, and now the whole thing is corrupt and needs to be restored from backup. Yes, users should read the release notes, they never do.
The corollary here is to be careful about setting your users up for a supply chain attack. Every dependency or external library you add is one more place for your application to be attacked. And just because the actual vulnerability is in SomeCoolLib.js, it’s still your app getting hacked. You chose that library, you’re now beholden to everything it gets wrong.
At the end of it all, I’d say the best app to write is the one you are interested in writing. The internet is littered with lots of good intentions and interesting starts. There is a lot less software which is actually feature complete and useful. If you lose interest, because you are so busy trying to please a whole bunch of idiots on the other side of the internet, you will never actually release anything. You do you, and fuck all the haters. If what you put out is interesting and useful, us users will show up and figure out how to use it. We’ll also bitch and moan, no matter how great your app is. It’s what users do. Do listen, feedback is useful. But, also remember that opinions are like assholes: everyone has one, and most of them stink.
A docker compose example with as few images as possible.
Yes, at least for hobby use. If it really needs something more complex than SQLite and an embedded HTTP server, it’s probably going to turn into a second job to keep it working properly.
No.
I like my services and stack.
From the *arrs and Jellyfin, HortusFox, Unifi Network Application (though it should run with just a SQL-Lite DB) over many things else.
Yes, databases are annoying but if the service that wants it is sane I have no problem doing it.
What does grind my gears is when services have many breaking changes (e.g. Immich). If it wasnt for that I would be more open to finally start with that and maybe install good and working immich service.
The things redditors mentioned are very good already. Primarily screenshots.
Please, please always add screenshots to let me have a general idea of the UI.
At the very least a demo instance if you can’t be bothered to add screenshots (yes, I have seen many services that would rather share a demo instance than screenshots…)
I’ve read this mentioned many times. Is it really that bad XD
It’s annoying to figure out if the program is any good if it’s reliant on a UI.
This comment is a bit silly. Databases just make sense for many services, although many could just use sqlite which would be fine (and many do). Redis etc is usually optional and might increase performance in some cases.
I wouldn’t be a fan of something requiring docker, but it’s often just the easiest way to deploy these types of services (and the easiest way to install it as a user).
Anyway, I’ll echo that clear, up-to-date documentation is nice. I shouldn’t have to search through actual code or the bug/issues section to find current information (but I get this is very challenging). And I’d rather projects didn’t make Discord a source of documentation (especially not the primary one).
I’ll add that having a NixOS module is a big plus for me, but I don’t expect the developers themselves to maintain this.
Unless you are a business with millions of users, I would be really skeptical of redis improving performance. At the end of the day, any simple KV store can replace it. It’s not like postgres where having sqlite in it’s place means not only a different performance profile, but also different semantics, sql dialect, rpc protocol, etc
One thing that makes a project good is knowing what it does, I’ve seen quite a few projects where they talk about all the features and technology and how to configure it but not a word about what it actually does, what problems it solves and so on.
I won’t self host your program if you don’t even tell me what it does, don’t make me search and clue together large parts of the documentation just to find if I want it. A simple explanation is enough but somehow I’ve seen quite a few programs that don’t have it.
Good documentation.