I do a lot of docker/k8s at work, and I use podman at home. Podman is very cool in theory but still rough around the edges. I recommend docker if you just want to get started, Podman is a little extra work.
I started with Docker and then migrated to Podman for the integrated Cockpit dashboard support. All my docker-compose files work transparently on top of rootful Podman so the migration was relatively easy. Things get finicky when you try to go rootless though.
I say try both. Rootful podman is gonna be closest to the Docker experience.
Docker. It’s been around longer, it’s more polished, easier to learn, simpler to use, supported everywhere and by everything, easy to find solutions when you search for help, doesn’t depend on systemd, compatible with every container image out there, and you can do things with it even if it’s not the “correct” way to do it while podman will tell you “no, you can’t do that”.
If your distro offers it, rootless podman + podman system service is the best setup, IMO. That will give you a docker command that is 1-to-1 compatible with docker and lets you use tools like docker-compose that expect a docker service socket. Then you can just follow tutorials that only explain things for docker.
My only issue with rootless is that SWAG doesn’t work with it, otherwise my other containers could be rootless. However, I heard connecting rootful and rootless containers is impossible so all my containers are rootful right now.
will it let you do rootless nfs mounts into the container? That’s the showstopper for me, as that is by far the best way to just make this all work within the context of my file storage.
What is rootless bring brought up so much? It’s a container, it’s isolated from the host anyway, what does it matter what user runs inside? And if something breaks into the container they can trash the app in it and the shared volumes anyway, even if they’re not root.
It depends on the use case. The most common security issue I have seen with docker is on Linux desktop systems: docker deamon runs as root and user wants to use it to test all kinds of containers. So they make the docker socket accessible to the user, to lazy to use “sudo docker” every time… Having access to the docker socket means having the same permissions as the one running the daemon: root . Your browser effectively now has root permissions. At this point you could just login as root to your desktop.
Defense in depth. If something escapes the container it’s limited to only what’s under that user and not the whole system. Having access to the whole system makes it easier for malware to hide/persist itself.
Way too dependent on the setup, a container with absolutely no outside access theoretically just has the kernel, but usually we want to communicate with our docker images not just run them
Because a container is only as isolated from the host as you want it to be.
Suppose you run a container and mount the entire filesystem into it. If that container is running as root, it can then read and write anything it likes (including password databases and /etc/sudo)
So what? If I mount / in the container and choose to run it as root that’s my business. Why would the containerization engine second-guess what I’m doing?
How would you like it if sudo told you “I can’t let you be root, you could read and write anything you like, including password databases and /etc/sudo”?
Nothing to stop running podman containers with full root access by creating & running them as root, you run them as whatever user you want. I’ve done it to troubleshoot containers on more than one occasion, usually when I want to play with VPN or privileged ports but too lazy to do it proper. The end goal for a lot of ppl, including myself, is to run as many things as non-root as possible. Why? Best practices around security have you give a service the minimal access & resources it needs to do it’s tasks. Some people allow traffic from the internet to their containers & they probably feel a little bit safer running those programs as non-root since it can create an extra layer that may need to be broken to fully compromise a system.
The point is to minimize privilege to the least possible - not to make it impossible to create higher privileged containers. If a container doesn’t need to get direct raw hardware access, manage low ports on the host network, etc. then why should I give it root and let it be able to do those things? Mapping it to a user, controlling what resources it has access to, and restricting it’s capabilities means that in the event that my container gets compromised, my entire host isn’t necessarily screwed.
We’re not saying “sudo shouldn’t be able to run as root” but that “by default things shouldn’t be run with sudo - and you need a compelling reason to swap over when you do”
There’s real usability benefits too. I’ve collected some anecdotes from Reddit:
Rootless podman is my first choice for using containers now, it works fantastically well in my experience. It’s so much nicer to have all my container related stuff like volumes, configs, the control socket, etc. in my home directory and standard user paths vs. scattered all over the system. Permission issues with bind mounts just totally disappear when you go rootless. It’s so much easier and better than the root privileged daemon.
and,
If you are on Linux, there is the fantastic podman option “–userns keep-id” which will make sure the uid inside+the container is the same as your current user uid.+
and,
Yeah in my experience with rootless you don’t need to worry about UID shenanigans anymore. Containers can do stuff as root (from their perspective at least) all they want but any files you bind mount into the container are still just owned/modified by your user account on the host system (not a root user bleeding through from the container).
finally,
The permissions (rwx) don’t change, but the uid/gid is mapped. E.g. uid 0 is the running user outside the container, by uid 1 will be mapped to 100000 (configurable), and say 5000 inside the container is mapped to 105000. I don’t remember the exact mapping but it works roughly like that.
I’d go Docker for the maturity. Podman is nice but I’ve definitely had some issues, and Buildah lacks any sort of caching and does unnecessary intermediate copies of the layers when pushing to a repository that really slows things down on larger apps/images.
… what? assuming you are using a Containerfile… what? It’s… the same as docker on layer caching. The --cache-to and --cache-from flags are particularly sweet.
Maybe they changed it since last year, but it wouldn’t cache layers for me. Everytime I’d rebuild the app, it would re-run all the actions from the Containerfile. So a whole npm install each build even though I only changed a source file. Building the exact same file with Docker cached every layer as expected, so a config change would only change the last layer and be basically instant vs 5 minutes.
The other issue with pushing to a registry was that it made a whole temporary tar of the image, then gzip it to disk again before starting to upload it. It blew up the disk space I had allocated to my VM really fast, and made uploading those images take minutes instead of seconds. Docker again seemingly does it all in a streaming fashion as it uploads, making it much faster.
This could have changed though, it’s evolving fast. Just didn’t fit my use case then. But because of those experiences, I’d say it’s probably a safer bet to learn Docker first since documentation is abundant, and there’s no little “oh I’m using Podman and have to use a slightly different syntax” gotchas to run into to make it hard for you.
Podman, rootless containers work well, and there is no central process running everything. I like that starting containers on boot is integrated with systemd.
Check my comment history for an example of a simple bind mount compose.yaml I use for developing a small Python project. It’s exactly the same as Docker Compose (since Podman Compose follows the Compose spec) but if you’re just getting started, it might be a good skeleton to build on.
Podman supports docker compose just fine. You have to run it as a service, so that it can expose a socket like docker does, but it supports doing exactly that
How do you automatically start podman containers? I currently just manually add systemd entries but that’s a lot more cumbersome than Docker which doesn’t require you to do anything at all.
That’s how you do it, you have to write systemd config files for each container. And because it was cumbersome they’ve come up with third party tools that write them for you and launch the container. 😉
Since I use Docker Compose with Podman, I just make a single systemd service to run Docker Compose on boot, thereby starting all my containers at once.
I have one Compose file per stack, which is an application and all of its containers, databases, etc. Pretty much the same way I organized things with just Docker.
I use Quadlet, which is now merged in podman. The only issue I had with it is running system systemd services as other (rootless) users, I can’t get it to create cid files that the users can access. In those cases only, I have to modify the generated services files, which defeats the purpose.
Docker if you are unsure how to begin. You can use docker compose with configs you find on the internet.
If your interested in podman I would start by using it though distrobox. Distrobox is a tool that allows other Linux environments on your host system and is really good for development
How do you make podman run an image that runs as an uid/gid that don’t exist on the host and needs to access host devices/volumes owned by uid/gid that don’t exist in the container?
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.
Rules:
Be civil: we’re here to support and learn from one another. Insults won’t be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it’s not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don’t duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
selfh.st Newsletter and index of selfhosted software and apps
I do a lot of docker/k8s at work, and I use podman at home. Podman is very cool in theory but still rough around the edges. I recommend docker if you just want to get started, Podman is a little extra work.
Don’t overthink this. Just start using something.
This is the best advice. Bloody hard for me to do, however. Not sure why.
definitely docker. it just has so much more documentation and community involvement.
Podman, dockerd is a single point of failure
my reaction
I started with Docker and then migrated to Podman for the integrated Cockpit dashboard support. All my docker-compose files work transparently on top of rootful Podman so the migration was relatively easy. Things get finicky when you try to go rootless though.
I say try both. Rootful podman is gonna be closest to the Docker experience.
Docker. It’s been around longer, it’s more polished, easier to learn, simpler to use, supported everywhere and by everything, easy to find solutions when you search for help, doesn’t depend on systemd, compatible with every container image out there, and you can do things with it even if it’s not the “correct” way to do it while podman will tell you “no, you can’t do that”.
If your distro offers it, rootless podman + podman system service is the best setup, IMO. That will give you a
dockercommand that is 1-to-1 compatible with docker and lets you use tools like docker-compose that expect a docker service socket. Then you can just follow tutorials that only explain things for docker.My only issue with rootless is that SWAG doesn’t work with it, otherwise my other containers could be rootless. However, I heard connecting rootful and rootless containers is impossible so all my containers are rootful right now.
will it let you do rootless nfs mounts into the container? That’s the showstopper for me, as that is by far the best way to just make this all work within the context of my file storage.
What is rootless bring brought up so much? It’s a container, it’s isolated from the host anyway, what does it matter what user runs inside? And if something breaks into the container they can trash the app in it and the shared volumes anyway, even if they’re not root.
It depends on the use case. The most common security issue I have seen with docker is on Linux desktop systems: docker deamon runs as root and user wants to use it to test all kinds of containers. So they make the docker socket accessible to the user, to lazy to use “sudo docker” every time… Having access to the docker socket means having the same permissions as the one running the daemon: root . Your browser effectively now has root permissions. At this point you could just login as root to your desktop.
Defense in depth. If something escapes the container it’s limited to only what’s under that user and not the whole system. Having access to the whole system makes it easier for malware to hide/persist itself.
Correct me if I’m wrong but containerization is enforced by the kernel, correct? If something escapes you’re pretty much screwed anyway.
There are many layers involved in preventing escapes from containers.
Way too dependent on the setup, a container with absolutely no outside access theoretically just has the kernel, but usually we want to communicate with our docker images not just run them
Because a container is only as isolated from the host as you want it to be.
Suppose you run a container and mount the entire filesystem into it. If that container is running as root, it can then read and write anything it likes (including password databases and /etc/sudo)
So what? If I mount / in the container and choose to run it as root that’s my business. Why would the containerization engine second-guess what I’m doing?
How would you like it if sudo told you “I can’t let you be root, you could read and write anything you like, including password databases and /etc/sudo”?
Nothing to stop running podman containers with full root access by creating & running them as root, you run them as whatever user you want. I’ve done it to troubleshoot containers on more than one occasion, usually when I want to play with VPN or privileged ports but too lazy to do it proper. The end goal for a lot of ppl, including myself, is to run as many things as non-root as possible. Why? Best practices around security have you give a service the minimal access & resources it needs to do it’s tasks. Some people allow traffic from the internet to their containers & they probably feel a little bit safer running those programs as non-root since it can create an extra layer that may need to be broken to fully compromise a system.
The point is to minimize privilege to the least possible - not to make it impossible to create higher privileged containers. If a container doesn’t need to get direct raw hardware access, manage low ports on the host network, etc. then why should I give it root and let it be able to do those things? Mapping it to a user, controlling what resources it has access to, and restricting it’s capabilities means that in the event that my container gets compromised, my entire host isn’t necessarily screwed.
We’re not saying “sudo shouldn’t be able to run as root” but that “by default things shouldn’t be run with sudo - and you need a compelling reason to swap over when you do”
There’s real usability benefits too. I’ve collected some anecdotes from Reddit:
and,
and,
finally,
If you’re running it for your own network, just use Docker. The tooling is way, way better for docker, especially for a beginner.
I’d go Docker for the maturity. Podman is nice but I’ve definitely had some issues, and Buildah lacks any sort of caching and does unnecessary intermediate copies of the layers when pushing to a repository that really slows things down on larger apps/images.
… what? assuming you are using a Containerfile… what? It’s… the same as docker on layer caching. The --cache-to and --cache-from flags are particularly sweet.
Maybe they changed it since last year, but it wouldn’t cache layers for me. Everytime I’d rebuild the app, it would re-run all the actions from the Containerfile. So a whole
npm installeach build even though I only changed a source file. Building the exact same file with Docker cached every layer as expected, so a config change would only change the last layer and be basically instant vs 5 minutes.The other issue with pushing to a registry was that it made a whole temporary tar of the image, then gzip it to disk again before starting to upload it. It blew up the disk space I had allocated to my VM really fast, and made uploading those images take minutes instead of seconds. Docker again seemingly does it all in a streaming fashion as it uploads, making it much faster.
This could have changed though, it’s evolving fast. Just didn’t fit my use case then. But because of those experiences, I’d say it’s probably a safer bet to learn Docker first since documentation is abundant, and there’s no little “oh I’m using Podman and have to use a slightly different syntax” gotchas to run into to make it hard for you.
This sounds like something on your end as I get cached builds every time, rootlessly even. Podman also supports cache mounts.
Docker because it just works. Podman has another 5 years (hopefully) to get this part right IMO.
On what host system?
Podman, rootless containers work well, and there is no central process running everything. I like that starting containers on boot is integrated with systemd.
i would like to try… but as far as i know, there is no “docker compose up -d”
Apparently it does! podman-compose
cool, thank you!
Check my comment history for an example of a simple bind mount
compose.yamlI use for developing a small Python project. It’s exactly the same as Docker Compose (since Podman Compose follows the Compose spec) but if you’re just getting started, it might be a good skeleton to build on.i have all my stacks on docker compose. if it follows the same specks, i would only need to convert volumes and networks
Podman supports docker compose just fine. You have to run it as a service, so that it can expose a socket like docker does, but it supports doing exactly that
How do you automatically start podman containers? I currently just manually add systemd entries but that’s a lot more cumbersome than Docker which doesn’t require you to do anything at all.
Check out: https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html
https://www.redhat.com/sysadmin/quadlet-podman
That’s how you do it, you have to write systemd config files for each container. And because it was cumbersome they’ve come up with third party tools that write them for you and launch the container. 😉
Since I use Docker Compose with Podman, I just make a single systemd service to run Docker Compose on boot, thereby starting all my containers at once.
That’s an interesting way to do it. Do you have everything in 1 compose file?
I have one Compose file per stack, which is an application and all of its containers, databases, etc. Pretty much the same way I organized things with just Docker.
I use Quadlet, which is now merged in podman. The only issue I had with it is running system systemd services as other (rootless) users, I can’t get it to create cid files that the users can access. In those cases only, I have to modify the generated services files, which defeats the purpose.
Docker if you are unsure how to begin. You can use docker compose with configs you find on the internet.
If your interested in podman I would start by using it though distrobox. Distrobox is a tool that allows other Linux environments on your host system and is really good for development
Whichever one is better supported by the containers you want to run.
They both support the same containers
Except in real life you’ll run into images that podman refuses to work with all the time.
Example? Ive definitely had compose projects that I had a hard time running with podman but all the individual containers seem to work just fine.
How do you make podman run an image that runs as an uid/gid that don’t exist on the host and needs to access host devices/volumes owned by uid/gid that don’t exist in the container?
You use podman unshare to
chownthe directories to the appropriate UID/GID in the container’s user namespace.This right here. Just found out about this last week after a long debug.