I'd still rather use docker. I don't mind that the daemon runs as root because there are some things that you need root for anyways! Like binding to privileged ports or setting up networks (use `internal: true` and the daemon will automatically set up iptables rules that limit traffic).
I deploy docker compose files with ansible so everything comes with built in security defaults like rootless, dropped caps, no new privileges, etc. I wish more containers supported running read only (its usually pretty easy to add, just overlooked) and distroless (common for go apps, less so otherwise).
There was a pretty good comment on reddit a while back with a list of hardenings for compose files [1]
Rootless is definitely the way to go. You can forward ports manually on the host if you really need to use privileged ports. I generally expose my containers through a reverse proxy, running bare metal on the host, and that completely bypasses the privileged port issue.
This whole privileged port thing is an outdated convention from the time when Linux ran on mainframes. Depending on your use case, it can be perfectly fine to lower it. I have set to `net.ipvX.ip_unprivileged_port_start = 80` on my server so that I can run rootless containers without extra privileges and have them bind to ports 80 and up.
Nits: It's a Unix convention (not just Linux), and Unix traditionally ran on minicomputers (not mainframes). Mainframes had their own specialized OSes, although some could run Unix in a partition or virtual machine.
The .container units get the job done, but if you want to manage a stack of containers and are familiar with Kubernetes, or just like YAML, I recommend using the .kube units instead.[1]
The '--userns=auto' argument is a useful isolation method in both rootless and rootful Podman containers. This allows rootful Podman to orchestrate privileged capabilities while running the container processes in an unprivileged namespace.
I've been doing this for a while now and it is by far much easier to work with and just as safe if configured properly. I think people just cargo cult thier way into wanting rootless containers for their long running services with no real look into if it's actually needed.
Rootless containers have their place, actual userspace workloads. Things like dev containers or normal users running containers but if one is creating users to run system level services then they are just adding extra steps imo.
Interesting... yeah if you think of Docker as an easy way to setup environments that's one thing but if you are intending airtight isolation so processes inside Docker can't "escape" most conventional use cases / discourse haven't really focused on that I think
Personally I still think this is not enough, and we really need full generalized (not AI-only) microvm support built into docker/podman, like yesterday.
Currently it's difficult to even get a hold of a properly configured minimal kernel (or time-consuming to try to build one) and all the right command-line incantations to even start a one-off microvm using say, qemu, with all the proper storage/networking/etc. bits one needs for production environments. Plus you need to keep that kernel updated very regularly.
I know there's projects like smolvm that try to make this simpler, but I've had some major problems with those solutions as well, and I just feel like the big boys need to step up and support this directly by now.
Containers and security are oxymorons. The flood of page cache cves (which can always be escalated/weaponized to an escape) from the other month is making deploying containers to prod untenable.
As for orchestration - a lot of folks think you need a completely new orchestration system for dealing with vms but we just simply re-use the existing infrastructure that already exists - the public clouds. Those companies have tens of thousands of engineers that are much better than the average engineer at this stuff, custom hardware, custom protocols and close to several decades of existing deployment.
I can build and ship a vm from my laptop/ci to prod on AWS/GCP in ~tens of second. Granted I come from the camp that thinks deploying full blown general purpose operating systems to prod is an increasingly incredibly risky practice.
Many such things have been built. Problem afaik is that the underlying hypervisor setup to support VMs isn't present, or isn't enabled by default, or is hard to figure out, on many machines. Plus you need nested virtualization when the thing you're using is a VM itself (also comes with possible pain). Docker runs because it has "ROAC" properties (Runs On Any Computer). That's not today true of VMs (of which micro-VMs, firecracker etc are a subset).
Rootless containers have privilege escalation risks of their own. The problem is the unprivileged_userns_clone sysctl, which rootless containers rely on, has historically been a big source of local privilege escalations. The most recent attack I'm aware of is CIFSwitch.
I deploy docker compose files with ansible so everything comes with built in security defaults like rootless, dropped caps, no new privileges, etc. I wish more containers supported running read only (its usually pretty easy to add, just overlooked) and distroless (common for go apps, less so otherwise).
There was a pretty good comment on reddit a while back with a list of hardenings for compose files [1]
1. https://www.reddit.com/r/selfhosted/comments/1pr74r4/comment...
It does run on IBM 370 type mainframes, but that port was done years after it first ran on the 386, and isn't in any way relevant to low port numbers.
[1]: https://docs.podman.io/en/stable/markdown/podman-kube.unit.5...
See the discussion here:
https://github.com/podman-container-tools/podman/discussions...
Rootless containers have their place, actual userspace workloads. Things like dev containers or normal users running containers but if one is creating users to run system level services then they are just adding extra steps imo.
Currently it's difficult to even get a hold of a properly configured minimal kernel (or time-consuming to try to build one) and all the right command-line incantations to even start a one-off microvm using say, qemu, with all the proper storage/networking/etc. bits one needs for production environments. Plus you need to keep that kernel updated very regularly.
I know there's projects like smolvm that try to make this simpler, but I've had some major problems with those solutions as well, and I just feel like the big boys need to step up and support this directly by now.
runsc (gvisor)
Kata Containers (virtualization)
both drop in replacements for runc (default OCI runtime)
there are others too. one based on libkrun : https://github.com/containers/crun/blob/main/krun.1.md
and these are just runc replacements.
with some minor scripting you can roll a CLI solution using any of gvisor, cloud-hypervisor, firecracker, muvm (libkrun).
As for orchestration - a lot of folks think you need a completely new orchestration system for dealing with vms but we just simply re-use the existing infrastructure that already exists - the public clouds. Those companies have tens of thousands of engineers that are much better than the average engineer at this stuff, custom hardware, custom protocols and close to several decades of existing deployment.
I can build and ship a vm from my laptop/ci to prod on AWS/GCP in ~tens of second. Granted I come from the camp that thinks deploying full blown general purpose operating systems to prod is an increasingly incredibly risky practice.