Yet another Hacker News

168 points psxuaw 3 hours ago 74 comments

shykes 2 hours ago | parent

Hi, I'm the creator of Docker and came across this post. It's always fun to see other engineers think about how you designed a system. I haven't been involved in developing Docker in many years, but it's a fun trip down memory lane.

I'm happy to answer any questions you may have.

irishcoffee 2 hours ago | parent

Oh wow, hi! I think my biggest “gripe” with docker is that everyone grabs it as a hammer and views many problems as a nail. Am I way off base with that? What was your original vision for how docker might be used in the wild?

skinfaxi 2 hours ago | parent

Given how things have changed since your creation and the way things are going, are there any things you would have done differently?

shykes 1 hour ago | parent

There are many. But there is an alternative universe where I remained CEO instead of transitioning to CTO, and continued pursuing my product vision obsessively instead of compromising on it because of lack of self-confidence and a desire to be loved. I think I would have avoided a lot of the mistakes that the company made - but on the other hand, I would have made a fresh set of mistakes instead, so who knows what the outcome would have been!

gradus_ad 2 hours ago | parent

General question. When you were designing Docker, did you anticipate the success it would achieve?

shykes 2 hours ago | parent

To give a sense of time: Docker launched in 2013, but it was one of many iterations of the same project, which I had been working on continuously since 2008. Over those 5 years, virtually nobody cared or understood what the hell I was working on. I lived in France and had no connection to the tech industry.

So, not only did I not anticipate this kind of success, I did not even have a reference frame for what success might even look like. All I knew was, I had an itch that I had to scratch, and I had nothing better to do :)

Waterluvian 2 hours ago | parent

Was it always called Docker? Do you think the branding played a larger role in success than you would have expected?

ak217 1 hour ago | parent

I believe Docker was always called Docker, but it grew directly out of the previous generation of the technology used by dotCloud (the company Solomon founded), which was using LXC with AUFS.

I think the breakthrough with Docker was bundling the AUFS-based image layer format with a centralized container image sharing registry. This was revolutionary at the time. But another huge element of success was Solomon and others evangelizing the technology relentlessly at meetups and conferences. I distinctly remember attending a demo in 2012 where the crowd was very excited to see the Docker workflow that we now take for granted, in large part because Solomon was so good at presenting it. At the time, I was working at a cloud PaaS company where we developed a similar technology on top of LXC and BTRFS (but no image registry). Docker really made me think hard about what we were doing, although I didn't end up making any moves.

https://jpetazzo.github.io/2017/02/24/from-dotcloud-to-docke...

hack1312 1 hour ago | parent

It used to be dotCloud iirc

memset 12 minutes ago | parent

Dotcloud was a PaaS:

https://news.ycombinator.com/item?id=2032821

Then it pivoted to docker:

https://news.ycombinator.com/item?id=6634002

ecliptik 1 hour ago | parent

In those years in the systems group I was working in at a fortune 500, we were very interested in this type of technology. Mostly using Solaris Zones and cgroups and trying to do this ourselves with limited results.

When Docker came out it solved so many issues we were having. Thank you for scratching that itch.

xtracto 1 hour ago | parent

I remember in 2013 the first time someone showed me Docker [im in Mexico]. My first thought is that we already had Vagrant for setting up and running virtual machines through console.

I didnt get it at that time haha.

layer8 2 hours ago | parent

If you were implementing an operating system (free from compatibility concerns), what would you do differently to better support lightweight containers?

figmert 2 hours ago | parent

I'm curious what made think of it in the first place? What was your thought process that made you think that these Linux primitives give the opportunity to make it a platform for distributing and deploying packaged applications and eliminating a lot of the issue around "it works my machine" argument?

You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)

bijmtir 1 hour ago | parent

Docker was basically a commercialisation of lxc and now we got nspawn going full circle.

iJohnDoe 1 hour ago | parent

I’m a huge fan of nspawn. Always wanted to use VMs instead of Docker and nspawn feels like a natural choice.

miladyincontrol 1 hour ago | parent

Its nice having the flexibility for both system containers and application containers. It does feel though that most people forget that nspawn does both, and that it only can do system containers.

iJohnDoe 52 minutes ago | parent

Agreed, I briefly forgot it can do both. Thanks for the reminder.

I went down the rabbit hole of Incus and once I realized it requires the full network aspects to be configured, then I figured using KVM, QEMU, Libvirt was more practical. Then stumbled into nspawn and it was basically ready to go.

Although, I haven’t done enough research to know if Incus provides certain useful capabilities.

radlad 35 minutes ago | parent

I'm a huge fan of Incus. VMs, LXC, and OCI containers all under the same Docker-like CLI interface. I run it on a local server and on a dedicated server in a local DC, and connect to both remotely using Nebula.

It includes project-level resource separation, advanced networking, clustering, snapshots, and more. Plus, Stéphane Graber is super responsive on the forums and issue tracker. I bounced off of Proxmox, and can't recommend Incus more for hypervisor-type scenarios.

lostlogin 1 hour ago | parent

The LXC way of getting an up per container is pretty great - I’m basic bitch with this sort of thing so I assume it’s less of an issue for the experts but really enjoyed the IP per container when I hit it.

shykes 1 hour ago | parent

This is incorrect.

Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.

The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".

As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.

bijmtir 58 minutes ago | parent

Was it based on openvz or wrapped lxc?

And just because you do something bad doesn’t mean we can’t call it what it was. A bad attempt at lying is still a bad attempt for example.

shykes 55 minutes ago | parent

Earlier incarnations of the project (2008-2011) were python based on openvz+aufs, with the very first prototypes (2007-08) wrapping vserver. We started the transition to lxc+aufs in 2011. Open-sourced the lxc wrapper in 2013. Replaced it with libcontainer in... 2014 I believe?

js2 55 minutes ago | parent

Huh, according to wikpedia, OpenVZ and Solaris Zones (Containers) were both released in 2005. My memory was that Solaris was first and OpenVZ was a copy cat, but I guess not.

stingraycharles 40 minutes ago | parent

“Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough”

Developers are a tough market to please, especially when your business revolves around open source software.

Unless you have an option to do private cloud hosting (like all these databases and other platforms do), it’s incredibly difficult to monetize.

In the case of docker, I think it’s in the nature of the product that it’s difficult to monetize. Best you can do is enterprise support contracts, but those aren’t good enough to justify all that VC funding.

zoeysmithe 24 minutes ago | parent

In the end, how were you able to monetize Docker?

okanat 2 hours ago | parent

Would you consider writing Docker/moby in a different language, if you were to create it scratch today?

shykes 2 hours ago | parent

Oh gosh, the choice of language is such a context-specific design decision. The first versions of what became Docker were written in Python. The choice of Go was a mix of nerdy excitement ("it's like the best of Python and C!"), and adoption strategy: back then devops tools were segmented by language runtime: ruby, python or Java mostly. Shipping a compiled binary allowed us to be adopted by all tribes.

None of these things can be transposed today, so I guess I can't answer...

For what it's worth, my current project (dagger) is written in Go.

Tenemo 2 hours ago | parent

Do you think e.g. students learning docker syntax at a uni is beneficial in 2026 or you think LLM-assisted programming means that Docker will become an abstraction under an abstraction and its setup will be mostly automated by LLMs? I find myself very rarely debugging or handwriting dockerfiles nowadays!

craftkiller 1 hour ago | parent

This is a strange question because you could learn Dockerfile syntax in a single night. There are only 18 directives, and the average Dockerfile uses like 6 of them. 90%+ of Dockerfiles is just "run this shell command" and "copy this file into the docker image". The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.

https://docs.docker.com/reference/dockerfile/

xyzzy_plugh 1 hour ago | parent

I'm gonna be real with you. No one needs to know docker syntax. Most engineers just copy and paste chunks of Dockerfiles around anyways.

Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.

Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.

LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.

edit: this isn't to say the syntax is good or bad (though there are several questionable parts of it) but extremely simple, and concise enough that you can just look up literally any command with great ease. It's surely a large part of its success, for better or for worse.

davnicwil 2 hours ago | parent

I'm a big fan of docker and mainly just want to say thanks for bringing it about.

I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...

After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.

It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.

I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)

Fabricio20 1 hour ago | parent

Thanks for developing such a valuable tool!

999900000999 1 hour ago | parent

Thank you.

For one, did you get rich beyond all comprehension and did you start a band with your wealth?

Second, do you still program ?

I was lucky enough to get in this industry when Docker was new. Sure beats trying to figure out why X happens on machine A and doesn’t on machine B.

shykes 58 minutes ago | parent

I did make a bunch of money with Docker, something I am eternally grateful for.

I did not start a band, but I did build a nice house, help my family etc.

I still program everyday, working on my second startup, https://dagger.io (please forgive the plug). We have a very active discord if you want to come say hi.

999900000999 41 minutes ago | parent

I work heavily with CI/CD. From a quick glance Dagger looks very promising and I might experiment with it this weekend.

I guess my follow up question is once you’re financially independent, what continues to motivate you?

Is it fun?

Do you feel like you’re creating tools that the industry needs.

I imagine if I ever get rich I’d work on music and small games.

Video games that you buy once and get to play forever, self host, etc. With a full open source commitment within 5 years or so. I’d probably create a game studio in a lower cost of living country though.

operator-name 1 hour ago | parent

I’d love to hear more about some limitations/tradeoffs of docker that you made at the time that you regret in hindsight. Was there anything that you wish didn’t get baked into the “default norms” for containers, or areas that would benefit from more attention and redesigns?

I’ve been following dagger and it’s been really interesting to see the evolution of the design, and new set of abstractions. There’s also nix which has seen popularity, with their approach of building layered oci containers.

potsandpans 59 minutes ago | parent

A while back you had this tweet which often gets cited, or at least _i_ personally think about it a lot.

> If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task!

How are you feeling about WASI today?

cole_dev 56 minutes ago | parent

Thanks for your innovation and helping developers! If Docker hasn't been your focus in recent years, what have you mainly been working on and what's been your favorite?

stroebs 49 minutes ago | parent

Were you around for the big Moby open sourcing? What went down from your perspective? If I recall, it was a fairly dramatic move that was not driven by the interests of Docker corp.

verdverm 47 minutes ago | parent

What learnings did you take from Docker when conceiving Dagger? Or is it more about unlocking BuildKit?

memset 19 minutes ago | parent

I was a dotcloud customer around 2012. You might have responded to one of my many support emails.

suprjami 2 hours ago | parent

Always nice to see someone demystifying a complex thing by learning the building blocks it's made from.

Knowing this, you might find rootless Podman's fork-exec method of starting containers to be more sensible. I certainly do. No root, no daemon, no socket.

If you want to learn more about the Linux Kernel namespaces which work together to make a container, this LWN series is really good: https://lwn.net/Articles/531114/

QGQBGdeZREunxLe 2 hours ago | parent

A good talk on the tech behind containers.

Containers From Scratch • Liz Rice • GOTO 2018

https://www.youtube.com/watch?v=8fi7uSYlOdc

tyingq 2 hours ago | parent

Poke around in bocker[1]. For technical people, it makes the same points the article makes in a way that's very clear.

[1] "Docker implemented in around 100 lines of bash" https://github.com/p8952/bocker

gzapp 2 hours ago | parent

That part about the threading was a huge source of issues back in the early days. Goroutines could hop threads and it's the threads namespace/cgroup configuration that gets inherited when forking..

One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.

dboreham 1 hour ago | parent

Never, ever, ever, write your own userland threading library.

(rule broken by almost every language in common use, unfortunately)

creata 12 minutes ago | parent

Where does this advice come from? Most web servers would use a lot more resources if they didn't use userland threading, wouldn't they?

j16sdiz 1 hour ago | parent

Every statement in the article make me ask: "what's weird about it?"

Everything in that list feel natural when you know how things evolves over many years. Maybe it is just me getting older and less curious.

paxy 1 hour ago | parent

A lot of people start using Docker as a black box and have the mental model of a VM, and when they start digging into the internals all the "weirdness" starts to exhibit itself.

dboreham 58 minutes ago | parent

These days they may as well have the mental model of Docker when they start using a VM :)

j16sdiz 41 minutes ago | parent

Maybe. That's why i start thinking i am getting old and less curious :(

stingraycharles 1 hour ago | parent

And not a single mention of the actual weird decision, a client / server model which complicates permission management by at least two orders of magnitude.

slowin 1 hour ago | parent

This is why I prefer Podman.

modeless 1 hour ago | parent

Yeah, the daemon is the real problem with Docker.

schmichael 8 minutes ago | parent

lxc never had a daemon but Canonical added one (lxd in 2015) and lxc forked it to incus to be community managed.

rkt (2013) was CoreOS's alternative to Docker and was daemon-less and failed.

Podman came along relatively late (2018) and benefited from a lot of Linux kernel and systemd improvements since Docker was designed.

So while arguably the client/server model is outdated for container management tools, I'm not sure it's weird. This is my workstation for example:

  schmichael@fedora:~$ ps -e -o comm= | rg 'd$' | wc -l
  67

Daemons are maybe the least weird paradigm you can choose for a Unix program.

assimpleaspossi 1 hour ago | parent

>>Images are just folders that Docker puts somewhere and then calls chroot (or pivot_root) to make that folder your container process’s new root directory.

And folders are graphical representations of directories--sometimes, not always--and not directories themselves. When will the newbies get this?

sophacles 1 hour ago | parent

Directory is a bad metaphor too.

altcognito 57 minutes ago | parent

The high level abstraction is more important than the low level implementation unless the low level implementation is leaking through, or there is a significant advantage conveyed by understanding the low level implementation.

I know a lot about how http works, and it comes in handy, but I can understand how juniors get away with not knowing it as fully as I do.

slowin 1 hour ago | parent

I prefer Podman most of the time, but Docker is/was amazing and a great DX put on top of the so called "5 different linux kernel features in trenchcoat". That's the selling point, it's super lightweight and takes full advantage of the OS features that were already there (or being developed) but were awkward to use and required tons of specialized knowledge.

kvirani 18 minutes ago | parent

Does Postman sidestep these weird things about Docker? Or just some of them?

kazinator 1 hour ago | parent

> And if you know where to look, you can even find that process in /proc.

This is very important for debugging! Not just that process, but the processes running in that Docker can be found in /proc.

Why this is useful is that you can run gdb outside of that docker and point it to a process in the docker, including setting the file system root to the docker one, so that it gdb reads the correct objects from the correct paths, to get the right symbols and addresses.

miladyincontrol 1 hour ago | parent

For as "terrible of a language" Go is alleged to be for a container runtime, its not like crun or youki are exactly faster in many meaningful ways unless you count churning containers running true and exiting.

And implying the system kernel still has to run the container kinda ignores kata containers and gvisor runtimes.

0x457 11 minutes ago | parent

It's about using go to develop container runtime being terrible and not that "if it was C it would be faster". OS runs the container, docker just does the setup and some managmenent.

0xbadcafebee 53 minutes ago | parent

Docker hasn't required root for 7 years, I wish people would stop repeating that. It uses a root daemon by default because it makes things easier. Also, Containers are only as fake as a "Desktop" in an OS, which isn't real either. It's a user, network & process namespace with a control bus and a bunch of other weird things. A Desktop is a Container with a GUI.

Other weird facts about OSes: a filesystem is a database, the Linux kernel is a monolithic program, "hidden" dot-files were a Unix bug that got carried forward, deleting a file doesn't remove the file from your machine, virtual memory is a lie, your computer can't run multiple programs at once (it's just running one at a time real fast), in 12 years most systems will crash due to one number getting too big, you technically never need to reboot because you can launch a different version of Linux while already running Linux, you can freeze a program to a file and then later resume running that same program in the same state (but nobody uses it), files aren't actually stored in directories, there's a maximum number of files (and size of file) on most filesystems (which you can use to lock up a machine, despite file quotas existing and nobody using them), you can lock up most systems by forking too many times / using too much RAM / accessing disk too much, there's a default number of open files and/or TCP ports which is artificially low and you can use to again lock up a machine, pipes are just two file descriptors connected to one memory buffer, an IP address is just one long number split up into bytes, etc.

fulafel 31 minutes ago | parent

> I decided to make my own container runtime for fun [...] Unlike VMs, containers are just processes

Try to make your own VM runtime next, you might be surprised!

schmichael 18 minutes ago | parent

> Go is kind of a terrible language for a container runtime

I love Go and use it for everything I can but will absolutely concede this point. It's clearly possible to write the most successful container runtimes on earth in Go, but you absolutely have to fight the language/runtime sometimes. Since containers are based on processes, handing off a container spec to a minimal rust runtime seems like what we should all be doing by now. crun has just proven good enough that AFAIK no one has RIIR.

adrianlee_dev 14 minutes ago | parent

This is a better way to understand it: we can think of a container as an isolated process. When deploying containers, having a clear understanding of the boundaries between containers and the host machine in terms of permissions, resources, and data is extremely helpful for troubleshooting issues.

thayne 7 minutes ago | parent

Today, I would definitely choose rust over go. And I lament that so much container software uses go rather than rust. But back when docker started rust was a lot less mature and probably wouldn't have been a good choice.