Netdev 0x16 - Keynote: It's Time to Replace TCP in the Datacenter

You can obviously build an RPC protocol on top of TCP, people have been doing it for decades. However there are things that are difficult to efficiently implement on top of TCP (multiplexing different logical streams over the same connection, message priorities) in the TCP model, and some things that are impossible to implement over TCP (for example, you don't really have any ability to influence what happens during congestion control scenarios). There are also a bunch of things about TCP that are fundamentally inefficient on modern hardware. Do these things matter if you're running some RPC service over the internet with a Python backend? Almost certainly not. But if you read John Ousterhout's paper or watch his Netdev talk^[1] it's also pretty clear that this isn't the use case he's been talking about. He's talking about RPC in the datacenter, where in theory modern hardware is capable of delivering RPC messages with microsecond scale latency but in practice no RPC systems built on TCP come anywhere close to that. The Netdev talk I've linked below is really good and John makes a really thorough presentation of his ideas and what he's actually trying to optimize for. [1]: ▶ this video

If you'd like to watch Ousterhoust explain Homa and his vision of replacing TCP: ▶ this video Spoiler: Ousterhout is right, and the data shows he is right. Maybe not everyone knows it yet, or it's the kind of "right" that not everyone will ever get to (and we'll settle for "good enough" for much longer), but he is almost certainly right. Not to disparage the poster of this article of course -- I always love a good hot take or contrarian piece.

If you want to watch Outsterhout in talk form: Netdev 0x16 - Keynote: It's time to replace TCP in the datacenter: ▶ this video Netdev is an amazing conference btw, it is insane that we get access to the trailblazing being shown off at conferences for free on places like YT.

I think there are a few misunderstandings there (the video is funny, btw). The kind of async IO I'm referring to exists one layer below whatever Apache and node are using. Both are event-driven, actually. They both use some flavor of epoll (see link ). Historically, buffered IO was sufficient to circumvent slow threads. Simply because buffered IO operations usually don't block (you merely memcpy, and the kernel flushes asynchronously in the background). That can only take you so far, however, and it's apparent today that hardware can go much further and that the gap is widening. It's a valid point, though, to question whether that additional performance is even needed. John Ousterhout ( ▶ this video ) is currently working on a new network protocol to alleviate some of the problems TCP creates in software wrt performance, and he also questions whether there is even a need for very fast networking for real-world applications. IMO, the mere existence of stuff like DPDK is proof enough. Many folks use it and would rather use the kernel if it could provide comparable performance.