CppCon 2014: Mike Acton "Data-Oriented Design and C++"

It clearly didn’t come out of game dev. Many people doing high performance work on either embedded or “big silicon” (amd64) in that era were fully aware of the importance of locality, branch prediction, etc But game dev, in particular Mike Acton, did an amazing job of making it more broadly known. His CppCon talk from 2014 [0] is IMO one of the most digestible ways to start thinking about performance in high throughput systems. In terms of heroes, I’d place Mike Acton, Fabian Giesen [1], and Bruce Dawson [2] at the top of the list. All solid performance-oriented people who’ve taken real time to explain how they think and how you can think that way as well. I miss being able to listen in on gamedev Twitter circa 2013 before all hell broke loose. [0] ▶ this video [1] link [2] link

This reminds of an amazing talk by Mike Acton on Data oriented design: ▶ this video And there is a related talk by Andrew Kelley implementing these ideas on Zig toolchain: link

A lot of great talks already mentioned. Thought I'd add a gem from Mike Acton: Data-Oriented Design and C++ ▶ this video

Most software in the industry is slow because it's doing a lot of stuff that it shouldn't. Often times additional "optimization" layers adds caching, but makes getting to the root of the issue harder. The biggest win is primarily getting rid of things you don't need and secondarily operating on things in batch. My playbook for optimizing in the real world is something like this: 1. Understand what you're actually trying to compute end-to-end. The bigger the chunk you're trying to optimize, the greater the potential for performance. 2. Sketch out what an optimal process would look like. What data do you need to fetch, what computation do you need to do on this, how often does this need to happen. Don't try to be clever and micro-optimize or cache computations. Just focus on only doing the things you need to do in a simple way. Use arrays a lot. 3. Understand what the current code is actually doing. How close to the sketch above are you? Are you doing a lot of I/O in the middle of the computation? Do you keep coming back to the same data? If you want to understand the limits of how fast computers are, and what optimal performance looks like I'd recommend two talks that come with a very different perspective from what you usually hear: 1. Mike Acton's talk at cppcon 2014 ▶ this video 2. Casey Muratori's talk about optimizing a grass planting algorithm link

Mike Acton presented a great visualization of this in his 2014 DOD talk -- here's a link to just that clip: ▶ this video

I think this is a valid critique. But then, most of the development (C aside) has been a victim to OOP. I have begun to think that OOP creates more problems than it solves. Added complexity and performance loss being the biggest. It's insane if you peek into a big projects where many developers came and went and each of them read an article about what it seemed an interesting design pattern and tried to implement them. You can have some kind of procedural programming if you use classes just to store data and use static methods. But the frameworks are OOP and there is no way around that unless you write your own frameworks. I am a big fan of Mike Acton's Data Oriented Architecture speeches, but I didn't find a way to implement something similar in C#. For those interested in something better than OOP, here's a few links: ▶ this video link link

A great talk which gets you thinking in this mindset is "Data-Oriented Design and C++" by Mike Acton ▶ this video As an example, if you are checking a boolean flag (1 bit) on an object, and it ends up being a cache miss (and x86_64 cache line size is 64 bytes), then your computer just went through all the expense of pulling in 512 bits from RAM yet it only used 1 of them. You are achieving 0.2% of the machine's possible throughput.

Just because it's asm doesn't mean it's fast, sure. Just because it's C doesn't mean it's faster than an implementation in your favourite scripting language. Having said that, compilers do pretty badly on C-to-simd optimisation. The best you get is loop vectorization if it's really simple logic. You can usually get some pretty good wins there. The fact you lay out your memory for simd usually is a win all of its own due to cache prefetching even if you don't actually use any simd instructions. Compilers need heuristics to manage cache when you know what you're trying to do, (eg when should it use non-temporal writes, for example?) Fast C code is written while having a really clear mental model of the underlying architecture and the assembly that the C will produce with -O3 (or whatever flag is relevant to your compiler) and then checked with -S or objdump -D, profiled with callgrind/cachegrind, perf, rdtsc etc... The compiler really can't "Do it for you" You /can/ use a compiler as one of your tools when /you/ do it. As Randy Hyde points out you can always beat the compiler because you can use its generated assembly language in every case you can't beat, so the absolute worst you get is a tie. So yeah, you can totally smoke clang, Intel, microsoft and gnu C compiler and get paid something for doing it in certain industries too. :-) Mike Acton being aggressively opinionated on the subject, but the lecture is really good (despite/because of) the bits you'll disagree with and th

I'm going to disagree with most of the comments here which say that you can't become faster, as I have become notably faster over the past couple of years (perhaps 1.5-2x faster!!). Basically, it was a fairly major coding style change that happened after working with others in my industry (game development) and seeing their prolific output of high quality code. My process for writing code used to be: - Identify problem. - Identify the abstractions that fit the problem - Implement those abstractions - Write code to solve problem using those abstractions. Now it is - Identify problem. - Write code to solve problem, pulling abstractions out if and only if they make sense and would actually be used . Honestly, I feel this leads to much better, cleaner and more understandable code, however, it tends not to mesh well with what some people consider best practices. The resulting code tends to not be particularly object oriented, tends to have large functions, and tends to perform very well due to having a very small distance from what the code says to what actually needs to be executed by the processor. For more on this approach, see [0]: link [1]: link [2]: ▶ this video

> cache-aware data structures and algorithms Maybe hardware- and access-aware more generally. One of Mike Acton's other talks has a "Is Data-Oriented Design even a thing?" section, which goes over what he means when he refers to DOD: ▶ this video

> Recently someone proposed this on LLVM Well, to be honest, it's not any extraordinary idea and it's not even mine. Mike Acton used this exact example in his CppCon video ( ▶ this video at 25:58). I just happened to know that LLVM's DenseMap has this problem. On the other comment, data locality is a property (well... it's not really a binary thing (i.e, an execution has it or it doesn't), but really a spectrum; but ok, it will do) of a program. For example, a simple definition which I like is in the paper "Improving Data Locality with Loop Transformations ( link ): > Data locality is the property that references to the same memory location or adjacent locations are reused within a short period of time. Changing the the layout is just a transformation to achieve this property. So, the two things are different in the sense that one is a property while the other is a transformation, but they're not different in the sense that data layout transformations are not relevant to data locality. Regarding cache locality, that is also a property that we could define let's based on this ( link ) say as: "The property that data brought into cache should be reused as much as possible before it is replaced". Data locality depends on the hardware on which the program is executed. Since we use caches in our hardware, to have data locality you usually need to need have cache locality so these two coincide. Finally, yes you're absolutely right (well, last time I checked...). LLVM has a bunch of

Casey Muratori is basically describing Data Oriented Programming. Perhaps the most famous talk about it was from CPPCON 2014 [1]. I come from an embedded background (microcontrollers and bare metal) so I'm pretty sympathetic to his argument. If you work in it long enough, it feels natural and "clean". Uncle bob's Clean Code probably feels natural to Java devs. Personally, my biggest gripe comes from experiencing people trying to introduce it to a team. Inevitably someone tries to enthusiastically implement some part of it and suddenly are writing 1 line functions everywhere. I think this is the type of thing Casey is also implicitly talking about when he's railing against the rules being brought up. [1] ▶ this video