Simple Code, High Performance

Thank you for the detailed explanation! It sounds like programming computers back then would have been a joy. Casey Muratori's talks[0] on performance and optimization sparked an interest in me to start putting thought into what the CPU actually does (which wasn't taught when I learned web development). Since discovering his talks, I've been on the hunt for more learning materials. [0] ▶ this video

The timing of this is great! I just watched Casey Muratori's lecture, "Simple Code, High Performance"[1] and its follow-up videos[2, 3]. I highly recommend watching it. It is about Software Optimization, and gives practical and high-level understanding of topics like CPU Architecture, SIMD etc. [1] ▶ this video [2] link [3] link

> Speak for yourself; most phones I see still take 2+ seconds to open some apps. Why is that though? I've written code since the 8 bit days of BBC micros and ZX Spectrums. Casey Muratori's (who has been on here a lot lately) talk nails it [1] - developers just don't realise how fast their code could and should run. And the fact that we have layers upon layers of frameworks and no one cares. [1] ▶ this video

Casey Muratori started a whole educational webseries about this - the fact that hardware is getting faster and faster but the user experience seems to be staying the same. ▶ this video He also has a "performance-aware programming" series: link

Most software in the industry is slow because it's doing a lot of stuff that it shouldn't. Often times additional "optimization" layers adds caching, but makes getting to the root of the issue harder. The biggest win is primarily getting rid of things you don't need and secondarily operating on things in batch. My playbook for optimizing in the real world is something like this: 1. Understand what you're actually trying to compute end-to-end. The bigger the chunk you're trying to optimize, the greater the potential for performance. 2. Sketch out what an optimal process would look like. What data do you need to fetch, what computation do you need to do on this, how often does this need to happen. Don't try to be clever and micro-optimize or cache computations. Just focus on only doing the things you need to do in a simple way. Use arrays a lot. 3. Understand what the current code is actually doing. How close to the sketch above are you? Are you doing a lot of I/O in the middle of the computation? Do you keep coming back to the same data? If you want to understand the limits of how fast computers are, and what optimal performance looks like I'd recommend two talks that come with a very different perspective from what you usually hear: 1. Mike Acton's talk at cppcon 2014 link 2. Casey Muratori's talk about optimizing a grass planting algorithm ▶ this video

For one real application of blue noise in particular, see ▶ this video (22:30 is a good starting point). Here Casy explains a grass planting algorithm for games, where white noise wouldn't be suitable.

It's Casey Muratori, he's an American gamedev, not a professor. The video was a recorded guest lecture for a uni in the Netherlands though. And it wasn't redone in assembly, it was C++ with SIMD intrinsics, which might as well just be assembly. ▶ this video

1. Single-thread performance This is hardly related to HPC. The closest thing maybe code optimization and competitive programming. First learn how to do anything fast in single process, then think about scale it up later. 2. Parallel programming Learn MPI. Specifically mpi4py. Once you go MPI, you never use python's multiprocessing module (it sucks). But stuffs are mostly about solving linear algebra problems, which I am not that interested in. 4. Distributed Computing Related to MPI. But nowaday distributed GPU computing become a thing, thing that I can think of is Dask. Don't know how to use that. link 5. Quantum Computing Completely unrelated to HPC, at least in the near future (10 to 30 years). Areas that is the most active in this field is molecular computation. If you don't have a phy/chem PHD, you are not able to read anything beyond college level stuff in this field. And those are only basics stuff like elementary schools' arithmetics. But write your own program to solve those quantum problems in classical computers is interesting, because people there only think about problems that cause scaling issue, not the others. That's why some may think that quantum computer is a replacement of supercomputers. If you can find ways and invent techniques that can reduce timecomplexity in terms of classical algorithm, you are welcome. And literally there is a lot of room for anyone because every quantum computer emulator is unoptimized and slow as hell. Google, Microsoft, IBM,

Blue noise is useful in games for generating more natural looking scenes. Here's an example of how to algorithmically place grass in the game "The Witness". ▶ this video So if you use white noise, it looks unnaturally clustered with patches. This is the part that discusses the noise. ▶ this video

There are no tricks, you just have to give up mountains of levels of abstraction accumulated over the years. ▶ this video