Clojure Concurrency - Rich Hickey

Rich presented Clojure to our local software development group in 2008, I've been working with Clojure (both as a hobbyist and eventually professionally) ever since. ▶ this video At that time I had been working with Java for years, both the conciseness of the code and the interactivity were flat-out amazing to me. It's saved me time and made me 100 times more productive. Congratulations to Rich, I am also extremely grateful for his work!

I started to write a response but remembered rich hickey talk I went to where he lays out problems with java style concurrency Clojure Concurrency - Rich Hickey ▶ this video Even though the talk is called clojure concurrency, first half of the talk is about the problems clojure solving in traditional concurrency. one my favorite talks I ever went to.

> I have yet to encounter a single problem that is solved more elegantly in Clojure than the equivalent Pythonic solution. A good place to start would be implementing Rich Hickey’s ant colony demo in both Python and Clojure. ▶ this video

> No, no, no. If you are writing multithreaded programs, you _will_ have locks, period. The question is whether you explicitly use them or if they are hidden somewhere in your libraries, frameworks, etc. Actually, the latter are much more difficult to debug. You say that and then proceed to call the article ridiculous. Well, I write and maintain large applications in Clojure. The code uses a multitude of threads for various purposes, with different lifetimes. And I've yet to see a single explicit lock. What's more important, though, is that I've yet to see a race condition, a deadlock, or data corruption, which I find pretty impressive. I do agree that you need to understand what is happening — but I disagree with your assumption that the basic tools are all on the same level. They are not, which Clojure clearly demonstrates. I won't go into details, because Rich Hickey explains it much, much better than I ever will be able to — here's an example of his talk: ▶ this video

I like Rich talks and his main point for his invention seems to be: to get concurrency right. Clojure Concurrency ▶ this video

This is maybe a bit too specific to Clojure and more generally about concurrency in it, but Rich Hickey introduced Clojure with an ant simulation demonstration. The video quality is not great, but there's an article (sadly not up anymore, but on the way back machine): link Video directly: ▶ this video

In the talk "Clojure Concurrency" [1], Rich Hickey demands that everyone in the room read "Java Concurrency in Practice" [2]. "It will scare the crap out of you." [1] ▶ this video [2] link

Well, when you pass a variable around, it doesn't and cannot change. This means that different threads can't get in eachothers' way anymore, for instance, but also that you can't make a big chunk of mistakes at all. They also have serious disadvantages : they can't be memory managed in the traditional way (since they tend to reuse other instances' memory in complex ways), and thus require a GC (refcounting can work, but ...). They are VERY allocation intensive, and they are worse than most non-persistent data structures. Assuming an O(1) allocator they can match non-persistent data structures in O-ness (ie. when making an invalid assumption that is quite popular in academia. In practice memory allocation is O(1) for small values, then O(n^2) once you get close to the system's memory capacity (scanning for holes in a long list) but don't go over it, and then O(oh fuck it let's just reboot this bloody BOAT ANCHOR) when crossing that line). Clojure is famous for having good persistent data structures. Rich Hickey went touring academia touting the benefits of immutable/persistent/functional data structures : ▶ this video There's also a famous book: link

I also find it to be the exact other way around. A lot of mathematical transformations, on the inside, break down neatly into sequential mutations, and you really do need to express them imperatively for maximum performance. Fortunately, you can look at the math and verify they compose appropriately, and use testing to verify you've implemented each step correctly. But if I'm doing anything involving state, particularly interdependent state like a simulation, functional algorithms make a lot of sense as a way to ensure that I always have a valid object handle (because I'm always returning a new, valid object as a result of a computation) and give you a somewhat easier route to refactor while maintaining the same semantics (because there are no rogue mutations or global state updates I need to conform to), as well as parallelizing updates to the object graph for performance. Clojure's semi-famous "ants" demo is a good example of this. ▶ this video link