Building Software Systems At Google and Lessons Learned
Stanford4 HN recommenders · recommended in 4 months, 2012–2022
lecture · intermediate · evergreen · 2012–2022 on HN
Jeff Dean of Google describes his experiences building Google's systems and how its infrastructure evolved over time. The Stanford talk covers search, scaling tipping points and lessons from building large-scale software.
HN recommends it as a classic look at systems engineering at Google, whose fundamentals have held up well despite its 2010 date.
Up next
5 mo on HNSteve Jobs, "Computers are like a bicycle for our minds." - Michael Lawrence Films
Michael Lawrence
5 HN recommender · 2010–2025
14 mo on HNThe Soul of Erlang and Elixir • Sasa Juric • GOTO 2019
GOTO Conferences
16 HN recommender · 2019–2024
11 mo on HN"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
12 HN recommender · 2014–2024
10 mo on HNErlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp
Erlang Solutions
6 HN recommender · 2014–2022
9 mo on HN"Turning the database inside out with Apache Samza" by Martin Kleppmann
Strange Loop Conference
9 HN recommender · 2015–2025
9 mo on HNThe Language of the System - Rich Hickey
ClojureTV
10 HN recommender · 2013–2025
9 mo on HNGoogle I/O 2012 - Go Concurrency Patterns
Google for Developers
9 HN recommender · 2012–2025
7 mo on HNConflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016
GOTO Conferences
3 HN recommender · 2017–2020
7 mo on HN"Unison: a new distributed programming language" by Paul Chiusano
Strange Loop Conference
7 HN recommender · 2020–2023
6 mo on HNPaxos lecture (Raft user study)
Diego Ongaro
6 HN recommender · 2013–2024
5 mo on HNHewitt, Meijer and Szyperski: The Actor Model (everything you wanted to know...)
jasonofthel33t
5 HN recommender · 2015–2025
5 mo on HNAdvanced Topics in Programming Languages: A Lock-Free Hash Table
GoogleTalksArchive
4 HN recommender · 2013–2024
5 comments from Hacker News
In addition to Linus's git talk, I really enjoyed Jeff Dean's EE380 retrospective on Building Systems at Google ( ▶ this video ). Many people have mentioned Jeff's basic premise elsewhere ("Design a system for 10x your current need, but not 100x, rewrite it before then") but this talk gave several useful examples where tipping points occurred (at least with Search).
Probably this one: link Or this one: ▶ this video (terrible audio) Both are very old, but extremely well aged. IMO, the secret behind Google Search is not the smartness of the algorithms, but how much of it is baked in those O(100ms) which takes Google machines to answer your query. That's why the links above are the true reason Google Search performs well. Natural Language Processing and Information Retrieval state of the art is far beyond what Google/Bing/Yahoo/Yandex/Baidu employ. But, it's far too expensive to serve it at QPS & latency required for decent UX.
There is a lecture from Jeff Dean about it: ▶ this video It's from 2010, but the fundamentals probably haven't changed.
Also recommend Jeff Dean's tech talk Building Software Systems At Google and Lessons Learned which references those latency numbers. Slides: link Video: ▶ this video Also, a previous thread on latency numbers: link
Google is famous for publishing a lot of material every year. A couple of related publications you might be interested in: 1. The Datacenter as a computer: link 2. The SRE book: link A couple of dated presentations: 1. Jeff Dean on building software systems at Google (2010) at Stanford: ▶ this video 2. Notes by James Hamilton on a talk by Jeff Dean on Google Datacenters in 2008: link -- Microsoft Research pretty much work on bleeding edge tech, as well and publish about some of their research pretty regularly. Here's a staring point to help anyone interested find relevant publications: link