How We've Scaled Dropbox

I found the Dropbox lecture [1] at Stanford one of the most riveting things ever. There is just so much technology behind Dropbox, it is staggering. There is a reason why it is so much better than iCloud sync, Google Drive, Box or OneDrive. [1] ▶ this video

imo the best way is to look at what other's have built. Here's some of my favorite talks that go from 0 users to millions. Dropbox - ▶ this video Instagram - link Slack (bonus. not as applicable but good reminder of why initial architecture does matter) - link

Dropbox is a lot more than a GUI on top of rsync. Even purely from an engineering standpoint (ignoring product & design) that's incorrect. You might enjoy this talk: ▶ this video

Applications dont really need to be well architected until they are hitting scale. Then the parts of their system that need to relieve pressure will need to be re-architected. This is almost like a case study and there are a lot of good talks on youtube from places like dropbox and facebook that explain the problem and solution. Example: ▶ this video If you dont want to do youtube case studies there are also books to read about distributed systems. Also reading about cloud architecture can help.

This is a talk about the evolution of Dropbox's architecture from 2012: ▶ this video . It is incredibly detailed (down to the exact sql schema they use for file metadata etc).

Postgres is >50x slower for range queries(example below) and is akin to using array-of-pointers (ie Java) whereas MySQL supports array-of-struct (C). Illustration from Dropbox scaling talk below. Sneak peek photo [1] (from [2]). Just imagine its literally 500-1000x more convoluted per B-tree leaf node. That's every Postgres table unless you CLUSTER periodically. [1]: link [2]: link Mind boggling how many people aren't aware of primary indexes in MySQL that is not supported at all in Postgres. For certain data layouts, Postgres pays either 2x storage (covering index containing every single column), >50x worse performance by effectively N+1 bombing the disk for range queries, or blocking your table periodically (CLUSTER). In Postgres the messiness loading primary data after reaching the B-tree leaf nodes pollutes caches and takes longer. This is because you need to load one 8kb page for every row you want, instead of one 8kb with 20-30 rows packed together. Example: Dropbox file history table. They initially used autoinc id for primary key in MySQL. This causes everybodys file changes to be mixed together in chronological order on disk in a B-Tree. The first optimization they made was to change the primary key to (ns_id, latest, id) so that each users (ns_id) latest versions would be grouped together on disk. Dropbox scaling talk: ▶ this video If a dropbox user has 1000 files and you can fit 20 file-version rows on each 8kb disk page (400bytes/row), the difference in performance

An important detail which this guide leaves out: The "actual location of the row" is usually the leaf node of another B-Tree. That is the primary B-Tree and it's indexed by the primary key. The major consequence is every non-primary index query involves dereferencing N pointers, which could mean loading N pages from disk/SSD to get leaf nodes spread out across the primary tree. Whereas if you query a range in the primary B-Tree directly, the rows are consecutive so you would only load N/M consecutive pages based on the ratio of rowsize to pagesize. That's why some people use composite keys for primary key, to get better data locality in the primary B-Tree index. See "How We've Scaled Dropbox"[1] to hear the same thing. At 48:36 they explain why they changed from PRIMARY KEY (id) to PRIMARY KEY (ns_id, latest, id). ns_id is kinda like user_id. So that change groups journal entries for users together on disk Specifically. PRIMARY KEY (id) orders things by creation date whereas PRIMARY KEY (ns_id, latest, id) orders things by ns_id primarily. [1]: ▶ this video

> Applications don't really need to be well architected until they are hitting scale. very True, 'a system Well architected' before hitting scale is considered OVER Engineering > Then the parts of their system that need to relieve pressure will need to be re-architected. > This is almost like a case study and there are a lot of good talks on youtube from places like dropbox and facebook that explain the problem and solution. Example: ▶ this video