What is in a Lucene index? Adrien Grand, Software Engineer, Elasticsearch
LuceneSolrRevolution3 HN recommenders · recommended in 3 months, 2019–2020
talk · intermediate · evergreen · 2019–2020 on HN
Elasticsearch engineer and Lucene committer Adrien Grand explains the design decisions inside a Lucene index and what problems it can and cannot solve. The talk goes into how Lucene stores data for fast search and analytics.
HN recommends it as an instructive, easy-to-follow look at Lucene internals from a core committer.
Up next
1 mo on HNTypeScript types can run DOOM
Michigan TypeScript
1 HN recommend · 2025–2025
3 mo on HNC++Now 2018: Matt Godbolt “What Else Has My Compiler Done For Me Lately?”
CppNow
3 HN recommender · 2018–2020
6 mo on HNCppCon 2017: Matt Godbolt “What Has My Compiler Done for Me Lately? Unbolting the Compiler's Lid”
CppCon
7 HN recommender · 2018–2022
11 mo on HN"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
12 HN recommender · 2014–2024
2 mo on HNThe economics of open source by C J Silverio | JSConf EU 2019
JSConf
2 HN recommender · 2019–2020
11 mo on HNThe Value of Values with Rich Hickey
InfoQ
10 HN recommender · 2014–2023
9 mo on HNAWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401)
Amazon Web Services
12 HN recommender · 2019–2025
9 mo on HN"Turning the database inside out with Apache Samza" by Martin Kleppmann
Strange Loop Conference
9 HN recommender · 2015–2025
7 mo on HNConflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016
GOTO Conferences
3 HN recommender · 2017–2020
6 mo on HN01 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2019)
CMU Database Group
6 HN recommender · 2020–2023
6 mo on HNIntroduction to NoSQL • Martin Fowler • GOTO 2012
GOTO Conferences
6 HN recommender · 2015–2021
5 mo on HNRich Hickey rants about HttpServletRequest in 1080p: Death By Specificity from Clojure Made Simple
Fred Overflow
5 HN recommender · 2020–2026
3 comments from Hacker News
I can super highly recommend talks by Adrien Grand on Lucene and Elasticsearch. They are super instructive yet easy to follow. ▶ this video link
This is excellent. I was recently reviewing Lucene concepts and found this video really good: ▶ this video Also this site has a series of Lucene articles that are pretty nice. The one on Term Vectors in particular: link Based on some quick research it seems like Lucene is already using a sorted skip data structure for the posting list, so I wonder why they had to do a custom implementation? Perhaps it has to do with their custom Document ID scheme and how they want to preserve order in the Posting List being different from the default behavior. It also sounds like searchers are searching on indexes as they're being written, and there is some custom coordination around visibility, which might require diverging from Lucene default behavior. Either way, pretty impressive!
This is extremely high level. A nice view of how lucene internals work by core committer Adrien Grand: link ▶ this video