intermediate
Inverted index
Understand term-to-document maps, postings lists, scoring, and refresh behavior as the core search data structure.
An inverted index maps each term to the list of documents (postings) that contain it — the core structure behind fast full-text retrieval. Instead of scanning every document for a word, the engine looks up the term and intersects posting lists.
term "keyboard" → [doc1, doc7, doc42]
term "wireless" → [doc1, doc9]
query AND → [doc1]
Each posting often carries positions and payloads for phrase queries and scoring. Segments are immutable on disk; new writes go to fresh segments until merge compacts them. Refresh exposes recent segments to search readers.
On interviews: explain why inverted indexes excel at token lookup but struggle with arbitrary substring or unindexed field scans; connect postings to BM25 scoring; mention merge and refresh as latency vs freshness knobs.
Common pitfalls: expecting fast `contains` on non-analyzed substrings; huge posting lists on stopword-like terms without filtering; not understanding that deletes are often tombstones until merge; conflating inverted index with B-tree primary-key lookup.
The trade-off is blazing token-oriented retrieval versus storage overhead, merge CPU, and the requirement to plan fields and analyzers up front.
Checklist:
- Define term → postings list mapping.
- Explain boolean/phrase query via list intersection.
- Tie segments, merge, and refresh to search visibility.
- Contrast with forward document scan and B-tree equality.
- Note scoring uses term frequency in postings.