intermediate

Inverted index

Understand term-to-document maps, postings lists, scoring, and refresh behavior as the core search data structure.

An inverted index maps each term to the list of documents (postings) that contain it — the core structure behind fast full-text retrieval. Instead of scanning every document for a word, the engine looks up the term and intersects posting lists.

					term "keyboard" → [doc1, doc7, doc42]
term "wireless" → [doc1, doc9]
query AND      → [doc1]
				

Each posting often carries positions and payloads for phrase queries and scoring. Segments are immutable on disk; new writes go to fresh segments until merge compacts them. Refresh exposes recent segments to search readers.

On interviews: explain why inverted indexes excel at token lookup but struggle with arbitrary substring or unindexed field scans; connect postings to BM25 scoring; mention merge and refresh as latency vs freshness knobs.

Common pitfalls: expecting fast `contains` on non-analyzed substrings; huge posting lists on stopword-like terms without filtering; not understanding that deletes are often tombstones until merge; conflating inverted index with B-tree primary-key lookup.

The trade-off is blazing token-oriented retrieval versus storage overhead, merge CPU, and the requirement to plan fields and analyzers up front.

Checklist:

  • Define term → postings list mapping.
  • Explain boolean/phrase query via list intersection.
  • Tie segments, merge, and refresh to search visibility.
  • Contrast with forward document scan and B-tree equality.
  • Note scoring uses term frequency in postings.