30% offEnding soon
FSD-026Frontend system design Premium

Design a Log Viewer for Millions of Events

Design a massive log viewer with ordered history, live tail, bounded buffers, pause and resume, search, virtualization, redaction, and recovery.

Advanced 62 min read

A large log viewer is an ordered investigation window, not a text area that keeps appending. The browser owns a bounded projection of one query generation. It merges historical pages with a resumable live cursor, rejects duplicates and stale events, exposes gaps, virtualizes the visible rows, and separates pause of presentation from pause of receipt and transport.

A production log surface may cover millions of stored events while receiving thousands more each second. The user expects to search, jump through time, expand a multiline record, select a row, copy safe fields, scroll away from the tail, and return without losing their place. Those interactions compete with an input stream that can outrun the main thread.

The browser should coordinate committed query state, ordered page merge, stream generations, bounded buffers, virtualized rendering, selection, search scope, and recovery status. The service should authorize every query, constrain cost, issue cursors, replay gaps, redact protected data, and keep the durable result.

Clarify the log investigation

Ask which query language and data sources are supported, whether order is ascending or descending, whether event IDs and sequence numbers exist, how long cursors remain valid, what Pause means, how much client memory is safe, whether multiline expansion changes row height, which fields can be copied or exported, and whether the reader needs a screen-reader review mode.

For this design, assume an authenticated operations tool. A committed query can load older history, move into live tail, pause presentation, resume from a bounded buffer or server cursor, expand structured details, and share a safe investigation link. Desktop is the primary investigation surface. Mobile supports bounded reading, filtering, selection, and pause or resume.

Building the log store, implementing the query language, and promising unlimited browser retention are outside scope.

The public interview prompt

Design a log viewer that can investigate millions of stored events and tail new events in real time. Explain:

  • draft and committed queries, generations, ranges, sort direction, and shareable state;
  • cursor pagination, stable event identity, overlap, deduplication, ordering, and gap recovery;
  • WebSocket or streaming transport, batching, backpressure, bounded queues, and overload policy;
  • distinct freeze-view, buffer-receipt, and stop-transport behaviors;
  • virtualized variable-height rows, scroll anchoring, row expansion, selection, and keyboard focus;
  • server search versus local highlight, structured fields, context lines, copy, and export;
  • loading, no-match, partial, stale, reconnecting, overflow, and authorization states;
  • redaction, tenant isolation, log injection, accessibility, performance, observability, and testing.

What the premium solution covers

The complete solution defines the ordered timeline invariant, historical and live protocols, a bounded backpressure model, three precise pause contracts, variable-height virtualization, query and local-search boundaries, safe structured rendering, accessibility, security, observability, testing, and rollout.

Original media follows a paused stream through bounded buffering and cursor recovery, compares the three meanings of Pause, maps historical pages to the live seam, and shows a virtualized log surface with a pinned selection. Two deterministic labs let the reader size a buffer and exercise duplicate, stale, gap, replay, pause, and resume events.

Premium solution

Continue with the complete system design

Unlock the architecture, state machine, API contracts, original diagrams, positioning model, accessibility decisions, performance budgets, testing plan, rollout strategy, scoring rubric, and interview walkthrough.

  • Detailed, beginner-friendly explanation
  • Production failure modes and trade-offs
  • Mobile-friendly architecture diagrams
  • Senior and staff-level interview signals
Unlock Premium

Frequently asked questions

How can a browser display millions of log events?
Keep the full result on the service and hold only bounded pages, indexes, and live buffers in the browser. Render a small virtual window, use stable event IDs, and evict data under an explicit memory policy.
What should Pause mean in a live log viewer?
Name the contract precisely. Freeze viewport stops automatic presentation, pause receipt keeps a bounded client buffer, and stop transport closes the channel. A product may offer all three, but one button should not hide which policy is active.
How should historical logs join a live stream?
The historical response returns a high watermark and opaque resume cursor. Open live mode from that cursor, deduplicate overlap by event identity, accept contiguous sequences, and replay or resnapshot when a gap appears.
Should a browser WebSocket be treated as backpressured?
No. The stable WebSocket API does not provide automatic backpressure. The application needs bounded queues, batching, admission policy, monitoring, and a recovery path when the consumer falls behind.
Should search run in the browser or on the server?
Server search defines which events belong to the result across the full time range. Client search only highlights or filters the bounded rows already loaded. Label these scopes so users do not mistake a local match count for a complete query.
How do you make a virtualized log viewer accessible?
Use semantic rows or a documented grid pattern, expose logical row count and present-row indices, keep keyboard focus stable, offer a non-streaming review mode, and avoid announcing every incoming event.