News

Inside Cerebras Knowledge: how Cerebras built its enterprise knowledge base

Cerebras Systems has published a detailed engineering write-up of Cerebras Knowledge, the internal knowledge base it built to connect staff to information scattered across the company (July 2026):

Adoption: Launched roughly three months ago, the tool now fields more than 15,000 questions a day from humans, automations and agents, making it one of Cerebras' most widely adopted internal tools.

Meeting data where it lives: Rather than force everything into a single "source of truth" platform, Cerebras extracts data directly from where it is generated — Slack threads, GitHub code, Google Docs edits and Jira metadata — requiring minimal change to existing behaviour.

Architecture: At the core sits a single Postgres table holding embeddings, raw summaries and metadata from every source. Each source defines what the data is, how to connect and how often to fetch; every row lands in the same embeddings table and is immediately queryable through one interface, with an access layer enforcing authentication, authorisation, auditing and analytics.

Hybrid Slack search: Vector search alone proved insufficient, so Cerebras fuses four techniques — full-text search for exact tokens (error strings, flags, hostnames), embedding search for paraphrase, inverse document frequency to separate signal from filler, and age decay so stale answers rank lower. No single scorer is trusted on its own.

Real-time ingestion and distillation: A Slack bot in Socket Mode streams every message over a WebSocket; on each event the ingest consumer re-fetches the whole thread and stores it as one row. An LLM then distils each thread into a normalised document — a searchable one-line question, a summary, the resolution, and the systems and code references mentioned — which is embedded instead of the raw transcript, sharply improving accuracy.

Bursting and code embeddings: To surface answers buried in long threads, Cerebras embeds "bursts" (consecutive messages from one author) that clear rarity, length and social-signal thresholds. For code, it uses the open-source CocoIndex framework to keep embeddings of repositories — some larger than 40 GB — current, splitting files along language-aware boundaries from coarse to fine.

The write-up was widely shared after Vercel's Drew Bredvick praised it on X as a clear reference for building an "ontology / knowledge base / company brain."

Read more: Cerebras · Drew Bredvick on X

More top stories