How Raft.build scales agent log analytics with ScopeDB

Raft is an agent-native workspace where AI agents work alongside people as first-class teammates in shared channels, threads, and tasks. Agents have their own identity, memory, and workspace: they claim tasks, hand work to one another, review changes, and even collaborate across company boundaries in joint channels. Every conversation in the room sets off a dense stream of activity underneath—API requests, message routing, database queries, and runtime events.

As more agents joined the rooms, the volume of operational traces and product events grew alongside them. PostgreSQL already carried Raft’s live messages, tasks, and transactional state—putting a high-concurrency stream of agent traces and long-retention event history into the same transactional database would quickly become untenable. At the same time, the team needed deeper visibility into what its agents were doing: before a release moves and in the hours after it lands, both engineers and agents need retained event history to verify how the system behaved.

Raft adopted ScopeDB as the shared source of evidence for its engineers and observability agents. Through a shared Raft channel between the two companies, ScopeDB CEO Yu Lei and Raft CTO Tenny Zhuang work alongside Raft’s observability agents—inspecting query patterns, checking materialized index coverage, and verifying release telemetry directly in the room without human-to-human relay or support tickets.

“We are ScopeDB’s customer, our tracing and observability run on them, but the shared room turned it into a two-way build: they harden their database against a real, high-concurrency agent workload, and we get an observability layer we can query and verify. Each side is the other’s proving ground.”

— Tenny Zhuang, CTO at Raft.build, in Don’t talk to me, talk to my agents

More agents, more events to investigate

One task can involve several agents and many exchanges. A teammate sees the conversation; the underlying system sees a sequence of requests, state changes, queries, retries, and handoffs across processes. Adding agents increases both the activity to record and the number of interactions the team may need to explain later.

The questions span different amounts of data. A failed operation needs a detailed trace. A release check needs a comparison across versions. A recurring problem may require looking back through weeks of events to see when it started and which operations it affects.

PostgreSQL owns Raft’s business state, including task changes and audit records. RisingWave consumes PostgreSQL changes to maintain live views such as inboxes and unread counts, while Redis handles communication and coordination between running services. ScopeDB sits alongside those workflow systems to store and analyze the high-volume stream of observations they produce.

Keep the records for questions that come later

Raft sends server traces and events to ScopeDB, and collects browser and daemon traces through its upload service. The tracing data includes service and deployment versions, trace and span identifiers, operation names, timestamps, and outcomes. A typed event table, raft.trace_events_v2, exposes frequently used fields directly to queries. Product behavior goes into a separate product.events table, with common fields such as time, platform, and app version alongside an object containing event-specific properties. You can also explore how Raft wires its trace contracts and ScopeDB writers in its public release mirror, botiverse/raft-source.

Retaining event-level records lets an investigation move directly from a high-level trend to the underlying cause. When a daily failure count spikes, an agent can drill down to the individual query that failed, inspect what the driver reported, and check whether the same pattern appears elsewhere.

Once events land in ScopeDB, new analyses can query the fields already present without building a new ingestion pipeline or precomputing a summary table. Filtering, text search, aggregation, and joins all run over the same stored data. When a field becomes important to a frequent query, an index can improve access to it in place without moving the event history into another system.

An agent can follow a failure across the system

Before a rollout, Raft’s observability agents check the release candidate’s trace fields on staging; after deployment, they compare production traces against the previous version or the same hours of the previous day to catch regressions early.

Suppose database errors spike after a release: which database operations have been failing, and are the failures concentrated in the new version? Raft records database query spans—covering core paths such as messages.insert, messages.search, and tasks.claim—with the database system, query name, execution phase, outcome, and error details, giving an investigating agent structured fields to compare across versions immediately:

The investigating agent starts by counting completed database query spans by version, query, and outcome over the last hour:

FROM raft.trace_events_v2
WHERE deployment_environment = 'production'
  AND span_name = 'server.db.query'
  AND row_kind = 'span_fact'
  AND event_time >= NOW() - 'PT1h'::interval
  AND event_time < NOW()
DISTINCT ON service_name, trace_id, span_id
GROUP BY service_version, db_system, query_name, outcome
AGGREGATE count() AS operations
ORDER BY operations DESC

ScopeQL’s pipeline syntax (FROM → WHERE → DISTINCT ON → GROUP BY → AGGREGATE) follows the same top-to-bottom sequence an agent uses to reason through a question. Chaining DISTINCT ON directly into GROUP BY deduplicates spans and compares failure rates across releases in one flat pipeline—without the nested subqueries or CTEs where agent-generated SQL often loses column scope.

The result determines the next question. When failures cluster in a specific query or release, the agent narrows the filter to outcome = 'error' and groups by execution phase, failure reason, SQLSTATE, and scrubbed driver message while pulling a representative trace ID:

FROM raft.trace_events_v2
WHERE deployment_environment = 'production'
  AND span_name = 'server.db.query'
  AND row_kind = 'span_fact'
  AND outcome = 'error'
  AND event_time >= NOW() - 'PT1h'::interval
  AND event_time < NOW()
DISTINCT ON service_name, trace_id, span_id
GROUP BY service_version, query_name, phase, reason, sqlstate, error_message
AGGREGATE
  count() AS failures,
  max_by(trace_id, event_time) AS latest_trace_id
ORDER BY failures DESC

A statement timeout (reason = 'statement_timeout', sqlstate = '57014') points to a different investigation from a connection or constraint error. Following latest_trace_id adds the surrounding request context; widening the time window checks whether the same failure predates the release, and running the same query after a fix confirms whether the pattern has cleared.

The agent returns the affected operation, versions, time range, supporting records, and the next check for the owner. ScopeDB supplies the evidence as the investigating agent works through each follow-up question.

The same database answers product questions

Raft also wants to know whether features are being used and whether people finish what they start. Its product event registry defines each event together with the question it is meant to answer.

For example, Raft lets a user change the model for several agents at once. The implementation records four events under one operation identifier:

Those records support several follow-up questions. Do people open the feature but leave before submitting? Are submitted operations completing only partially? Does that vary by app version or platform?

Because product.events pairs typed columns (uuid, app_version, platform, event, `timestamp`) with a semi-structured properties object, an agent can deduplicate event deliveries by uuid, roll up all four lifecycle steps by operation_id, and aggregate the feature funnel in a single ScopeQL pipeline:

FROM product.events
WHERE deployment_environment = 'production'
  AND event IN (
    'bulk_model_change_opened',
    'bulk_model_change_closed',
    'bulk_model_change_submitted',
    'bulk_model_change_finished'
  )
  AND `timestamp` >= NOW() - 'PT168h'::interval
DISTINCT ON uuid
SELECT
  app_version,
  platform,
  event,
  properties['operation_id']::string AS operation_id,
  properties['outcome']::string AS outcome
GROUP BY app_version, platform, operation_id
AGGREGATE
  max(if(event = 'bulk_model_change_closed', 1, 0)) AS closed_early,
  max(if(event = 'bulk_model_change_submitted', 1, 0)) AS submitted,
  max(if(outcome = 'all_succeeded', 1, 0)) AS all_succeeded,
  max(if(outcome IN ('partial', 'none_succeeded'), 1, 0)) AS needs_attention
GROUP BY app_version, platform
AGGREGATE
  count() AS opened_operations,
  sum(closed_early) AS abandoned_before_submit,
  sum(submitted) AS submitted_operations,
  sum(all_succeeded) AS completed_cleanly,
  sum(needs_attention) AS partial_or_failed
ORDER BY opened_operations DESC

This query highlights two capabilities that keep product analysis agile. First, schema on the fly lets Raft keep stable dimensions in typed columns while reading feature-specific fields (operation_id, outcome) directly from the properties object—shipping a new product event requires no table migration. Second, chaining DISTINCT ON uuid into two consecutive GROUP BY ... AGGREGATE stages deduplicates at-least-once event writes, rolls raw events up into per-operation states, and summarizes the funnel by release and platform on the spot, without building a separate ETL job or precomputing a funnel table.

Grow the history without replacing the database

The initial pressure came from log volume, but retaining useful history means planning for more of it. More agents produce more events. Longer retention gives investigators a better view of recurring problems. More people and agents asking questions adds query demand alongside continued ingestion.

ScopeDB separates durable storage from compute. Event history stays in shared object storage, while compute capacity can expand to handle more ingestion or analytical work. Separate compute groups can serve those workloads against the same committed tables. Capacity can be reduced after a busy period while the history remains available.

That architecture lets a team grow its event analysis within the same database instead of planning another migration each time the workload outgrows its initial capacity. Selective investigations use indexes for targeted lookups, while broader analyses scale with additional compute as data volume and query patterns evolve.

For Raft, the benefit is continuity: the records collected while agents work remain available when the team needs to verify that work. Whether an observability agent is comparing post-release traces across versions, an engineer is diagnosing a statement timeout, or the product team is checking whether users finish a new workflow, the evidence lives in one place. The team can keep the history and change the question.

If you want a closer look at how ScopeDB scales compute independently of stored history—so busy investigations get extra capacity without inflating baseline costs—dive into Cloud Elasticity.

Ready to try ScopeQL on your own event data? Start with the quickstart to create an event table, load semi-structured records, and run your first pipeline queries in minutes.