JavaScript SDK 0.3.0

Released .

JavaScript SDK 0.3.0 brings conditional Catalog changes, complete statement snapshots, and explicit write deadlines to TypeScript and JavaScript applications. The release is available as scopedb on npm. For a new application, start with the Node.js SDK guide.

What's new

  • Read, preview, and replace complete Catalog definitions with ETags to protect against concurrent changes. Definitions preserve extension fields and support views, computed columns, and destructive-change controls.
  • Observe statement status, progress, results, and errors through one snapshot. Snapshots are isolated from caller mutation, and concurrent observations cannot overwrite a terminal result with an older state.
  • Inspect structured server errors and interrupted write outcomes before deciding whether to retry. Unknown Append outcomes remain unretried by default; .retryUnknown(true) adds an explicit opt-in for applications that can accept duplicates.
  • Use a 4 MiB default Append batch target, gzip compression for direct appends, and retries bounded by count and elapsed time. Retries honor server refusal and Retry-After.
  • Receive row and column diagnostics for malformed results, preserve columns named __proto__, and reject duplicate column names during object conversion. Cancelled flushes retain delivery outcomes for the next report without double counting.

Upgrade from 0.2

The changes below apply when upgrading from the published scopedb 0.2.2 release.

Update the dependency

Node.js 20 remains the minimum supported version, and the package remains ESM-only. Update the dependency and commit the package manifest and lockfile:

npm install scopedb@0.3.0

The package and import names remain scopedb. CommonJS applications can load it with await import("scopedb") inside an async function.

Replace Ingest with Append

Previous APIReplacement in 0.3
Client.insert() / Client.ingest()client.table(name).append(ndjson)
Client.ingestStream()client.table(name).appendStream()
IngestStream / IngestStreamBuilderAppendStream / AppendStreamBuilder

Append writes rows whose fields match an existing destination table. It does not accept a ScopeQL transform. Move field renaming, conversion, filtering, and routing into your application before sending. Using the client from the connection example, this snippet requires an existing events table with occurred_at timestamp and name string columns:

const source = { ts: "2026-10-23T12:00:00Z", name: "page_view" };
const stream = client.table("events").appendStream().build();
await stream.send({
  occurred_at: new Date(source.ts).toISOString(),
  name: source.name,
});
const result = await stream.shutdown();
console.log("Written rows:", result?.num_rows_inserted ?? 0);

Date.toISOString() retains milliseconds. Keep an already valid RFC 3339 string when sub-millisecond precision matters, and follow the Append JSON encoding rules for structured values, missing columns, and exact integer input.

For a transformation that needs ScopeQL, append raw events to an object column in a staging table, then run a separate FROM ... SELECT ... INSERT INTO ... statement. These are separate writes; track processed batches before retrying the transformation.

Review these delivery contracts when replacing an Ingest stream:

  • send() and sendAll() queue rows; they do not confirm delivery. Await flush() to settle admitted writes while keeping the stream open, or shutdown() when finished.
  • Direct append() returns AppendRowsResult. In the default stop policy, flush() and shutdown() return AppendRowsResult | null; read num_rows_inserted, with null for an empty barrier. Continue mode returns AppendDeliveryReport; inspect outcome, committedRows, failedRows, and unknownRows.
  • Reports cover activity since the previous returned report. stats() counters cover the stream's lifetime and are not reset by a flush. Aborting a flush stops local waiting; its delivery outcomes remain in the next report.
  • Keep each batch within 8 MiB of uncompressed NDJSON and 200,000 rows. Reduce old batch-byte targets above that limit. The default target is 4 MiB.
  • The default is four concurrent batches per stream. Set .maxConcurrentBatches(1) for sequential requests or to reduce pressure across producers. Query result ordering still requires ORDER BY.
  • Streams retry confirmed temporary rejections. Unknown outcomes remain unretried unless .retryUnknown(true) is enabled, which can duplicate rows. Client timeouts and cancellations are never replayed. .maxRetries(0) disables retries; direct append() makes one attempt.

For long-lived best-effort sinks, use the continue-mode telemetry recipe. For application-owned retry loops, follow the retry guidance for confirmed rejections.

Update Catalog reads and mutations

Replace paginated list*, fetch*, and iterate* calls with a resource selector followed by get(). Reads return a complete definition and opaque etag. In the table below, catalog is client.catalog().

Previous APIReplacement in 0.3
List databasescatalog.get()
List schemas in a databasecatalog.database(db).get()
List tables in a schemacatalog.schema(db, schema).get()
Fetch a table / describe() / tableSchema()table.catalog().get()
table.apply()table.catalog().apply(definition)
table.drop()table.catalog().delete(options)
distinctOn / distinct_ondefinition.distinct with on and by
dataRetentionDays / data_retention_daysdefinition.retention_days
partitionBy / clusterBydefinition.partition_by / cluster_by
Column dataTypeColumn data_type

Definitions use snake_case fields and preserve extensions. apply() replaces the entire selected resource, including children; omitted fields are cleared. Read the definition, edit it, and carry its ETag through a dry run and the mutation. Potentially destructive changes require allowDestructive: true. Follow the catalog examples for a complete read, preview, and conditional update. Mutations never retry automatically.

Update statements and query limits

Previous APIReplacement in 0.3
withExecTimeout("30s")withMaxElapsedSeconds(30)
withMaxParallelism(...)Removed
handle.lastStatus()handle.snapshot()
handle.status() / fetchOnce()handle.refresh()
handle.fetch()handle.wait()
Handle progress() / resultSet()Inspect a snapshot or call wait()
StatementStatus*StatementSnapshot*
FetchOptionsWaitOptions

Execution limits use positive seconds instead of duration strings. withMaxElapsedSeconds() includes queueing time. withMaxEstimatedSeconds() separately rejects statements whose estimated duration exceeds a limit; it does not control parallelism. Omitted limits use service defaults.

snapshot() is synchronous and local; refresh() observes the server while the statement is active. Both return caller-owned copies. cancel() returns a complete snapshot, which may still be active or already finished; call wait() when necessary. Confirmed cancellation makes wait() throw a ScopeDBError with kind === "StatementCancelled". Aborting a local signal only stops waiting.

When matching snapshots, read failed.error instead of failed.message. Cancelled snapshots no longer carry a message. Retain the handle after a failed status read instead of resubmitting. For a submission whose response might be lost, supply a known UUID with withStatementId() and reconcile through client.statementHandle(id).

Review write deadlines

Append attempts and Catalog mutations now default to a 30-second timeout, including reading the response body. Override it with ClientOptions.writeTimeoutMs, a direct write's timeoutMs, or the stream's .attemptTimeoutMs(). Omit the option to inherit the default, use a positive integer for milliseconds, or use null to disable that attempt deadline. Zero is invalid.

Streams also have a five-minute per-batch delivery budget covering requests and retry delays, configured with .maxElapsedTimeMs(). Disabling the attempt deadline does not disable that budget or caller cancellation. Query waiting has no extra SDK deadline unless you supply a signal; service execution limits still apply.

A timeout does not roll back a write. Inspect Append outcome and writeDetails, or Catalog's catalog_commit_outcome_unknown error code, before deciding whether to retry. See timeouts and error handling.

Rename remaining fields and methods

Previous APIReplacement in 0.3
ClientOptions.tokenapiKey
table.withDatabase() / withSchema()client.table(name, { database, schema })
jsonRows() / intoValues() / intoObjects()rawRows() / toValues() / toObjects()
IntoOptionsResultOptions
executeOne()(await statement.execute()).first()
batchBytes / flushIntervaltargetBatchBytes / flushIntervalMs
maxPendingBytes / maxInFlightRequestsmaxBufferedBytes / maxConcurrentBatches
attemptTimeoutattemptTimeoutMs
Append result append_statestatus
AppendRowsError.appendStateoutcome
Separate stats().lastFailure message/status fieldsstats().lastFailure.error
u_intuint
Low-level Client transport methodsTable.append(), Statement.submit(), and handle methods

Table.identifier() now always includes database and schema, defaulting to scopedb and public. Integer results still default to exact bigint; use integerMode: "string" for JSON output. Preserve original NDJSON text when importing integers that JavaScript numbers cannot represent exactly.