Cloud Elasticity: Three ScopeDB Design Principles (Part 3)

Total Cost of Ownership

In an era when data grows beyond massive, the Total Cost of Ownership (TCO) has become a critical metric for evaluating data infrastructure. Among the infrastructure, the cost of databases, where you store and query the data, stands out as a big challenge.

Conventional database solutions often force a difficult trade-off: provision for peak load and suffer high waste during off-peak hours, or provision for average load and risk performance degradation or outages during traffic spikes.

This rigidity stems from the coupled architecture of storage and compute found in most Shared Nothing systems. Because every compute node maintains a local replica of the data, scaling in is risky and operationally expensive—data must be rebalanced to remaining nodes before a node can be safely removed.

Consequently, engineering teams typically resort to Static Provisioning: they estimate the maximum possible traffic (peak load), add a safety buffer, and provision infrastructure to that level 24/7.

As illustrated above, the Real Workload (solid green line) fluctuates significantly throughout the day. However, due to the difficulty of scaling, the Static Provisioning (red dashed line) must remain constant at the peak level. The entire area between the red line and the blue curve represents Wasted Capacity—expensive compute and storage resources that are paid for but sit idle for the majority of the day. Ideally, an infrastructure should follow the Elastic Provisioning curve (solid blue line), closely fitting the actual demand.

Tiered Storage: A Pioneer in Cost Efficiency

To address the prohibitive cost of storing petabytes of data on high-performance block storage (like EBS), many traditional database vendors introduced "Tiered Storage." This technique attempts to offer the best of both worlds by offloading older, less frequently accessed (”cold”) data to cheaper object storage (S3), while keeping "hot" data on local high-performance disks.

While this was a pioneering attempt that reduced the cost of storing cold data, it only patched the legacy Shared Nothing architecture rather than fixing the real issue. These systems were originally built with the core assumption that data locality is essential. In the bare-metal world, reading from a local SSD was orders of magnitude faster than the network. However, in the cloud, network bandwidth has exploded, while the operational cost of managing stateful storage on compute nodes has become a major bottleneck.

Because Tiered Storage treats S3 as a cold archive rather than a primary access layer, it introduces significant limitations:

The Rehydration Penalty: If your active query data set is already large enough, the database often cannot read data that has been offloaded directly. Instead, because the storage engine was written with the assumption that data would be processed on local disks, the database must "rehydrate" the data, i.e., download entire partitions back to the local disk, before the storage engine can process them. This introduces massive, unnecessary throughput spikes, especially if most of the data would be filtered out.
Resource Contention: Even if the data were eventually offloaded to a cold archive, all data ingestion would initially run on local disks. Every compute node is responsible for managing some data partitions and holding their leader replica. When triggering the background download process, it competes for IOPS with real-time ingestion and critical queries.
Limited Elasticity: Despite the offload, compute nodes remain stateful. They are still the "owners" of the data partitions they manage. A new node cannot serve user queries immediately. It triggers a complex data rebalancing process to shuffle data shards to local disks. Removing a node online is impossible. You must either stop the service to repartition the data or mirror the cluster to a new partitioned cluster. This is why you have to use Static Provisioning to survive a traffic spike.

S3 as a Primary Storage Layer

ScopeDB takes a fundamentally different approach: Zero Local Disks for Storage. In our architecture, S3 is not just a cold archive; it is the primary storage layer and the single source of truth. Compute nodes are effectively stateless, persisting no data locally but committing directly to S3.

Database engineers often worry that object storage's high latency (Time-To-First-Byte) makes it unsuitable for ad-hoc queries. This is valid only if you treat S3 like a slow local disk. ScopeDB, however, treats S3 as a massive-throughput network service. We have re-engineered the entire storage stack to exploit the unique characteristics of object storage:

Deterministic I/O Planning: We never use expensive and slow S3 LIST operations to scan for data. Instead, ScopeDB maintains a dedicated, transactional metadata service that tracks the precise location, statistics, and heuristic lightweight indexes of every data block. Before a query execution begins, the planner knows which objects and byte ranges are required, converting partition downloads into byte-range fetches.
Throughput over Latency: Analytics workloads are typically bound by throughput, not latency. ScopeDB's vectorized execution engine is designed to hide the latency of individual S3 calls through massive parallelism. Instead of issuing sequential reads, the engine issues dozens of concurrent, asynchronous read requests to saturate the available network bandwidth. By pipelining these requests, the engine ensures that the compute cores are always fed with data, regardless of S3's TTFB.
Aggressive Data Pruning: The fastest I/O is the one you don't perform. By combining adaptive expression indexes with our optimized columnar layout, ScopeDB aggressively prunes irrelevant data. We don't just skip files; we use range requests to fetch only the specific columns and pages needed for the query. This drastically reduces the amount of data traveling over the network.
Ephemeral Acceleration: While S3 ensures durability and infinite scalability, ScopeDB supports opt-in multi-tier caches for ultimate speed. Unlike the persistent storage in Shared Nothing architectures, ScopeDB's cache is ephemeral. It automatically adapts to hot data to provide sub-second latency, but it is not required for correctness or availability. If a node fails, no data is lost; a new node simply connects to S3 and begins serving queries immediately, warming up its cache in the background.

This Zero Local Disks architecture unlocks capabilities that go far beyond simple cost savings. Because storage and compute are completely decoupled, ScopeDB offers true workload isolation. You can spin up separate compute slots for different tasks—one for high-throughput ingestion, another for ad-hoc data science queries, and a third for scheduled reporting—all operating on the exact same dataset in S3 without replication lag or resource contention.

Stateless nodes can simplifies operations even further. There are no "bad disks" to replace or "degraded nodes" to nurse back to health. If a compute node underperforms, it can be terminated instantly and replaced. This robustness allows ScopeDB to leverage cost-effective Spot Instances for batch processing with zero risk of data loss, further driving down the Total Cost of Ownership.

Extreme Elasticity with Serverless Compute

With storage safely delegated to S3, ScopeDB's compute layer becomes truly elastic and serverless. We introduce the concept of "Compute Slots" to isolate workloads and maximize resource utilization.

Workload Isolation: You can assign dedicated compute slots to different teams or tasks. For example, one group for steady-state ingestion, another for the DevOps team's ad-hoc troubleshooting, and a third for scheduled reporting. A heavy query from the data science team will never impact the ingestion pipeline or the CEO's dashboard.
Auto-Scaling: Compute slots can automatically scale out (add more nodes) when query load increases and scale in (remove nodes) when the load subsides.
Scale-to-Zero: For sporadic workloads, such as a nightly report, the resource group can scale to zero when not in use. You pay absolutely nothing for computing while idle.

This elasticity means ScopeDB adapts to your business in real-time. In our customer's scenario, the observability platform automatically scales up to a few nodes during the day and scales in to exactly one node at night when developers get off work.

Summary

This post concludes our series on the three design principles of ScopeDB.

Insight In No Time: We eliminated the ETL tax, enabling real-time analysis on raw events with adaptive indexes.
Schema On The Fly: We introduced a flexible data schema solution that adapts to your data's evolution without downtime.
Cloud Elasticity: We leveraged the cloud to separate storage and compute, delivering a data solution with cost efficiency and scalability nearly on par with the cloud itself.

ScopeDB is built from the ground up to be the analytical database for the cloud era. You write raw events and immediately get insight for your application. It handles the volume, variety, and velocity of modern data while keeping your costs under control.

Ready to experience true cloud elasticity? Contact us to start your free trial or schedule a demo.