From Tendermint, QLDB, and DoltDB to Three Separated SQLite Databases: An 8-Year Blockchain Storage Retrospective

Spend enough years building infrastructure and you start to appreciate a recurring irony of software engineering: after seven or eight years of putting the industry's heaviest, most fashionable distributed databases and search clusters through their paces in production, the architecture that ultimately gives you the highest throughput and the greatest freedom is often the simplest, most old school design in the room.
In this month's upgrade of ArcBlock Chain into a next-generation Agentic Layer 1, alongside our full-stack DID refresh, unified composable assets, and declarative DSL smart contracts, we completed a ground-up simplification of our storage engine.
We retired the heavy external database servers and standalone search clusters we ran for years, returned to high-concurrency SQLite (in WAL mode), and introduced a physically separated Three-DB Architecture: Ledger DB (append-only verifiable truth), State DB (deterministic current state projection), and Index DB (disposable query read model).
Why would a modern Layer 1 blockchain make that move after eight years in production? To answer that, it helps to walk through the five storage generations we actually lived through.
Eight Years, Five Storage Generations: From Tendermint, QLDB, and DoltDB to Radical Simplicity
Many developers entering crypto today assume a blockchain's storage engine is static from day one. When you operate a live network since 2018, however, your storage layer evolves continuously alongside real application workloads and shifting cloud infrastructure. Over the past eight years, ArcBlock Chain's storage architecture progressed through five distinct stages:

Stage 1: Starting on Tendermint Engine and LevelDB
In the earliest days of ArcBlock Chain, our consensus and networking layers were built on top of the Tendermint engine. As anyone who built chains in that era remembers, Tendermint came coupled with LevelDB (GoLevelDB) under the hood.
That stack helped us ship our initial block pipeline quickly, but the limitations of tightly coupling a raw key-value store to the consensus engine surfaced fast. Pure key-value APIs are too low-level for expressive multi-dimensional state queries, forcing application code to hand-roll prefix scans. Worse, whenever automated test suites, lightweight local nodes, and high-availability production nodes needed different storage media, having storage hardwired into the consensus engine became a bottleneck.
Stage 2: Building a Storage Abstraction Across Memory, File, and Database Backends
To break that coupling, we made an early architectural decision that paid dividends for years: we abstracted both current state (StateDB) and query indexing (IndexDB) behind clean driver interfaces supporting memory, file, and db backends out of the box.
That abstraction gave the chain immediate flexibility. Automated tests and local sandboxes could spin up in milliseconds on a pure in-memory backend and vanish cleanly after the run; lightweight single-machine deployments could persist directly to local files; and production nodes could plug into managed enterprise databases without changing a single line of upper-layer transaction pipeline code.
Stage 3: Running AWS QLDB in Production
With a clean storage abstraction in place, we looked for a production cloud database that combined high reliability with native cryptographic verifiability. When Amazon Web Services launched AWS QLDB (Quantum Ledger Database), its built-in append-only cryptographic journal and verifiable history digest matched our ledger requirements naturally.
We implemented a production QLDB driver behind our storage abstraction and ran AWS QLDB as the core production backing store for an extended period, supporting live applications and asset flows reliably.
Stage 4: Migrating to DoltDB after QLDB's Retirement, Paired with Elasticsearch
The biggest risk of any proprietary cloud service is vendor lifecycle churn. When AWS announced that QLDB would be retired, we had to choose a new production engine.
Because we valued state versioning and historical auditability, we turned to DoltDB, the MySQL-compatible database built around Git-style versioned commits. Throughout these stages, we also ran a dedicated Elasticsearch cluster alongside our state database to power block explorer search, wallet transaction histories, and multi-dimensional asset queries.
On a whiteboard, "DoltDB + Elasticsearch" looked like a powerhouse combination. After running it in production, however, the engineering "complexity tax" became impossible to ignore 🤣:
- Paying a steep latency tax for unused general-purpose features: A blockchain transaction pipeline is a strictly linear, deterministic state machine. We never branch on-chain state like a Git repo and manually
mergeit back, yet every transaction was paying the cross-process network and commit-lock overhead of maintaining a full Git-style SQL version tree. - Using a sledgehammer for structured lookups in Elasticsearch: When we audited real query traffic from wallets and explorers, we found that 95%+ of queries were exact lookups by DID address, asset factory, transaction type, or timestamp range, with zero need for natural-language tokenization or fuzzy relevance scoring! Maintaining a memory-hungry JVM Elasticsearch cluster (and even padding big-integer token balances with leading zeros just so text indices would sort numbers properly) was classic
overkill. - Heavy operational footprint blocking lightweight deployment: If running a chain requires orchestrating a relational database server, a versioned storage layer, and an Elasticsearch cluster, you can never pack a sovereign blockchain into a few megabytes of memory on an edge node or let a developer spin one up with zero external dependencies on a laptop.
Stage 5: Returning to SQLite with a Physically Separated 3-DB Architecture
Once we looked at those numbers honestly, the right move was clear: strip away every heavy external service we did not actually need, and return to SQLite in Write-Ahead Logging (WAL) mode, the most battle-tested, zero-dependency embedded relational engine in the world.
Dropping cross-process connection pools, JVM garbage collection, and heavy version-tree commits cut our transaction commit latency by an order of magnitude and reduced memory usage from gigabytes to tens of megabytes. Instead of dumping every table into a single SQLite file, however, we designed a physically separated Three-DB Architecture from first principles.
Why Three Separate SQLite Databases? Physical Separation of Ledger, State, and Index
In many traditional blockchain nodes, raw transactions, current account balances, and secondary search indices live inside the same database instance. If an index table bloats or corrupts, or if a protocol upgrade changes a state table column, the entire node has to go offline for a high-risk database migration.
In the upgraded ArcBlock Chain, even though all three layers use SQLite under the hood, we split them physically into three orthogonal database files with completely different lifecycles:

1. Ledger DB (Append-Only Verifiable Ledger · The Single Source of Truth)
Ledger DB is the only irreplaceable crown jewel of the node. It has one job: recording every cryptographically signed raw transaction (txRaw) in strictly monotonic sequence order, along with its hash-chain digest (entryHash), deterministic state write-set fingerprint (writeSetDigest), and BFT multi-signed Checkpoint Merkle roots.
For backups and disaster recovery, Ledger DB is the only database file that ever needs to be backed up. Because it is an immutable append-only log, recent hot segments stay in fast local SQLite storage while older historical segments can be archived automatically as verifiable SegmentBundle files into object storage (such as S3 or Cloudflare R2), keeping active disk usage compact regardless of how many years the chain runs.
2. State DB (Current State Projection · 100% Rebuildable by Replay)
State DB holds the current snapshot of account balances, NFT ownership, staking pools, and active contracts. In our architecture, State DB is not an independent source of truth; it is a pure mathematical projection produced by replaying Ledger DB from genesis (or from the latest verified checkpoint):
State(N) = Replay(Ledger[1 .. N])Separating State DB from Ledger DB physically gives engineers two superpowers:
- Instant Self-Healing via Deterministic Replay: Even if a power failure corrupts
statedb.sqliteor an operator deletes the file outright, the node simply replaysLedger DBfrom the latest checkpoint to reconstruct the exact current state, matching the network's cryptographic state commitment bit for bit. - Protocol Upgrades Without Fragile Data Migration Scripts: In the past, evolving a state table schema meant writing nerve-wracking production migration scripts. Today, because every signed raw transaction is preserved untouched in
Ledger DB, upgrading state projection logic is as clean as replaying the ledger into a freshState DB.
3. Index DB (Query Read Model · Zero-Backup Derived View)
Index DB is the high-speed reading room built for wallets, block explorers, dApp frontends, and AI agents. Using SQLite's native composite B-tree indices and native numeric sorting, it handles multi-dimensional queries by sender, recipient, transaction type, time window, and asset tag without breaking a sweat, replacing our old Elasticsearch cluster completely.
Because Index DB is physically isolated from both Ledger DB and State DB:
- Heavy Analytics Queries Never Slow Down Core Consensus Writes: No matter how many complex filter queries hit
Index DB, they never contend for write locks onLedger DBorState DB. - Zero Backup Overhead and Seconds-Fast Rebuild: You never waste storage backing up bulky secondary indices. Whenever a product team wants to add new query dimensions, you update the indexer schema, delete
indexdb.sqlite, and let it rebuild in the background while core transaction settlement continues uninterrupted 😝.
The Deeper You Know Complexity, the More You Respect Simplicity
Walking through five generations of storage over eight years, from Tendermint and LevelDB to our custom multi-backend abstraction, AWS QLDB, DoltDB with Elasticsearch, and finally three clean, single-purpose SQLite files, reinforced a timeless lesson: the highest form of systems architecture is not seeing how many components you can add, but knowing how much you can take away.
By distilling our storage layer into the Ledger + State + Index Three-DB Architecture, a full verifiable sovereign blockchain can process hundreds of instant-finality transactions per second on a standard server, run with zero external database dependencies inside a lightweight ARC (Agentic Realm Computer) node, or even execute inside serverless edge containers like Cloudflare Workers and Durable Objects.
Shedding heavy infrastructure baggage so that the ledger is purely the ledger, state is purely state, and indexing is purely indexing is what lets ArcBlock Chain run light, fast, and verifiable as a next-generation Agentic Layer 1.
References and Further Reading
- When Every NFT Is Born an Account, a Container, and a Contract: ArcBlock Chain's Unified Asset Design
- Why Our Smart Contracts Are Intentionally Non-Turing-Complete, and Why Every Contract Is an On-Chain NFT
- ArcBlock Chain: A Next-Generation Agentic Layer 1 Built for a Multi-Chain World
- ArcBlock Chain Upcoming Upgrade: Seamless Migration and Zero-Protocol-Fee Web Bridge Plan
- Three Generations of OCAP: From Chain Access Protocol to AFS
- SQLite Write-Ahead Logging (WAL)
Referenced here
Products
- ARC active
The runtime for Blocklets. It gives a developer a place to run an application described as a Blocklet, together with the resources that Blocklet declares it needs.
- ArcBlock Chain active
The application-focused Layer 1 for identity, assets and agreements. ArcBlock Chain brings these operations into the protocol, with ABT as the public network’s native token.
- DID Wallet renamed
Create and manage a digital identity (DID) and its associated credentials in one app.