XChain Platform UTXO Tracker: Architecture

Position in the Data Pipeline

flowchart TD
    NODE["Coin Node<br>(bitcoind / litecoind / dogecoind)"]
    TRACKER["xchain-utxo-tracker"]
    LEVELDB[("LevelDB")]
    ENCODER["xchain-encoder"]
    PSBT["PSBT<br>(signed+broadcast by caller)"]

    NODE -->|"JSON-RPC polling"| TRACKER
    TRACKER -->|"UTXO/balance queries for encoder"| LEVELDB
    TRACKER --> ENCODER
    ENCODER --> PSBT

The UTXO tracker sits between the coin node and the encoder. It continuously polls the coin node for new blocks, parses every transaction, and maintains a LevelDB index of all unspent outputs. The encoder queries the tracker’s API to find spendable inputs when constructing transactions.

Unlike the decoder (which extracts XChain ACTION data), the UTXO tracker indexes all transaction outputs regardless of whether they carry XChain data.

Internal Components

flowchart TD
    API["api.js<br>Express + JSON-RPC server<br>REST endpoints + JSON-RPC methods<br>Loads env vars, creates tracker, starts API"]
    ORCH["XChainUtxoTracker<br>Main orchestrator class<br>Block polling loop (1s interval)<br>Reorg detection → block processing → mempool updates"]
    LEVELUP["LevelUpDb<br>(LevelDB storage)"]
    CONN["Blockchain Connector<br>(RPC client)"]
    DECODER["Block Decoder<br>(parsing)"]
    MEMLEVEL["memory-level<br>(mempool in-memory)"]
    CRYPTONET["CryptoNetworks<br>(network params)"]
    BUFUTILS["bufferutils<br>(binary encoding)"]

    API --> ORCH
    ORCH --> LEVELUP
    ORCH --> CONN
    ORCH --> DECODER
    LEVELUP --> MEMLEVEL
    CONN --> CRYPTONET

Source Files

LevelDB Key Schema

All data is stored in a single LevelDB instance using single-byte prefix keys. Keys and values are raw binary Buffers for compactness.

Key/Value Layouts

Two string keys are also used as checkpoints:

• LAST_BLOCK_HEIGHT: hex-encoded current tip height
• LAST_BLOCK_HASH: hex-encoded current tip hash

Key Design Principles

O key (output index): The scriptHash comes first in the key, enabling efficient range scans to answer “what UTXOs does this address have?” by scanning all O keys with a given scriptHash prefix.

H key (output hint): Maps an outpoint (txHash8 + index) back to its scriptHash. When processing an input that spends an output, the tracker reads the H hint to find the scriptHash, then deletes the corresponding O record. Without H, the tracker would need to scan all O records to find the one being spent.

K/M keys (deleted archives): When a UTXO is spent, the O and H records are deleted, but copies are saved as K and M records keyed by blockHash. If a reorg rolls back that block, the K/M records are restored to O/H. After DEFAULT_UNDO_BLOCKS (BTC: 12, LTC: 48, DOGE: 120) subsequent blocks, the K/M records are purged.

txHash8 truncation: Transaction hashes are truncated to 8 bytes in index keys (T, I, O, H, J, K, M, W). The full 32-byte hash is stored in O values for API responses. 8-byte truncation provides sufficient uniqueness for index lookups while halving key sizes.

Block Processing Loop

The main loop runs continuously in XChainUtxoTracker.start():

flowchart TD
    TOP(("while (keepParsing)"))
    S1["1. Poll getblockchaininfo()<br>every 1 second"]
    S2["2. Wait for node sync<br>(verificationprogress >= 0.99)"]
    Q3{"3. Caught up with tip?"}
    MEMPOOL["Trigger mempool updates<br>every 60 seconds"]
    Q4{"4. New blocks available?"}
    STEPA["a. Fill prefetch queue<br>(up to 10 blocks ahead)"]
    STEPB["b. Fetch next block hash<br>and hex data"]
    STEPC["c. Decode block via<br>XChainBlockDecoder"]
    STEPD["d. Verify chain continuity<br>(prevHash == last stored hash)"]
    QREORG{"prevHash mismatch?"}
    REORG["Enter reorg handling<br>(see Reorg Handling below)"]
    STEPE["e. Begin LevelDB batch transaction<br>(if first block in batch)"]
    STEPF["f. Two-pass transaction processing:<br>Pass 1: insert all outputs (O + H + S + W records)<br>Pass 2: process all inputs<br>(delete spent O/H, create K/M/I/J)"]
    STEPG["g. Record block metadata<br>(B, T, N records)"]
    QH{"h. Batch complete (200 blocks)<br>or at chain tip?"}
    COMMIT["Commit batch transaction atomically<br>Cleanup aged K/M records<br>(blocks older than UNDO_BLOCKS)<br>Save checkpoint<br>(LAST_BLOCK_HEIGHT, LAST_BLOCK_HASH)<br>Calculate and display ETA<br>from rolling 1000-block window"]

    TOP --> S1 --> S2 --> Q3
    Q3 -->|yes| MEMPOOL --> Q4
    Q3 -->|no| Q4
    Q4 -->|yes| STEPA --> STEPB --> STEPC --> STEPD --> QREORG
    QREORG -->|mismatch| REORG --> TOP
    QREORG -->|match| STEPE --> STEPF --> STEPG --> QH
    QH -->|yes| COMMIT --> TOP
    QH -->|no| TOP
    Q4 -->|no| TOP

Two-Pass Transaction Processing

Within each block, transactions are processed in two passes:

1. Pass 1; Outputs: All transaction outputs are inserted first (O, H, S, and W records). O is the unspent output index; H is the output hint used to locate O during spend processing; S records the first block height a scriptHash is seen; W is the creation-block reverse index used to clean up phantom UTXOs on reorg. Inserting outputs first ensures that when an output is created and spent within the same block, the output exists in the batch transaction before the input tries to delete it.

2. Pass 2; Inputs: All transaction inputs are processed. For each input, the tracker reads the H hint to find the scriptHash, deletes the O and H records, and creates K/M archive records (for reorg undo), I records (spent input), and J records (input hint for cleanup).

Concurrent Block Prefetch

To minimize RPC idle time, the tracker maintains a prefetch queue of up to PREFETCH_SIZE (10) blocks. For non-AuxPoW chains, block hashes and hex data are fetched in two batch HTTP requests. For AuxPoW chains (Dogecoin/Litecoin with HogEx), blocks are fetched individually because AuxPoW headers must be stripped before parsing.

Batch Writes

LevelDB writes are accumulated in an in-memory transaction (Map of put/del operations) and flushed atomically via db.batch() every 200 blocks or when the tracker reaches the chain tip (or when heap usage exceeds 2 GB). This minimizes write amplification and ensures all-or-nothing semantics; a crash mid-batch loses at most 200 blocks of progress, which are re-indexed on restart.

Reorg Handling

When the tracker detects that an incoming block’s previousHash does not match the stored LAST_BLOCK_HASH, it enters the reorg verification loop:

1. Walk back: Starting from the tracker’s tip, walk back one block at a time, comparing the tracker’s stored block hash with what the coin node reports for that height.
2. Find fork point: Continue until the hashes match; this is the fork point.
3. Rollback: For each rolled-back block:
   • Restore deleted outputs from K/M archive records → O/H records
   • Delete the block’s B, T, N records
   • Delete I/J records created by that block
   • Remove Z/S records associated with that block
4. Reset state: Clear the prefetch queue, update LAST_BLOCK_HEIGHT/LAST_BLOCK_HASH, reset the lastBlocks array.
5. Resume: Normal forward indexing resumes from the fork point.

flowchart TD
    DETECT["Incoming block's previousHash does not match stored LAST_BLOCK_HASH"]
    WALK["1. Walk back one block at a time,<br>comparing tracker hash to node hash"]
    FORK["2. Find fork point where hashes match"]
    ROLLBACK["3. Rollback each rolled-back block:<br>restore K/M archives to O/H records<br>delete B, T, N records<br>delete I/J records<br>remove Z/S records"]
    RESET["4. Reset state:<br>clear prefetch queue, update LAST_BLOCK_HEIGHT/LAST_BLOCK_HASH,<br>reset lastBlocks array"]
    RESUME["5. Resume normal forward indexing from the fork point"]

    DETECT --> WALK --> FORK --> ROLLBACK --> RESET --> RESUME

The undo window is determined per chain: BTC 12 blocks, LTC 48 blocks, DOGE 120 blocks (overridable via XCHAIN_UNDO_BLOCKS_<COIN>). Reorgs exceeding the configured window throw an error and require a full re-index.

Mempool Tracking

When the tracker is caught up with the chain tip, it updates the mempool every 60 seconds:

1. Call getrawmempool to get all current transaction IDs
2. Sort and diff against the in-memory mempool database:
   • Remove entries no longer in the node’s mempool
   • Skip entries already indexed
3. Fetch raw transaction hex for new entries in batches of 1000
4. Parse transactions and insert outputs/inputs into the mempool database (memory-level-backed, in-memory only)

Mempool data is not written to the persistent LevelDB. When a mempool transaction confirms (its block is processed), the confirmed UTXO is written to the main database and the mempool entry is naturally superseded.

The mempoolBusy flag prevents concurrent mempool updates.

flowchart TD
    A["Every 60 seconds, once caught up with the chain tip"] --> B["getrawmempool: fetch current txids"]
    B --> C["Sort and diff against the in-memory mempool DB"]
    C --> D["Remove entries no longer in the node's mempool"]
    C --> E["Skip entries already indexed"]
    D --> F["Fetch raw tx hex for new entries (batches of 1000)"]
    E --> F
    F --> G["Parse transactions, insert outputs/inputs into the mempool DB"]
    G -.->|"next cycle"| A

Balance Calculation

getBalanceInfo(address) computes balances by:

1. Converting the address to a scriptPubKey via bitcoinjs-lib
2. Hashing the scriptPubKey with SHA-256 to get the scriptHash
3. Range-scanning O records in both the main DB and mempool DB for that scriptHash
4. For each confirmed output: checking if it’s being spent in the mempool (via I record lookup)
5. Accumulating confirmed balance, pending balance, received total, and UTXO counts using BigInt arithmetic
6. Converting satoshi BigInt values to decimal strings via satoshiToDecimalString(). No floating-point involved

Bootstrap

For new deployments, syncing from block 0 can take a long time. The tracker supports:

• Backup (getbootstrap): Creates a compressed tar+gzip archive of the LevelDB data directory using tar, pv (for progress), and pigz (parallel gzip). Original size stored in the gzip comment for progress reporting. A .sha256 sidecar is written next to the archive so the restore path below can verify it.

• Restore (restorebootstrap): Accepts two archive layouts and validates the archive before the destructive /data wipe, so a wrong-layout or checksum-failing archive aborts with the live store intact:

  • Single-layer (what getbootstrap above produces): the archive is the LevelDB data directory itself. It is verified against its published <archive>.sha256 sidecar, and a missing sidecar fails closed unless BOOTSTRAP_RESTORE_ALLOW_UNVERIFIED=1.
  • Wrapper (what the node’s BootstrapService publishes): an outer tar whose members are data.tar.gz plus data.sha256. It is detected by those member names, unwrapped to a temp dir, and the inner data.tar.gz is checksum-verified against the bundled data.sha256 before becoming the effective restore source. No external .sha256 sidecar is required or consulted for this layout.

  After extraction the tracker resumes normal polling from the restored height. Tip continuity is checked lazily by the reorg path on the next block, not eagerly at restore time.

Both operations run as background tasks tracked by UUID, with progress queryable via getbootstrapstatus / getbootstraprestorestatus.

Copyright © 2025–2026 Dankest, LLC

Based on XChain Platform by Dankest, LLC – https://dankest.llc

Licensed under the GNU Affero General Public License v3.0 (AGPL-3.0-or-later) with a commercial license available for proprietary use.

You may use, modify, and distribute this material under the terms of the License. See LICENSE and NOTICE for full terms. See the licensing overview.