commit 2bf3c97e786d97129c396a12a1e3e2f28978d97d
parent a89fcba93b1adb383dfaafe1356245aafeff7aa9
Author: Joris Hartog <jorishartog@hotmail.com>
Date: Tue, 25 Aug 2026 14:45:39 +0200
Use compact storage size for block consensus
Diffstat:
17 files changed, 188 insertions(+), 310 deletions(-)
diff --git a/README.md b/README.md
@@ -180,6 +180,12 @@ with `docker compose down -v`, because consensus launch profiles cannot be
changed in place. The five-slot committee is also a consensus reset: volumes
created by the earlier three-slot protocol must likewise be recreated.
+The next release also introduces compact snapshot v6 and deliberately provides
+no old-chain migration. All nodes must participate in the coordinated network
+reset: preserve wallet/configuration files, remove or archive the old chain and
+UI databases, and create or join the new genesis. See the operator playbooks for
+non-Compose reset commands.
+
Stop the network while keeping chain data:
```sh
@@ -204,7 +210,7 @@ Use your iuna wallet address as the worker username. Accepted shares become PoW
## Operator Docs
-- [Genesis and candidate rehearsal](docs/genesis.md)
+- [Protocol](docs/protocol.md)
- [Operator failure playbooks](docs/operator-playbooks.md)
- [Security review checklist](docs/security-review.md)
diff --git a/docs/genesis.md b/docs/genesis.md
@@ -1,236 +0,0 @@
-# Genesis And Candidate Rehearsal
-
-This is operator documentation for bootstrapping an iuna local/test network or the mainnet-candidate network. Most users should join an existing bootnode instead of creating genesis.
-
-The mainnet-candidate genesis is not disposable by default. It is the genesis that can become mainnet if the candidate passes the agreed stability window and release gates. In that case, mined coins, UTXOs, tickets, and chain history remain on the same ledger; promotion is a coordinated release and network-identity cutover, not a second genesis.
-
-Keep the management UI bound to `127.0.0.1`. Only the P2P listener should be internet-facing.
-
-## Candidate Manifest
-
-Before creating the mainnet-candidate genesis, publish one manifest in the release notes or operator coordination channel:
-
-```text
-version:
-git commit:
-release tag:
-genesis operator:
-genesis start time:
-genesis hash:
-stability window:
-promotion policy:
-bootnodes:
-checksums:
-```
-
-Fill it with:
-
-- the exact release artifact version and git commit;
-- the release tag, once created;
-- the genesis operator and UTC start time;
-- the genesis hash after the first node starts;
-- the agreed no-reset stability window, for example one week;
-- whether a healthy candidate will be promoted to mainnet without a second genesis;
-- every public bootnode as `<host>:<p2p-port>`;
-- a link or pasted copy of `downloads/SHA256SUMS`.
-
-Do not start the stability window until the fresh genesis exists, at least one published bootnode is reachable, and independent operators have verified the release checksums.
-
-## Release Artifacts
-
-Build and deploy from the candidate commit:
-
-```sh
-cargo test --locked
-cargo check --locked --manifest-path fuzz/Cargo.toml
-cargo run --locked --manifest-path fuzz/Cargo.toml --bin p2p_envelope -- -runs=256 fuzz/corpus/p2p_envelope
-cargo run --locked --manifest-path fuzz/Cargo.toml --bin compact_snapshot -- -runs=256 fuzz/corpus/compact_snapshot
-cargo run --locked --manifest-path fuzz/Cargo.toml --bin domain_json -- -runs=256 fuzz/corpus/domain_json
-cargo run --locked --manifest-path fuzz/Cargo.toml --bin stratum_request -- -runs=256 fuzz/corpus/stratum_request
-cargo run --locked --manifest-path fuzz/Cargo.toml --bin wallet_config -- -runs=256 fuzz/corpus/wallet_config
-cargo run --locked --manifest-path fuzz/Cargo.toml --bin vdf_proof -- -runs=16 fuzz/corpus/vdf_proof
-cargo test --locked domain::adversarial_tests:: -- --ignored
-cargo test --locked --release --test properties -- --ignored
-./deployment.sh <version>
-```
-
-The deployment script writes release packages to `downloads/` and creates `downloads/SHA256SUMS`.
-It runs `256` iterations per general fuzz target and `16` iterations for the
-slower VDF proof fuzz target by default; set `IUNA_FUZZ_RUNS` or
-`IUNA_VDF_FUZZ_RUNS` to a positive integer only for an explicitly documented
-emergency redeploy.
-
-Verify the files before publishing them:
-
-```sh
-shasum -a 256 -c downloads/SHA256SUMS
-```
-
-On Linux, `sha256sum -c downloads/SHA256SUMS` is equivalent.
-
-Publish the checksums with the release artifacts. A node operator should be able to download the artifact and verify it before starting a candidate node.
-
-## Create Genesis
-
-Genesis requires a fresh wallet path and a fresh chain database. Use a new data directory for the candidate genesis:
-
-```sh
-iuna --genesis --data-dir ~/.iuna-candidate-genesis --p2p 0.0.0.0:9444 --http 127.0.0.1:18661
-```
-
-For local source builds, use the same flags through Cargo:
-
-```sh
-cargo run -- --genesis --data-dir ~/.iuna-candidate-genesis --p2p 0.0.0.0:9444 --http 127.0.0.1:18661
-```
-
-Open `http://127.0.0.1:18661`, set the wallet password if prompted, write down the recovery phrase, and finish setup. The process prints the wallet file, config file, chain database, management UI, and P2P listener on startup.
-
-Genesis bootstraps the chain with a 1 IUNA burn, creates launch tickets for the first blocks, measures an initial VDF delay, and leaves the starter wallet with spendable IUNA for early testing.
-
-After startup, record the genesis hash from block `0` in the local UI or authenticated blocks endpoint:
-
-```sh
-curl -s 'http://127.0.0.1:18661/api/blocks?before_height=1&limit=1'
-```
-
-If `curl` returns an auth response, collect the same value from the local UI or include the browser session cookie in the request.
-
-## Publish Bootnodes
-
-Publish at least one stable public P2P address:
-
-```text
-your-host.example:9444
-```
-
-For every bootnode:
-
-- allow inbound TCP traffic on the P2P port;
-- keep the HTTP management UI local-only;
-- publish the address operators should use with `--join`;
-- record the operator or owner in the candidate manifest.
-
-If a bootnode address changes during the stability window, treat it as an operational incident and update the manifest. Do not hide bootnode churn from candidate notes.
-
-## Join Nodes
-
-Start joining nodes with a fresh data directory and a published bootnode:
-
-```sh
-iuna --data-dir ~/.iuna-candidate --p2p 0.0.0.0:9445 --http 127.0.0.1:18661 --join your-host.example:9444
-```
-
-For a public peer, configure the reachable P2P listener and announce address in Settings or with CLI flags:
-
-```sh
-iuna --data-dir ~/.iuna-candidate --p2p 0.0.0.0:9445 --p2p-announce your-node.example:9445 --http 127.0.0.1:18661 --join your-host.example:9444
-```
-
-For a private wallet-only node, leave inbound P2P disabled in Settings and connect outbound to bootnodes.
-
-## Backup And Restore Drill
-
-Before the stability window starts, every core operator should prove they can restore their node.
-
-Back up:
-
-- `wallet.json`;
-- `config.json`;
-- `chain.sqlite3`;
-- `ui_data.sqlite3`;
-- the release artifact used to run the node;
-- the matching `SHA256SUMS` entry;
-- the wallet recovery phrase, stored separately from the machine.
-
-With the default data directory these files live under `~/.iuna`. With `--data-dir`, they live under that directory unless `--wallet` or `--chain-db` overrides the path.
-
-Restore rehearsal:
-
-1. Stop the node.
-2. Copy the backup into a new restore data directory.
-3. Start the same release binary with `--data-dir <restore-dir>`.
-4. Confirm the wallet address, local height, tip hash, and peers match expectations.
-5. If restoring without chain state, start with a published bootnode and let the node sync:
-
-```sh
-iuna --data-dir <restore-dir> --join your-host.example:9444
-```
-
-Never use `--genesis` to recover a node. `--genesis` is only for creating a fresh network.
-
-## No-Reset Stability Window
-
-For the mainnet-candidate network, treat unplanned resets as launch-blocking incidents unless they were explicitly scheduled before the window started. Operators may mine and transact during this window with the expectation that the ledger can become mainnet if the candidate passes.
-
-Start the window only after:
-
-- genesis hash and start time are published;
-- release artifacts and `SHA256SUMS` are published;
-- at least one bootnode is reachable from an independent host;
-- at least one non-genesis node has synced from the published bootnode;
-- backup/restore has been rehearsed by core operators.
-
-During the window, record daily:
-
-- height and tip hash from at least two nodes;
-- peer count and stale peer count;
-- last block age;
-- finalizer mode, recovery blocks, and fallback frequency;
-- mempool size and rejected block or transaction errors;
-- any node restart, bootnode change, restore, rollback, or manual chain deletion.
-
-Use [operator failure playbooks](operator-playbooks.md) for stalled height, divergent tips, old snapshots, missing burn committee signatures, recovery blocks, and corrupted local persistence.
-
-Exit criteria:
-
-- no unplanned reset during the full agreed window;
-- fresh nodes can still sync from genesis through published bootnodes;
-- release artifact checksums are independently verified;
-- backup/restore rehearsal has passed;
-- all launch-blocking incidents are fixed or explicitly deferred before mainnet.
-
-## Upgrade And Rollback
-
-For routine candidate upgrades, preserve local state and replace only the
-software artifact:
-
-1. Stop the node.
-2. Back up `wallet.json`, `config.json`, `chain.sqlite3`, and `ui_data.sqlite3`.
-3. Verify the new release artifact against the published `SHA256SUMS`.
-4. Start the new binary with the same `--data-dir`, `--wallet`, `--chain-db`,
- P2P, HTTP, and Stratum settings.
-5. Confirm the wallet address, genesis hash, local height, tip hash, launch
- profile, peer count, and last block age.
-
-Do not start upgrades with `--genesis`. Do not delete `chain.sqlite3` during a
-candidate-to-mainnet promotion. The promoted release must load the existing
-candidate chain database and continue from the current tip.
-
-If the upgraded node fails before it mines or accepts blocks under new rules,
-rollback is a software rollback:
-
-1. Stop the upgraded node.
-2. Restart the previous verified binary with the same data directory.
-3. Confirm the node resumes the same height and tip it had before the upgrade.
-4. Keep peers connected and let normal sync catch up if the network advanced
- while the node was offline.
-
-If the upgraded node has already accepted blocks that older binaries reject, do
-not silently roll back. Treat that as a possible hard-fork or release incident:
-preserve chain/UI databases, stop public bootnode churn, compare tips across
-operators, and publish a decision before asking operators to delete or replace
-chain data.
-
-## Promotion To Mainnet
-
-If the candidate passes the stability window, publish a promotion notice instead of a new genesis plan. The notice should include:
-
-- candidate genesis hash;
-- promoted tip height and tip hash;
-- final candidate release tag and mainnet release tag;
-- bootnodes that will remain online through the cutover;
-- whether the P2P network ID changes from `iuna-mainnet-candidate` to `iuna-mainnet-v1`;
-- the exact upgrade window for operators.
-
-Do not delete chain data when promoting. Nodes should keep `chain.sqlite3`, wallet files, and UI data, then upgrade or restart with the promoted release. A P2P network ID change fences upgraded mainnet nodes away from old candidate binaries, but it must not change genesis, launch profile rules, or any existing ledger state.
diff --git a/docs/operator-playbooks.md b/docs/operator-playbooks.md
@@ -117,6 +117,35 @@ Avoid:
- deleting the chain database before capturing the startup error;
- using `--genesis` to recover an old node. `--genesis` is only for creating a fresh network.
+## Coordinated Snapshot V6 Reset
+
+The next release accepts compact local snapshot format v6 only and does not migrate earlier chain history. This is a planned consensus/network reset, not a corrupted-database incident. All operators must agree on the release, genesis, network identity, bootnodes, and start time before bringing public nodes back online.
+
+Before upgrading:
+
+1. Stop the node and preserve `chain.sqlite3` if it is needed as historical evidence.
+2. Keep `wallet.json` and `config.json`; verify that their backups are readable.
+3. Archive or remove only `chain.sqlite3` and `ui_data.sqlite3` from the node's configured data directory.
+4. Start exactly one designated node with `--genesis`, record its genesis hash, and publish that hash with the release commit and checksums.
+5. Start every other node without `--genesis` and join a trusted published bootnode.
+
+Example for a default data directory, retaining the old databases as evidence:
+
+```sh
+mv ~/.iuna/chain.sqlite3 ~/.iuna/chain.sqlite3.pre-v6 2>/dev/null || true
+mv ~/.iuna/ui_data.sqlite3 ~/.iuna/ui_data.sqlite3.pre-v6 2>/dev/null || true
+iuna --join <trusted-peer-addr:port>
+```
+
+For the disposable Compose testnet, `docker compose down -v` removes all volumes and the configured bootstrap creates the fresh genesis on the next start.
+
+Avoid:
+
+- running multiple independent `--genesis` nodes;
+- copying an old snapshot blob into a v6 database;
+- deleting or replacing wallets as part of the chain reset;
+- starting before the published genesis hash and release checksum are available.
+
## No Burn Committee Signatures
Symptoms:
diff --git a/docs/protocol.md b/docs/protocol.md
@@ -21,7 +21,7 @@ The current mainnet-candidate parameter set is intentionally close to Bitcoin wh
- launch profile ID: `iuna-mainnet-candidate`;
- launch profile hash: `eb2f67e9d735474859ceb1fe124fe270977214f6e2f4cd855a4d8c3b5ecac558`;
- target block time: `10 minutes`;
-- maximum serialized block size: `1,000,000` bytes;
+- maximum compact stored block-body size: `1,000,000` bytes;
- maximum transaction items per block: `1,000`;
- maximum pending transactions per node: `10,000`;
- maximum pending transaction pool bytes per node: `8 MiB`;
@@ -49,7 +49,11 @@ Changing any value in this section requires a conscious mainnet-candidate reset
If the mainnet-candidate network is promoted to mainnet, the candidate genesis, chain history, UTXOs, tickets, and launch profile remain intact. A later P2P network ID change to `iuna-mainnet-v1` is only a peer-network cutover unless it is accompanied by an explicitly announced hard fork or reset.
-Block size is checked from the node's canonical serialized block representation after parsing, so alternate JSON whitespace or key order cannot make a block count smaller. Transaction selection and fee-rate policy use compact economic transaction size: addresses, hashes, signatures, and Stratum headers count as their decoded byte lengths, and numeric fields count as compact base-128 varint widths. That keeps hex text and JSON decimal formatting from making transactions look larger or smaller economically than their protocol data.
+The consensus block-size limit is the exact number of bytes produced by the compact snapshot v6 block-body encoder when the block is appended to its parent chain. The encoder's reference tables are seeded by genesis allocations and extended in chain order, so all nodes calculate the same context-dependent size. The snapshot header, launch profile, block-count field, SQLite row metadata, and SQLite page overhead are not charged to an individual block.
+
+The compact representation stores binary hashes, addresses, signatures, and VDF data instead of their hexadecimal text. It uses base-128 varints for integers, chain-wide references for repeated addresses and protocol IDs, a single shared owner and signature for transaction inputs, implicit burn change where possible, and transaction indexes for burns repeated by the burn-bundle section. Heights, parent hashes, and block hashes are reconstructed from chain order and canonical block contents rather than repeated in each stored block body. Burns benefit most from this layout, followed by transfers and mine actions.
+
+P2P messages and management API responses still use JSON. Their byte length is not the consensus block size. Transaction fee-rate ordering uses a separate compact economic transaction weight, so changing JSON whitespace, key order, or hexadecimal formatting cannot change consensus size or fee priority.
## Coins and Transactions
@@ -308,7 +312,7 @@ When a node builds a block, the flow is:
5. Fill remaining block space with valid fee-paying transfers, additional burns, and mine actions ordered by fee rate. Mine actions are limited to `2` actions per anchor.
6. Bind the VDF seed to the five burn-attestation slot hashes, using default hashes for missing slots. Slot `0` uses the synthetic finalizer attestation hash instead of a separate burn-bundle signature.
-Blocks are bounded by transaction count and serialized byte size. The mainnet-candidate maximum block size is `1,000,000` bytes.
+Blocks are bounded by transaction count and exact compact stored block-body size. The mainnet-candidate maximum is `1,000,000` bytes. Block admission checks the finished block with the parent chain's compact reference context. Block construction uses the same encoder while reserving mandatory anchor and attested burns before filling the remaining space.
## Fork Choice
@@ -324,6 +328,12 @@ Genesis is explicit. A normal node without a chain starts in setup mode and wait
The genesis flow bootstraps the mainnet-candidate network with an initial burn ticket and a fixed `1 IUNA` initial reward for the genesis wallet. New nodes fetch and validate chain snapshots from peers, then continue with normal block validation.
+## Local Chain Persistence And Reset Boundary
+
+The local `chain.sqlite3` database stores one atomically replaced compact snapshot blob plus independently checked tip height and tip hash metadata. Snapshot format v6 is the only accepted local format in the next release; older compact snapshot versions are deliberately not decoded or migrated.
+
+This persistence change is paired with a coordinated network reset. Every node must start the next release without its previous chain and UI databases, then either create the agreed new genesis or join a trusted peer on that new chain. Wallet and configuration files are not chain state and should be retained. Detailed recovery and reset commands are in [Operator Failure Playbooks](operator-playbooks.md).
+
## What This Design Is Trying to Achieve
iuna is trying to make these things true at the same time:
diff --git a/docs/security-review.md b/docs/security-review.md
@@ -22,7 +22,7 @@ Primary code:
- `src/domain/ledger_consensus.rs`
- `src/domain/ledger_reveal.rs`
- `src/domain/ticket.rs`
-- `src/domain/vdf.rs`
+- `src/domain/vdf/mod.rs`
- `src/domain/protocol.rs`
Evidence already in the tree:
@@ -30,7 +30,7 @@ Evidence already in the tree:
- adversarial consensus tests in `src/domain/adversarial_tests.rs`;
- release soak test in `tests/properties.rs`;
- protocol rules documented in `docs/protocol.md`;
-- candidate rehearsal and rollback process in `docs/genesis.md`.
+- reset, joining, recovery, and rollback procedures in `docs/operator-playbooks.md`.
### Transaction And Mempool Validation
@@ -85,6 +85,8 @@ Evidence already in the tree:
- P2P tests in `src/adapters/p2p/tests.rs`;
- compact snapshot malformed-input tests in `src/adapters/chain_store/compact.rs`;
+- compact block-size boundary tests proving that selection and consensus use the
+ same snapshot v6 block-body encoder;
- fuzz targets for `p2p_envelope`, `compact_snapshot`, and `domain_json`.
### Wallet, Key Storage, And HTTP Auth
@@ -221,10 +223,10 @@ see which revision was tested.
- Public exposure: verify bootnodes expose only the intended P2P and optional
Stratum ports, and that the management UI remains bound to a local or
otherwise protected address.
-- Candidate manifest: `docs/genesis.md` contains the manifest template and
- operating procedure. For a specific candidate, verify and publish the real
- genesis hash, network ID, bootnodes, checksums, promotion policy, rollback
- instructions, release tag, and git commit before the stability window starts.
+- Candidate manifest: release coordination must publish the real genesis hash,
+ network ID, bootnodes, checksums, reset and rollback instructions, release tag,
+ and git commit before the stability window starts. The protocol and operator
+ playbooks are the maintained in-tree references.
- Release evidence: keep successful release-gate logs from the exact tagged
candidate revision.
diff --git a/src/adapters/chain_store.rs b/src/adapters/chain_store.rs
@@ -7,10 +7,10 @@ use std::{
use anyhow::{Context, Result};
use rusqlite::{Connection, OptionalExtension, params};
-use crate::domain::ChainSnapshot;
-
-mod compact;
-use compact::{decode_compact_snapshot, encode_compact_snapshot};
+use crate::{
+ compact::{decode_compact_snapshot, encode_compact_snapshot},
+ domain::ChainSnapshot,
+};
#[cfg(feature = "fuzzing")]
pub fn fuzz_decode_compact_snapshot(bytes: &[u8]) -> Result<ChainSnapshot> {
diff --git a/src/adapters/chain_store/compact.rs b/src/adapters/chain_store/compact.rs
@@ -3,8 +3,8 @@ use std::collections::BTreeMap;
use anyhow::{Context, Result, bail};
use crate::domain::{
- Block, BurnBundleSection, BurnBundleSignature, ChainSnapshot, FinalizerMode, LaunchProfile,
- LeaderProof, MaskedBurn, OutPoint, Transaction, TxInput, TxOutput,
+ Amount, Block, BurnBundleSection, BurnBundleSignature, ChainSnapshot, FinalizerMode,
+ LaunchProfile, LeaderProof, MaskedBurn, OutPoint, Transaction, TxInput, TxOutput,
};
const COMPACT_SNAPSHOT_MAGIC: &[u8] = b"IUNA-SNAPSHOT";
@@ -15,12 +15,51 @@ const MAX_COMPACT_SNAPSHOT_BLOCKS: usize = 10_000;
const MAX_COMPACT_VEC_ITEMS: usize = 10_000;
const MAX_COMPACT_BYTE_FIELD: usize = 8 * 1024 * 1024;
-#[derive(Default)]
+#[derive(Clone, Debug, Default)]
struct EncodeTables {
addresses: BTreeMap<String, u64>,
protocol_ids: BTreeMap<String, u64>,
}
+#[derive(Clone, Debug, Default)]
+pub(crate) struct CompactBlockContext {
+ tables: EncodeTables,
+}
+
+impl CompactBlockContext {
+ pub(crate) fn for_chain(
+ genesis_allocations: &BTreeMap<String, Amount>,
+ blocks: &[Block],
+ ) -> Result<Self> {
+ let mut context = Self::default();
+ for address in genesis_allocations.keys() {
+ context.tables.register_address(address);
+ }
+ for block in blocks {
+ let mut writer = CompactWriter::default();
+ encode_block_body(&mut writer, block, &mut context.tables)?;
+ context.tables.register_protocol_id(&block.hash);
+ }
+ Ok(context)
+ }
+
+ pub(crate) fn block_size_bytes(&self, block: &Block) -> Result<usize> {
+ let mut tables = self.tables.clone();
+ let mut writer = CompactWriter::default();
+ encode_block_body(&mut writer, block, &mut tables)?;
+ Ok(writer.into_inner().len())
+ }
+
+ pub(crate) fn append_block(&mut self, block: &Block) -> Result<()> {
+ let mut tables = self.tables.clone();
+ let mut writer = CompactWriter::default();
+ encode_block_body(&mut writer, block, &mut tables)?;
+ tables.register_protocol_id(&block.hash);
+ self.tables = tables;
+ Ok(())
+ }
+}
+
#[derive(Default)]
struct DecodeTables {
addresses: Vec<String>,
diff --git a/src/adapters/http/ui.rs b/src/adapters/http/ui.rs
@@ -208,7 +208,7 @@ pub(super) fn ui_block(
.map(|bundle: &UiBurnBundle| bundle.byte_size)
.sum::<usize>();
let total_bytes = block
- .serialized_size_bytes()
+ .json_size_bytes()
.unwrap_or_else(|_| transaction_bytes.saturating_add(burn_bundle_bytes));
UiBlock {
height: block.height,
diff --git a/src/domain/adversarial_tests.rs b/src/domain/adversarial_tests.rs
@@ -969,8 +969,8 @@ fn mini_block_verdict(ledger: &Ledger, block: &Block, now_ms: u64) -> MiniBlockV
if block.transactions.len() > ledger.launch_profile.max_block_transactions {
return MiniBlockVerdict::TooManyTransactions;
}
- if block
- .serialized_size_bytes()
+ if ledger
+ .consensus_block_size_bytes(block)
.ok()
.is_none_or(|bytes| bytes > ledger.launch_profile.max_block_bytes)
{
@@ -2454,8 +2454,10 @@ fn independent_mini_validator_matches_consensus_for_block_prechecks() {
);
let mut size_limited_ledger = ledger.clone();
- size_limited_ledger.launch_profile.max_block_bytes =
- block.serialized_size_bytes().unwrap().saturating_sub(1);
+ size_limited_ledger.launch_profile.max_block_bytes = ledger
+ .consensus_block_size_bytes(&block)
+ .unwrap()
+ .saturating_sub(1);
assert_mini_validator_agrees(
&size_limited_ledger,
block.clone(),
@@ -4234,7 +4236,7 @@ fn performance_budget_block_validation_rejects_count_and_byte_overflow() {
"block over transaction-count budget validated"
);
- let block_bytes = block.serialized_size_bytes().unwrap();
+ let block_bytes = harness.ledger.consensus_block_size_bytes(&block).unwrap();
let mut byte_limited = harness.ledger.clone();
byte_limited.launch_profile.max_block_bytes = block_bytes;
byte_limited
@@ -4379,11 +4381,10 @@ fn blockspace_flood_stays_bounded_by_transaction_count_and_bytes() {
block.transactions.len(),
max_test_transactions
);
+ let block_bytes = ledger.consensus_block_size_bytes(&block).unwrap();
assert!(
- block.serialized_size_bytes().unwrap() <= MAX_BLOCK_BYTES,
- "block exceeded byte limit under flood: {} > {}",
- block.serialized_size_bytes().unwrap(),
- MAX_BLOCK_BYTES
+ block_bytes <= MAX_BLOCK_BYTES,
+ "block exceeded byte limit under flood: {block_bytes} > {MAX_BLOCK_BYTES}"
);
ledger
diff --git a/src/domain/block.rs b/src/domain/block.rs
@@ -106,10 +106,10 @@ impl Block {
}
}
- pub fn serialized_size_bytes(&self) -> Result<usize> {
+ pub fn json_size_bytes(&self) -> Result<usize> {
serde_json::to_vec(self)
.map(|bytes| bytes.len())
- .context("failed to serialize block for size check")
+ .context("failed to serialize block as JSON for size check")
}
pub fn required_burns(&self) -> Vec<&Transaction> {
@@ -377,7 +377,7 @@ mod tests {
}
#[test]
- fn serialized_block_size_uses_canonical_node_representation() {
+ fn json_block_size_uses_canonical_node_representation() {
let compact_wire_json = format!(
r#"{{"height":1,"prev_hash":"{}","timestamp_ms":1,"miner":"{}","reward":0,"vdf_rounds":1,"vdf_output":"out","leader_proof":null,"transactions":[],"hash":"{}"}}"#,
"0".repeat(64),
@@ -391,7 +391,7 @@ mod tests {
assert_eq!(block.finalizer_mode, FinalizerMode::Ticket);
assert_eq!(block.finalizer_rank, 0);
assert_eq!(block.burn_bundle_section, BurnBundleSection::default());
- assert_eq!(block.serialized_size_bytes().unwrap(), canonical_json_len);
+ assert_eq!(block.json_size_bytes().unwrap(), canonical_json_len);
assert!(canonical_json_len > compact_wire_json.len());
}
}
diff --git a/src/domain/ledger_apply.rs b/src/domain/ledger_apply.rs
@@ -102,6 +102,7 @@ impl Ledger {
apply_finalizer_ticket_effects(self.tip(), &block, &mut tickets)?;
tickets.extend(tickets_created_by_block(&block, &self.launch_profile)?);
credit_reward_outputs(&mut utxos, &block, &reward_committee)?;
+ self.compact_block_context.append_block(&block)?;
self.utxos = utxos;
self.utxo_lineage = utxo_lineage;
self.lineage_values = lineage_values;
@@ -190,7 +191,7 @@ impl Ledger {
if block.transactions.len() > self.launch_profile.max_block_transactions {
bail!("block has too many transactions");
}
- if block.serialized_size_bytes()? > self.launch_profile.max_block_bytes {
+ if self.consensus_block_size_bytes(block)? > self.launch_profile.max_block_bytes {
bail!("block exceeds max block size");
}
ensure_mine_anchor_limit(block.height, &block.transactions)?;
diff --git a/src/domain/ledger_chain.rs b/src/domain/ledger_chain.rs
@@ -2,6 +2,8 @@ use std::collections::{BTreeMap, BTreeSet};
use anyhow::{Result, bail};
+use crate::compact::CompactBlockContext;
+
use super::fork::{ForkChoice, ForkPoint, ForkQuality};
use super::genesis::{build_genesis_block, utxos_after_genesis, validate_genesis_block};
use super::ledger_ops::validate_genesis_allocations;
@@ -69,6 +71,11 @@ impl Ledger {
let genesis = build_genesis_block(&genesis_allocations, genesis_transactions);
let utxos = utxos_after_genesis(&genesis_allocations, &genesis)?;
let tickets = genesis_tickets(&genesis_allocations, &genesis, &launch_profile)?;
+ let compact_block_context = if genesis_allocations.is_empty() {
+ CompactBlockContext::default()
+ } else {
+ CompactBlockContext::for_chain(&genesis_allocations, std::slice::from_ref(&genesis))?
+ };
Ok(Self {
chain: vec![genesis],
genesis_allocations: genesis_allocations.clone(),
@@ -85,6 +92,7 @@ impl Ledger {
initial_vdf_rounds: vdf_rounds,
vdf_rounds,
launch_profile,
+ compact_block_context,
})
}
@@ -129,6 +137,8 @@ impl Ledger {
bail!("chain snapshot genesis does not match its allocations and transactions");
}
let utxos = utxos_after_genesis(&genesis_allocations, &genesis)?;
+ let compact_block_context =
+ CompactBlockContext::for_chain(&genesis_allocations, std::slice::from_ref(&genesis))?;
let mut ledger = Self {
chain: vec![genesis],
@@ -146,6 +156,7 @@ impl Ledger {
initial_vdf_rounds: vdf_rounds,
vdf_rounds,
launch_profile,
+ compact_block_context,
};
ledger.tickets = genesis_tickets(
&ledger.genesis_allocations,
diff --git a/src/domain/ledger_mempool.rs b/src/domain/ledger_mempool.rs
@@ -2,7 +2,8 @@ use anyhow::{Context, Result, bail};
use serde::Serialize;
use super::ledger_ops::{
- apply_transaction, ensure_transaction_fits_empty_block, transaction_has_missing_inputs,
+ apply_transaction, compact_block_context, ensure_transaction_fits_empty_block,
+ transaction_has_missing_inputs,
};
use super::transaction::{Transaction, transaction_inputs_spent_by};
use super::{
@@ -45,7 +46,11 @@ impl Ledger {
transaction.verify_signature()?;
self.validate_transaction_terms(&transaction)?;
- ensure_transaction_fits_empty_block(&transaction, self.launch_profile.max_block_bytes)?;
+ ensure_transaction_fits_empty_block(
+ compact_block_context(self),
+ &transaction,
+ self.launch_profile.max_block_bytes,
+ )?;
self.validate_mine_anchor_available(&transaction)?;
if transaction_inputs_spent_by(&transaction, &self.pending) {
diff --git a/src/domain/ledger_ops.rs b/src/domain/ledger_ops.rs
@@ -1,6 +1,8 @@
use std::collections::{BTreeMap, BTreeSet};
use anyhow::{Context, Result, bail};
+
+use crate::compact::CompactBlockContext;
use ed25519_dalek::{Signature, Verifier, VerifyingKey};
use super::hex::hex_hash;
@@ -11,11 +13,21 @@ use super::transaction::Transaction;
use super::vdf::vdf_solution_placeholder;
use super::{
Amount, BURN_COMMITTEE_SIZE, Block, BlockSelection, BurnCommitteeMember, BurnTicket,
- FinalizerMode, LeaderProof, LeaderProofPayload, MINE_REWARD, OutPoint, PUBLIC_KEY_BYTES,
- RECOVERY_BLOCK_DELAY_MS, SIGNATURE_BYTES, TxInput, TxOutput, decode_hex_array,
- validate_address, validate_hash, validate_protocol_id, validate_signature,
+ FinalizerMode, LeaderProof, LeaderProofPayload, Ledger, MINE_REWARD, OutPoint,
+ PUBLIC_KEY_BYTES, RECOVERY_BLOCK_DELAY_MS, SIGNATURE_BYTES, TxInput, TxOutput,
+ decode_hex_array, validate_address, validate_hash, validate_protocol_id, validate_signature,
};
+pub(super) fn compact_block_context(ledger: &Ledger) -> &CompactBlockContext {
+ &ledger.compact_block_context
+}
+
+impl Ledger {
+ pub(crate) fn consensus_block_size_bytes(&self, block: &Block) -> Result<usize> {
+ self.compact_block_context.block_size_bytes(block)
+ }
+}
+
pub(super) fn validate_genesis_allocations(
genesis_allocations: &BTreeMap<String, Amount>,
) -> Result<()> {
@@ -68,6 +80,7 @@ pub(super) fn validate_genesis_burn_transaction(transaction: &Transaction) -> Re
}
pub(super) fn estimated_block_selection_size_bytes(
+ context: &CompactBlockContext,
selection: &BlockSelection,
recovery: bool,
burn_bundle_section: &BurnBundleSection,
@@ -95,18 +108,23 @@ pub(super) fn estimated_block_selection_size_bytes(
transactions: selection.transactions.clone(),
hash: "f".repeat(64),
};
- block.serialized_size_bytes()
+ context.block_size_bytes(&block)
}
pub(super) fn ensure_transaction_fits_empty_block(
+ context: &CompactBlockContext,
transaction: &Transaction,
max_block_bytes: usize,
) -> Result<()> {
let selection = BlockSelection {
transactions: vec![transaction.clone()],
};
- if estimated_block_selection_size_bytes(&selection, false, &BurnBundleSection::default())?
- > max_block_bytes
+ if estimated_block_selection_size_bytes(
+ context,
+ &selection,
+ false,
+ &BurnBundleSection::default(),
+ )? > max_block_bytes
{
bail!("transaction exceeds max block size");
}
diff --git a/src/domain/ledger_pending.rs b/src/domain/ledger_pending.rs
@@ -5,7 +5,7 @@ use anyhow::{Context, Result, bail};
use super::ledger_mempool::pending_pool_item_bytes;
use super::ledger_ops::{
apply_spendable_pending_transaction, apply_transaction, best_selectable_burn_from_index,
- best_selectable_transaction_index, ensure_transaction_fits_empty_block,
+ best_selectable_transaction_index, compact_block_context, ensure_transaction_fits_empty_block,
estimated_block_selection_size_bytes, transaction_has_missing_inputs,
validate_transaction_inputs, validate_transaction_outputs,
};
@@ -109,6 +109,7 @@ impl Ledger {
required_burn_signature: Option<&str>,
burn_bundle_section: &BurnBundleSection,
) -> Result<BlockSelection> {
+ let block_context = compact_block_context(self);
let mut utxos = self.utxos.clone();
let mut remaining = self.valid_pending_transactions();
let mut selected = Vec::new();
@@ -135,13 +136,7 @@ impl Ledger {
if let Some(index) = anchor_index {
let tx = remaining.remove(index);
let signature = tx.signature().to_string();
- self.select_required_anchor_burn(
- tx,
- required_burn_owner,
- burn_bundle_section,
- &mut utxos,
- &mut selected,
- )?;
+ self.select_required_anchor_burn(tx, required_burn_owner, &mut utxos, &mut selected)?;
if required_burn_signatures.contains(&signature) {
selected_required_burn_signatures.insert(signature);
}
@@ -163,18 +158,6 @@ impl Ledger {
bail!("attested burns do not fit within the block transaction count limit");
}
let signature = tx.signature().to_string();
- let mut candidate = BlockSelection {
- transactions: selected.clone(),
- };
- candidate.transactions.push(tx.clone());
- if estimated_block_selection_size_bytes(
- &candidate,
- required_burn_owner.is_some(),
- burn_bundle_section,
- )? > self.launch_profile.max_block_bytes
- {
- bail!("attested burns do not fit in the block");
- }
apply_transaction(&tx, &mut utxos).context("attested burn is not spendable")?;
selected.push(tx);
selected_required_burn_signatures.insert(signature);
@@ -187,6 +170,19 @@ impl Ledger {
bail!("attested burn {missing} is not pending");
}
+ let required_selection = BlockSelection {
+ transactions: selected.clone(),
+ };
+ if estimated_block_selection_size_bytes(
+ block_context,
+ &required_selection,
+ required_burn_owner.is_some(),
+ burn_bundle_section,
+ )? > self.launch_profile.max_block_bytes
+ {
+ bail!("required block content does not fit in the block");
+ }
+
while selected.len() < self.launch_profile.max_block_transactions {
let Some(index) = best_selectable_transaction_index(&remaining, &utxos, None) else {
break;
@@ -197,6 +193,7 @@ impl Ledger {
};
candidate.transactions.push(tx.clone());
if estimated_block_selection_size_bytes(
+ block_context,
&candidate,
required_burn_owner.is_some(),
burn_bundle_section,
@@ -215,7 +212,6 @@ impl Ledger {
&self,
tx: Transaction,
required_burn_owner: Option<&str>,
- burn_bundle_section: &BurnBundleSection,
utxos: &mut BTreeMap<OutPoint, TxOutput>,
selected: &mut Vec<Transaction>,
) -> Result<()> {
@@ -230,18 +226,6 @@ impl Ledger {
if selected.len() >= self.launch_profile.max_block_transactions {
bail!("required block anchor burn does not fit within the transaction count limit");
}
- let mut candidate = BlockSelection {
- transactions: selected.clone(),
- };
- candidate.transactions.push(tx.clone());
- if estimated_block_selection_size_bytes(
- &candidate,
- required_burn_owner.is_some(),
- burn_bundle_section,
- )? > self.launch_profile.max_block_bytes
- {
- bail!("required block anchor burn does not fit in the block");
- }
apply_transaction(&tx, utxos).context("required block anchor burn is not spendable")?;
selected.push(tx);
Ok(())
@@ -312,7 +296,11 @@ impl Ledger {
pub(super) fn validate_new_transaction(&self, transaction: &Transaction) -> Result<()> {
self.validate_transaction_terms(transaction)?;
- ensure_transaction_fits_empty_block(transaction, self.launch_profile.max_block_bytes)?;
+ ensure_transaction_fits_empty_block(
+ compact_block_context(self),
+ transaction,
+ self.launch_profile.max_block_bytes,
+ )?;
self.validate_mine_anchor_available(transaction)?;
let mut utxos = self.utxos_after_spendable_pending()?;
apply_transaction(transaction, &mut utxos)
diff --git a/src/domain/ledger_state.rs b/src/domain/ledger_state.rs
@@ -7,6 +7,7 @@ use super::{
Amount, Block, BurnTicket, LaunchProfile, LineageOwnerValues, OutPoint, Transaction, TxOutput,
UtxoLineageRoot,
};
+use crate::compact::CompactBlockContext;
#[derive(Clone, Debug)]
pub struct Ledger {
@@ -25,6 +26,7 @@ pub struct Ledger {
pub(super) initial_vdf_rounds: u64,
pub(super) vdf_rounds: u64,
pub(super) launch_profile: LaunchProfile,
+ pub(super) compact_block_context: CompactBlockContext,
}
pub(super) fn unix_now_ms() -> u64 {
diff --git a/src/lib.rs b/src/lib.rs
@@ -1,3 +1,5 @@
pub mod adapters;
pub mod app;
+#[path = "adapters/chain_store/compact.rs"]
+pub(crate) mod compact;
pub mod domain;