Skip to main content

​
Purpose and Scope

The test framework provides infrastructure for running standardized Ethereum JSON test fixtures to validate core protocol behavior.
Its goal is to verify that transaction processing, block validation, state transitions, and consensus rules operate correctly across a range of hard-fork configurations. The test fixtures use data from the tests directory included in the StableNet codebase and are executed directly on top of the StableNet implementation. For EVM-specific testing tools, see EVM Testing Tools. For general build and CI/CD flows, see Build and CI/CD.

​
Test Type Overview

The test framework supports four major test categories to validate different aspects of the protocol.
Test TypePurposeJSON StructureExecution
State TestsValidate transaction processing without block contextstJSON containing pre-state, transaction, and post-stateExecute a single transaction on top of genesis
Block TestsValidate full blocks including header/body and state transitionsbtJSON containing genesis and a block sequenceInsert blocks sequentially
Transaction TestsValidate RLP encoding and sender derivationTransaction RLP with expected sender per forkDecode and verify sender
Difficulty TestsValidate difficulty calculation rulesParent header and expected difficultyCompare against consensus engine output

​
Test Infrastructure Architecture

​
testMatcher Pattern Matching

testMatcher controls test selection, configuration overrides, and expected-failure handling via regular-expression rules.
Pattern TypeMethodPurpose
skiploadpatskipLoad(pattern)Skip JSON loading itself to reduce memory usage
slowpatslow(pattern)Exclude in -short mode or on 32-bit environments
failpatfails(pattern, reason)Mark known failing tests
configpatconfig(pattern, cfg)Override chain config for specific tests
runonlylistpatrunonly(pattern)Run only tests matching the specified pattern

​
Running State Tests

State tests execute a single transaction against a pre-state and validate the resulting state root and logs, allowing transaction processing logic to be verified in isolation.

​
StateTest Implementation Overview

State tests generate and run subtests for each fork combination and input index. Key methods:
MethodPurpose
Subtests()Enumerate all fork/index combinations
Run(...)Execute and verify results
RunNoVerify(...)Execute without verification (for benchmarking)
checkError(...)Validate whether an error was expected

​
Execution Flow Summary

  1. Parse fork string and load chain config
  2. Create genesis block and set up pre-state
  3. Convert the transaction into a core.Message
  4. Create EVM context and execute the transaction
  5. Commit state and compute state root
  6. Compare against expected state root, logs, and exceptions

​
Running Across Storage Schema Variants

Each State test is executed across the following four combinations:
  • hash / trie: hash-based scheme without snapshots
  • hash / snap: hash-based scheme with snapshots
  • path / trie: path-based scheme without snapshots
  • path / snap: path-based scheme with snapshots
This validates consistency across both StateDB and snapshot layers.

​
Running Block Tests

Block tests insert a block sequence into an actual chain and validate the final state and header chain to confirm end-to-end blockchain behavior.

​
BlockTest Implementation Overview

Block tests include the following tasks:
  • Create a genesis block from test data
  • Insert test blocks sequentially
  • Handle valid vs invalid blocks
  • Validate final state and header chain
If a block JSON lacks BlockHeader, transactions, and UncleHeaders, that block is treated as invalid, and insertion failure is the expected behavior.

​
Execution Flow Summary

  1. Commit genesis state to the DB
  2. Initialize the blockchain including the consensus engine and caches
  3. Insert test blocks sequentially
  4. Validate account state against the final StateDB
  5. Validate header consistency by walking backward from the chain head

​
Transaction Tests

Transaction tests validate that RLP decoding and sender derivation behave correctly under different fork rules. The framework verifies:
  1. Whether RLP decoding succeeds
  2. Sender recovery via ECDSA signatures
  3. Validity of intrinsic gas calculation
  4. Transaction hash consistency

​
Fork Configuration

The test framework predefines chain configurations for all Ethereum hard forks. Examples include:
ForkKey Characteristics
FrontierInitial release
ByzantiumDifficulty bomb delay
IstanbulGas cost adjustments
LondonEIP-1559 BaseFee
MergeTransition to PoS
CancunBlob transactions
AnzeonWBFT consensus
Fork strings may include EIP overrides such as "Berlin+1884+1283", and are configured dynamically at runtime.

​
Test Directory Structure

Test data is organized using a standardized directory structure. 
tests/
├── testdata/
│   ├── BlockchainTests/
│   ├── GeneralStateTests/
│   ├── TransactionTests/
│   ├── RLPTests/
│   ├── BasicTests/
│   └── LegacyTests/
├── spec-tests/
│   └── fixtures/
│       ├── blockchain_tests/
│       └── state_tests/
└── evm-benchmarks/
    └── benchmarks/

​
EVM Command Integration

The cmd/evm tool provides a CLI interface for running Block tests.

​
blocktest Command Overview

Key flags include:
  • --run: select tests via regular expression
  • --machine: output JSON-based EVM traces
  • --dump: dump final state after execution

​
Benchmarking Support

State tests support benchmark execution for performance measurement. Benchmarks measure:
  • Gas throughput (mgas/s)
  • Total execution time
  • Effective throughput accounting for gas refunds
This enables quantitative analysis of how EVM and StateDB changes impact performance.

​
Error Handling and Verification

​
Expected Exception Handling

Tests determine success by comparing expected exceptions with actual outcomes.
ExpectedActualResult
No exceptionNo errorPass
Exception expectedError occursPass
No exceptionError occursFail
Exception expectedNo errorFail
The current implementation does not strictly compare error strings; it focuses primarily on whether an error occurred.

​
Diagnostic Output on Failure

When a test fails, the following information is provided:
  • Full EVM execution trace for the failing test
  • Step-by-step operation logs in JSON format
  • Gas-limit-based constraints to prevent excessive output
This enables precise analysis of protocol behavior issues.