> ## Documentation Index > Fetch the complete documentation index at: https://developers.stellar.org/llms.txt > Use this file to discover all available pages before exploring further. # Fuzz Testing > Increase confidence in a contract's correctness with fuzz testing. The [fuzzing example] demonstrates how to fuzz test Soroban contracts with [`cargo-fuzz`] and customize the input to fuzz tests with the [`arbitrary`] crate. It also demonstrates how to adapt fuzz tests into reusable property tests with the [`proptest`] and [`proptest-arbitrary-interop`] crates. It builds on the [timelock example]. [![Open in Codespaces](https://github.com/codespaces/badge.svg)][open-in-github-codespaces] [![Open in Codeanywhere](https://codeanywhere.com/img/open-in-codeanywhere-btn.svg)][open-in-code-anywhere] [open-in-github-codespaces]: https://github.com/codespaces/new?repo=stellar/soroban-examples&editor=web [open-in-code-anywhere]: https://app.codeanywhere.com/#https://github.com/stellar/soroban-examples [fuzzing example]: https://github.com/stellar/soroban-examples/tree/v23.0.0/fuzzing [`cargo-fuzz`]: https://docs.rs/cargo-fuzz [`arbitrary`]: https://docs.rs/arbitrary [`proptest`]: https://docs.rs/proptest [`proptest-arbitrary-interop`]: https://docs.rs/proptest-arbitrary-interop [timelock example]: https://github.com/stellar/soroban-examples/tree/v23.0.0/timelock ## Run the Example First go through the [setup] process to get your development environment configured, then clone the `v23.0.0` tag of `soroban-examples` repository: [setup]: https://developers.stellar.org/docs/build/smart-contracts/getting-started/setup.md ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} git clone -b v23.0.0 https://github.com/stellar/soroban-examples ``` You will also need the `cargo-fuzz` tool, and to run `cargo-fuzz` you will need a nightly Rust toolchain: ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} cargo install cargo-fuzz rustup install nightly ``` To run one of the fuzz tests, navigate to the `fuzzing` directory and run the `cargo fuzz` subcommand with the `nightly` toolchain: ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} cd fuzzing cargo +nightly fuzz run fuzz_target_1 ``` > [!NOTE] > > If you're developing on MacOS you may need to add the `--sanitizer=thread` flag in order to fix some [known linking errors](https://github.com/stellar/rs-soroban-sdk/issues/1056). You should see output that begins like this: ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} $ cargo +nightly fuzz run fuzz_target_1 Compiling soroban-fuzzing-contract v0.0.0 (/home/azureuser/data/stellar/soroban-examples/fuzzing) Compiling soroban-fuzzing-contract-fuzzer v0.0.0 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz) Finished release [optimized + debuginfo] target(s) in 23.74s Finished release [optimized + debuginfo] target(s) in 0.07s Running `fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1 ...` INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 886588732 INFO: Loaded 1 modules (1093478 inline 8-bit counters): 1093478 [0x55eb8e2c7620, 0x55eb8e3d2586), INFO: Loaded 1 PC tables (1093478 PCs): 1093478 [0x55eb8e3d2588,0x55eb8f481be8), INFO: 105 files found in /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/corpus/fuzz_target_1 INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: seed corpus: files: 105 min: 32b max: 61b total: 3558b rss: 86Mb #2 pulse ft: 8355 exec/s: 1 rss: 307Mb #4 pulse cov: 8354 ft: 11014 corp: 1/32b exec/s: 2 rss: 313Mb #8 pulse cov: 8495 ft: 12420 corp: 4/128b exec/s: 4 rss: 315Mb ``` The rest of this tutorial will explain how to set up this fuzz test, interpret this output, and remedy fuzzing failures. ## Background: Fuzz Testing and Rust Fuzzing is a kind of testing where new inputs are repeatedly fed into a program in hopes of finding unexpected bugs. This style of testing is commonly employed to increase confidence in the correctness of security-sensitive software. In Rust, fuzzing is most often performed with the [`cargo-fuzz`] tool, which drives LLVM's [`libfuzzer`], though other fuzzing tools are available. [`libfuzzer`]: https://llvm.org/docs/LibFuzzer.html Soroban has built-in support for fuzzing Soroban contracts with `cargo-fuzz`. `cargo-fuzz` is a mutation-based fuzzer: it runs a test program, passing it generated input; while the program is executing, the fuzzer monitors which branches the program takes, and which functions it executes; after execution the fuzzer uses this information to make decisions about how to *mutate* the previously-used input to create new input that might discover more branches and functions; it then runs the test again with new input, repeating this process for potentially millions of iterations. In this way `cargo-fuzz` is able to automatically explore execution paths through the program that may never be seen by other types of tests. If a fuzz tests panics or hard-crashes, `cargo-fuzz` considers it a failure and provides instructions for repeating the test with the failing inputs. Fuzz testing is typically an exploratory and interactive process, with the programmer devising schemes for producing input that will stress the program in interesting ways, observing the behavior of the fuzz test, and iterating on the test itself. Resolving a fuzz testing failure typically involves capturing the problematic input in a unit test. The fuzz test itself may or may not be kept, depending on determinations about the cost of maintaining the fuzzer vs the likelihood of it continuing to find bugs in the future. While fuzzing non-memory-safe software tends to be more lucrative than fuzzing Rust software, it is still relatively common to find panics and other logic errors in Rust through fuzzing. In Rust, multiple fuzzers are maintained by the [`rust-fuzz`] GitHub organization, which also maintains a "trophy case" of Rust bugs found through fuzzing. [`rust-fuzz`]: https://github.com/rust-fuzz ## About the Example The example used for this tutorial is based on the [`timelock`] example program, with some changes to demonstrate fuzzing. [`timelock`]: https://github.com/stellar/soroban-examples/tree/v23.0.0/timelock The contract, `ClaimableBalanceContract`, allows one party to deposit an arbitrary quantity of a token to the contract, specifying additionally: the `claimants`, addresses that may withdraw from the contract; and the `time_bound`, a specification of when those claimants may withdraw from the account. The `TimeBound` type looks like ```rust theme={"theme":{"light":"github-light","dark":"github-dark"}} #[derive(Clone, Debug)] #[contracttype] pub enum TimeBoundKind { Before, After, } #[derive(Clone, Debug)] #[contracttype] pub struct TimeBound { pub kind: TimeBoundKind, pub timestamp: u64, } ``` `ClaimableBalanceContract` has two methods, `deposit` and `claim`: ```rust theme={"theme":{"light":"github-light","dark":"github-dark"}} pub fn deposit( env: Env, from: Address, token: Address, amount: i128, claimants: Vec
, time_bound: TimeBound, ); pub fn claim( env: Env, claimant: Address, amount: i128, ); ``` `deposit` may only be successfully called once, after which `claim` may be called multiple times until the balance is completely drained, at which point the contract becomes dormant and may no longer be used. ## Fuzz Testing Setup For these examples, the fuzz tests have been created for you, but normally you would use the `cargo fuzz init` command to create a fuzzing project as a subdirectory of the contract under test. To do that you would navigate to the contract directory, in this case, `soroban-examples/fuzzing`, and execute ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} cargo fuzz init ``` A `cargo-fuzz` project is its own crate, which lives in the `fuzz` subdirectory of the crate being tested. This crate has its own `Cargo.toml` and `Cargo.lock`, and another subdirectory, `fuzz_targets`, which contains Rust programs, each its own fuzz test. Our `soroban-examples/fuzzing` directory looks like * `Cargo.toml` - this is the contract's manifest * `Cargo.lock` * `src` * `lib.rs` - this is the contract code * `fuzz` - this is the fuzzing crate * `Cargo.toml` - this is fuzzing crate's manifest * `Cargo.lock` * `fuzz_targets` * `fuzz_target_1.rs` - this is a single fuzz test * `fuzz_target_2.rs` There are special considerations to note in the configuration of both the [contract's manifest] and the [fuzzing crate's manifest]. [contract's manifest]: https://github.com/stellar/soroban-examples/tree/v23.0.0/fuzzing/Cargo.toml [fuzzing crate's manifest]: https://github.com/stellar/soroban-examples/tree/v23.0.0/fuzzing/fuzz/Cargo.toml Within the contract's manifest one must specificy the crate type as both "cdylib" and "rlib": ```toml title="fuzzing/Cargo.toml" theme={"theme":{"light":"github-light","dark":"github-dark"}} [package] name = "soroban-fuzzing-contract" version = "0.0.0" edition = "2021" publish = false rust-version = "1.89.0" [lib] crate-type = ["cdylib", "rlib"] doctest = false [features] testutils = [] ``` In most examples, a Soroban contract will only be a "cdylib", a Rust crate that is compiled to a dynamically loadable Wasm module. For fuzzing though, the fuzzing crate needs to be able to link to the contract crate as a Rust library, an "rlib". > [!NOTE] > > Note that cargo has a \[feature/bug that inhibits LTO]\[lto] of cdylibs when a crate is both a "cdylib" and "rlib". This can be worked around by building the contract with either `soroban contract build` or `cargo rustc --crate-type cdylib` instead of the typical `cargo build`. [lto]: https://github.com/stellar/soroban-docs/pull/476 The contract crate must also provide the "testutils" feature. When "testutils" is activated, the Soroban SDK's [`contracttype`] macro emits additional code needed for running fuzz tests. [`contracttype`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/attr.contracttype.html Within the fuzzing crate's manifest one must turn on the "testutils" features in both the contract crate and the `soroban-sdk` crate: ```toml title="fuzzing/fuzz/Cargo.toml" theme={"theme":{"light":"github-light","dark":"github-dark"}} [package] name = "soroban-fuzzing-contract-fuzzer" version = "0.0.0" publish = false edition = "2021" rust-version = "1.79.0" [package.metadata] cargo-fuzz = true [dependencies] libfuzzer-sys = "0.4" soroban-sdk = { version = "23.0.1", features = ["testutils"] } soroban-env-host = { version = "23.0.1" } soroban-ledger-snapshot = { version = "23.0.1" } [dependencies.soroban-fuzzing-contract] path = ".." features = ["testutils"] ``` ## A Simple Fuzz Test First let's look at [`fuzz_target_1.rs`]. This fuzz test does two things: it first deposits an arbitrary amount, then it claims an arbitrary amount. [`fuzz_target_1.rs`]: https://github.com/stellar/soroban-examples/tree/v23.0.0/fuzzing/fuzz/fuzz_targets/fuzz_target_1.rs Again, you can run this fuzzer from the `soroban-examples/fuzzing` directory with the following command: ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} cargo +nightly fuzz run fuzz_target_1 ``` The entry point and setup code for Soroban contract fuzz tests will typically look like: ```rust title="fuzzing/fuzz/fuzz_targets/fuzz_target_1.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} #[derive(Arbitrary, Debug)] struct Input { deposit_amount: i128, claim_amount: i128, } fuzz_target!(|input: Input| { let env = Env::default(); env.mock_all_auths(); env.ledger().set(LedgerInfo { timestamp: 12345, protocol_version: 1, sequence_number: 10, network_id: Default::default(), base_reserve: 10, min_temp_entry_ttl: u32::MAX, min_persistent_entry_ttl: u32::MAX, max_entry_ttl: u32::MAX, }); // Turn off the CPU/memory budget for testing. env.cost_estimate().budget().reset_unlimited(); // ... do fuzzing here ... } ``` Instead of a `main` function, `cargo-fuzz` uses a special entry point defined by the [`fuzz_target!`] macro. This macro accepts a Rust closure that accepts `input`, any Rust type that implements the [`Arbitrary`] trait. Here we have defined a struct, `Input`, that derives `Arbitrary`. [`fuzz_target!`]: https://docs.rs/libfuzzer-sys/latest/libfuzzer_sys/macro.fuzz_target.html [`arbitrary`]: https://docs.rs/arbitrary/latest/arbitrary/trait.Arbitrary.html `cargo-fuzz` will be responsible for generating `input` and repeatedly calling this closure. To test a Soroban contract, we must set up an [`Env`]. Note that we have disabled the CPU and memory budget: this will allow us to fuzz arbitrarily complex code paths without worrying about running out of budget; we can assume that running out of budget during a transaction always correctly fails, canceling the transaction; it is not something we need to fuzz. [`env`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Env.html Refer to the [`fuzz_target_1.rs`] source code for additional setup for this contract. This fuzzer performs two steps: deposit, then claim: ```rust title="fuzzing/fuzz/fuzz_targets/fuzz_target_1.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} // Deposit, then assert invariants. { let _ = timelock_client.try_deposit( &depositor_address, &token_contract_id, &input.deposit_amount, &vec![&env, claimant_address.clone()], &TimeBound { kind: TimeBoundKind::Before, timestamp: 123456, }, ); assert_invariants(&env, &timelock_contract_id, &token_client, &input); } // Claim, then assert invariants. { let _ = timelock_client.try_claim(&claimant_address, &input.claim_amount); assert_invariants(&env, &timelock_contract_id, &token_client, &input); } ``` There are a number of potential strategies for writing fuzz tests. The strategy in this test is to make arbitrary, possibly weird and unrealistic, calls to the contract, disregarding whether those calls succeed or fail, and then to make assertions about the state of the contract. Because there are many potential failure cases for any given contract call, we don't want to write a fuzz test by attempting to interpret the success or failure of any given call: that path leads to duplicating the contract's logic within the fuzz test. Instead we just want to ensure that, regardless of what happened during execution, the contract is never left in an invalid state. Notice the use of the `try_` client function to invoke the contract. Each contract function can be invoked with a `try_` variant, which captures any errors, including panics and crashes, and returns the value on success or an error otherwise. Without using the `try_` variant, a panic from within a contract will immediately cause the fuzz test to fail, but in most cases a panic within a contract does not indicate a bug - it is simply how a Soroban contract cancels a transaction. `try_` returns a `Result`, but here we discard it. Finally, the `assert_invariants` function is where we make any assertions we can about the state of the contract: ```rust title="fuzzing/fuzz/fuzz_targets/fuzz_target_1.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} /// Directly inspect the contract state and make assertions about it. fn assert_invariants( env: &Env, timelock_contract_id: &Address, token_client: &TokenClient, input: &Input, ) { // Configure the environment to access the timelock contract's storage. env.as_contract(timelock_contract_id, || { let storage = env.storage().persistent(); // Get the two datums owned by the timelock contract. let is_initialized = storage.has(&DataKey::Init); let claimable_balance = storage.get::<_, ClaimableBalance>(&DataKey::Balance); // Call the token client to get the balance held in the timelock contract. // This consumes contract execution budget. let actual_token_balance = token_client.balance(timelock_contract_id); // There can only be a claimaible balance after the contract is initialized, // but once the balance is claimed there is no balance, // but the contract remains initialized. // This is a truth table of valid states. assert!(match (is_initialized, claimable_balance.is_some()) { (false, false) => true, (false, true) => false, (true, true) => true, (true, false) => true, }); assert!(actual_token_balance >= 0); if let Some(claimable_balance) = claimable_balance { assert!(claimable_balance.amount > 0); assert!(claimable_balance.amount <= input.deposit_amount); assert_eq!(claimable_balance.amount, actual_token_balance); assert!(claimable_balance.claimants.len() > 0); } }); } ``` ## Interpreting `cargo-fuzz` Output If you run `cargo-fuzz` with `fuzz_target_1`, from inside the `soroban-examples/fuzzing` directory, you will see output similar to: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} $ cargo +nightly fuzz run fuzz_target_1 Compiling soroban-fuzzing-contract v0.0.0 (/home/azureuser/data/stellar/soroban-examples/fuzzing) Compiling soroban-fuzzing-contract-fuzzer v0.0.0 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz) Finished release [optimized + debuginfo] target(s) in 25.18s Finished release [optimized + debuginfo] target(s) in 0.08s Running `fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1 -artifact_prefix=/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/artifacts/fuzz_target_1/ /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/corpus/fuzz_target_1` INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1384064486 INFO: Loaded 1 modules (1122058 inline 8-bit counters): 1122058 [0x561f6ecd4fc0, 0x561f6ede6eca), INFO: Loaded 1 PC tables (1122058 PCs): 1122058 [0x561f6ede6ed0,0x561f6ff05f70), INFO: 173 files found in /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/corpus/fuzz_target_1 INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: seed corpus: files: 173 min: 32b max: 61b total: 6039b rss: 83Mb #4 pulse cov: 4848 ft: 10214 corp: 1/32b exec/s: 2 rss: 313Mb #8 pulse cov: 8507 ft: 11743 corp: 4/128b exec/s: 4 rss: 315Mb #16 pulse cov: 8512 ft: 12393 corp: 10/320b exec/s: 8 rss: 319Mb thread '' panicked at 'assertion failed: claimable_balance.amount > 0', fuzz_targets/fuzz_target_1.rs:130:13 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ==6102== ERROR: libFuzzer: deadly signal #0 0x561f6ae3a431 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x1c80431) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) #1 0x561f6e3855b0 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x51cb5b0) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) #2 0x561f6e35c08a (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x51a208a) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) #3 0x7fce05f5e08f (/lib/x86_64-linux-gnu/libc.so.6+0x4308f) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee) #4 0x7fce05f5e00a (/lib/x86_64-linux-gnu/libc.so.6+0x4300a) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee) #5 0x7fce05f3d858 (/lib/x86_64-linux-gnu/libc.so.6+0x22858) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee) ... #27 0x561f6e3847b9 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x51ca7b9) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) #28 0x561f6ad98346 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x1bde346) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) #29 0x7fce05f3f082 (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee) #30 0x561f6ad9837d (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x1bde37d) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) NOTE: libFuzzer has rudimentary signal handlers. Combine libFuzzer with AddressSanitizer or similar for better crash reports. SUMMARY: libFuzzer: deadly signal MS: 0 ; base unit: 0000000000000000000000000000000000000000 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x5d,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0xff,0x5f,0x5f,0x52,0xff, \000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000]\000\000\000\000\000\000\000\000\377__R\377 artifact_prefix='/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/artifacts/fuzz_target_1/'; Test unit written to /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 Base64: AAAAAAAAAAAAAAAAAAAAAAAAXQAAAAAAAAAA/19fUv8= ──────────────────────────────────────────────────────────────────────────────── Failing input: fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 Output of `std::fmt::Debug`: Input { deposit_amount: 0, claim_amount: -901525218878596739118967460911579136, } Reproduce with: cargo fuzz run fuzz_target_1 fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 Minimize test case with: cargo fuzz tmin fuzz_target_1 fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 ──────────────────────────────────────────────────────────────────────────────── Error: Fuzz target exited with exit status: 77 ``` This is a fuzzing failure, indicating a bug in either the fuzzer or the program. The details will be different. Here is the same output, with less important lines trimmed: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} thread '' panicked at 'assertion failed: claimable_balance.amount > 0', fuzz_targets/fuzz_target_1.rs:130:13 ... Failing input: fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 Output of `std::fmt::Debug`: Input { deposit_amount: 0, claim_amount: -901525218878596739118967460911579136, } Reproduce with: cargo fuzz run fuzz_target_1 fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 Minimize test case with: cargo fuzz tmin fuzz_target_1 fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 ``` The first line here is printed by our Rust program, and indicates exactly where the fuzzer panicked. The later lines indicate how to reproduce this failing case. The first thing to do when you get a fuzzing failure is copy the command to reproduce the failure, so that you can use it to debug: ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} cargo +nightly fuzz run fuzz_target_1 fuzz/artifacts/fuzz_target_1/crash-04704b1542f61a21a4649e39023ec57ff502f627 ``` Notice though that we need to tell `cargo` to use the nightly toolchain with the `+nightly` flag, something that `cargo-fuzz` doesn't print in its version of the command. Another thing to notice is that by default, `cargo-fuzz` / `libfuzzer` does not print names of functions in its output, as in the stack trace: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} ==6102== ERROR: libFuzzer: deadly signal #0 0x561f6ae3a431 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x1c80431) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) ... #28 0x561f6ad98346 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x1bde346) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) #29 0x7fce05f3f082 (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee) #30 0x561f6ad9837d (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x1bde37d) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) ``` Depending on how your system is set up, you may or may not have this problem. In order to print stack traces, `libfuzzer` needs the `llvm-symbolizer` program. On Ubuntu-based systems this can be installed with the `llvm-dev` package: ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} sudo apt install llvm-dev ``` After which `libfuzzer` will print demangled function names instead of addresses: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} ==6323== ERROR: libFuzzer: deadly signal #0 0x557c9da6a431 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:87:3 #1 0x557ca0fb55b0 in fuzzer::PrintStackTrace() /home/azureuser/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.5/libfuzzer/FuzzerUtil.cpp:210:38 #2 0x557ca0f8c08a in fuzzer::Fuzzer::CrashCallback() /home/azureuser/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.5/libfuzzer/FuzzerLoop.cpp:233:18 #3 0x557ca0f8c08a in fuzzer::Fuzzer::CrashCallback() /home/azureuser/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.5/libfuzzer/FuzzerLoop.cpp:228:6 #4 0x7ff19e84d08f (/lib/x86_64-linux-gnu/libc.so.6+0x4308f) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee) #5 0x7ff19e84d00a in __libc_signal_restore_set /build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/internal-signals.h:86:3 #6 0x7ff19e84d00a in raise /build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:48:3 #7 0x7ff19e82c858 in abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:79:7 ... #23 0x557c9daee89a in fuzz_target_1::assert_invariants::hd6d4f9549b01c31c /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/fuzz_targets/fuzz_target_1.rs:103:5 #24 0x557c9daee89a in fuzz_target_1::_::run::hac1117cb3dfecb2b /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/fuzz_targets/fuzz_target_1.rs:69:9 #25 0x557c9daecea6 in rust_fuzzer_test_input /home/azureuser/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.5/src/lib.rs:297:60 ... #37 0x557c9d9c8346 in main /home/azureuser/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.5/libfuzzer/FuzzerMain.cpp:20:30 #38 0x7ff19e82e082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #39 0x557c9d9c837d in _start (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x1bde37d) (BuildId: 6a95a932984a405ebab8171dddc9f812fdf16846) ``` To continue, our program has a bug that should be easy to fix by inspecting the error and making a slight modification to the source. Once the bug is fixed, the fuzzer will run continuously, producing output that looks like ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} $ cargo +nightly fuzz run fuzz_target_1 Compiling soroban-fuzzing-contract v0.0.0 (/home/azureuser/data/stellar/soroban-examples/fuzzing) Compiling soroban-fuzzing-contract-fuzzer v0.0.0 (/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz) Finished release [optimized + debuginfo] target(s) in 24.91s Finished release [optimized + debuginfo] target(s) in 0.08s Running `fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1 -artifact_prefix=/home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/artifacts/fuzz_target_1/ /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/corpus/fuzz_target_1` INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 1619748028 INFO: Loaded 1 modules (1122061 inline 8-bit counters): 1122061 [0x5647a55b9080, 0x5647a56caf8d), INFO: Loaded 1 PC tables (1122061 PCs): 1122061 [0x5647a56caf90,0x5647a67ea060), INFO: 173 files found in /home/azureuser/data/stellar/soroban-examples/fuzzing/fuzz/corpus/fuzz_target_1 INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: seed corpus: files: 173 min: 32b max: 61b total: 6039b rss: 85Mb #2 pulse ft: 8067 exec/s: 1 rss: 312Mb #4 pulse cov: 8068 ft: 10709 corp: 1/32b exec/s: 2 rss: 315Mb #8 pulse cov: 8476 ft: 11498 corp: 5/160b exec/s: 4 rss: 317Mb #16 pulse cov: 8512 ft: 12362 corp: 9/288b exec/s: 8 rss: 320Mb #32 pulse cov: 8516 ft: 13290 corp: 19/608b exec/s: 10 rss: 326Mb #64 pulse cov: 8516 ft: 13311 corp: 27/864b exec/s: 21 rss: 340Mb #128 pulse cov: 8540 ft: 13536 corp: 37/1196b exec/s: 25 rss: 365Mb #175 INITED cov: 8540 ft: 13580 corp: 42/1387b exec/s: 29 rss: 382Mb #177 NEW cov: 8545 ft: 13821 corp: 43/1419b lim: 48 exec/s: 29 rss: 384Mb L: 32/48 MS: 1 ChangeASCIIInt- #178 NEW cov: 8545 ft: 13824 corp: 44/1451b lim: 48 exec/s: 29 rss: 384Mb L: 32/48 MS: 1 ChangeBinInt- #229 NEW cov: 8545 ft: 13826 corp: 45/1483b lim: 48 exec/s: 38 rss: 401Mb L: 32/48 MS: 1 ChangeByte- #256 pulse cov: 8545 ft: 13826 corp: 45/1483b lim: 48 exec/s: 36 rss: 410Mb #361 NEW cov: 8545 ft: 13830 corp: 46/1521b lim: 48 exec/s: 40 rss: 451Mb L: 38/48 MS: 5 ShuffleBytes-CMP-EraseBytes-CopyPart-ChangeBinInt- DE: "\005\000\000\000"- NEW_FUNC[1/1]: 0x5647a2964640 in rand::rngs::adapter::reseeding::ReseedingCore$LT$R$C$Rsdr$GT$::reseed_and_generate::ha760ded93293681c /home/azureuser/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rand-0.7.3/src/rngs/adapter/reseeding.rs:235 #368 NEW cov: 8557 ft: 13842 corp: 47/1566b lim: 48 exec/s: 40 rss: 454Mb L: 45/48 MS: 2 CrossOver-InsertRepeatedBytes- #512 pulse cov: 8557 ft: 13842 corp: 47/1566b lim: 48 exec/s: 46 rss: 502Mb #850 NEW cov: 8557 ft: 13843 corp: 48/1610b lim: 48 exec/s: 53 rss: 591Mb L: 44/48 MS: 2 CopyPart-ChangeBit- #1024 pulse cov: 8557 ft: 13843 corp: 48/1610b lim: 48 exec/s: 56 rss: 645Mb #1796 NEW cov: 8557 ft: 13863 corp: 49/1642b lim: 53 exec/s: 71 rss: 669Mb L: 32/48 MS: 1 ChangeBinInt- #1913 NEW cov: 8557 ft: 13864 corp: 50/1675b lim: 53 exec/s: 73 rss: 669Mb L: 33/48 MS: 2 ShuffleBytes-InsertByte- #3749 REDUCE cov: 8557 ft: 13864 corp: 50/1670b lim: 68 exec/s: 98 rss: 669Mb L: 39/48 MS: 1 EraseBytes- ... ``` And this output will continue until the fuzzer is killed with `Ctrl-C`. Next, let's look at a single line of fuzzer output: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} #177 NEW cov: 8545 ft: 13821 corp: 43/1419b lim: 48 exec/s: 29 rss: 384Mb L: 32/48 MS: 1 ChangeASCIIInt- ``` The most important column here is `cov`. This is a cumulative measure of branches covered by the fuzzer. When this number stops increasing the fuzzer has probably explored as much of the program as it can. The other columns are described in the [`libfuzzer` documentation][lfout]. [lfout]: https://llvm.org/docs/LibFuzzer.html#output Finally, lets look at this warning: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes. ``` By default, `libfuzzer` only generates input up to 4096 bytes. In a lot of cases, this is probably reasonable, but `cargo-fuzz` can increase the `max_len` by appending the argument after `--`: ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} cargo +nightly fuzz run fuzz_target_1 -- -max_len=20000 ``` All the options to libfuzzer can be listed with ```sh theme={"theme":{"light":"github-light","dark":"github-dark"}} cargo +nightly fuzz run fuzz_target_1 -- -help=1 ``` See the [`libfuzzer` documentation] for more. [`libfuzzer` documentation]: https://llvm.org/docs/LibFuzzer.html#output ## Accepting Soroban Types as Input with the `SorobanArbitrary` Trait Inputs to the `fuzz_target!` macro must implement the [`Arbitrary`] trait, which accepts bytes from the fuzzer driver and converts them to Rust values. Soroban types though are managed by the host environment, and so must be created from an [`Env`] value, which is not available to the fuzzer driver. The [`SorobanArbitrary`] trait, implemented for all Soroban contract types, exists to bridge this gap: it defines a *prototype* pattern whereby the `fuzz_target` macro creates prototype values that the fuzz program can convert to contract values with the standard soroban conversion traits, [`FromVal`] or [`IntoVal`]. [`sorobanarbitrary`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/testutils/arbitrary/trait.SorobanArbitrary.html [`fromval`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/trait.FromVal.html [`intoval`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/trait.IntoVal.html The types of prototypes are identified by the associated type, `SorobanArbitrary::Prototype`: ```rust theme={"theme":{"light":"github-light","dark":"github-dark"}} pub trait SorobanArbitrary: TryFromVal + IntoVal + TryFromVal { type Prototype: for<'a> Arbitrary<'a>; } ``` Types that implement `SorobanArbitrary` include: * `i32`, `u32`, `i64`, `u64`, `i128`, `u128`, [`I256`], [`U256`], `()`, and `bool`, * [`Error`], * [`Bytes`], [`BytesN`], [`Vec`], [`Map`], * [`Address`], [`Symbol`], * [`Val`], [`i256`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.I256.html [`u256`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.U256.html [`error`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/xdr/enum.Error.html [`bytes`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Bytes.html [`bytesn`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.BytesN.html [`vec`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Vec.html [`map`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Map.html [`address`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Address.html [`symbol`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Symbol.html [`val`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Val.html All user-defined contract types, those with the [`contracttype`] attribute, automatically derive `SorobanArbitrary`. Note that `SorobanArbitrary` is only derived when the "testutils" Cargo feature is active. This implies that, in general, to make a Soroban contract fuzzable, the contract crate must define a "testutils" Cargo feature, that feature should turn on the "soroban-sdk/testutils" feature, and the fuzz test, which is its own crate, must turn that feature on. ## A More Complex Fuzz Test The [`fuzz_target_2.rs`] example, demonstrates the use of `SorobanArbitrary`, the advancement of time, and more advanced fuzzing techniques. [`fuzz_target_2.rs`]: https://github.com/stellar/soroban-examples/tree/v23.0.0/fuzzing/fuzz/fuzz_targets/fuzz_target_2.rs This fuzz test takes a much more complex input, where some of the values are user-defined types exported from the contract under test. This test is structured as a simple interpreter, where the fuzzing harness provides arbitrarily-generated "steps", where each step is either a `deposit` command or a `claim` command. The test then treats each of these steps as a separate transaction: it maintains a snapshot of the blockchain state, and for each step creates a fresh environment in which to execute the contract call, simulating the advancement of time between each step. As in the previous example, assertions are made after each step. The input to the fuzzer looks, in part, like: ```rust title="fuzzing/fuzz/fuzz_targets/fuzz_target_2.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} #[derive(Arbitrary, Debug)] struct Input { addresses: [
::Prototype; NUM_ADDRESSES], #[arbitrary(with = |u: &mut Unstructured| u.int_in_range(0..=i128::MAX))] token_mint: i128, steps: RustVec, } #[derive(Arbitrary, Debug)] struct Step { #[arbitrary(with = |u: &mut Unstructured| u.int_in_range(1..=u64::MAX))] advance_time: u64, command: Command, // `Command` not shown here - see the full source. } ``` This shows how to use the `SorobanArbitrary::Prototype` associated type to define inputs to the fuzzer. A Soroban [`Address`] can only be created with an [`Env`], so cannot be generated directly by the `Arbitrary` trait. Instead we use the fully-qualified name of the `Address` prototype, `
::Prototype`, to ask for `Address`'s prototype instead. Then when our fuzzer needs the `Address` we instantiate it with the [`FromVal`] trait: ```rust title="fuzzing/fuzz/fuzz_targets/fuzz_target_2.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} let depositor_address = Address::from_val(&env, &input.addresses[cmd.depositor_index]); ``` *** The contract we are fuzzing is a *timelock* contract, where calculation of time is crucial for correctness. So our testing must account for the advancement of time. The contract defines a `TimeBound` type and accepts it in the `deposit` method: ```rust title="fuzzing/src/lib.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} #[derive(Clone, Debug)] #[contracttype] pub struct TimeBound { pub kind: TimeBoundKind, pub timestamp: u64, } #[contractimpl] impl ClaimableBalanceContract { pub fn deposit( env: Env, from: Address, token: Address, amount: i128, claimants: Vec
, time_bound: TimeBound, ) { ... } } ``` In our fuzzer, one of the possible commands issued each step is a `DepositCommand`: ```rust title="fuzzing/fuzz/fuzz_targets/fuzz_target_2.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} #[derive(Arbitrary, Debug)] struct DepositCommand { #[arbitrary(with = |u: &mut Unstructured| u.int_in_range(0..=NUM_ADDRESSES - 1))] depositor_index: usize, amount: i128, // This is an ugly way to get a vector of integers in range #[arbitrary(with = |u: &mut Unstructured| { u.arbitrary_len::().map(|len| { (0..len).map(|_| { u.int_in_range(0..=NUM_ADDRESSES - 1) }).collect::, _>>() }).and_then(|inner_result| inner_result) })] claimant_indexes: RustVec, time_bound: ::Prototype, } ``` Notice that this command again uses the `SorobanArbitrary::Prototype` associated type to accept a `TimeBound` as input. To advance time we maintain a [`LedgerSnapshot`], defined in the [`soroban-ledger-snapshot`] crate. For each step we call [`Env::from_snapshot`] to create a fresh environment to execute the step, then [`Env::to_snapshot`] to create a new snapshot to use in the following step. [`ledgersnapshot`]: https://docs.rs/soroban-ledger-snapshot/latest/soroban_ledger_snapshot/struct.LedgerSnapshot.html [`soroban-ledger-snapshot`]: https://docs.rs/soroban-ledger-snapshot [`env::from_snapshot`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Env.html#method.from_snapshot [`env::to_snapshot`]: https://docs.rs/soroban-sdk/latest/soroban_sdk/struct.Env.html#method.to_snapshot Here is a simplified outline of how this works. See the full source code for details. ```rust title="fuzzing/fuzz/fuzz_targets/fuzz_target_2.rs" theme={"theme":{"light":"github-light","dark":"github-dark"}} let snapshot = { let init_ledger = LedgerInfo { timestamp: 12345, protocol_version: 1, sequence_number: 10, network_id: Default::default(), base_reserve: 10, min_temp_entry_ttl: u32::MAX, min_persistent_entry_ttl: u32::MAX, max_entry_ttl: u32::MAX, }; LedgerSnapshot::from(init_ledger, None) }; let mut prev_env = Env::from_snapshot(init_snapshot); for step in &config.input.steps { // Advance time and create a new env from snapshot. let curr_env = { let mut snapshot = prev_env.to_snapshot(); snapshot.ledger.sequence_number += 1; snapshot.ledger.timestamp = snapshot.ledger.timestamp.saturating_add(step.advance_time); let env = Env::from_snapshot(snapshot); env.cost_estimate().budget().reset_unlimited(); env }; step.command.exec(&config, &curr_env); assert_invariants(&config, &prev_env, &curr_env); prev_env = curr_env; } ``` ## Converting a Fuzz Test to a Property Test In addition to fuzz testing, Soroban supports property testing in the style of quickcheck, by using the [`proptest`] and [`proptest-arbitrary-interop`] crates in conjunction with the `SorobanArbitrary` trait. Property tests are similar to fuzz tests in that they generate randomized input. Property tests though do not instrument their test cases or mutate their input based on feedback from previous tests. Thus they are a weaker form of test. The great benefit of property tests though is that they can be included in standard Rust test suites and require no extra tooling to execute. One might take advantage of this by interactively fuzzing to discover deep bugs, then convert fuzz tests to property tests to help prevent regressions. The [`proptest.rs`] file is a translation of `fuzz_target_1.rs` to a property test. [`proptest.rs`]: https://github.com/stellar/soroban-examples/tree/v23.0.0/fuzzing/src/proptest.rs