Publishing History Archives - Stellar Developer Docs

If you want to run a Full Validator, you need to set up your node to publish a history archive covering the network’s complete history. You can host an archive using a blob store such as Amazon’s S3 or Digital Ocean’s spaces, or you can simply serve a local archive directly via an HTTP server such as Nginx or Apache. If you’re setting up a Basic Validator, you can skip this section. No matter what kind of node you’re planning to run, make sure to set it up to get history, which is covered in Environment Preparation. Getting to a complete archive takes two stages:

Configuring your node to publish, described in the sections below. This fills the archive forward from the moment your node starts.
Completing the archive, which backfills everything from before that point.

One archive per nodeEach node must publish to its own dedicated archive. Writing to the same archive from multiple nodes is not supported and will result in undefined behavior, potentially including data loss. If you’re running multiple Full Validators (as Tier 1 organizations do), configure a separate archive for each — for example, history-a.example.com, history-b.example.com, and history-c.example.com.

Caching and History Archives

The primary cost of running a validator will very likely be egress bandwidth. A crucial part of your strategy to manage those costs should be caching. You can significantly reduce these data transfer costs by using common caching techniques or a CDN. Three simple rules apply to caching the History archives:

Do not cache the archive state file .well-known/stellar-history.json (Cache-Control: no-cache)
Do not cache HTTP 4xx responses (Cache-Control: no-cache)
Cache everything else for as long as possible (> 1 day)

Local History Archive Using nginx

Here, we’ll demonstrate how you can configure your node to store its history files in the local filesystem, and then publish that history using nginx for our webserver software. First, you must add a history configuration stanza to your /etc/stellar/stellar-core.cfg configuration file.

[HISTORY.local]
get="cp /mnt/xvdf/stellar-core-archive/node_001/{0} {1}"
put="cp {0} /mnt/xvdf/stellar-core-archive/node_001/{1}"
mkdir="mkdir -p /mnt/xvdf/stellar-core-archive/node_001/{0}"

Then, you must run Stellar Core’s new-hist command to create the local history archive.

sudo -u stellar stellar-core --conf /etc/stellar/stellar-core.cfg new-hist local

This command creates the history archive structure:

$ tree -a /mnt/xvdf/stellar-core-archive/
/mnt/xvdf/stellar-core-archive
└── node_001
    ├── history
    │   └── 00
    │       └── 00
    │           └── 00
    │               └── history-00000000.json
    └── .well-known
        └── stellar-history.json

6 directories, 2 files

Now that the history archive’s file structure is ready, you can configure a virtual host in nginx to serve the local archive.

server {
  listen 80;
  root /mnt/xvdf/stellar-core-archive/node_001/;

  server_name history.example.com;

  # do not cache 404 errors
  error_page 404 /404.html;
  location = /404.html {
    add_header Cache-Control "no-cache" always;
  }

  # do not cache history state file
  location ~ ^/.well-known/stellar-history.json$ {
    add_header Cache-Control "no-cache" always;
    try_files $uri =404;
  }

  # cache entire history archive for 1 day
  location / {
    add_header Cache-Control "max-age=86400";
    try_files $uri =404;
  }
}

Amazon S3 History Archive

Now, let’s demonstrate a configuration where your node stores its history files using Amazon’s S3 service. You can then publish that history using an Amazon S3 static site, or again use nginx for your webserver software. This time, using nginx, we’ll include some proxy and CDN configuration, as well. Start by adding a history configuration stanza to your /etc/stellar/stellar-core.cfg configuration file.

[HISTORY.s3]
get='curl -sf http://history.example.com/{0} -o {1}' # Cached HTTP endpoint
put='aws s3 cp --region us-east-1 {0} s3://bucket.name/{1}' # Direct S3 access

Then, you must run Stellar Core’s new-hist command to create and initialize the S3 archive.

sudo -u stellar stellar-core --conf /etc/stellar/stellar-core.cfg new-hist s3

These S3 history files can be served with something as simple as an Amazon S3 static site. Optionally, you may want to place a reverse proxy and CDN in front of the S3 static site (we’ll use nginx for this example).

server {
  listen 80;
  root /srv/nginx/history.example.com;
  index index.html index.htm;

  server_name history.example.com;

  # use google nameservers for lookups
  resolver 8.8.8.8 8.8.4.4;

  # bucket.name s3 static site endpoint
  set $s3_bucket "bucket.name.s3-website-us-east-1.amazonaws.com";

  # do not cache 404 errors
  error_page 404 /404.html;
  location = /404.html {
    add_header Cache-Control "no-cache" always;
  }

  # do not cache history state file
  location ~ ^/.well-known/stellar-history.json$ {
    add_header Cache-Control "no-cache" always;
    proxy_intercept_errors on;
    proxy_pass  http://$s3_bucket;
    proxy_read_timeout 120s;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host            $s3_bucket;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
  }

  # cache history archive for 1 day
  location / {
    add_header Cache-Control "max-age=86400";
    proxy_intercept_errors on;
    proxy_pass  http://$s3_bucket;
    proxy_read_timeout 120s;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host            $s3_bucket;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
  }
}

Backfilling a History Archive

Given the choice, it’s best to configure your history archive prior to your node’s initial sync with an existing network. That way your validator’s history publishes as you join, and subsequently sync with, the network. However, if you have not published an archive during the node’s initial sync, the steps required to start publishing from an existing validator are quite straightforward. This is the first of the two stages on the way to a Full Validator; completing the archive follows. First, you’ll need to stop your stellar-core instance:

systemctl stop stellar-core # modify this if not using systemctl

Then, add the history archive configuration to your node’s /etc/stellar/stellar-core.cfg configuration file.

[HISTORY.local]
get="cp /mnt/xvdf/stellar-core-archive/node_001/{0} {1}"
put="cp {0} /mnt/xvdf/stellar-core-archive/node_001/{1}"
mkdir="mkdir -p /mnt/xvdf/stellar-core-archive/node_001/{0}"

Next, you must run Stellar Core’s new-hist command to create and initialize the local archive. (This is done as the stellar user.)

sudo -u stellar stellar-core --conf /etc/stellar/stellar-core.cfg new-hist local

Now you can start your Stellar Core instance again:

systemctl start stellar-core # modify this if not using systemctl

As you allow your node to join the network again, you can watch it start publishing a few checkpoints to the newly created archive.

2019-04-25T12:30:43.275 GDUQJ [History INFO] Publishing 1 queued checkpoints [16895-16895]: Awaiting 0/0 prerequisites of: publish-000041ff

At this stage your validator is publishing each new checkpoint as the network advances, which lets other nodes catch up from your archive as far back as the point where you started publishing. Everything before that point is still missing, and filling it in is the second stage.

Complete History Archive

This is the second stage. A Full Validator publishes an archive reaching back to the network’s first ledger. The stellar-archivist command line tool gives you what you need to close the gap.

Installing stellar-archivist

Using the SDF package repositories, you can install it by running:

apt-get install stellar-archivist-rs

The tool is also published on crates.io. For detailed usage, please run stellar-archivist-rs --help (or stellar-archivist --help if installed from crates.io).

Where to run stellar-archivist

stellar-archivist is a standalone tool. It does not link against or communicate with stellar-core; all it needs is read and write access to the archive itself. That access can be a local filesystem path (file://), an HTTP(S) source, or a cloud object store (s3://, gcs://, azblob://, b2://), so you can run it from any machine that can reach your archive.

Run stellar-archivist on a separate host from your validator to avoid resource contention.Do not stop your validator to work on its archive. stellar-archivist operates on the archive files directly and does not interfere with live publishing. Your node should stay online and keep publishing checkpoints throughout.

Choosing a command

Your goal	Command
Find out whether an archive is complete, and where the gaps are	`scan` — read-only
Fix missing or corrupt files in an archive you already have	`repair`
Build an archive from scratch, or backfill a range of history	`mirror`

All three accept --low and --high to restrict the work to a ledger range; both bounds are rounded outward to the nearest checkpoint.

Checking file contents with `--verify`

By default, every command checks only that files exist. Adding --verify also checks that their contents are sound, which requires significantly more CPU and memory. Leave it off for routine completeness checks and for fixing missing files. Turn it on to find and fix corruption, where a file exists but its contents are wrong.

Scanning an archive

scan is read-only. It walks the archive and reports the files that are missing, which is how you find out whether your archive has a gap and exactly where it is.¹

stellar-archivist-rs scan file:///mnt/xvdf/stellar-core-archive/node_001 --report report.json

2026-08-21T22:05:55.175197Z  INFO Starting scan of file:///mnt/xvdf/stellar-core-archive/node_001
2026-08-21T22:05:55.177412Z  INFO Processing 32 checkpoints from 63 (0x0000003f) to 2047 (0x000007ff)
2026-08-21T22:05:55.179433Z ERROR Missing results/00/00/00/results-000000bf.xdr.gz
2026-08-21T22:05:55.190684Z ERROR Failed to process bucket/8c/e5/7e/bucket-8ce57e5f8c2ee574af879dde28cf87a0d936a5a1ea905b377409f275a4e73a1c.xdr.gz: Hash mismatch: got 49ec4c32c208f6eb53d0e847bc8a4819544a71ca04c20f47737ba0842f133370
2026-08-21T22:05:55.191706Z ERROR Missing transactions/00/00/05/transactions-000005bf.xdr.gz
2026-08-21T22:05:55.195509Z ERROR Missing scp/00/00/03/scp-0000037f.xdr.gz
2026-08-21T22:05:55.206930Z ERROR Missing scp/00/00/06/scp-0000067f.xdr.gz
2026-08-21T22:05:55.209163Z ERROR Missing history/00/00/06/history-000006ff.json
2026-08-21T22:05:55.221592Z ERROR Failed to process bucket/d6/2e/37/bucket-d62e37771e7ae305d981317d710cd9f6bdf254d16caa315955217deddaaaab05.xdr.gz: Hash mismatch: got 49ec4c32c208f6eb53d0e847bc8a4819544a71ca04c20f47737ba0842f133370
...
2026-08-21T22:05:55.237827Z  INFO Scan complete: 378 files found, 11 missing or corrupt
2026-08-21T22:05:55.237835Z ERROR 1 history file(s) missing or corrupt
2026-08-21T22:05:55.237840Z ERROR 1 transactions file(s) missing or corrupt
2026-08-21T22:05:55.237845Z ERROR 1 results file(s) missing or corrupt
2026-08-21T22:05:55.237849Z ERROR 6 bucket file(s) missing or corrupt
2026-08-21T22:05:55.237854Z  WARN 2 optional scp file(s) missing or corrupt
2026-08-21T22:05:55.238022Z  WARN Archive is incomplete: 11 issue(s) found
error: Archive issues found

--report writes the same findings as JSON:

{
  "version": 1,
  "well_known": null,
  "files": {
    "1471": ["transactions"],
    "1663": ["scp"],
    "1791": ["history"],
    "191": ["results"],
    "895": ["scp"]
  },
  "buckets": [
    "0e3f43118dc8761a94102e2f043f7994a0a12078220b2873dce162fd1d195a50",
    "344dd4abedf73ee3d65d07702e0a1a065b710fcfabc9202821ac50c16e5db898",
    "8ce57e5f8c2ee574af879dde28cf87a0d936a5a1ea905b377409f275a4e73a1c",
    "8e0eb1c3e797336973e5e7d3591b5bf6385de2d6d990d81a9732d76d61d62cf9",
    "d62e37771e7ae305d981317d710cd9f6bdf254d16caa315955217deddaaaab05",
    "dc1031942e58d472b8f7f9630250a296426c767afdca92fb687450adb87e1700"
  ],
  "checkpoints": [],
  "summary": {
    "succeeded": 378,
    "skipped": 0,
    "failed": 11,
    "retries": 0
  }
}

files maps a checkpoint ledger to the file types broken at that checkpoint.
buckets lists bucket hashes that are missing or failed their SHA-256 check.
checkpoints lists cross-file and hash chain failures, which are re-fetched whole.
well_known is the checkpoint to restore .well-known/stellar-history.json from, or null when it is healthy.

Repairing an archive

Once a scan has told you what is broken, repair fixes it in place: it identifies missing or broken files at the destination and re-fetches them from a known-good source — such as the SDF public history archive.

stellar-archivist-rs repair \
  https://history.stellar.org/prd/core-testnet/core_testnet_001 \
  file:///mnt/xvdf/stellar-core-archive/node_001

2026-08-21T22:05:55.831816Z  INFO Starting repair from https://history.stellar.org/prd/core-testnet/core_testnet_001 to file:///mnt/xvdf/stellar-core-archive/node_001 with 32 workers
2026-08-21T22:05:56.036395Z  INFO Processing 32 checkpoints from 63 (0x0000003f) to 2047 (0x000007ff)
2026-08-21T22:05:56.103469Z  INFO Progress: 32/32 checkpoints processed
2026-08-21T22:05:56.103510Z  INFO Repair main pass completed: 395 files processed, 0 failed
2026-08-21T22:05:56.103520Z  INFO Repair file retry completed: 0 files processed, 0 failed
2026-08-21T22:05:56.103526Z  INFO Repair checkpoint retry completed: 0 files processed, 0 failed

Repair runs in stages and reports each one separately: a main pass that audits the destination, a file retry that re-fetches everything recorded as broken, and a checkpoint retry for cross-file and chain failures.

The JSON report written by scan can be handed straight back to repair as a plan, using --plan. Repair then re-fetches exactly those files.

stellar-archivist-rs repair \
  https://history.stellar.org/prd/core-testnet/core_testnet_001 \
  file:///mnt/xvdf/stellar-core-archive/node_001 \
  --plan report.json

A final scan confirms the archive has been repaired:

stellar-archivist-rs scan file:///mnt/xvdf/stellar-core-archive/node_001

2026-08-21T22:05:57.627359Z  INFO Starting scan of file:///mnt/xvdf/stellar-core-archive/node_001
2026-08-21T22:05:57.629405Z  INFO Processing 32 checkpoints from 63 (0x0000003f) to 2047 (0x000007ff)
2026-08-21T22:05:57.686713Z  INFO Progress: 32/32 checkpoints processed
2026-08-21T22:05:57.686747Z  INFO Scan complete: 395 files found, 0 missing or corrupt

Add --dry-run to report what would be repaired without writing anything. Combined with --report, this writes a plan you can review and apply later.

Mirroring an archive

mirror copies a source archive to a destination. It is the command to reach for when backfilling: run it once to build a complete archive from scratch, or scope it with --low and --high to fill in just the range of history your validator did not publish itself. For example, run the following command against an empty destination:

stellar-archivist-rs mirror \
  https://history.stellar.org/prd/core-testnet/core_testnet_001 \
  file:///mnt/xvdf/stellar-core-archive/node_001 \
  --high 1023

2026-08-20T17:09:23.266746Z  INFO Starting mirror from https://history.stellar.org/prd/core-testnet/core_testnet_001 to file:///mnt/xvdf/stellar-core-archive/node_001 with 32 workers
2026-08-20T17:09:23.664272Z  INFO No existing archive at destination; starting a fresh mirror
2026-08-20T17:09:23.664300Z  INFO Processing 16 checkpoints from 63 (0x0000003f) to 1023 (0x000003ff)
2026-08-20T17:09:23.850821Z  INFO Progress: 16/16 checkpoints processed
2026-08-20T17:09:23.850848Z  INFO Mirror completed: 196 files copied, 0 failed, 0 skipped
2026-08-20T17:09:23.850855Z  INFO No existing .well-known, creating new one at checkpoint 1023 (0x000003ff)
2026-08-20T17:09:23.851784Z  INFO Updated destination .well-known to checkpoint 1023 (0x000003ff)

Afterwards, the destination contains a new archive covering history up to checkpoint 1023.

Mirroring is resumable. Running it again against the same destination with a higher --high picks up from the destination’s current checkpoint and copies only what is missing.Files already present at the destination are skipped rather than re-fetched. Use --overwrite to re-fetch and replace them instead.Filling in a range older than your archive’s latest checkpoint (recorded in .well-known/stellar-history.json) requires --overwrite:

stellar-archivist-rs mirror \
  https://history.stellar.org/prd/core-testnet/core_testnet_001 \
  file:///mnt/xvdf/stellar-core-archive/node_001 \
  --low 63 --high 895 --overwrite

You should now have a complete history archive being written by your full validator. Congratulations!

The sample output on this page was captured with --verify, which is why it reports hash mismatches alongside the missing files. ↩

​Caching and History Archives

​Local History Archive Using nginx

​Amazon S3 History Archive

​Backfilling a History Archive

​Complete History Archive

​Installing stellar-archivist

​Where to run stellar-archivist

​Choosing a command

​Checking file contents with --verify

​Scanning an archive

​Repairing an archive

​Mirroring an archive

Footnotes

Caching and History Archives

Local History Archive Using nginx

Amazon S3 History Archive

Backfilling a History Archive

Complete History Archive

Installing stellar-archivist

Where to run stellar-archivist

Choosing a command

Checking file contents with `--verify`

Scanning an archive

Repairing an archive

Mirroring an archive