> ## Documentation Index
> Fetch the complete documentation index at: https://developers.stellar.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Import

> This document outlines methods to perform inital backfill when setting up hubble.

This document outlines methods to perform inital backfill when setting up hubble.

## Use SDF ETL Data as a Source

## 1. Export Data to Cloud Storage and Load Data into BigQuery

* Use the [EXPORT](https://cloud.google.com/bigquery/docs/reference/standard-sql/export-statements) command provided by Google Cloud Platform (GCP) to export your dataset in the required format (e.g., Avro, Parquet).
* Use the [LOAD](https://cloud.google.com/bigquery/docs/reference/standard-sql/load-statements) command to load the exported files into your BigQuery dataset.

#### Example:

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
EXPORT DATA OPTIONS(
  uri='gs://my-bucket/history-transactions/*',
  format='PARQUET',
  overwrite=true) AS
SELECT * FROM crypto-stellar.crypto_stellar.history_transactions;

LOAD DATA INTO mydataset.transactions
  FROM FILES(
    format='PARQUET',
    uris = ['gs://my-bucket/history-transactions/*']
  )
  WITH PARTITION COLUMNS;
```

## Use BigQuery API / Console to Mirror SDF's Dataset

The Stellar Development Foundation provides public access to fully transformed Stellar network data through the public datasets and tables in GCP BigQuery. Instructions on how to access this data can be found in the [Connecting](https://developers.stellar.org/docs/data/analytics/hubble/developer-guide/connecting-to-bigquery.md) section.

Use the [Create Table Copy statement](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_table_copy) to copy data across datasets.

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
CREATE [ OR REPLACE ] TABLE [ IF NOT EXISTS ] table_name
COPY source_table_name
...
[OPTIONS(table_option_list)]
```

## Use Galexie as a Source

[Galexie](https://developers.stellar.org/docs/data/indexers/build-your-own/galexie) is a tool for extracting, processing, and exporting Stellar ledger metadata to external storage, creating a data lake of pre-processed ledger metadata. This is an upstream data source to Hubble, useful when you have a custom Stellar-ETL pipeline.

**Steps**

1. Ensure you provide the correct value for datastore-path in Stellar-ETL [command flags](https://github.com/stellar/stellar-etl/tree/master?tab=readme-ov-file#common-flags). This represents the name of the bucket where your Galexie instance outputs ledger close metadata.
2. Set up an [orchestration system like Airflow](https://developers.stellar.org/docs/data/analytics/hubble/developer-guide/scheduling-and-orchestration/getting-started.md).
3. In your Airflow instance, trigger the history\_table\_export DAG for older dates.