# Hugging Face

> **Beta:**
>
> The `Hugging Face` external catalog is available in beta. Its syntax and behavior may change as the implementation evolves.

Hugging Face external catalogs expose datasets hosted on the [Hugging Face Hub](https://huggingface.co/datasets) as tables. Files are read in place from the Hub: nothing is copied into Altertable.

## Requirements

- One or more dataset repositories on the Hugging Face Hub, stored as Parquet, CSV or JSON files
- For private or gated datasets, a Hugging Face [access token](https://huggingface.co/settings/tokens) with read access to those datasets

Hugging Face catalogs are read-only in Altertable.

## Connect a Hugging Face Catalog

1. Open **Catalogs** in the target environment.
2. Click **New catalog** and choose **Hugging Face**.
3. Paste an access token, or leave it empty for public datasets.
4. Select the file format of the datasets.
5. Map each table name to a dataset location, written as `owner/dataset[@revision][/path]`.
6. Test and create the catalog.

| Location                      | Reads                                             |
| ----------------------------- | ------------------------------------------------- |
| `acme/train-records`          | Every file of the default branch                  |
| `acme/train-records/data`     | Every file under `data/` on the default branch    |
| `acme/train-records@v3/data`  | Every file under `data/` at the `v3` tag          |
| `acme/train-records@~parquet` | The Parquet conversion maintained by Hugging Face |

The token is only sent to the dataset repositories mapped in the catalog.

## Query Example

```sql
SELECT language, stage, label, count(*) AS n
FROM hf.main.train_records
GROUP BY ALL
ORDER BY ALL;
```

## Query Other Revisions

Every mapped table also gets a `<table>_at(rev)` table function that reads the same dataset at any branch, tag or commit:

```sql
SELECT block, label, count(*) AS n
FROM hf.main.train_mix_at('v2')
GROUP BY ALL;
```

Pin to a commit hash for fully reproducible results, since branches and tags can move.

## Performance

Each query reads the dataset files from the Hub, so large scans are bound by download speed and by [Hugging Face rate limits](https://huggingface.co/docs/hub/rate-limits), and queries you run again download the same files again.

For a dataset you query repeatedly, copy it into an Altertable catalog once and keep the copy in sync instead:

1. Open the Altertable catalog that should hold the copy — not this Hugging Face catalog, which is read-only.
2. On its **Ingest** tab, click **New synchronizer** in **Synchronizers**.
3. Choose **External catalog** as the source type, select this Hugging Face catalog, and pick the schema and tables to copy.
4. Choose the destination schema and a schedule.

Each run replaces the copied tables with the latest data from the Hub, so queries read the local copy instead of downloading the dataset again.