# Ingest with HTTP API

The HTTP API provides a lightweight interface for ingesting data into Altertable catalogs.

## Goal

Choose the right ingest endpoint, load records or files into a lakehouse table, and query the table to confirm that the data arrived.

## Prerequisites

- Your lakehouse username and password from [Credentials](/docs/lakehouse/authentication.md).
- A target catalog and schema.
- JSON records or a CSV, JSON, or Parquet file to ingest.

The API supports these ingest paths:

- Append event-style JSON records (up to 32 MB per request)
- Upload CSV, JSON, and Parquet files via `/upload` (up to 100 GB per request)
- Upsert CSV, JSON, and Parquet files by primary key

For terminal workflows, the [CLI](/docs/developer-tooling/cli.md) wraps these same ingest paths with `altertable append`, `altertable upload`, and `altertable upsert`.

## Steps

### 1. Choose an ingest mode

| Mode     | Use it for                                               | Maximum request size  |
| -------- | -------------------------------------------------------- | --------------------- |
| `append` | Streaming events and operational JSON records            | 32 MB                 |
| `upload` | Creating, appending, or overwriting a table from a file  | 100 GB                |
| `upsert` | Updating and inserting file rows by a stable primary key | See the API reference |

### 2. Append JSON records

Append is designed for streaming events and operational records without managing ingestion jobs. You can send individual records, but microbatching several records usually gives you much higher throughput. Append records with cURL or SDKs:

### cURL

```bash
curl -X POST "https://api.altertable.ai/append?catalog=my_catalog&schema=main&table=events" \
  -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \
  -d '{"timestamp":"2026-01-01","event":"click","value":42}'
```

### JavaScript

```typescript
import { AltertableLakehouseClient } from '@altertable/lakehouse';

const client = new AltertableLakehouseClient({
  username: '{{LAKEHOUSE_USERNAME}}',
  password: '{{LAKEHOUSE_PASSWORD}}',
});

await client.append({
  catalog: 'my_catalog',
  schema: 'main',
  table: 'events',
  body: {
    Single: { timestamp: '2026-01-01', event: 'click', value: 42 },
  },
});
```

### Python

```python
from altertable_lakehouse import Client

client = Client(
    username="{{LAKEHOUSE_USERNAME}}",
    password="{{LAKEHOUSE_PASSWORD}}",
)

client.append(
    catalog="my_catalog",
    schema="main",
    table="events",
    data={"timestamp": "2026-01-01", "event": "click", "value": 42},
)
```

### Ruby

```ruby
require 'altertable/lakehouse'

client = Altertable::Lakehouse::Client.new(
  username: '{{LAKEHOUSE_USERNAME}}',
  password: '{{LAKEHOUSE_PASSWORD}}'
)

client.append(
  catalog: 'my_catalog',
  schema: 'main',
  table: 'events',
  payload: { timestamp: '2026-01-01', event: 'click', value: 42 }
)
```

### Go

```go
import (
	"context"
	"log"

	altertable "github.com/altertable-ai/altertable-lakehouse-go"
)

client, err := altertable.NewClient(altertable.Config{
	Username: "{{LAKEHOUSE_USERNAME}}",
	Password: "{{LAKEHOUSE_PASSWORD}}",
})
if err != nil {
	log.Fatal(err)
}

_, err = client.Append(context.Background(), altertable.AppendParams{
	Catalog: "my_catalog",
	Schema:  "main",
	Table:   "events",
}, altertable.AppendRequest{
	Single: altertable.AppendPayload{
		"timestamp": "2026-01-01",
		"event":     "click",
		"value":     42,
	},
})
if err != nil {
	log.Fatal(err)
}
```

### PHP

```php
<?php

use Altertable\Lakehouse\Config\LakehouseConfig;
use Altertable\Lakehouse\LakehouseClient;
use Altertable\Lakehouse\Models\AppendPayload;
use Altertable\Lakehouse\Models\AppendRequest;

$config = LakehouseConfig::builder()
    ->withCredentials('{{LAKEHOUSE_USERNAME}}', '{{LAKEHOUSE_PASSWORD}}')
    ->build();

$client = new LakehouseClient($config);

$client->append(
    'my_catalog',
    'main',
    'events',
    AppendRequest::single(new AppendPayload(
        data: [[
            'timestamp' => '2026-01-01',
            'event' => 'click',
            'value' => 42,
        ]],
        columns: ['timestamp', 'event', 'value'],
    )),
);
```

### Rust

```rust
use altertable_lakehouse::{AltertableClient, AppendRequest};
use serde_json::json;
use std::collections::HashMap;

let client = AltertableClient::builder()
    .credentials("{{LAKEHOUSE_USERNAME}}", "{{LAKEHOUSE_PASSWORD}}")
    .build()?;

let payload = HashMap::from([
    ("timestamp".to_string(), json!("2026-01-01")),
    ("event".to_string(), json!("click")),
    ("value".to_string(), json!(42)),
]);

client
    .append(
        "my_catalog",
        "main",
        "events",
        None,
        &AppendRequest::Single(payload),
    )
    .await?;
```

### CLI

```bash
# Install
curl -fsSL https://install.altertable.ai | sh

# Configure credentials
altertable profile --configure

# Upload a local file
altertable upload \
  --catalog my_catalog \
  --schema main \
  --table events \
  --mode overwrite \
  --format parquet \
  --file events.parquet
```

Append is eventually consistent. The endpoint returns as soon as the data is acknowledged, and the new rows are queryable in the catalog within a few seconds.

Each request body can be up to **32 MB**. For larger file uploads, use `/upload` with `mode=append` instead.

The endpoint acknowledges valid JSON before insertion runs. If a row still cannot be inserted after retries—for example, because a value is incompatible with the current table schema—Altertable writes it to `altertable.main.quarantined_appends` instead of dropping it silently.

Append also supports smart migrations:

| Input                             | Behavior                                                                  |
| --------------------------------- | ------------------------------------------------------------------------- |
| The table does not exist          | Altertable infers the schema from the JSON payload and creates the table. |
| The payload includes a new column | Altertable automatically adds the missing column to the table.            |

### 3. Upload files

`/upload` uploads CSV, JSON, or Parquet files and supports several ingestion modes via the `mode` query parameter:

- **`create`**: Create a new table from the file
- **`append`**: Append rows to an existing table (no `primary_key` required)
- **`overwrite`**: Replace the table with the uploaded data

Use `/upload` when you want to load a file without matching rows by primary key. The `mode` parameter is required.

Request bodies can be up to **100 GB**, so `/upload` is the right choice for large file uploads. Use `/append` for small, event-style JSON payloads.

From the [CLI](/docs/developer-tooling/cli.md), upload a CSV file with:

```bash
altertable upload --catalog my_catalog --schema main --table users --mode create --format csv --file users.csv
```

### 4. Upsert by primary key

`/upsert` updates existing rows and inserts new rows in the same request. Use it when the file has a stable primary key column:

```json
[
  {
    "id": "user_123",
    "email": "alice@example.com",
    "plan": "pro"
  },
  {
    "id": "user_456",
    "email": "bob@example.com",
    "plan": "team"
  }
]
```

Upload `users.json` with the `primary_key` parameter:

```bash
curl "https://api.altertable.ai/upsert?catalog=my_catalog&schema=main&table=users&primary_key=id" \
  -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \
  --data-binary @users.json
```

The same endpoint also accepts Parquet and CSV files. Parquet is usually the best choice for larger batches.

From the [CLI](/docs/developer-tooling/cli.md), upsert a CSV file by primary key with:

```bash
altertable upsert --catalog my_catalog --schema main --table users --primary-key id --format csv --file users.csv
```

## Verification

Query the target table after the ingest request finishes:

```bash
altertable query "SELECT * FROM my_catalog.main.users LIMIT 10"
```

For append requests, allow a few seconds for eventual consistency. Confirm that the returned rows match the records or file you sent.

## Troubleshooting

- **The request is unauthorized:** refresh the lakehouse credentials used to build `ALTERTABLE_BASIC_AUTH_TOKEN`.
- **A new append row is missing:** wait a few seconds, then query the table again because append is eventually consistent.
- **An accepted row is missing:** query `altertable.main.quarantined_appends` for the target catalog, schema, and table. Inspect `error` and `payload`, then correct and resend the row.
- **The file is too large:** use `/upload` for files up to 100 GB instead of sending them to `/append`.
- **Upsert updates the wrong rows:** confirm that `primary_key` names a stable, populated column in the uploaded file.

## Next steps

Use the reference page for endpoint details, payload shapes, and full examples:

- [POST /append](/docs/api-references/lakehouse.md#post-append)
- [POST /upload](/docs/api-references/lakehouse.md#post-upload)
- [POST /upsert](/docs/api-references/lakehouse.md#post-upsert)

The same reference also covers related query, validation, and task endpoints:

- [POST /query](/docs/api-references/lakehouse.md#post-query)
- [`GET /query/{query_id}`](/docs/api-references/lakehouse.md#get-queryquery-uuid)
- [`DELETE /query/{query_id}`](/docs/api-references/lakehouse.md#delete-queryquery-uuid)
- [`GET /tasks/{task_id}`](/docs/api-references/lakehouse.md#get-taskstask-uuid)
- [POST /validate](/docs/api-references/lakehouse.md#post-validate)