# Altertable > Altertable combines lakehouse storage, DuckDB SQL, and access to external data. Use the same data from agents, applications, SQL clients, and data apps. --- url: https://altertable.ai/docs/administration/access-control.md --- # Access control Access control defines who can open an organization, use an environment, and read or write catalog data. Altertable applies the same role model to team members and service accounts, so humans and automation can be scoped with the same rules. Roles are assigned on three resource levels: | Level | What it controls | Common use | | ------------ | ------------------------------------------------------------------ | ----------------------------------------------------------- | | Organization | Organization-wide access, administration, and billing | Admins and users who need access across every environment | | Environment | Dashboard access and default catalog access inside one environment | Production, staging, or customer-specific access boundaries | | Catalog | Read or write access to one Altertable or external catalog | Narrow data access for teams, agents, and service accounts | Higher-level roles can cascade to lower-level resources. For example, an organization reader can read data across all environments, while a custom per-environment role lets you choose access separately for each environment and catalog. ## Roles Altertable exposes a small set of roles at each level: | Scope | Role | Behavior | | ------------ | ---------------------- | --------------------------------------------------------------- | | Organization | Admin | Manage members, access, billing, and organization settings | | Organization | Writer | Read and write data in every environment | | Organization | Reader | View data and dashboards in every environment | | Organization | Custom per environment | Set access separately for each environment and catalog | | Environment | Writer | Read and write data in that environment | | Environment | Reader | View data and dashboards in that environment | | Environment | Custom per catalog | Open the environment, then set read or write access per catalog | | Catalog | Writer | Read and write data in that catalog | | Catalog | Reader | Read data in that catalog | Use organization roles for broad access, environment roles when each environment represents a meaningful boundary, and catalog roles when a person or service needs only a specific data source. For [Terraform](/docs/administration/infrastructure-as-code.md), **Custom per environment** corresponds to `organization:member`, and **Custom per catalog** corresponds to `environment:member`. Combining `organization:member` with `environment:writer` grants write access to the specified environment without organization-wide write access. ## Team members Team members are managed from **Organization settings**. Users with organization management access can invite members, review pending invitations, and update roles from a member detail page. ## Service accounts Service accounts are organization-scoped principals for automation such as CI/CD, dbt jobs, scheduled imports, and machine-to-machine API access. After creating a service account, open its detail page and review its roles before using it. Narrow or expand access with the same organization, environment, and catalog roles used for team members. ## Recommended patterns - Give most people the narrowest organization role that matches their normal work. - Use custom per-environment access when production and staging should have different audiences. - Use catalog roles for sensitive datasets, shared customer environments, and automation that should not read every catalog. - Prefer service accounts over shared human credentials for scheduled jobs and deployment pipelines. ## Related guides - **[Workers](/docs/lakehouse/compute/workers.md)**: Where queries and source connectivity run - **[Authentication](/docs/administration/lakehouse-credentials.md)**: Credentials for lakehouse clients - **[MCP](/docs/agents/mcp.md)**: Governed agent access to interact with your lakehouse --- url: https://altertable.ai/docs/administration/encryption.md --- # Encryption Altertable encrypts customer data in transit and at rest. See [Security and compliance](/security-compliance) for the security overview, key-management information, and SOC 2 Type II attestation. ## Credentials Database and object-storage credentials are encrypted before storage and used when Altertable connects to the source. Use [lakehouse credentials](/docs/administration/lakehouse-credentials.md) for clients that query or ingest through Altertable. Encryption protects stored credentials, but it does not determine what the credential holder can access. Use [access control](/docs/administration/access-control.md) to scope members and service accounts. ## Lakehouse storage Altertable-managed storage uses Cloudflare R2 and Amazon S3, with encryption keys managed by Altertable. If you connect your own bucket, review that provider’s encryption and key-management settings alongside its access policy. Workers write to distributed object storage buckets over TLS 1.3. Each bucket stores data encrypted at rest with AES-256 and Altertable-managed keys. See [Buckets](/docs/lakehouse/storage/buckets.md) for managed and connected storage options, and [Network access](/docs/administration/network.md) for reaching restricted sources. To query Amazon S3 table buckets, connect an [S3 Tables external catalog](/docs/lakehouse/external-catalogs/s3-tables.md). --- url: https://altertable.ai/docs/administration.md --- # Administration Altertable accounts are split into organizations and environments. An organization is the company account: members, billing, and the environments you create. An environment is an isolated workspace for one stage of work, with its own cloud region, compute, catalogs, and analyses. The core pieces are: - **Organizations own the account**: Members, roles, billing, service accounts, and management API keys live on the organization. Create one organization per company or isolated tenant. - **Environments isolate work**: Create an environment for each stage of your workflow (development, staging, production) or for a customer-specific boundary. Each environment has its own cloud provider, region, compute allocation, and lakehouse. ## Organization An organization is the top-level account. It holds: - Team members and invitations - The plan, usage, and compute units you allocate to environments - Service accounts and management API keys - Organization-wide integrations and shared AI context Organization roles control who can administer the account and whether someone can read or write data across every environment. Use [access control](/docs/administration/access-control.md) to assign organization, environment, and catalog roles. Manage members, the plan, and environments from **Organization settings**. ## Environment An environment is the workspace you open in the app. Data, queries, data apps, insights, tasks, and knowledge all live inside one environment. Create a new environment when you need a separate stage or audience: - **production**, **staging**, and **development** for the same product - A dedicated environment for a customer or team that should not share catalogs Each environment has: - A slug used in app URLs and APIs (`/app/{organization}/{environment}`) - A cloud provider and region for that environment's infrastructure - Compute units allocated from the organization plan - Its own catalogs, credentials, and analyses Environment roles let people open one environment without granting access to the others. Catalog roles can narrow that further to a single data source. ## How they fit together ```text Organization ├── Members, billing, service accounts └── Environments ├── production │ catalogs, compute, insights, tasks └── staging catalogs, compute, insights, tasks ``` Provision environments from Organization settings or with [Terraform](/docs/administration/infrastructure-as-code.md). Query and analyze inside an environment through the [lakehouse](/docs/lakehouse.md). ## Administration guides - [Security and access](/docs/administration/security.md): Control access, identities, and encryption across Altertable organizations, environments, and catalogs. - [Access control](/docs/administration/access-control.md): Manage organization, environment, and catalog access for team members, service accounts, and production vs staging workspaces. - [Network](/docs/administration/network.md): How Altertable workers reach your catalogs. Allow these egress IP addresses, or connect through an SSH tunnel when the database is not publicly reachable. - [Principals](/docs/administration/principals.md): Manage Altertable principals for people and automation so access to organizations, environments, and data stays governed. - [Lakehouse credentials](/docs/administration/lakehouse-credentials.md): Altertable generates secure credentials for each environment. Access them to connect via the HTTP API, Arrow Flight SQL, or Postgres adapter. - [Encryption](/docs/administration/encryption.md): Encrypts data in transit and at rest, including credentials for databases and connected object storage buckets. - [Limits](/docs/administration/limits.md): Altertable implements soft limits on storage, agents, and SQL sessions across pricing tiers, with hard limits for memory and large table scans. - [Terraform](/docs/administration/infrastructure-as-code.md): Manage Altertable environments, buckets, catalogs, service accounts, members, credentials, and roles with the Terraform provider. --- url: https://altertable.ai/docs/administration/infrastructure-as-code.md --- # Terraform The Altertable Terraform provider manages Altertable resources from configuration: environments, storage buckets, catalogs, service accounts, organization members, lakehouse credentials, and role grants. Use it to review platform changes as plans, keep environments reproducible, and provision access for automation alongside the rest of your infrastructure. It works with Terraform `1.0+` and OpenTofu. Use Terraform for declarative infrastructure state. Use the [CLI](/docs/lakehouse/clients/cli.md) for interactive checks, one-off operations, and scripts that need to inspect or act on existing Altertable resources. ## Setup Add the provider to `required_providers` and configure it with a management API key: ```terraform terraform { required_version = ">= 1.0" required_providers { altertable = { source = "altertable-ai/altertable" } } } provider "altertable" { api_key = var.altertable_management_api_key } ``` Issue a management API key from **Organization settings → Management API keys**, then pass it as `api_key` or set the `ALTERTABLE_API_KEY` environment variable. ## Resources | Resource | Manages | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `altertable_environment` | Environments and their cloud provider | | `altertable_bucket` | Object-storage buckets connected to an environment, backing lakehouse catalogs | | `altertable_catalog` | Native databases (`engine = "altertable"`) and external connections such as Postgres, MySQL, BigQuery, or Snowflake | | `altertable_service_account` | Service accounts for automation | | `altertable_user` | Organization members, invited by email | | `altertable_credential` | Lakehouse credentials for a user or service account in an environment | | `altertable_role_set` | The complete set of role grants for one principal | Similarly, data sources exist for buckets, catalogs, credentials, environments, service accounts, and users. A special `altertable_whoami` data source looks up the identity behind the configured key. Full argument reference for every resource and data source lives on the [Terraform Registry](https://registry.terraform.io/providers/altertable-ai/altertable/latest/docs). ## Example A complete setup: an environment with a native and an external catalog, and a service account with scoped roles and a lakehouse credential for CI. ```terraform resource "altertable_environment" "production" { name = "Production" cloud_provider = "aws" cloud_provider_region = "us-east-1" } resource "altertable_catalog" "warehouse" { environment_id = altertable_environment.production.id engine = "altertable" name = "Warehouse" } resource "altertable_catalog" "analytics" { environment_id = altertable_environment.production.id engine = "postgres" name = "Analytics" postgres_config = { host = "db.example.com" port = 5432 database = "analytics" username = "altertable" password = var.pg_password } } resource "altertable_user" "teammate" { email = "teammate@example.com" } resource "altertable_role_set" "teammate" { user_id = altertable_user.teammate.id roles = [ { role = "organization:member" }, { role = "environment:writer", resource_id = altertable_environment.production.id }, ] } resource "altertable_service_account" "ci" { label = "CI Deploy" } resource "altertable_role_set" "ci" { service_account_id = altertable_service_account.ci.id roles = [ { role = "organization:member" }, { role = "environment:writer", resource_id = altertable_environment.production.id }, ] } resource "altertable_credential" "ci" { principal_type = "service_account" principal_id = altertable_service_account.ci.id environment_id = altertable_environment.production.id label = "CI" } output "ci_password" { value = altertable_credential.ci.password sensitive = true } ``` A role set declares the complete set of grants for a single principal: one `altertable_role_set` per user or service account, mixing organization-wide roles with roles scoped to a `resource_id`. See [Access control](/docs/administration/access-control.md) for the available roles. ## Import existing resources Bring resources created in the app under Terraform with `import` blocks (Terraform `1.12+`) and resource identities: ```terraform import { to = altertable_environment.production identity = { id = "7d3e1b9a-2c4f-4a6b-9e8d-5f0a1b2c3d4e" } } ``` Environments, buckets, catalogs, service accounts, credentials, and users support identity import. Nested resources use composite identities — a catalog, for example, is identified by its `environment_id` and `id`. A user is imported by its raw `user_id`. ## Source code The provider is published on the [Terraform Registry](https://registry.terraform.io/providers/altertable-ai/altertable), and its source code is available at [github.com/altertable-ai/terraform-provider-altertable](https://github.com/altertable-ai/terraform-provider-altertable). --- url: https://altertable.ai/docs/administration/lakehouse-credentials.md --- # Lakehouse credentials Altertable automatically generates lakehouse username and lakehouse password credentials for each environment in your account. ## Goal Retrieve environment-specific lakehouse credentials and verify them against the HTTP query endpoint. ## Prerequisites - [Access to the Altertable organization and environment](/docs/administration/access-control.md) you want to connect to. - Permission to view its **Credentials** section. ## Steps ### 1. Copy the credentials Open **Organization settings → API keys & credentials**, then select the intended environment. Copy the lakehouse username and lakehouse password generated for the environment. ### 2. Choose a client interface These credentials work across all client interfaces for connecting to the lakehouse. Note that authentication methods may vary by interface—refer to the specific documentation for each client interface for details. - [HTTP API](/docs/api-references/lakehouse.md): send the credentials with HTTP Basic Auth. - [Arrow Flight SQL](/docs/lakehouse/clients/arrow-flight-sql.md): pass the username and password to a compatible client over TLS. - [Postgres adapter](/docs/lakehouse/clients/postgres.md): use the credentials in a TLS-enabled Postgres connection. - [Altertable CLI](/docs/lakehouse/clients/cli.md): follow the setup guide to configure an authenticated local profile. ### 3. Configure the client Follow the selected interface guide. Keep the username and password paired with the environment where Altertable generated them. ## Verification Set your lakehouse credentials, then run a query that needs no existing table: ```bash export ALTERTABLE_USERNAME='{{LAKEHOUSE_USERNAME}}' export ALTERTABLE_PASSWORD='{{LAKEHOUSE_PASSWORD}}' curl https://api.altertable.ai/query \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H 'Content-Type: application/json' \ -d '{"statement":"SELECT 1"}' ``` A valid connection returns a JSONL response for `SELECT 1`. ## Troubleshooting - **Authentication fails:** copy both credentials again and make sure neither value includes surrounding whitespace. - **Credentials work in another environment:** return to the intended environment before opening **Credentials**. - **A database client cannot connect:** confirm that the hostname, port, and TLS settings match the selected interface guide. - **A CLI profile expired:** [log in again](/docs/lakehouse/clients/cli.md#2-log-in) for that profile. **Continue in Altertable:** [Open API keys & credentials](/app/settings/api-keys-and-credentials) ## Next steps - [Query with HTTP API](/docs/lakehouse/query/http-api.md) - [Connect with Arrow Flight SQL](/docs/lakehouse/clients/arrow-flight-sql.md) - [Connect with the Postgres adapter](/docs/lakehouse/clients/postgres.md) - [Install the Altertable CLI](/docs/lakehouse/clients/cli.md) --- url: https://altertable.ai/docs/administration/limits.md --- # Limits Altertable applies both soft and hard limits for resource management across the platform. ## Soft Limits Altertable's pricing tiers implement soft limits on storage, agents, and SQL sessions. These limits provide predictable pricing while maintaining flexibility for growing analytical workloads. ## Hard Limits Altertable enforces hard technical memory limits that cannot be exceeded. This ensures system stability and prevents resource exhaustion. [Contact us](mailto:contact@altertable.ai) if your workloads require additional memory capacity. ## Exceeding Limits Soft limits can be exceeded through usage-based billing. For hard limits, please [contact our team](mailto:contact@altertable.ai) to discuss custom solutions or roadmap timing. ## Lakehouse requests | Operation | Maximum request body | | ----------------------------------------------------------------------------- | -------------------- | | [Append JSON records](/docs/api-references/lakehouse.md#post-append) | 32 MiB | | [Upload CSV, JSON, or Parquet](/docs/api-references/lakehouse.md#post-upload) | 100 GiB | | [Upsert from a file](/docs/api-references/lakehouse.md#post-upsert) | 100 GiB | For larger append batches, split the records or [upload a file in append mode](/docs/api-references/lakehouse.md#post-upload). **Continue in Altertable:** [View usage](/app/settings/usage) --- url: https://altertable.ai/docs/administration/network.md --- # Network Managed [Workers](/docs/lakehouse/compute/workers.md) connect to your databases and warehouses over the network. If inbound traffic is restricted, allow Altertable's egress IP addresses. If the catalog is not reachable from the public internet, you can also connect through an [SSH tunnel](/docs/lakehouse/external-catalogs.md). ## Egress IP addresses Managed Workers connect from a fixed set of public IPv4 addresses. Allow these addresses so Altertable can reach the catalog. Self-hosted Workers use your own network instead. They do not egress from the addresses below. Allow every address. Firewalls that require a prefix can use `/32`. ```text 46.224.237.164 46.4.253.101 94.130.221.56 188.40.149.154 138.199.242.155 138.199.241.17 ``` ### Where to allow them - **[MongoDB Atlas](/docs/lakehouse/external-catalogs/mongodb.md)**: [IP Access List](https://www.mongodb.com/docs/atlas/security/ip-access-list/) under **Network Access**. Add each address (or `address/32`). - **[Snowflake](/docs/lakehouse/external-catalogs/snowflake.md)**: include the addresses in a network rule and attach it to the user Altertable uses. - **Self-hosted databases**: allow inbound TCP from these addresses on the database port, or connect through an [SSH tunnel](/docs/lakehouse/external-catalogs.md). --- url: https://altertable.ai/docs/administration/principals.md --- # Principals Principals are identities that can be granted access in Altertable. A principal can be a person using the app or a service account used by automation. Altertable applies the same [access control](/docs/administration/access-control.md) model to both types: assign roles at the organization, environment, or catalog level depending on how much access the principal needs. ## Principal types | Type | Use for | Managed from | | --------------- | ----------------------------------------------------------------------- | --------------------- | | Team member | Humans who log in to the Altertable app | Organization settings | | Service account | Automation such as CI, dbt jobs, scripts, and machine-to-machine access | Organization settings | ## Service accounts Use service accounts instead of shared human accounts for scheduled jobs and integrations. They make ownership clearer, can be scoped narrowly, and can be rotated without affecting a person's login. Typical uses include: - Running ingestion or transformation jobs. - Connecting BI tools or scripts. - Managing deployments that need API or lakehouse credentials. - Giving an AI client or workflow access to a specific environment or catalog. Use the [CLI](/docs/lakehouse/clients/cli.md) to test a service account profile from a terminal before wiring it into CI. After creating a service account, review its roles before using it in production. Prefer catalog-level or environment-level access when the job does not need the full organization. Issue a management API key for a service account when automation needs the Management API. The key acts as that service account, so its requests use the same role assignments and resource scope. Store the key in a secret manager and rotate it independently of human credentials. ## Learn more - **[Access control](/docs/administration/access-control.md)**: role scopes and recommended patterns. - **[Authentication](/docs/administration/lakehouse-credentials.md)**: credentials for querying the lakehouse. - **[MCP](/docs/agents/mcp.md)**: Governed agent access to interact with your lakehouse. --- url: https://altertable.ai/docs/administration/security.md --- # Security and access Set the policies that determine who and what can access Altertable. Start with access control and principals, then review how data and credentials are protected. ## Choose the control - [Access control](/docs/administration/access-control.md) defines the organization, environment, and catalog roles a member or service account receives. - [Principals and credentials](/docs/administration/principals.md) explains human and machine identities and their authentication methods. - [Network access](/docs/administration/network.md) explains how Workers reach restricted data sources through allowlists or SSH tunnels. - [Encryption](/docs/administration/encryption.md) explains protection for stored data and connections. When connecting an agent, [MCP scopes](/docs/agents/mcp.md#pin-a-scope) limit the tools it can use. Resource permissions still determine which data and objects those tools can access. --- url: https://altertable.ai/docs/agents/ask.md --- # Ask Altertable Ask Altertable is the conversational interface for querying and understanding data in Altertable. It can inspect your lakehouse, use semantic models and knowledge, run SQL, create analyses, and remember feedback from previous work. Use Ask Altertable for exploratory questions, follow-up analysis, and operational investigations where the right query is not obvious yet. For repeatable reporting, save the result as an [insight](/docs/data-apps/insights.md). In the Altertable app, open **Ask Agent** to start a conversation. ## What Ask Altertable can use Ask Altertable works from the same governed context as the rest of Altertable: - Catalog schemas and table metadata from your [lakehouse](/docs/lakehouse.md). - [Semantic models](/docs/agents/knowledge/semantic-layer.md) that define business metrics and dimensions. - [Knowledge entries](/docs/agents/knowledge/entries.md), connected repositories, and organization context. - [Memories](/docs/agents/knowledge/memories.md) created from previous agent runs and feedback. - Existing insights, dashboards, notifications, and task history. ## Use in Slack Ask Altertable is also available in Slack. An organization administrator connects Slack from **Organization settings → Integrations**. Invite the connected app to the channel where you want to ask questions, then mention it with your question. Continue in its thread to ask follow-up questions. The Slack agent uses the configured environment and its configured data access role for connected channels. An administrator chooses both when connecting Slack and can change them in **Organization settings → Integrations**. Include the time range in your question, and check returned queries and source data before sharing the result with the channel. ## Good prompts Start with the business question and include the time period, entity, or segment you care about: ```text Which customers had the largest drop in weekly usage after the pricing change? ``` ```text Are checkout errors related to payment provider latency over the last 24 hours? ``` ```text How has trial-to-paid conversion changed over the last three months? ``` Ask Altertable may run SQL, inspect schemas, or ask clarifying questions before answering. When the answer depends on an assumption, ask it to show the query or explain the fields it used. ## Learn more - **[SQL Explorer](/docs/lakehouse/query/sql-explorer.md)**: run and refine SQL directly. - **[Semantic Layer](/docs/agents/knowledge/semantic-layer.md)**: define reusable business concepts. - **[Memories](/docs/agents/knowledge/memories.md)**: review the context Ask Altertable learns over time. - **[MCP](/docs/agents/mcp.md)**: connect external AI clients to the same data and tools. --- url: https://altertable.ai/docs/agents/cli.md --- # Use the CLI with agents Use MCP for interactive Altertable operations. Add the CLI when your agent needs local files, ingestion commands, scripts, or CI. The CLI does not replace MCP, and each connection has its own authentication. > **Before using the Altertable CLI:** > > Install the CLI if you haven’t already: > > ```bash > curl -fsSL https://install.altertable.ai | sh > altertable login > ``` > > If `altertable` isn’t found after installation, open a new terminal. Run `altertable login` to log in to your organization and environment. Follow the [CLI setup guide](/docs/lakehouse/clients/cli.md#2-log-in) to configure credentials and verify access. ## Prepare a profile Check the profile before using the CLI alongside MCP: ```bash altertable profile show altertable profile status altertable --agent query "SELECT 1 AS connected" ``` Verify that the CLI profile and MCP connection target the intended organization and environment. ## Return structured results Use the global `--agent` option before the subcommand: ```bash altertable --agent query "SELECT table_catalog, table_schema, table_name FROM information_schema.tables LIMIT 10" ``` This returns structured JSON without a pager, colors, or terminal styling. Discover tables before querying them and bound exploratory results with `LIMIT`. For commands and options, see the [CLI reference](/docs/api-references/cli.md). --- url: https://altertable.ai/docs/agents.md --- # Agents Use Altertable's agent in the app, or connect your own AI client through MCP. Agents can inspect catalogs, run SQL, and create analyses or data apps within the access you grant. If you are new to Altertable, [set up with your agent](/docs/getting-started.md) to connect MCP and get a first answer from your stack. The CLI is optional for terminal commands and local files. ## Start with a question Use [Ask Altertable](/docs/agents/ask.md) for questions in the Altertable app. If you already work in Claude, ChatGPT, or another AI client, [connect that agent](/docs/agents/setup.md) instead. - [Ask Altertable](/docs/agents/ask.md): Ask questions about your Altertable data with an agent that uses shared lakehouse context, SQL, and saved business definitions. - [Use the CLI with agents](/docs/agents/cli.md): Use the Altertable CLI alongside MCP when your agent needs terminal commands, local files, ingestion, or structured query results. - [Connect your agent](/docs/agents/setup.md): Connect Codex, Claude, ChatGPT, Cursor, or another compatible client to Altertable with MCP, workflow skills, and optional terminal access. - [Business context](/docs/agents/knowledge.md): Manage semantic layers, memories, repositories, and business context that help teams and agents understand lakehouse data. - [Model Context Protocol (MCP)](/docs/agents/mcp.md): Connect MCP-compatible AI clients to Altertable so agents can query lakehouse data through governed, scoped tools. For recurring questions, [schedule a task](/docs/automation/tasks.md). --- url: https://altertable.ai/docs/agents/knowledge/entries.md --- # Knowledge entries Knowledge entries are durable reference materials that AI agents can search while answering questions or running tasks. Use them for context that should be remembered exactly, such as metric definitions, operating procedures, product terminology, customer notes, or repository documentation. Knowledge entries complement [memories](/docs/agents/knowledge/memories.md). Entries are written or connected by your team; memories are learned from agent runs and feedback. ## Entry types You can add knowledge in two common ways: - **Direct entries**: notes, explanations, and small documents created inside Altertable. - **Repository syncs**: connected GitHub or GitLab repositories that make Markdown, text files, and dbt metadata available as searchable context. Use direct entries for business rules and lightweight guidance. Use repository syncs when the source of truth already lives in code, analytics projects, or operational documentation. ## Import skills as knowledge You can upload a Markdown skill file or sync a repository of skills to make its guidance available to Altertable agents. ### Upload a Markdown skill 1. Open **Knowledge** and choose **Add knowledge**. 2. Select **File**, then **Choose file** to upload the skill's `.md` file. For a standalone upload, use a descriptive filename such as `monthly-revenue-analysis.md`. 3. Set **Title** to a specific name such as "Monthly revenue analysis". Include the task and topic you would search for to make the entry easier to find. 4. Choose whether the entry is available in the current environment or in all environments. 5. Review **Automatically include in AI sessions**, then select **Add knowledge**. Keep the original `SKILL.md` filename in its skill directory, as required by the [Agent Skills format](https://agentskills.io/specification). Rename only the copy you upload as knowledge. The upload creates a knowledge entry from that file. To update it later, edit the entry and replace the attachment. If the skill references other documents, add those separately or sync its repository. ### Sync a skills repository 1. Connect GitHub or GitLab in **Settings** → **Integrations**. See [knowledge integrations](/docs/agents/knowledge/integrations.md). 2. In **Knowledge**, choose **Add knowledge** → **Repository**. 3. Select the provider, repository, and branch. For example, select [`altertable-ai/skills`](https://github.com/altertable-ai/skills) if it is available through your GitHub connection. 4. Choose the environment scope and select **Add repository**. Repository sync imports supported Markdown and text files, including `SKILL.md` files. It skips local development skills under `.agents/skills/`, `.claude/skills/`, and `.codex/skills/` at any depth. Keep skills you want to share as knowledge outside those directories, for example at `skills/revenue-analysis/SKILL.md`. Sync the repository again after changing the source files to refresh its knowledge entries. ### How imported skills affect agents Imported skills are indexed as knowledge. Agents can find them through knowledge search, including the `search_entities` MCP tool, and may use their instructions when answering questions or running tasks. Review the content before importing it: development conventions or conflicting instructions can steer an agent away from your intended analysis. Turning off **Automatically include in AI sessions** keeps an entry out of the initial session context, but agents can still find it through search. The environment scope controls where the knowledge is available. This imports the skill's content as reference material; it does not install an agent plugin or register a callable skill. To install Altertable skills in an external agent, follow [Agent setup](/docs/agents/setup.md). ## dbt repositories Altertable detects dbt projects through `dbt_project.yml` and extracts knowledge from their source files. For Git-based sync, commit `target/manifest.json` alongside each project's source files on the connected branch to include model lineage and semantic metadata. Also commit `target/semantic_manifest.json` to preserve generated metric and semantic definitions in a dedicated knowledge entry for each project. Both paths are relative to the project's `dbt_project.yml`. Syncing a repository does not run dbt to generate these artifacts or publish executable metrics. Follow the [dbt repository setup](/docs/lakehouse/modeling/dbt.md#sync-project-knowledge) to generate and refresh the artifacts, preserve metric definitions, and enrich matching semantic models. ## What to include Good entries are specific and reusable: - How your company defines revenue, activation, churn, and retention. - Which tables are authoritative for common analyses. - Known caveats in source systems or ETL jobs. - Product terminology and internal names that do not appear in schemas. - Investigation playbooks for recurring incidents or workflows. Keep entries current. Stale guidance can be worse than missing guidance because agents may confidently apply the wrong rule. ## Learn more - **[Semantic Layer](/docs/agents/knowledge/semantic-layer.md)**: define metrics and dimensions directly on models. - **[Memories](/docs/agents/knowledge/memories.md)**: understand learned context from agent work. - **[Ask Altertable](/docs/agents/ask.md)**: use knowledge during conversational analysis. --- url: https://altertable.ai/docs/agents/knowledge.md --- # Business context Knowledge gives AI agents the business context they need to interpret data correctly. It includes semantic models, durable knowledge entries, connected repositories, organization context, and memories learned from prior work. Start with a [knowledge entry](/docs/agents/knowledge/entries.md) for written business context. Add a [Semantic Layer model](/docs/agents/knowledge/semantic-layer.md) when you need governed metrics and dimensions. - [Semantic Layer](/docs/agents/knowledge/semantic-layer.md): Define shared metrics, dimensions, and business concepts in Altertable for governed analytics and AI workflows. - [Memories](/docs/agents/knowledge/memories.md): Use Altertable memories to preserve context from prior agent work so future analysis starts with reusable knowledge. - [Knowledge entries](/docs/agents/knowledge/entries.md): Create and manage knowledge entries and repositories in Altertable so teams and agents can reuse trusted business context. - [Repository integrations](/docs/agents/knowledge/integrations.md): Connect source providers like GitHub and GitLab so Altertable can sync repository files into your knowledge base. --- url: https://altertable.ai/docs/agents/knowledge/integrations/gitlab-self-hosted.md --- # GitLab (self-hosted) [GitLab](https://about.gitlab.com/) self-managed instances can connect to Altertable through an OAuth application. Because a self-hosted instance runs on your own domain, you register the OAuth application yourself and provide its credentials to Altertable. The connection is set up in two stages: you create an OAuth application in GitLab, then enter its credentials in Altertable and authorize the connection. ## Prerequisites - A self-hosted GitLab instance reachable from Altertable (for example `https://gitlab.your-company.com`). - Permission to create an OAuth application on that instance (user, group, or admin application depending on your setup). - Access to create connections in your Altertable environment. ## 1. Create the OAuth application on GitLab Register an OAuth application on your GitLab instance so it can authorize Altertable. 1. In GitLab, open the OAuth application settings for your user, group, or instance, depending on where you have permission to register applications. 2. Choose **Add new application**. 3. Set a **Name**, such as `Altertable Knowledge`. 4. Set `https://app.altertable.ai/gitlab-app/callback` as a **Redirect URI**. 5. Enable **Confidential**. 6. Select the required **Scopes**: - `read_api` — read access to the GitLab API. - `read_user` — read the authenticating user's profile. - `read_repository` — read-only access to repository contents. 7. Save the application. 8. Copy the generated **Application ID** and **Secret**. The secret is shown only once. ## 2. Complete the connection on Altertable Go to the Altertable connection screen to complete authorization. 1. In Altertable, open **Settings** → **Integrations**, find the **GitLab** row, then choose **Connect GitLab** → **Self-hosted GitLab**. 2. Enter your **Instance URL**, then paste the **Application ID** and **Secret** from GitLab. 3. Choose **Connect** to start the OAuth authorization flow. 4. Approve the authorization request on your GitLab instance when prompted. 5. Altertable confirms the connection once authorization succeeds. ## Troubleshooting - **404 after clicking Connect:** double-check the **Instance URL** for typos, and — unless a reverse proxy serves GitLab under a subpath — enter the plain origin (for example `https://gitlab.example.com`), not a full page URL such as `https://gitlab.example.com/dashboard/projects`. - **Redirect URI mismatch:** the URI registered in GitLab must match the one Altertable provides exactly, including scheme and trailing path. - **Authorization fails:** confirm the application scopes include `read_api`, `read_user`, and `read_repository`, and that the instance is reachable from Altertable. - **Instance not reachable:** verify the instance URL and that any firewall or network policy allows access from Altertable. --- url: https://altertable.ai/docs/agents/knowledge/integrations.md --- # Repository integrations Connect a source provider to sync Markdown and text files from your repositories into Altertable [knowledge](/docs/agents/knowledge.md), so AI agents can reference them. Open **Settings** → **Integrations** in Altertable to enable a provider: - **GitHub** - **GitLab** — GitLab.com (Cloud) or a self-hosted instance Self-hosted GitLab requires registering your own OAuth application. See [GitLab (self-hosted)](/docs/agents/knowledge/integrations/gitlab-self-hosted.md) for instructions. - [GitLab (self-hosted)](/docs/agents/knowledge/integrations/gitlab-self-hosted.md): Connect a self-hosted GitLab instance to Altertable through an OAuth application to sync knowledge from your repositories. --- url: https://altertable.ai/docs/agents/knowledge/memories.md --- # Memories Memories are learned context from agent runs and feedback. AI agents use them to adapt to your data model, business terminology, and team preferences over time. Use memories for patterns that emerge through work, such as recurring metric caveats, useful investigation steps, or feedback about which findings matter. ## Memories and knowledge entries Memories complement [knowledge entries](/docs/agents/knowledge/entries.md), but they are created differently. Knowledge entries are durable reference material your team adds or syncs from GitHub. Memories are observations agents create and refine while they analyze data, generate notifications, and receive feedback. Use knowledge entries when the source of truth should be written down exactly. Let memories capture what the system learns from repeated workflows. ## What memories help with - **Preferences**: important KPIs, preferred insight types, and analysis style. - **Feedback**: which [notifications](/docs/automation/notifications.md) were useful or noisy. - **Business context**: product names, segments, seasonality, and internal terminology. - **Working patterns**: investigation steps that worked well in past analyses. ## How memories stay relevant Agents do not keep every observation forever. Memories have relevance based on importance, recency, and how often they are used. Frequently used memories stay active. Similar memories can be consolidated into a clearer pattern. Memories that stop being useful gradually fade and can be removed. ## Manage memories Memories are scoped by environment. In organization settings, open the memories list for an environment to search, edit, or remove saved memories. ## Learn more - **[Ask Altertable](/docs/agents/ask.md)**: natural-language analysis that creates and uses memories - **[Knowledge entries](/docs/agents/knowledge/entries.md)**: add durable reference material for AI agents - **[Notifications](/docs/automation/notifications.md)**: review findings so agents learn what matters --- url: https://altertable.ai/docs/agents/knowledge/semantic-layer.md --- # Semantic Layer The semantic layer defines business-friendly models over lakehouse tables. It gives teams and AI agents shared names for measures, dimensions, timestamps, filters, and relationships so the same metric is computed the same way everywhere. Use semantic models when analysts, dashboards, and AI agents should work with concepts like `monthly recurring revenue`, `active accounts`, or `signup cohort` instead of rewriting raw SQL each time. ## What a model contains A semantic model is attached to a table or view and can define: - **Measures**: aggregations such as revenue, active users, order count, or average latency. - **Dimensions**: fields used for grouping and filtering, such as plan, region, customer tier, or status. - **Timestamps**: time fields used for trends and time grains. - **Relations**: joins to other models so analyses can cross business entities. - **Formatting**: display hints such as currency, percentages, and date formats. ## Where models are used Semantic models power [semantic insights](/docs/data-apps/insights.md#1-semantic-insights), Ask Altertable, tasks, and connected clients. They also make data discovery easier because important tables carry explicit business meaning. Each model also carries an [endorsement](/docs/lakehouse/endorsement.md) status that tells agents which tables to prefer. ## Enrich models from dbt A connected dbt knowledge repository can enrich existing models with descriptions, column metadata, supported measures and dimensions, and relationships. For Git-based sync, commit `target/manifest.json` in the dbt project and sync the repository to apply this enrichment. For this enrichment, models must already exist in Altertable and match the manifest's catalog, schema, and table names. See the [dbt integration](/docs/lakehouse/modeling/dbt.md#enrich-existing-semantic-models) for setup and limitations. See [dbt artifact requirements](/docs/lakehouse/modeling/dbt.md#which-files-need-to-be-committed) to make generated metric definitions available as knowledge. ## Learn more - **[Insights](/docs/data-apps/insights.md)**: build semantic and SQL analyses. - **[Ask Altertable](/docs/agents/ask.md)**: ask natural-language questions over models and data. - **[Knowledge entries](/docs/agents/knowledge/entries.md)**: add supporting business documentation. --- url: https://altertable.ai/docs/agents/mcp.md --- # Model Context Protocol (MCP) Altertable's MCP server gives AI clients a governed way to interact with your lakehouse. ## Goal Connect an MCP-compatible AI client to Altertable, authorize one environment, and verify that the client can discover Altertable tools and catalogs. ## Prerequisites - An Altertable account with [access to the organization and environment](/docs/administration/access-control.md) you want to query. - An AI client that supports hosted MCP servers over streamable HTTP and OAuth 2.0. Clients that only support local `stdio` servers cannot connect to Altertable's hosted MCP server. ## Steps ### 1. Add the server | Setting | Value | | -------------- | ------------------------------- | | Server URL | `https://mcp.altertable.ai/mcp` | | Transport | `Streamable HTTP` | | Authentication | `OAuth 2.0` | 1. Add `https://mcp.altertable.ai/mcp` as an MCP server in your client. 2. Start the connection or authorization flow. 3. Log in to Altertable in the browser when prompted. 4. Choose the organization and environment the client should access. 5. Approve the connection and return to your client. After approval, the client uses an OAuth bearer token for MCP tool discovery and execution. Access is scoped to the environment selected during authorization. You must belong to its organization and have permission to use that environment. Tools are filtered by the granted tier, resource permissions, enabled product features, and connected integrations. Reconnect and authorize again to change the environment or scope. Start with `https://mcp.altertable.ai/mcp` and grant [Read-only](/docs/agents/mcp.md#pin-a-scope) access for exploration. Choose [Analyst](/docs/agents/mcp.md#pin-a-scope) when you need to create analyses or data apps. Choose [Full access](/docs/agents/mcp.md#pin-a-scope) only when you need lakehouse writes or schema changes. #### Pin a scope Choose the least access your workflow needs during authorization. The default endpoint accepts all three tiers. To request a specific access level up front, use a pinned endpoint: | Access | Endpoint | | ---------------------------------------------- | ------------------------------------------ | | [Read-only](/docs/agents/mcp.md#pin-a-scope) | `https://mcp.altertable.ai/mcp/read-only` | | [Analyst](/docs/agents/mcp.md#pin-a-scope) | `https://mcp.altertable.ai/mcp/analyst` | | [Full access](/docs/agents/mcp.md#pin-a-scope) | `https://mcp.altertable.ai/mcp/read-write` | The [glossary](/docs/api-references/glossary.md#mcp-access-names) maps each display label to its endpoint and OAuth scope. A pinned endpoint requires its exact tier, even if your token grants broader access. A mismatch returns `403 insufficient_scope`. Use the default endpoint when the client should choose access during authorization. For example, to add a read-only connection in Claude Code: ```bash claude mcp add --transport http altertable-mcp https://mcp.altertable.ai/mcp/read-only ``` ### 2. Let the client discover OAuth settings Compatible clients should discover OAuth settings automatically and follow the metadata URL in the server’s `WWW-Authenticate` challenge. Pinned endpoints provide path-specific metadata. Altertable supports protected resource metadata, authorization server discovery, and dynamic client registration. | Purpose | URL | | ----------------------------- | ------------------------------------------------------------------ | | MCP server endpoint | `https://mcp.altertable.ai/mcp` | | Protected resource metadata | | | Authorization server metadata | | When a client calls an authenticated MCP method without a valid token, Altertable may return a `WWW-Authenticate: Bearer ...` challenge that points the client to the protected resource metadata. ### 3. Review tool permissions MCP tools can list catalogs and semantic models, query the lakehouse, validate SQL, view insights and dashboards, and create outputs such as insights, memories, notifications, and scheduled tasks when the granted tier and resource permissions allow it. For the exact tool schema, see [MCP tools](/docs/api-references/mcp-tools.md). ## Verification Ask the connected client: ```text Use Altertable to list my available catalogs. ``` The client should discover the Altertable tools, request authorization if needed, and return catalogs from the environment selected during authorization. ## Troubleshooting - **Client cannot connect**: confirm it supports hosted MCP over streamable HTTP and OAuth. - **Discovery does not start**: use exactly `https://mcp.altertable.ai/mcp` (or `/mcp/read-only`, `/mcp/analyst`, or `/mcp/read-write` if you pinned a scope). - **Browser login loops**: finish authorization in the browser window opened by the client. - **No tools appear**: reconnect so the client reruns discovery, authorization, and tool listing. - **Wrong environment**: reconnect and choose the correct environment during authorization. - **403 on a pinned endpoint**: the token's granted scope doesn't match the endpoint (e.g. a read-only token used against `/mcp/read-write`). Reauthorize with the matching scope, or connect to the bare `https://mcp.altertable.ai/mcp` endpoint instead. **Continue in Altertable:** [Configure MCP server](/app/settings/mcp-servers/new) ## Next steps - [Review MCP tools](/docs/api-references/mcp-tools.md) to see the exact schemas and permissions. - [Set up an AI client](/docs/agents/setup.md) with client-specific instructions. - [Install the Altertable CLI](/docs/lakehouse/clients/cli.md) for terminal automation and structured agent output. --- url: https://altertable.ai/docs/agents/setup/chatgpt.md --- # Set up ChatGPT The Altertable plugin gives ChatGPT access to Altertable Skills and the hosted MCP server. ## Install the Altertable plugin 1. Open ChatGPT and go to Settings → Plugins. 2. Click **Add** → **Add a marketplace**. 3. Enter `https://github.com/altertable-ai/skills`. 4. Click **Add marketplace**. ## Connect and authorize Altertable MCP 1. Find the Altertable plugin and click **Install plugin**. 2. ChatGPT will try to connect Altertable MCP and open the Altertable authorization page: - Select your organization. - Select your environment. - Keep the mode set to [Read-only](/docs/agents/mcp.md#pin-a-scope) unless ChatGPT needs to change persistent resources. 3. Click **Authorize**, then return to ChatGPT. 4. Start a new ChatGPT chat. Altertable MCP should appear as connected, and the Altertable Skills should be available. ## Verify Start a new chat and enter: ```text @Altertable list my available catalogs ``` ChatGPT should use Altertable, call `initialize`, and then inspect the available catalogs. ## Use Prefix a question with `@Altertable`: ```text @Altertable analyze web traffic for the last 30 days and call out meaningful changes ``` You can also use `/ask-altertable` to let Altertable choose the right workflow or `/query-altertable` for a direct data question: ```text /query-altertable show daily signups for the last 30 days ``` ## Tools - [Altertable Skills](/docs/agents/setup.md#skills-included) guide ChatGPT through exploration, SQL, funnels, web traffic, insights, and platform operations. - [Altertable MCP](/docs/agents/mcp.md) provides governed access to catalogs, queries, insights, findings, and tasks. ## Update Open Settings → Plugins to check the Altertable marketplace for updates. Install an available update, then start a new ChatGPT chat so it loads the latest skills. ## Troubleshooting - **Plugin is missing:** return to Settings → Plugins, confirm that the Altertable marketplace is present, and install the plugin again. - **Skills do not appear:** confirm that the plugin is installed, then start a new ChatGPT chat. - **MCP is not connected:** open the Altertable plugin and reconnect Altertable MCP. - **Authorization does not finish:** complete the authorization page, select an organization and environment, and then return to ChatGPT. - **Wrong environment is connected:** disconnect Altertable MCP and authorize it again with the correct environment. --- url: https://altertable.ai/docs/agents/setup/claude-code.md --- # Set up Claude Code The Altertable Claude plugin installs Altertable Skills and registers the hosted MCP server. ## Quick start 1. Open Claude Code from the root of the project where you want to use Altertable. 2. Add the Altertable marketplace: ```bash claude plugin marketplace add altertable-ai/skills ``` 3. Install the plugin: ```bash claude plugin install altertable@altertable-ai ``` 4. Start a new Claude Code session so it discovers the plugin. 5. Ask Claude to analyze a real workflow: ```text /altertable:ask-altertable analyze my signup funnel from landing page to account creation ``` The first Altertable tool call opens OAuth authorization in your browser. Choose the organization and environment Claude should access, then return to Claude Code. ## Verify Open `/plugin` and confirm that **Altertable** is installed. Its entry includes the installed plugin version. Open `/mcp` to list configured servers, connect to Altertable, or inspect connection errors. Then ask: ```text Use Altertable to list my available catalogs. ``` Claude should call `initialize` first and then inspect the catalogs. ## Tools Use the router when you are unsure which skill applies: ```text /altertable:use-altertable ``` ### When to add the CLI The plugin already provides [Altertable Skills](/docs/agents/setup.md#skills-included) and [Altertable MCP](/docs/agents/mcp.md). Install the [Altertable CLI](/docs/lakehouse/clients/cli.md) only when Claude needs to run Altertable from the shell, such as in CI, ingestion workflows, or scripts that consume JSON. Claude can use the CLI alongside MCP. ## Update Each plugin release is versioned. Check the installed version with `/plugin`, then refresh the marketplace when a new Altertable Skills release is available: ```bash claude plugin marketplace update altertable-ai ``` ## Troubleshooting - **Plugin is missing:** open `/plugin`, confirm that the Altertable marketplace is present, then reinstall the plugin. - **Skills do not appear:** start a new Claude Code session after installation or update. - **MCP tools do not appear:** open `/mcp` to inspect or reconnect the Altertable server, or run `claude mcp list` from the shell. Reinstall the plugin if the server is missing. - **Authorization does not finish:** complete the browser flow opened by Claude and choose an organization and environment. - **Wrong environment is connected:** disconnect the MCP server and authorize it again with the correct environment. - **Claude Code has broader installation problems:** run `/doctor` for general diagnostics. --- url: https://altertable.ai/docs/agents/setup/claude.md --- # Set up Claude The Altertable plugin gives Claude access to Altertable Skills and the hosted MCP server. ## Install the Altertable plugin 1. Open Claude and go to Settings → Plugins. 2. Click **Add** → **Add marketplace**. 3. Select **Add from a repository**. The official Anthropic listing is pending review. 4. Enter `https://github.com/altertable-ai/skills`. 5. Keep auto-sync enabled to receive updates automatically. 6. Click **Sync**. The Altertable plugin should appear immediately. ## Connect and authorize Altertable MCP 1. Find the Altertable plugin, open the **Connectors** tab, and click **Connect** for the `altertable-mcp` connector. 2. When Claude asks whether you trust the destination, click **Yes**. 3. On the authorization page: - Select your organization. - Select your environment. - Keep the mode set to [Read-only](/docs/agents/mcp.md#pin-a-scope) unless Claude needs to change persistent resources. 4. Click **Authorize**, then return to Claude. 5. Start a new Claude chat. Altertable MCP should appear as connected, and the Altertable Skills should be available. ## Verify Start a new chat and enter: ```text /ask-altertable list my available catalogs ``` Claude should use the Altertable skill, call `initialize`, and then inspect the available catalogs. ## Use Prefix a question with `/ask-altertable` when you want Altertable to choose the right workflow: ```text /ask-altertable analyze web traffic for the last 30 days and call out meaningful changes ``` Use `/query-altertable` for a direct data question: ```text /query-altertable show daily signups for the last 30 days ``` ## Tools - [Altertable Skills](/docs/agents/setup.md#skills-included) guide Claude through exploration, SQL, funnels, web traffic, insights, and platform operations. - [Altertable MCP](/docs/agents/mcp.md) provides governed access to catalogs, queries, insights, findings, and tasks. ## Update When auto-sync is enabled, Claude syncs updates from the Altertable marketplace automatically. You can also return to Settings → Plugins, open the Altertable marketplace, and click **Sync**. ## Troubleshooting - **Plugin is missing:** return to Settings → Plugins and sync the Altertable marketplace again. - **Skills do not appear:** confirm that the plugin is installed, then start a new Claude chat. - **MCP is not connected:** open the plugin's **Connectors** tab and reconnect `altertable-mcp`. - **Authorization does not finish:** complete the authorization page, select an organization and environment, and then return to Claude. - **Wrong environment is connected:** disconnect `altertable-mcp` and authorize it again with the correct environment. --- url: https://altertable.ai/docs/agents/setup/codex.md --- # Set up Codex The Altertable Codex plugin packages Altertable Skills and the hosted MCP configuration. ## Quick start 1. Open Codex from the root of the project where you want to use Altertable. If **Altertable** is available in your Plugins directory, install it there and skip steps 2 and 3. 2. Add the Altertable marketplace: ```bash codex plugin marketplace add altertable-ai/skills ``` 3. Install the plugin: ```bash codex plugin add altertable@altertable-ai ``` 4. Start a new Codex session so it loads the plugin. 5. Ask Codex to analyze a real workflow: ```text Analyze web traffic for the last 30 days and call out meaningful changes. ``` The first Altertable tool call opens OAuth authorization in your browser. Choose the organization and environment Codex should access, then return to Codex. ## Verify List installed plugins and MCP servers: ```bash codex plugin list codex mcp list ``` Then ask: ```text Use Altertable to list my available catalogs. ``` Codex should call `initialize` first and then inspect the catalogs. ## Tools - [Altertable Skills](/docs/agents/setup.md#skills-included) provide workflows for exploration, SQL, funnels, web traffic, insights, and platform operations. - [Altertable MCP](/docs/agents/mcp.md) provides governed access to catalogs, queries, insights, findings, and tasks. - The optional [Altertable CLI](/docs/lakehouse/clients/cli.md) supports shell automation, CI, ingestion, SQL, and structured agent output. If you only need the MCP server, add it without installing the plugin: ```bash codex mcp add altertable-mcp --url https://mcp.altertable.ai/mcp ``` Do not add a duplicate when `codex mcp list` already shows the server. ### Choose between Skills, MCP, and the CLI Use Skills for workflow guidance, MCP for interactive Altertable operations, and the CLI for terminal automation, CI, ingestion, or structured JSON. The CLI is optional and installed separately; Codex can use it alongside MCP. ## Update Refresh released skills from the configured marketplace: ```bash codex plugin marketplace upgrade altertable-ai ``` Then open `/plugins` to install the available Altertable update. ## Troubleshooting - **Plugin is missing:** run `codex plugin marketplace list`, refresh the Altertable marketplace, and install the plugin again. - **Skills do not appear:** start a new Codex session after installation or update. - **MCP server is missing:** install the plugin again or add the hosted server directly. - **Authorization does not finish:** complete the browser flow opened by Codex and choose an organization and environment. - **Wrong environment is connected:** remove the Altertable MCP connection and authorize it again with the correct environment. --- url: https://altertable.ai/docs/agents/setup/cursor.md --- # Set up Cursor The Altertable Cursor plugin packages Altertable Skills and the hosted MCP configuration. Cursor loads it as a [Cursor Plugin](https://cursor.com/docs/plugins) from `.cursor-plugin/plugin.json`. ## Quick start 1. Open the project where you want to use Altertable. If **Altertable** is available in Customize → Plugins, install it there and skip steps 2 and 3. 2. Add the Altertable marketplace from Settings → Customize → Browse Marketplace → Add Marketplace, then enter `https://github.com/altertable-ai/skills`. From the Agent CLI: ```bash agent plugin marketplace add https://github.com/altertable-ai/skills ``` Team and Enterprise admins can instead import `https://github.com/altertable-ai/skills` from Dashboard → Plugins. 3. Install **Altertable** from Customize → Plugins, or type `/plugin` in the Agent CLI and install it from the Marketplace tab. 4. Start a new agent session so Cursor loads the plugin. 5. Ask Cursor to analyze a real workflow: ```text Analyze my signup funnel from landing page to account creation. ``` The first Altertable tool call opens OAuth authorization in your browser. Choose the organization and environment Cursor should access, then return to Cursor. ## Verify Ask Cursor: ```text Use Altertable to list my available catalogs. ``` Cursor should call `initialize` first and then inspect the catalogs. ## Tools - [Altertable Skills](/docs/agents/setup.md#skills-included) provide workflows for exploration, SQL, funnels, web traffic, insights, and platform operations. - [Altertable MCP](/docs/agents/mcp.md) provides governed access to catalogs, queries, insights, findings, and tasks. - The optional [Altertable CLI](/docs/lakehouse/clients/cli.md) supports shell automation, CI, ingestion, SQL, and structured agent output. Install at user scope when the plugin should follow you across projects. Install at project scope when every contributor should use the same version. ### Choose between Skills, MCP, and the CLI Use Skills for workflow guidance, MCP for interactive Altertable operations, and the CLI for terminal automation, CI, ingestion, or structured JSON. The CLI is optional and installed separately; Cursor can use it alongside MCP. ## Update Refresh the marketplace, then install the available Altertable update from Customize → Plugins or `/plugin`. ```bash agent plugin marketplace update altertable-ai ``` ## Troubleshooting - **Plugin is missing:** open Customize → Plugins, confirm that the Altertable marketplace is present, then install **Altertable**. - **Skills do not appear:** start a new agent session after installation or update. - **MCP server is disabled:** open Customize and enable the Altertable MCP server. - **Authorization does not finish:** complete the browser flow opened by Cursor and choose an organization and environment. - **Wrong environment is connected:** remove the Altertable MCP connection and authorize it again with the correct environment. --- url: https://altertable.ai/docs/agents/setup.md --- # Connect your agent Give your AI client access to Altertable through MCP. Install the Altertable plugin where supported to include workflow skills alongside the connection. For a complete first-time setup, start with [agent onboarding](/docs/getting-started.md). ## Let your agent configure the connection If the agent can read documentation and run terminal commands, paste: ```text Read https://altertable.ai/docs/getting-started/prompt.md and help me connect this agent to Altertable. Stop after verifying the connection. ``` The setup prompt detects the client, preserves existing configuration, and verifies authentication. If your client cannot install its own integration, follow its guide below. ## Choose your client - [Claude](/docs/agents/setup/claude.md): Install the Altertable plugin in Claude, connect the hosted MCP server, and verify access to your data catalogs. - [ChatGPT](/docs/agents/setup/chatgpt.md): Install the Altertable plugin in ChatGPT, connect the hosted MCP server, and verify access to your data catalogs. - [Claude Code](/docs/agents/setup/claude-code.md): Install the Altertable plugin in Claude Code, load its bundled skills and MCP configuration, and verify access to your data catalogs. - [Codex](/docs/agents/setup/codex.md): Install the Altertable plugin from its Codex marketplace, connect the hosted MCP server, and verify access to your data catalogs. - [Cursor](/docs/agents/setup/cursor.md): Install the Altertable Cursor plugin, connect the hosted MCP server, and verify access to your data catalogs. - [Other agents](/docs/agents/setup/other-agents.md): Install Altertable Skills in any compatible agent without a native plugin, then add the hosted MCP server and verify catalog access. ## Skills, MCP, and CLI | Tool | Purpose | | ------ | ------------------------------------------------------------- | | Skills | Guide the agent through Altertable workflows and tool choices | | MCP | Give the client authenticated access to Altertable operations | | CLI | Run terminal commands, load local files, and automate scripts | MCP uses browser authorization. Choose the organization, environment, and access scope when connecting. See [MCP configuration](/docs/agents/mcp.md) for transport, OAuth, and read-only access. Install and authenticate the [CLI](/docs/lakehouse/clients/cli.md) separately when the agent needs terminal workflows. See [Use the CLI with agents](/docs/agents/cli.md) for structured output and profile checks. ## Skills included The [Altertable skills repository](https://github.com/altertable-ai/skills) packages the workflow instructions and client plugin manifests. Skills cover data exploration, ingestion, analysis, data apps, automation, and business context. Use the client's update mechanism to refresh the installed plugin or skills package. Keep the installed version aligned with the workflows you use. ## Verify access Ask the agent to initialize Altertable and list available catalogs. Complete browser authorization if prompted. Confirm that it sees the intended environment before asking it to query or change data. Continue with the [onboarding conversation](/docs/getting-started.md#set-up-with-your-agent) to connect your stack and answer a first question. --- url: https://altertable.ai/docs/agents/setup/other-agents.md --- # Set up other agents Use this guide when the agent does not have a dedicated setup guide for [Claude](/docs/agents/setup/claude.md), [ChatGPT](/docs/agents/setup/chatgpt.md), [Claude Code](/docs/agents/setup/claude-code.md), [Codex](/docs/agents/setup/codex.md), or [Cursor](/docs/agents/setup/cursor.md). The portable installer adds [Altertable Skills](/docs/agents/setup.md#skills-included) only. Add the hosted MCP server separately when the client needs live catalog and query access. ## Quick start 1. Open the project where you want to use Altertable. 2. Install the portable skills: ```bash npx skills add altertable-ai/skills ``` 3. Add a streamable HTTP MCP server named `altertable-mcp` at `https://mcp.altertable.ai/mcp`. See [MCP](/docs/agents/mcp.md) for transport and authorization details. 4. Start a new agent session so it discovers the skills and MCP server. 5. Ask the agent to analyze a real workflow: ```text Analyze my signup funnel from landing page to account creation. ``` The first Altertable tool call opens OAuth authorization in your browser. Choose the organization and environment the agent should access, then return to the client. ## Verify Ask the agent: ```text Use Altertable to list my available catalogs. ``` The agent should call `initialize` first and then inspect the catalogs. ## Tools - [Altertable Skills](/docs/agents/setup.md#skills-included) provide workflows for exploration, SQL, funnels, web traffic, insights, and platform operations. - [Altertable MCP](/docs/agents/mcp.md) provides governed access to catalogs, queries, insights, findings, and tasks. - The optional [Altertable CLI](/docs/lakehouse/clients/cli.md) supports shell automation, CI, ingestion, SQL, and structured agent output. ### Choose between Skills, MCP, and the CLI Use Skills for workflow guidance, MCP for interactive Altertable operations, and the CLI for terminal automation, CI, ingestion, or structured JSON. The CLI is optional and installed separately; the agent can use it alongside MCP. ## Update Re-run the installer when a new [Altertable Skills release](https://github.com/altertable-ai/skills/releases) is available: ```bash npx skills add altertable-ai/skills ``` ## Troubleshooting - **Skills do not appear:** start a new agent session after installation or update. - **MCP tools do not appear:** add `https://mcp.altertable.ai/mcp` in the client and reconnect. The skills installer does not register the server. - **Authorization does not finish:** complete the browser flow opened by the client and choose an organization and environment. - **Wrong environment is connected:** remove the Altertable MCP connection and authorize it again with the correct environment. --- url: https://altertable.ai/docs/api-references/cli.md --- # CLI command reference Use this reference for command syntax, options, arguments, and examples. > **Before using the Altertable CLI:** > > Install the CLI if you haven’t already: > > ```bash > curl -fsSL https://install.altertable.ai | sh > altertable login > ``` > > If `altertable` isn’t found after installation, open a new terminal. Run `altertable login` to log in to your organization and environment. Follow the [CLI setup guide](/docs/lakehouse/clients/cli.md#2-log-in) to configure credentials and verify access. CLI version: 1.13.3 ## Global options | Name | Type | Description | Details | | --------------------- | ------- | ----------------------------------------------------------------------------------------------------------- | ------------------- | | `--help`, `-h` | boolean | Show this help | Optional; global | | `--version`, `-v` | boolean | Show the Altertable CLI version | Optional; root-only | | `--debug`, `-d` | boolean | Enable debug output | Optional; global | | `--json` | boolean | Machine-readable JSON output | Optional; global | | `--agent` | boolean | Agent-friendly preset: structured JSON output, no pager or terminal styling | Optional; global | | `--no-agent` | boolean | Disable agent mode, including automatic detection | Optional; global | | `--no-color` | boolean | Disable terminal colors and styling | Optional; global | | `--profile` | string | Use a named profile for this command only | Optional; global | | `--connect-timeout` | string | HTTP connect timeout in seconds (default 30) | Optional; global | | `--read-timeout` | string | HTTP read timeout in seconds (default 60; 0 = no limit for streams) | Optional; global | | `--ignore-ssl-errors` | boolean | Skip TLS certificate verification for HTTPS requests (for development only, e.g. self-signed local servers) | Optional; global | ## Platform ### altertable login Sign in with your browser (OAuth) and store the session. ```bash altertable login [options] ``` #### Options | Name | Type | Description | Details | | ----------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | | `--control-plane-url` | string | Control-plane server root to log in against; saved to the profile only on success (the CLI appends /oauth and /rest/v1). Default: | Optional; command | | `--data-plane-url` | string | Data-plane base URL saved to the profile only on successful login. Default: | Optional; command | | `--allow-insecure-http` | boolean | Allow http\:// URLs other than localhost for --control-plane-url (for development only) | Optional; command | | `--replace-profile` | boolean | Store the login session in the current profile instead of switching profiles | Optional; command | | `--org` | string | Preselect this organization in the browser authorization page | Optional; command; Value: SLUG | | `--env` | string | Preselect this environment in the browser authorization page | Optional; command; Value: SLUG | | `--service-account` | string | Create an environment-scoped service account with this label and store its access token instead of your login session | Optional; command; Value: LABEL | | `--scope` | string | Request a service account limited to `ro` or `rw` on every catalog, or to specific catalogs, e.g. `analytics:ro,staging:rw` (requires --service-account). The catalog access the server actually grants is reported on success. | Optional; command; Value: MODE\|CATALOG:MODE,... | #### Examples ```bash altertable login ``` ```bash altertable login --org acme --env staging ``` ```bash altertable login --replace-profile ``` ```bash altertable login --service-account "CI Bot" --scope analytics:ro ``` ```bash altertable login --service-account "CI Bot" --scope ro ``` ### altertable logout Remove stored credentials and settings for all profiles. ```bash altertable logout [options] ``` #### Options | Name | Type | Description | Details | | ------------------ | ------- | ------------------------------------------------------- | ----------------- | | `--except-current` | boolean | Keep the current profile and remove every other profile | Optional; command | #### Examples ```bash altertable logout ``` ```bash altertable logout --except-current ``` ### altertable profile Manage named profiles and stored credentials. ```bash altertable profile configure|list|show|status|switch|current|env|delete ``` #### Examples ```bash altertable profile show ``` ```bash altertable profile configure ``` ```bash altertable profile configure acme_production --api-key atm_xxx --env production ``` ```bash altertable profile configure --scope lakehouse ``` ```bash altertable profile list ``` ```bash altertable profile switch ``` ```bash altertable profile status ``` ```bash eval "$(altertable profile env acme_staging)" ``` ```bash altertable --profile acme_staging profile show ``` ### altertable profile configure Create or update a profile's credentials and settings ```bash altertable profile configure [options] [NAME] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | -------------------------------------------------- | ----------------- | | `` | positional | Profile name (default: selected or active profile) | Optional; command | #### Options | Name | Type | Description | Details | | ----------------------- | ------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------ | | `--user` | string | Lakehouse username (global) | Optional; command | | `--password` | string | Lakehouse password (global) | Optional; command | | `--basic-token` | string | Pre-encoded HTTP Basic token | Optional; command | | `--api-key` | string | Management API key for the --env environment | Optional; command | | `--env` | string | Target environment (management API keys are per-environment) | Optional; command | | `--password-stdin` | boolean | Read the lakehouse password from stdin | Optional; command | | `--api-key-stdin` | boolean | Read the management API key from stdin | Optional; command | | `--data-plane-url` | string | Data-plane base URL; can be set alone or with a credential (default: ) | Optional; command | | `--control-plane-url` | string | Control-plane server root; the CLI appends /rest/v1 (default: ) | Optional; command | | `--allow-insecure-http` | boolean | Allow http\:// URLs other than localhost (not recommended) | Optional; command | | `--scope` | enum | Limit the interactive wizard to one plane (default: both) | Optional; command; Values: management, lakehouse | #### Examples ```bash altertable profile configure ``` ```bash altertable profile configure acme_production --api-key atm_xxx --env production ``` ```bash altertable profile configure acme_staging --user alice --password-stdin ``` ```bash altertable profile configure --scope lakehouse ``` ### altertable profile list List configured profiles ```bash altertable profile list ``` ### altertable profile show Show a profile's stored identity, auth, and endpoints ```bash altertable profile show [NAME] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | -------------------------------------------------- | ----------------- | | `` | positional | Profile name (default: selected or active profile) | Optional; command | ### altertable profile status Verify stored credentials and show the profile ```bash altertable profile status [NAME] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | -------------------------------------------------- | ----------------- | | `` | positional | Profile name (default: selected or active profile) | Optional; command | ### altertable profile switch Interactively switch the active profile ```bash altertable profile switch [NAME] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | ------------ | ----------------- | | `` | positional | Profile name | Optional; command | ### altertable profile current Show the active profile name ```bash altertable profile current ``` ### altertable profile env Print shell exports for a profile ```bash altertable profile env [NAME] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | -------------------------------------- | ----------------- | | `` | positional | Profile name (default: active profile) | Optional; command | ### altertable profile delete Delete a profile ```bash altertable profile delete [options] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | ------------ | ----------------- | | `` | positional | Profile name | Required; command | #### Options | Name | Type | Description | Details | | ------- | ------- | ---------------- | ----------------- | | `--yes` | boolean | Confirm deletion | Optional; command | ### altertable catalog Manage catalogs (databases and connections) in the current environment. ```bash altertable catalog ``` ```bash altertable catalog create ``` #### Examples ```bash altertable catalog ``` ```bash altertable catalog create Analytics ``` ### altertable catalog create Create an Altertable catalog. ```bash altertable catalog create ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | ------------ | ----------------- | | `` | positional | Catalog name | Required; command | #### Examples ```bash altertable catalog create Analytics ``` ### altertable api Management REST API — HTTP invoker and OpenAPI spec. ```bash altertable api [options] ``` ```bash altertable api route|spec ``` #### Arguments | Name | Type | Description | Details | | ------------ | ---------- | --------------------------------- | ----------------- | | `` | positional | Path under /rest/v1, e.g. /whoami | Required; command | #### Options | Name | Type | Description | Details | | ------------------- | ------ | --------------------------------------------------------------------------- | -------------------------------------------------------- | | `--method`, `-X` | enum | HTTP method override (default GET, or POST when fields/input are provided) | Optional; command; Values: GET, POST, PATCH, DELETE, PUT | | `--raw-field`, `-f` | string | String request parameter key=value (repeatable; gh api -f semantics) | Optional; Repeatable; command | | `--field`, `-F` | string | Typed request parameter key=value (true, false, null, integers; repeatable) | Optional; Repeatable; command | | `--input` | string | File to use as the request body; use - for stdin | Optional; command | | `--env` | string | Replace {environment\_id} in the path | Optional; command | | `--format` | enum | Serialized output format; use global --json for JSON | Optional; command; Values: csv, markdown | #### Examples ```bash altertable api /whoami ``` ```bash altertable api route ``` ```bash altertable api /environments/production/connections ``` ```bash altertable api /service_accounts -X POST -F label="CI Bot" ``` ### altertable api route List management API paths and methods from the bundled OpenAPI spec. ```bash altertable api route [OPERATION] ``` #### Arguments | Name | Type | Description | Details | | ------------- | ---------- | ---------------------------------------------------- | ----------------- | | `` | positional | Optional operationId to inspect, e.g. createDatabase | Optional; command | #### Examples ```bash altertable api route ``` ```bash altertable api route createDatabase ``` ### altertable api spec Print the bundled management OpenAPI specification (YAML by default; JSON with --json). ```bash altertable api spec ``` #### Examples ```bash altertable api spec ``` ```bash altertable api spec --json ``` ### altertable doctor Diagnose local configuration and Altertable connectivity. ```bash altertable doctor [options] ``` #### Options | Name | Type | Description | Details | | ----------- | ------- | --------------------------------------------------------------- | ----------------- | | `--offline` | boolean | Inspect local configuration without contacting Altertable APIs. | Optional; command | #### Examples ```bash altertable doctor ``` ```bash altertable doctor --offline ``` ```bash altertable --json doctor ``` ### altertable app Create, develop, lint, build, and upgrade local data apps; validate, publish, and update data apps. ```bash altertable app create|dev|build|lint|upgrade|validate|publish|update ``` #### Examples ```bash altertable app create my-app ``` ```bash altertable app dev ``` ```bash altertable app dev --file index.tsx ``` ```bash altertable app lint ``` ```bash altertable app build ``` ```bash altertable app upgrade ``` ```bash altertable app validate --file index.tsx ``` ```bash altertable app publish --title 'Revenue explorer' --file index.tsx ``` ```bash altertable app update APP-1 --file index.tsx ``` ### altertable app create Create a local data app project with a live connection check. ```bash altertable app create [options] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | ---------------------- | ----------------- | | `` | positional | App name in kebab-case | Required; command | #### Options | Name | Type | Description | Details | | ------------------- | ------- | -------------------------------------------------------------------------- | ----------------- | | `--dir` | string | Destination directory (default: ./\). | Optional; command | | `--without-profile` | boolean | Create an offline scaffold with organization and environment placeholders. | Optional; command | #### Examples ```bash altertable app create product-pulse ``` ```bash altertable app create product-pulse --dir ./apps/product-pulse ``` ```bash altertable --profile production app create product-pulse ``` ### altertable app dev Preview a local data app directory, or a single-file data app, with lakehouse access. ```bash altertable app dev [options] ``` #### Options | Name | Type | Description | Details | | -------- | ------ | --------------------------------------------------------------------------------------------------------------------------------- | ----------------- | | `--dir` | string | App directory (default: current directory). | Optional; command | | `--file` | string | Path to a single-file React data app (index.tsx) to preview instead of --dir. Queries are declared in the file via defineDataApp. | Optional; command | | `--port` | string | Local dev server port (1–65535; default: app setting, or 25837 for --file). | Optional; command | #### Examples ```bash altertable app dev ``` ```bash altertable app dev --port 3022 ``` ```bash altertable --profile staging app dev --dir ./my-app ``` ```bash altertable app dev --file ./index.tsx ``` ### altertable app build Typecheck and build a local data app without Altertable credentials. ```bash altertable app build [options] ``` #### Options | Name | Type | Description | Details | | ------- | ------ | ------------------------------------------- | ----------------- | | `--dir` | string | App directory (default: current directory). | Optional; command | #### Examples ```bash altertable app build ``` ```bash altertable app build --dir ./my-app ``` ### altertable app lint Lint a local data app project: format, lint rules, types, contracts, build, and client credential boundary. ```bash altertable app lint [options] ``` #### Options | Name | Type | Description | Details | | ------------- | ------- | -------------------------------------------------------------------- | ----------------- | | `--dir` | string | App directory (default: current directory). | Optional; command | | `--lakehouse` | boolean | Run each operation’s declared checks against the selected lakehouse. | Optional; command | #### Examples ```bash altertable app lint ``` ```bash altertable app lint --lakehouse ``` ### altertable app upgrade Update a local data app's package to the CLI's tested version. ```bash altertable app upgrade [options] ``` #### Options | Name | Type | Description | Details | | ------- | ------ | ------------------------------------------- | ----------------- | | `--dir` | string | App directory (default: current directory). | Optional; command | #### Examples ```bash altertable app upgrade ``` ```bash altertable app upgrade --dir ./my-app ``` ### altertable app validate Type-check and bundle a data app from source without saving. ```bash altertable app validate [options] ``` #### Options | Name | Type | Description | Details | | -------- | ------ | ----------------------------------------------------------------------------------------------------- | ----------------- | | `--file` | string | Path to the single-file React source (index.tsx). Queries are declared in the file via defineDataApp. | Required; command | #### Examples ```bash altertable app validate --file index.tsx ``` ### altertable app publish Create a data app from source. ```bash altertable app publish [options] ``` #### Options | Name | Type | Description | Details | | --------------- | ------ | ----------------------------------------------------------------------------------------------------- | ----------------- | | `--title` | string | Title of the data app. | Required; command | | `--file` | string | Path to the single-file React source (index.tsx). Queries are declared in the file via defineDataApp. | Required; command | | `--description` | string | Optional description of the data app. | Optional; command | #### Examples ```bash altertable app publish --title 'Revenue explorer' --file index.tsx ``` ### altertable app update Update a data app from source. Omitted fields are left unchanged. ```bash altertable app update [options] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | ------------------------------------- | ----------------- | | `` | positional | Data app slug to update (e.g. APP-1). | Required; command | #### Options | Name | Type | Description | Details | | --------------- | ------ | ------------------------------------------------------------------------------------------------------------ | ----------------- | | `--title` | string | New title of the data app. | Optional; command | | `--file` | string | Path to replacement single-file React source (index.tsx). Replaces the queries with those the file declares. | Optional; command | | `--description` | string | New description of the data app. | Optional; command | #### Examples ```bash altertable app update APP-1 --file index.tsx ``` ```bash altertable app update APP-1 --title 'Revenue explorer' --file index.tsx ``` ### altertable update Update Altertable CLI to the latest release. Aliases: upgrade ```bash altertable update [options] [VERSION] ``` #### Arguments | Name | Type | Description | Details | | ----------- | ---------- | ---------------------------------------------- | ----------------- | | `` | positional | Specific version to install (default: latest). | Optional; command | #### Options | Name | Type | Description | Details | | --------- | ------- | ----------------------------------------------- | ----------------- | | `--check` | boolean | Check for an update without installing it. | Optional; command | | `--force` | boolean | Reinstall or downgrade to the selected release. | Optional; command | #### Examples ```bash altertable update ``` ```bash altertable update --check ``` ```bash altertable update 1.2.0 --force ``` ### altertable completion Generate or install shell completion scripts. ```bash altertable completion ``` ```bash altertable completion generate|install ``` #### Examples ```bash altertable completion install ``` ```bash altertable completion install zsh ``` ```bash altertable completion generate bash > ~/.local/share/bash-completion/completions/altertable ``` ### altertable completion generate Generate a shell completion script. ```bash altertable completion generate ``` #### Arguments | Name | Type | Description | Details | | --------- | ---------- | -------------------------------- | ------------------------------------------ | | `` | positional | Shell to generate completion for | Required; command; Values: bash, fish, zsh | #### Examples ```bash altertable completion generate bash ``` ```bash altertable completion generate zsh > ~/.local/share/zsh/site-functions/_altertable ``` ### altertable completion install Install shell completion for the current shell. ```bash altertable completion install [options] [BASH|FISH|ZSH] ``` #### Arguments | Name | Type | Description | Details | | --------- | ---------- | --------------------------------------------------------- | ------------------------------------------ | | `` | positional | Shell to install completion for (default: detected shell) | Optional; command; Values: bash, fish, zsh | #### Options | Name | Type | Description | Details | | --------- | ------- | --------------------------------------------------------------- | ----------------- | | `--no-rc` | boolean | Write the completion file without updating shell startup files. | Optional; command | #### Examples ```bash altertable completion install ``` ```bash altertable completion install fish ``` ```bash altertable completion install zsh --no-rc ``` ## Ingest ### altertable append Append JSON rows to a table. ```bash altertable append [options] ``` ```bash altertable append status ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | ------------------------------------------- | ----------------- | | `` | positional | JSON object, array, or @file (@- for stdin) | Required; command | #### Options | Name | Type | Description | Details | | -------- | ------- | -------------------------------------------------------- | ----------------- | | `--to` | string | Destination as catalog.schema.table | Required; command | | `--sync` | boolean | Wait for the append operation to finish before returning | Optional; command | #### Examples ```bash altertable append '{"event":"checkout_completed","user_id":"usr_123","revenue":99}' --to analytics.main.events ``` ```bash altertable append status ``` ### altertable append status Fetch status for an append operation. ```bash altertable append status ``` #### Arguments | Name | Type | Description | Details | | ------------- | ---------- | ---------------------------- | ----------------- | | `` | positional | Append id returned by append | Required; command | ### altertable upload Upload a file to create, append to, or overwrite a table. ```bash altertable upload [options] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | -------------------- | ----------------- | | `` | positional | Local file to upload | Required; command | #### Options | Name | Type | Description | Details | | ---------- | ------ | ----------------------------------------------------- | ----------------------------------------------------------------------- | | `--to` | string | Destination as catalog.schema.table | Required; command | | `--format` | enum | Input format; inferred from the filename when omitted | Optional; command; Values: csv, json, parquet | | `--mode` | enum | create, append, or overwrite | Optional; command; Values: create, append, overwrite; Default: "create" | #### Examples ```bash altertable upload users.csv --to analytics.main.users ``` ```bash altertable upload orders.parquet --to analytics.main.orders --mode overwrite ``` ### altertable upsert Upload a file and match existing rows by primary key. ```bash altertable upsert [options] ``` #### Arguments | Name | Type | Description | Details | | -------- | ---------- | -------------------- | ----------------- | | `` | positional | Local file to upload | Required; command | #### Options | Name | Type | Description | Details | | ---------- | ------ | ----------------------------------------------------- | --------------------------------------------- | | `--to` | string | Destination as catalog.schema.table | Required; command | | `--format` | enum | Input format; inferred from the filename when omitted | Optional; command; Values: csv, json, parquet | | `--key` | string | Column name used to match existing rows | Required; command | #### Examples ```bash altertable upsert users.csv --to analytics.main.users --key id ``` ## Query ### altertable query Run SQL queries against the lakehouse. ```bash altertable query [options] ``` ```bash altertable query show|cancel ``` #### Arguments | Name | Type | Description | Details | | ------------- | ---------- | --------------------------------------------- | ----------------- | | `` | positional | SQL statement to run, or @file (@- for stdin) | Required; command | #### Options | Name | Type | Description | Details | | ---------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | | `--format` | enum | Result format: csv, jsonl, and parquet stream from the API; markdown is rendered by the CLI. Use global --json for structured JSON | Optional; command; Values: csv, jsonl, parquet, markdown | | `--layout` | enum | Human layout: auto, table, or line | Optional; command; Values: auto, table, line | | `--columns` | string | Comma-separated columns to show | Optional; command | | `--max-width` | string | Maximum display width for table columns | Optional; command; Default: "32" | | `--compute-size` | string | Compute size for the query | Optional; command; Default: "AUTO" | | `--dialect` | string | Source SQL dialect to transpile from (server default: DuckDB) | Optional; command | | `--catalog` | string | Catalog name (optional; can also come from the session) | Optional; command | | `--schema` | string | Schema name (optional; can also come from the session) | Optional; command | | `--output` | string | Write result bytes/text to this path instead of stdout | Optional; command | | `--query-id` | string | Optional stable query id | Optional; command | | `--session-id` | string | Optional session id | Optional; command | | `--pager` | enum | Pager mode for human output: auto, always, or never | Optional; command; Values: auto, always, never; Default: "auto" | #### Examples ```bash altertable query "SELECT id, email, plan FROM analytics.main.users LIMIT 10" ``` ```bash altertable query "SELECT event, timestamp FROM analytics.main.events ORDER BY timestamp DESC LIMIT 10" --json ``` ```bash altertable query "SELECT 1" --format csv --output results.csv ``` ```bash altertable query @report.sql --format csv --output results.csv ``` ```bash altertable query show ``` ### altertable query show Fetch metadata for a completed or running query. ```bash altertable query show ``` #### Arguments | Name | Type | Description | Details | | ------------ | ---------- | ---------------------------- | ----------------- | | `` | positional | Query id returned by the API | Required; command | ### altertable query cancel Cancel a running query. ```bash altertable query cancel [options] ``` #### Arguments | Name | Type | Description | Details | | ------------ | ---------- | ------------------ | ----------------- | | `` | positional | Query id to cancel | Required; command | #### Options | Name | Type | Description | Details | | -------------- | ------ | ------------------------------ | ----------------- | | `--session-id` | string | Session id that owns the query | Required; command | ### altertable schema List schemas, tables, and columns for a catalog. ```bash altertable schema [options] ``` #### Arguments | Name | Type | Description | Details | | ----------- | ---------- | ------------ | ----------------- | | `` | positional | Catalog name | Required; command | #### Options | Name | Type | Description | Details | | ---------- | ---- | ---------------------------------------------------- | --------------------------------------------------------------- | | `--format` | enum | Serialized output format; use global --json for JSON | Optional; command; Values: csv, markdown | | `--pager` | enum | Pager mode for human output: auto, always, or never | Optional; command; Values: auto, always, never; Default: "auto" | #### Examples ```bash altertable schema analytics ``` ```bash altertable schema analytics --json ``` ### altertable duckdb Open a DuckDB shell attached to lakehouse catalogs (all of them by default). ```bash altertable duckdb [CATALOG] ``` #### Arguments | Name | Type | Description | Details | | ----------- | ---------- | -------------------------------------------- | ----------------- | | `` | positional | Catalog to attach (defaults to all catalogs) | Optional; command | #### Examples ```bash altertable duckdb ``` ```bash altertable duckdb analytics ``` --- url: https://altertable.ai/docs/api-references/glossary.md --- # Glossary | Term | Meaning | | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | [Organization](/docs/administration.md#organization) | Members, billing, and administration shared across environments. | | [Environment](/docs/administration.md#environment) | An isolated scope with its own catalogs, credentials, compute allocation, and analyses. | | [Catalog](/docs/lakehouse/catalogs.md) | A namespace of schemas and tables. Managed catalogs store ingested data. External catalogs expose connected sources. | | [Worker](/docs/lakehouse/compute/workers.md) | Compute that executes queries and caches data. | | [Bucket](/docs/lakehouse/storage/buckets.md) | Object storage for catalog files, separate from a scheduled file import. | | [Ask Altertable](/docs/agents/ask.md) | The conversational agent. Some app controls call it **Ask Agent**. | | [Ask Altertable in Slack](/docs/agents/ask.md#use-in-slack) | The same agent accessed through the organization's Slack integration. | | [Cousteau](/docs/automation/cousteau.md) | Autonomous product investigations. | | [Data app](/docs/data-apps.md) | An interactive application built over your data. Use for new reporting. | | [Insight](/docs/data-apps/insights.md) | A saved query or chart. Existing dashboards combine insights into layouts. | | [Lakehouse credentials](/docs/administration/lakehouse-credentials.md) | An environment's username and password for SQL and ingestion clients. | | [Product Analytics API key](/docs/product-analytics/authentication.md) | A key for Product Analytics event and identity ingestion. | | [`altertable-mcp`](/docs/agents/mcp.md) | The default MCP server name in the agent plugin. A manually added connection can use another name. | ## MCP access names Each access tier has a display name, a pinned endpoint, and an OAuth scope: | Access | Endpoint | OAuth scope | | ---------------------------------------------- | ------------------------------------------ | ------------- | | [Read-only](/docs/agents/mcp.md#pin-a-scope) | `https://mcp.altertable.ai/mcp/read-only` | `mcp:ro` | | [Analyst](/docs/agents/mcp.md#pin-a-scope) | `https://mcp.altertable.ai/mcp/analyst` | `mcp:analyst` | | [Full access](/docs/agents/mcp.md#pin-a-scope) | `https://mcp.altertable.ai/mcp/read-write` | `mcp:rw` | Resource permissions still apply. Use the matching scope for a pinned endpoint. See [MCP setup](/docs/agents/mcp.md#pin-a-scope). --- url: https://altertable.ai/docs/api-references.md --- # Reference Use these references when you need endpoint details, authentication headers, request payloads, or generated tool schemas. For terminal workflows, use the [CLI](/docs/lakehouse/clients/cli.md) to get started, then consult the [CLI reference](/docs/api-references/cli.md) for commands and options. - [Lakehouse API](/docs/api-references/lakehouse.md): Reference for the Altertable HTTP API, including authentication and endpoints for query, file upload, upserts, append, validation, and query lifecycle. - [Product Analytics API](/docs/api-references/product-analytics.md): Product Analytics HTTP API reference for tracking events, identifying users, and authenticating HTTP requests from your applications. - [OpenTelemetry API](/docs/api-references/opentelemetry.md): OTLP/HTTP reference for logs, traces, and metrics, including authentication, encodings, compression, and response statuses. - [MCP tools](/docs/api-references/mcp-tools.md): Full reference of MCP tools available when connected to Altertable. Covers orientation, schema exploration, querying, previewing, and write actions. - [CLI command reference](/docs/api-references/cli.md): Browse every Altertable CLI command, global option, argument, and example, with stable links for terminal workflows and automation. - [SDKs](/docs/api-references/sdks.md): Find the Altertable client libraries for lakehouse queries, ingestion, Product Analytics, and data apps, with links to setup guides and source repositories. - [Glossary](/docs/api-references/glossary.md): Understand Altertable names for catalogs, environments, agents, credentials, compute, storage, and MCP access labels, endpoints, and OAuth scopes. --- url: https://altertable.ai/docs/api-references/lakehouse.md --- # Lakehouse API ## Quick reference - **Base URL** https\://api.altertable.ai - **Authentication** [HTTP Basic Auth](#authentication) - **Terminal workflows** [Use the CLI](/docs/lakehouse/clients/cli.md) * [Query with SQL](#post-query) * [Estimate a query](#post-explain) * [Upload a file](#post-upload) * [Upsert by primary key](#post-upsert) Use the Altertable HTTP API to run SQL, estimate query scans, upload files, upsert by primary key, append event-style JSON records, validate SQL, and request autocomplete suggestions without opening a database protocol connection. If you are working from a terminal, the [CLI](/docs/lakehouse/clients/cli.md) wraps the query, upload, upsert, and append endpoints with profile-based authentication. [Download the deployed OpenAPI contract](https://api.altertable.ai/openapi/lakehouse.json) for machine-readable request and response schemas. ## Authentication Authenticate every request with HTTP Basic Auth using your environment’s [lakehouse credentials](/docs/administration/lakehouse-credentials.md). Base64-encode `lakehouse_username:lakehouse_password` and store the result for the request examples below: ```bash ALTERTABLE_BASIC_AUTH_TOKEN=$(printf '%s' "{{LAKEHOUSE_USERNAME}}:{{LAKEHOUSE_PASSWORD}}" | base64 | tr -d '\n') ``` Then send the encoded value in the `Authorization` header: ```http Authorization: Basic ``` The [Python and Ruby SDKs](/docs/api-references/sdks.md) construct the header from your lakehouse username and password during client initialization. ## POST `/query` Execute SQL and stream the result as JSON Lines. **Authentication:** HTTP Basic Auth\ **Permission:** Query access to referenced catalogs | Parameter | Type | Location | Required | Description | | -------------- | ---------------------------------------------------- | -------- | -------- | ----------------------------------------------------------------------------------------------------- | | `statement` | string | body | Yes | The SQL statement to execute. | | `catalog` | string | body | No | Catalog to use for unqualified table names. | | `schema` | string | body | No | Schema to use for unqualified table names. | | `session_id` | string | body | No | Reuse an existing query session. | | `timezone` | string | body | No | IANA timezone for the session. Defaults to UTC. | | `compute_size` | XS \| S \| M \| L \| XL \| 2XL \| 3XL \| 4XL \| AUTO | body | No | Choose a compute tier or infer one from the SQL. AUTO cannot be combined with session\_id. | | `dialect` | string | body | No | Source SQL dialect to transpile to DuckDB before execution. | | `limit` | integer | body | No | Limit the number of returned rows. | | `offset` | integer | body | No | Skip this many rows before returning results. | | `format` | default \| csv \| jsonl \| parquet | body | No | Choose the streamed output format. Defaults to the Altertable metadata, column names, and row stream. | | `ephemeral` | boolean | body | No | Create an ephemeral session. Defaults to false. | | `sanitize` | boolean | body | No | Automatically limit returned rows. Defaults to false. | | `params` | object \| array | body | No | Named ($name) or positional ($1) bind values. Values must be JSON scalars. | | `requested_by` | string | body | No | Caller identifier recorded in query logs. | | `query_id` | UUID | body | No | Assign a query identifier instead of generating one. | ### Response format The default response uses **JSONL** (JSON Lines) for efficient streaming: 1. **First line**: query metadata, including the session, query, and Worker identifiers 2. **Second line**: an array of column names 3. **Remaining lines**: one JSON array per result row, in column order Set `format` to `csv`, `jsonl`, or `parquet` when you need a standard data stream without the Altertable metadata and column-name lines. The `jsonl` format emits one JSON object per row. ### Response statuses | Status | Description | | ------ | --------------------------------------------------------------------- | | 200 | Execute a SQL query and return streaming data in the requested format | | 400 | Invalid SQL query or sanitization failed | | 401 | Authentication required | ### Request ```curl curl -X POST https://api.altertable.ai/query \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ -H "Content-Type: application/json" \ -d '{"statement":"SELECT event, COUNT(*) AS count FROM product_analytics.main.events GROUP BY event"}' ``` ### Response (200 OK) ```json {"statement":"SELECT event, COUNT(*) AS count FROM product_analytics.main.events GROUP BY event","rows_limit":null,"rows_offset":null,"init_time_ms":0,"session_id":"123e4567-e89b-12d3-a456-426614174000","query_id":"123e4567-e89b-12d3-a456-426614174000","worker_slug":"worker-abc","connections_errors":{}} ["event","count"] ["signup",42] ["login",123] ``` ## POST `/upload` Upload a CSV, JSON, or Parquet file to create, append to, or replace a table. **Authentication:** HTTP Basic Auth\ **Permission:** Write access to the destination catalog | Parameter | Type | Location | Required | Description | | --------- | ----------------------------------------------- | -------- | -------- | ---------------------------------------------------- | | `catalog` | string | query | Yes | Catalog to ingest data into. | | `schema` | string | query | Yes | Schema within the catalog. | | `table` | string | query | Yes | Table to create or insert into. | | `mode` | create \| append \| create\_append \| overwrite | query | Yes | How the uploaded data changes the destination table. | ### Request body Binary file data. The server infers CSV, JSON, or Parquet from the `Content-Type` header when present, otherwise from magic bytes in the payload: - **CSV**: Comma-separated values with header row - **JSON**: JSON array of objects or JSONL (one JSON object per line) - **Parquet**: Apache Parquet columnar format (most efficient) ### Ingestion modes - **`create`**: Create a new table with the uploaded data (fails if the table already exists) - **`append`**: Append the uploaded data to an existing table (preserves existing data) - **`create_append`**: Create the table if missing; otherwise append to it - **`overwrite`**: Drop the existing table and recreate it with the uploaded data (replaces all data) Request bodies can contain up to **100 GiB**. ### Response statuses | Status | Description | | ------ | -------------------------------------- | | 200 | File uploaded successfully | | 400 | Invalid request (e.g., malformed file) | | 401 | Authentication required | ### Request #### Create a CSV table ```curl curl "https://api.altertable.ai/upload?catalog=my_catalog&schema=main&table=users&mode=create" \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ --data-binary @users.csv ``` #### Append JSON to a table ```curl curl "https://api.altertable.ai/upload?catalog=my_catalog&schema=main&table=events&mode=append" \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ --data-binary @events.json ``` ### Response (200 OK) ```http HTTP/1.1 200 OK ``` ## POST `/upsert` Merge uploaded CSV, JSON, or Parquet rows into a table by primary key. **Authentication:** HTTP Basic Auth\ **Permission:** Write access to the destination catalog | Parameter | Type | Location | Required | Description | | -------------- | ------ | -------- | -------- | ------------------------------------------------------------------------------------------------------ | | `catalog` | string | query | Yes | Catalog to ingest data into. | | `schema` | string | query | Yes | Schema within the catalog. | | `table` | string | query | Yes | Table to merge rows into. | | `primary_key` | string | query | Yes | Column or comma-separated columns used to match existing rows. | | `cursor_field` | string | query | No | Cursor column or comma-separated columns. Higher values win on primary-key collisions; NULL is lowest. | The request body contains binary CSV, JSON, or Parquet data and can be up to **100 GiB**. Matching rows are updated and new rows are inserted. ### Response statuses | Status | Description | | ------ | -------------------------------------------------------------- | | 200 | File upserted successfully | | 400 | Invalid request (e.g., malformed file or missing primary\_key) | | 401 | Authentication required | ### Request ```curl curl "https://api.altertable.ai/upsert?catalog=my_catalog&schema=main&table=users&primary_key=id" \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ --data-binary @updates.parquet ``` ### Response (200 OK) ```http HTTP/1.1 200 OK ``` ## POST `/append` Append one JSON object or a batch of objects without managing pipelines or offsets. **Authentication:** HTTP Basic Auth\ **Permission:** Write access to the destination catalog | Parameter | Type | Location | Required | Description | | --------- | ------- | -------- | -------- | ----------------------------------------------- | | `catalog` | string | query | Yes | Catalog to append data to. | | `schema` | string | query | Yes | Schema within the catalog. | | `table` | string | query | Yes | Table to append data to. | | `sync` | boolean | query | No | Wait for processing to finish before returning. | The request body can be one JSON object or an array of objects, up to **32 MiB**. For larger files, use `/upload` with `mode=append`. Use `task_id` with [`GET /tasks/{task_id}`](/docs/api-references/lakehouse.md#get-tasks-task-uuid) to poll asynchronous processing. When `sync=true`, the response waits until the append task finishes. The response confirms that Altertable accepted the records for asynchronous processing. Rows that still fail after retries are written to `altertable.main.quarantined_appends` with the destination catalog, schema, table, error, original payload, and retry count. ### Automatic table management - Infer column types from JSON values - Create the table when it does not exist - Add columns when later payloads contain new fields - Make accepted data queryable within seconds Numbers become `BIGINT` or `DOUBLE`, strings become `VARCHAR`, booleans become `BOOLEAN`, arrays and objects become `JSON`, and RFC3339 timestamps become `TIMESTAMP`. You can also create the table manually when you need explicit types, constraints, or defaults. Append respects the existing schema and only adds missing columns. ### Response statuses | Status | Description | | ------ | ------------------------------------------- | | 200 | Append data to a table | | 400 | Invalid request | | 401 | Authentication required | | 403 | Catalog not found, or no write access to it | ### Response fields | Field | Type | Description | Details | | --------------- | ----------------------- | ----------------------------------------------------------------------------------------------- | -------- | | `error_code` | null \| AppendErrorCode | | Optional | | `error_message` | string \| null | Human-readable error message if the data was not successfully appended | Optional | | `ok` | boolean | Indicates whether the data was successfully enqueued for processing | Required | | `task_id` | string \| null | Task ID that can be used to track the processing status of this batch via GET /tasks/{task\_id} | Optional | ### Request ```curl curl "https://api.altertable.ai/append?catalog=my_catalog&schema=main&table=events" \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ -H "Content-Type: application/json" \ -d '{"event":"signup","user_id":123}' ``` ### Response (200 OK) ```json { "ok": true, "task_id": "019d68d4-c4cb-7e40-ad50-fd4b68c2045e", "error_code": null } ``` ## POST `/explain` Estimate the files, bytes, and rows a SQL statement will scan without running it. **Authentication:** HTTP Basic Auth\ **Permission:** Query access to referenced catalogs | Parameter | Type | Location | Required | Description | | -------------- | ------- | -------- | -------- | --------------------------------------------- | | `statement` | string | body | Yes | SQL statement to estimate. | | `catalog` | string | body | No | Catalog to use for unqualified table names. | | `schema` | string | body | No | Schema to use for unqualified table names. | | `session_id` | string | body | No | Existing session to use for the estimate. | | `include_plan` | boolean | body | No | Include the raw EXPLAIN plan in the response. | The response reports per-table row estimates and, when available, file and byte estimates after filter pruning. It also returns totals across all scanned tables and connection errors for catalogs that could not be attached. Each table uses the [TableScanEstimate](#tablescanestimate) response type. ### Response statuses | Status | Description | | ------ | ------------------------------------ | | 200 | Explain result for the SQL statement | | 400 | Invalid request parameters | | 401 | Authentication required | ### Response fields | Field | Type | Description | Details | | ------------------------ | -------------------- | -------------------------------------------------------------- | -------- | | `connections_errors` | object | Connection errors for attached catalogs | Required | | `error` | string \| null | Error message if the explain failed | Optional | | `plan` | array | Raw EXPLAIN JSON plan (wrapped in array for API compatibility) | Optional | | `scanned_bytes_estimate` | integer \| null | Estimated bytes to scan after filter pruning | Optional | | `scanned_files_estimate` | integer \| null | Estimated files to scan after filter pruning | Optional | | `statement` | string | The statement that was explained | Required | | `tables` | TableScanEstimate\[] | Per-table scan estimates | Required | | `total_bytes` | integer \| null | Total bytes across all table scans | Optional | | `total_files` | integer \| null | Total files across all table scans | Optional | ### Request ```curl curl https://api.altertable.ai/explain \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ -H "Content-Type: application/json" \ -d '{"statement":"SELECT * FROM my_catalog.main.events WHERE timestamp >= current_date - 7","include_plan":false}' ``` ### Response (200 OK) ```json { "connections_errors": {}, "statement": "SELECT * FROM my_catalog.main.events WHERE timestamp >= current_date - 7", "tables": [] } ``` ## POST `/validate` Check a SQL statement for syntax and semantic issues without executing it. **Authentication:** HTTP Basic Auth\ **Permission:** Query access to referenced catalogs | Parameter | Type | Location | Required | Description | | ------------ | ------ | -------- | -------- | --------------------------------------- | | `statement` | string | body | Yes | SQL statement to validate. | | `catalog` | string | body | No | Catalog to use for validation. | | `schema` | string | body | No | Schema to use for validation. | | `session_id` | string | body | No | Existing session to use for validation. | ### Response statuses | Status | Description | | ------ | --------------------------------------- | | 200 | Validation result for the SQL statement | | 400 | Invalid request parameters | | 401 | Authentication required | ### Response fields | Field | Type | Description | Details | | -------------------- | -------------- | ----------------------------------------- | -------- | | `connections_errors` | object | Connection errors for attached catalogs | Required | | `error` | string \| null | Error message if the statement is invalid | Optional | | `statement` | string | The statement that was validated | Required | | `valid` | boolean | Whether the SQL statement is valid | Required | ### Request ```curl curl https://api.altertable.ai/validate \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ -H "Content-Type: application/json" \ -d '{"statement":"SELECT 1 AS connected"}' ``` ### Response (200 OK) ```json { "connections_errors": {}, "statement": "SELECT 1 AS connected", "valid": true } ``` ## POST `/autocomplete` Return ranked SQL completions for a partial statement. **Authentication:** HTTP Basic Auth\ **Permission:** Query access to referenced catalogs | Parameter | Type | Location | Required | Description | | ----------------- | ------- | -------- | -------- | -------------------------------------------------------------------- | | `statement` | string | body | Yes | Partial SQL statement. | | `catalog` | string | body | No | Catalog used to scope suggestions. | | `schema` | string | body | No | Schema used to scope suggestions. | | `session_id` | string | body | No | Existing session used for catalog context. | | `max_suggestions` | integer | body | No | Maximum suggestions to return. Defaults to 20 and cannot exceed 200. | The response includes the original statement, connection errors by catalog when present, and suggestions with their type, score, and replacement start position. See [AutocompleteSuggestion](#autocompletesuggestion) for the fields in each suggestion. ### Response statuses | Status | Description | | ------ | ---------------------------- | | 200 | SQL autocomplete suggestions | | 400 | Invalid request parameters | | 401 | Authentication required | ### Response fields | Field | Type | Description | Details | | -------------------- | ------------------------- | ----------- | -------- | | `connections_errors` | object | | Required | | `statement` | string | | Required | | `suggestions` | AutocompleteSuggestion\[] | | Required | ### Request ```curl curl https://api.altertable.ai/autocomplete \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" \ -H "Content-Type: application/json" \ -d '{"statement":"SELECT ma","catalog":"my_catalog","schema":"main","max_suggestions":20}' ``` ### Response (200 OK) ```json { "connections_errors": {}, "statement": "SELECT ma", "suggestions": [] } ``` ## GET `/query/{query_id}` Inspect a running or completed query. **Authentication:** HTTP Basic Auth\ **Permission:** Access to the query | Parameter | Type | Location | Required | Description | | ---------- | ---- | -------- | -------- | ------------------------------------------------- | | `query_id` | UUID | path | Yes | Query identifier returned when execution started. | ### Response statuses | Status | Description | | ------ | ----------------------- | | 200 | Query log | | 401 | Authentication required | | 404 | Query not found | ### Response fields Use the `query_id` returned by [POST /query](/docs/api-references/lakehouse.md#post-query). Running-query progress fields are included when available. The optional `stats` object uses [QueryStats](#querystats). | Field | Type | Description | Details | | ------------------ | ------------------ | ----------- | -------------- | | `client_interface` | SessionKind | | Required | | `duration_ms` | integer \| null | | Optional | | `end_time` | string \| null | | Optional | | `error` | string \| null | | Optional | | `fingerprint` | integer \| null | | Optional | | `query` | string | | Required | | `requested_by` | string \| null | | Optional | | `session_catalog` | string \| null | | Optional | | `session_id` | string | | Required | | `session_schema` | string \| null | | Optional | | `start_time` | string | | Required | | `stats` | null \| QueryStats | | Optional | | `threads` | integer | | Required | | `user_agent` | string \| null | | Optional | | `uuid` | string | | Required | | `visible` | boolean | | Required | | `percentage` | number | | When available | | `rows_processed` | integer | | When available | | `total_rows` | integer | | When available | ### Request ```curl curl "https://api.altertable.ai/query/" \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" ``` ### Response (200 OK) ```json { "uuid": "123e4567-e89b-12d3-a456-426614174000", "start_time": "2026-10-28T10:00:00Z", "query": "SELECT 1", "client_interface": "HTTP", "visible": true, "session_id": "123e4567-e89b-12d3-a456-426614174000", "threads": 2 } ``` ## DELETE `/query/{query_id}` Cancel a running query. **Authentication:** HTTP Basic Auth\ **Permission:** Access to the query | Parameter | Type | Location | Required | Description | | ---------- | ---- | -------- | -------- | --------------------------- | | `query_id` | UUID | path | Yes | Query identifier to cancel. | ### Response statuses | Status | Description | | ------ | --------------------------- | | 200 | Query cancellation response | | 401 | Authentication required | | 404 | Query not found | ### Response fields | Field | Type | Description | Details | | ----------- | ------- | ----------- | -------- | | `cancelled` | boolean | | Required | | `message` | string | | Required | ### Request ```curl curl -X DELETE "https://api.altertable.ai/query/" \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" ``` ### Response (200 OK) ```json { "cancelled": true, "message": "Query cancellation requested" } ``` ## GET `/tasks/{task_id}` Inspect an asynchronous task returned by the append API. **Authentication:** HTTP Basic Auth\ **Permission:** Access to the task | Parameter | Type | Location | Required | Description | | --------- | ---- | -------- | -------- | ---------------------------------------------------- | | `task_id` | UUID | path | Yes | Task identifier returned by an asynchronous request. | Task status is `pending` or `completed`. ### Response statuses | Status | Description | | ------ | ----------------------- | | 200 | Task status | | 400 | Invalid task ID | | 401 | Authentication required | ### Response fields | Field | Type | Description | Details | | --------- | ---------- | -------------------------- | -------- | | `status` | TaskStatus | Current status of the task | Required | | `task_id` | string | The task ID | Required | ### Request ```curl curl "https://api.altertable.ai/tasks/" \ -H "Authorization: Basic $ALTERTABLE_BASIC_AUTH_TOKEN" ``` ### Response (200 OK) ```json { "task_id": "019d68d4-c4cb-7e40-ad50-fd4b68c2045e", "status": "completed" } ``` ## Nested response types The endpoint responses use these nested types. [Query status](/docs/api-references/lakehouse.md#get-query-query-uuid) includes query statistics, [explain](/docs/api-references/lakehouse.md#post-explain) returns table scan estimates, and [autocomplete](/docs/api-references/lakehouse.md#post-autocomplete) returns suggestions. ### AutocompleteSuggestion | Field | Type | Description | Details | | ------------------ | -------------- | ----------- | -------- | | `extra_char` | string \| null | | Optional | | `suggestion` | string | | Required | | `suggestion_score` | integer | | Required | | `suggestion_start` | integer | | Required | | `suggestion_type` | string | | Required | ### TableScanEstimate | Field | Type | Description | Details | | ------------------------ | --------------- | -------------------------------------------------------------------------- | -------- | | `estimated_rows` | integer | Estimated number of rows in the table | Required | | `filters` | string \| null | Filters applied to the table | Optional | | `scanned_bytes_estimate` | integer \| null | Estimated bytes to scan after filter pruning based on column min/max stats | Optional | | `scanned_files_estimate` | integer \| null | Estimated files to scan after filter pruning based on column min/max stats | Optional | | `table_name` | string | Name of the table | Required | | `total_bytes` | integer \| null | Total bytes across all data files | Optional | | `total_files` | integer \| null | Total number of data files for this table | Optional | ### QueryStats | Field | Type | Description | Details | | --------- | -------------------- | ----------- | -------- | | `caching` | null \| CachingStats | | Optional | | `scan` | null \| ScanStats | | Optional | ### CachingStats | Field | Type | Description | Details | | ------------------- | ------- | ----------- | -------- | | `data_bytes_hits` | integer | | Required | | `data_bytes_misses` | integer | | Required | | `data_hits` | integer | | Required | | `data_misses` | integer | | Required | ### ScanStats | Field | Type | Description | Details | | ------------------------ | ------- | ----------- | -------- | | `estimated_result_rows` | integer | | Required | | `estimated_scanned_rows` | integer | | Required | ### SessionKind `ArrowFlightSQL`, `HTTP`, `Postgres`. ### AppendErrorCode `invalid-data`. ### TaskStatus `pending`, `completed`. --- url: https://altertable.ai/docs/api-references/mcp-tools.md --- # MCP tools Use this reference to look up tool inputs and outputs. For connection, OAuth authorization, environment selection, and access scopes, see [MCP setup](/docs/agents/mcp.md). This page lists public tools and their minimum scopes. Discovery in your client also depends on permissions, enabled features, and connected integrations. Behavior hints describe tool properties; the granted scope and resource permissions determine access. ## `ask` Asks the Altertable Agent an analytical question about the data in the current environment and returns its answer or a question for the user (no streaming). Never answer that question yourself: ask the user, with your own question tool if you have one, then continue with chat\_id and their answer as the prompt. Use it for multi-step analysis the user wants the agent to handle end to end. The agent is read-only: it explores catalogs, runs read-only queries, and reads existing insights, dashboards, documentation, and public GitHub repositories, but never creates, updates, or deletes them or any data. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":true,"readOnlyHint":false,"title":"Ask Agent"} ``` [Input and output schemas for `ask`](/docs/api-references/mcp-tools/ask.md) ## `ask_question` Ask one question when the user must decide and the request or context does not settle it. Offer exactly three distinct options, recommended first. The user replies in their next message with an option or their own answer. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":true,"title":"Ask Question"} ``` [Input and output schemas for `ask_question`](/docs/api-references/mcp-tools/ask_question.md) ## `comment_finding` Add a comment to a finding discussion thread. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Comment Finding"} ``` [Input and output schemas for `comment_finding`](/docs/api-references/mcp-tools/comment_finding.md) ## `create_dashboard` Creates a new dashboard. Expects a title plus optional description, widgets, variables, and labels. Widgets can reference saved insights by slug, embed inline insight definitions, show text, or group charts in sections. Returns the dashboard ID, slug, and URL. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Create Dashboard"} ``` [Input and output schemas for `create_dashboard`](/docs/api-references/mcp-tools/create_dashboard.md) ## `create_data_app` Creates a data app from a single index.tsx file. Only React, `@altertable/data-app`, and `@altertable/data-app-react` imports are allowed. The file is bundled before save, and declares the SQL queries the app runs by ID. The source defines its title and description. Returns the slug and URL. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Create Data App"} ``` [Input and output schemas for `create_data_app`](/docs/api-references/mcp-tools/create_data_app.md) ## `create_insight` Creates a new insight. Dispatches on the `kind` field and expects the matching definition: `funnel` + `funnel_definition`, `journeys` + `journeys_definition`, `segmentation` + `segmentation_definition`, `retention` + `retention_definition`, `semantic` + `semantic_definition`, or `sql` + `sql_definition` + `sql_parameters`. Returns the insight ID and slug. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Create Insight"} ``` [Input and output schemas for `create_insight`](/docs/api-references/mcp-tools/create_insight.md) ## `create_knowledge` Create an environment knowledge entry, loaded into this environment's agent session context along with organization knowledge. Only call this tool when the human explicitly asks to save a knowledge entry. The title and content must be either text the human provided directly, or text fetched verbatim from a source the human explicitly pointed at. Before creating the entry, ensure the human's intent to save it as environment knowledge is explicit; if the intended scope, title, or content is unclear, ask for clarification. Never paraphrase, summarize, synthesize, infer, translate, reformat, or generate either field yourself. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Create Knowledge"} ``` [Input and output schemas for `create_knowledge`](/docs/api-references/mcp-tools/create_knowledge.md) ## `create_memory` Store a high-signal fact, decision, finding, or reusable technique worth recalling in a later session, either because the user asked you to remember it or because it will save work next time. Skip routine or low-value details. Set source\_slug (and entities) so search\_memory can retrieve it. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Create Memory"} ``` [Input and output schemas for `create_memory`](/docs/api-references/mcp-tools/create_memory.md) ## `create_sandbox` Creates one short-lived workspace sandbox and returns its id and expiry. Pass repository slugs to clone repositories already connected to this workspace into /workspace/\, or omit them for an empty sandbox. The workspace CLI in the sandbox is already signed in for this environment. An analyst session is read-only. The sandbox expires on its own and does not save its changes back to the organization. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":true,"readOnlyHint":false,"title":"Create Sandbox"} ``` [Input and output schemas for `create_sandbox`](/docs/api-references/mcp-tools/create_sandbox.md) ## `create_task` Creates one task in this workspace when the user asks to save it. An ask task runs its instructions on the given targets and starts active. A dlt\_run task records where a pipeline will load and is saved paused, so this call does not run it. Provide a cron schedule, event triggers, or both. Never send secret values. Returns the task slug and URL. Minimum scope: `mcp:rw` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Create Task"} ``` [Input and output schemas for `create_task`](/docs/api-references/mcp-tools/create_task.md) ## `delete_data_app` Deletes a data app by slug. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Delete Data App"} ``` [Input and output schemas for `delete_data_app`](/docs/api-references/mcp-tools/delete_data_app.md) ## `document_semantic_model` Sets the natural-language description on the persisted semantic model for a table. Pass `catalog` from `list_catalogs` (same value as its `catalog_name`); `schema` and `table` match the lakehouse identifiers. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":false,"title":"Document semantic model"} ``` [Input and output schemas for `document_semantic_model`](/docs/api-references/mcp-tools/document_semantic_model.md) ## `document_semantic_model_dimension` Sets the natural-language description on one dimension of a persisted semantic model. `projection` is the dimension name: alias (`as`) if set, otherwise the column name (same as listed under Columns/dimensions in get\_catalog output). Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":false,"title":"Document semantic model dimension"} ``` [Input and output schemas for `document_semantic_model_dimension`](/docs/api-references/mcp-tools/document_semantic_model_dimension.md) ## `execute_insight` Executes exactly one insight and returns its query results as structured data for further reasoning. Use this to read an insight's numbers, series, or rows. Pass `insight` for a saved insight or `definition` for a non-persisted insight, never both. A definition does not create or update a saved insight. Definitions dispatch on the `kind` field and expect the matching fields: `funnel` + `funnel_definition`, `journeys` + `journeys_definition`, `segmentation` + `segmentation_definition`, `retention` + `retention_definition`, `semantic` + `semantic_definition`, or `sql` + `sql_definition` + `sql_parameters` (x\_axis\_columns / y\_axis\_columns from SQL aliases). Definition-based funnel, journeys, segmentation, and retention kinds also require `from`, `to`, and `interval` inside `definition`. Preferred flow for new SQL results: validate the statement with `query_lakehouse`, then pass `sql_definition` and `sql_parameters` inside `definition` (visualization\_options alone is not enough). Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Execute Insight"} ``` [Input and output schemas for `execute_insight`](/docs/api-references/mcp-tools/execute_insight.md) ## `execute_lakehouse` Executes a SQL statement against the lakehouse using the DuckDB dialect, including writes (INSERT/UPDATE/DELETE/DDL). Result output is bounded at 50000 tokens. If `truncation` is set, re-issue the same statement with `offset: ` to fetch the next page. Add a deterministic `ORDER BY` to any query that may be paginated; without it, continuation can duplicate or skip rows (especially for GROUP BY or joins). For read-only queries, prefer the `query_lakehouse` tool instead. Before writing your first statement, load the `query-altertable` skill by reading `altertable://skills/query-altertable` with `read_resource`, unless it is already loaded. Minimum scope: `mcp:rw` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":true,"readOnlyHint":false,"title":"Execute Lakehouse"} ``` [Input and output schemas for `execute_lakehouse`](/docs/api-references/mcp-tools/execute_lakehouse.md) ## `explain_sql` Check the DuckDB execution plan and estimated cost for one or more SQL statements before running them, to catch expensive queries early. Before writing your first statement, load the `query-altertable` skill by reading `altertable://skills/query-altertable` with `read_resource`, unless it is already loaded. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Explain DuckDB SQL"} ``` [Input and output schemas for `explain_sql`](/docs/api-references/mcp-tools/explain_sql.md) ## `generate_dlt_pipeline` Writes the Python dlt pipeline that loads an external source into a lakehouse schema, and proves it runs in a sandbox before returning it. Call this when the user wants to ingest a source Altertable does not already hold. Returns a validated script plus the names of the source credentials it reads, or the reason no correct pipeline could be written. Pass `last_run_error` to repair a pipeline whose task run failed. This spawns a sandbox and takes several minutes, so tell the user it is running and how long it takes before calling it, and call it once the target and the source are settled, not to explore options. Minimum scope: `mcp:rw` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":true,"readOnlyHint":false,"title":"Generate dlt Pipeline"} ``` [Input and output schemas for `generate_dlt_pipeline`](/docs/api-references/mcp-tools/generate_dlt_pipeline.md) ## `get_catalog` Returns structured data for one or more lakehouse catalogs (altertable databases or external connections). Normally omit `level`: narrow with `schemas` and `tables`, and the tool automatically returns detailed schema for a small selection or a compact table overview for a large one. Use `level: overview` only when you explicitly want names and descriptions without columns; `columns` and `full` are explicit schema-depth overrides. After choosing a table, use profile\_columns to inspect observed nulls, cardinality, uniqueness, ranges, and common values. Each table includes explicit semantic endorsement (draft, verified, excluded) labels. Tables whose semantic endorsement is `excluded` are omitted from the output. A column reported with `contains_pii` holds personally identifiable data: prefer aggregates over it and do not echo its values back to the user unless they ask for them. Typical flow: call with the default to discover tables. If more than one table could represent the requested concept, first inventory the schema without `tables` at `level: overview`; then pass `tables: [...]` for the specific tables you intend to query. When a table reports `estimated_row_count`, treat it as an approximate row count: on large tables keep query\_lakehouse gentle with aggregates, selective filters, and LIMIT instead of scanning or returning every row. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Get Catalog"} ``` [Input and output schemas for `get_catalog`](/docs/api-references/mcp-tools/get_catalog.md) ## `get_finding` Returns one finding by slug. Accepts a finding slug (DSC-*) or the slug of its inbox notification (NTF-*). Returns the finding body, review status, related task and context slugs, and comments. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"Get Finding"} ``` [Input and output schemas for `get_finding`](/docs/api-references/mcp-tools/get_finding.md) ## `get_neighbors` Walk one hop over stored entity\_edges (structural associations and similar\_to). Given an indexed entity slug, returns the entities directly connected to it in either direction as compact refs, plus the source identity. Find slugs via `search_entities`; read full content via `altertable://ontology/entities/{slug}`. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"Get Neighbors"} ``` [Input and output schemas for `get_neighbors`](/docs/api-references/mcp-tools/get_neighbors.md) ## `initialize` REQUIRED FIRST STEP: CALL THIS BEFORE ANY OTHER TOOL. Returns information about the current organization, environment, and knowledge-entry context the agent must use to decide what to do and how to behave. Skipping this risks acting on the wrong scope or missing mandatory specific guidance. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"Initialize session context"} ``` [Input and output schemas for `initialize`](/docs/api-references/mcp-tools/initialize.md) ## `list_catalogs` Lists all data catalogs in the current environment: external connections (Postgres, Snowflake, etc.) and Altertable databases. Each entry includes slug (use in task target\_slugs), catalog\_name (use with get\_catalog), name, engine, read\_only, and description when one is set. For full schema, tables, and columns, call get\_catalog with that catalog\_name. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"List Catalogs"} ``` [Input and output schemas for `list_catalogs`](/docs/api-references/mcp-tools/list_catalogs.md) ## `list_dashboards` Lists all dashboards in the organization. Dashboards are collections of insights presented together. Supports optional query filtering to search by title or description. Returns dashboard metadata including slug, title, description, and creation/update timestamps. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"List Dashboards"} ``` [Input and output schemas for `list_dashboards`](/docs/api-references/mcp-tools/list_dashboards.md) ## `list_events` Lists the Product Analytics event names tracked in the current environment, each with the names of the properties recorded on it. Use it to discover which events and properties exist before querying or charting them. Filter by a substring of the event name, or by when an event was last seen. Returns names only, never event counts or property values. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"List Events"} ``` [Input and output schemas for `list_events`](/docs/api-references/mcp-tools/list_events.md) ## `list_insights` Lists all insights (charts) in the organization. Insights are saved queries of these kinds: funnel, journeys, segmentation, logssegmentation, semantic, sql, retention. Supports optional query filtering to search by title or description. Returns insight metadata including slug, title, description, chart kind, and creation/update timestamps. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"List Insights"} ``` [Input and output schemas for `list_insights`](/docs/api-references/mcp-tools/list_insights.md) ## `list_repositories` Lists Git repositories connected to the organization. Use the returned slugs with `create_sandbox` to clone specific repositories into a sandbox. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"List Repositories"} ``` [Input and output schemas for `list_repositories`](/docs/api-references/mcp-tools/list_repositories.md) ## `list_semantic_metrics` Lists the semantic metrics of this environment and, when one can be queried, a runnable query for it. Prefer these metrics over computing a business metric yourself, and run the query with query\_lakehouse. Use a metric's expression only to explain it or tell metrics apart, e.g. a built-in filter. To combine, break down or filter, add metrics, DIMENSIONS or WHERE inside SEMANTIC\_VIEW(...) using the dimension members listed on that metric, or other columns of the metric alias or of the aliases in those members, e.g. DIMENSIONS date\_trunc('month', \) AS month. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"List Semantic Metrics"} ``` [Input and output schemas for `list_semantic_metrics`](/docs/api-references/mcp-tools/list_semantic_metrics.md) ## `list_tasks` Lists recurring tasks in the environment (schedules, ask tasks, dlt pipelines, etc.). Supports optional filters by full-text query, task kind (same names as create\_task `type`), target slug, and labels. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"List Tasks"} ``` [Input and output schemas for `list_tasks`](/docs/api-references/mcp-tools/list_tasks.md) ## `list_user_traits` Lists the names of user traits recorded by Product Analytics in the current environment. Use it to discover which trait names exist before filtering or segmenting by them. Returns trait names only, never trait values. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"List User Traits"} ``` [Input and output schemas for `list_user_traits`](/docs/api-references/mcp-tools/list_user_traits.md) ## `profile_columns` Profiles selected columns from one lakehouse table using one bounded, cached sample query. Use this after get\_catalog and before SQL when the answer depends on repeated versus unique values, null or blank handling, entity versus event grain, ordering stored values, or interpreting an ambiguous measure. Returns counts, cardinality, sample-scoped uniqueness, ranges, and common values. Observed nulls and blanks are evidence about representation; do not exclude nulls or blanks unless the request or documented semantics require it. Always pass only the load-bearing columns; at most 25 columns can be profiled per call. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Profile Columns"} ``` [Input and output schemas for `profile_columns`](/docs/api-references/mcp-tools/profile_columns.md) ## `query_lakehouse` Executes a read-only SQL query against the lakehouse using the DuckDB dialect. Result output is bounded at 50000 tokens. If `truncation` is set, re-issue the same statement with `offset: ` to fetch the next page. Add a deterministic `ORDER BY` to any query that may be paginated; without it, continuation can duplicate or skip rows (especially for GROUP BY or joins). To write data, use the `execute_lakehouse` tool instead. Before writing your first statement, load the `query-altertable` skill by reading `altertable://skills/query-altertable` with `read_resource`, unless it is already loaded. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Query Lakehouse"} ``` [Input and output schemas for `query_lakehouse`](/docs/api-references/mcp-tools/query_lakehouse.md) ## `read_resource` Reads an `altertable://` resource and returns its contents. `altertable://skills/{name}` returns a skill playbook and an `additional_files` manifest; `altertable://skills/{name}/{directory}/{filename}` returns a linked skill file (e.g. `altertable://skills/analyze-product-behavior/references/match-recognize.md`). `altertable://ontology/entities/{slug}` returns an entity in full; `altertable://ontology/entities/{slug}/neighbors` lists what it connects to; `altertable://ontology` describes the entity types. Use a URI already in context; otherwise find slugs with `search_entities`. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"Read Resource"} ``` [Input and output schemas for `read_resource`](/docs/api-references/mcp-tools/read_resource.md) ## `render_data_app` Renders one saved data app in the inline Data App Viewer. Use this when the user asks to open, show, or interact with a data app. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Render Data App"} ``` [Input and output schemas for `render_data_app`](/docs/api-references/mcp-tools/render_data_app.md) ## `render_insight` Renders exactly one insight in the inline Insight Viewer so the user can see the chart. Returns query results plus the visualization metadata the client needs to display the insight. Use this when the user asks to show, visualize, chart, inspect, or render a saved or non-persisted insight. Pass `insight` for a saved insight or `definition` for a non-persisted insight, never both. A definition does not create or update a saved insight. Definitions dispatch on the `kind` field and expect the matching fields: `funnel` + `funnel_definition`, `journeys` + `journeys_definition`, `segmentation` + `segmentation_definition`, `retention` + `retention_definition`, `semantic` + `semantic_definition`, or `sql` + `sql_definition` + `sql_parameters` (x\_axis\_columns / y\_axis\_columns from SQL aliases). Definition-based funnel, journeys, segmentation, and retention kinds also require `from`, `to`, and `interval` inside `definition`. Preferred flow for new SQL results: validate the statement with `query_lakehouse`, then pass `sql_definition` and `sql_parameters` inside `definition` (visualization\_options alone is not enough). Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Render Insight"} ``` [Input and output schemas for `render_insight`](/docs/api-references/mcp-tools/render_insight.md) ## `report_finding` Reports a finding worth human attention from the current investigation. Persists the finding and notifies users through enabled delivery channels. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Report Finding"} ``` [Input and output schemas for `report_finding`](/docs/api-references/mcp-tools/report_finding.md) ## `run_sandbox_command` Runs one command inside an existing workspace sandbox and returns stdout, stderr, and the exit code. The command can modify that sandbox. It does not run on the host. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":true,"readOnlyHint":false,"title":"Run Sandbox Command"} ``` [Input and output schemas for `run_sandbox_command`](/docs/api-references/mcp-tools/run_sandbox_command.md) ## `search_docs` Search the Altertable platform documentation for information about features, insight types, usage and best practices. Returns relevant documentation snippets with titles, content, and URLs. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Search Docs"} ``` [Input and output schemas for `search_docs`](/docs/api-references/mcp-tools/search_docs.md) ## `search_entities` Hybrid lexical and semantic search across indexed Altertable entities. Returns matching entity slugs, titles, content snippets, an `rrf_score` for in-response ordering, a `semantic_similarity` (absolute relevance, present when the semantic branch matched), and `matched_branches` indicating which retrieval branches (lexical, semantic) fired per hit. Use `node_types` to scope by entity type; its schema lists every searchable type, and any other value is rejected. Read `altertable://ontology` for how types relate. Read a hit's `resource_uri` for its full content. Call `get_neighbors` to inspect directly related entities. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"Search Entities"} ``` [Input and output schemas for `search_entities`](/docs/api-references/mcp-tools/search_entities.md) ## `search_memory` Search for memories, ranked by reciprocal rank fusion (RRF), with importance as a bounded boost. Supports both semantic (vector) and keyword search. Note: Returned memories count as accessed, at most once per day. Tip: Use this tool proactively at the start of a workflow to auto-recall relevant context. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":true,"title":"Search Memory"} ``` [Input and output schemas for `search_memory`](/docs/api-references/mcp-tools/search_memory.md) ## `take_screenshot` Use this to see how a supported entity renders, given its slug. It captures the shared view of the entity as an inline image. Auto-creates a short-lived share token if none exists. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":false,"openWorldHint":true,"readOnlyHint":false,"title":"Take Screenshot"} ``` [Input and output schemas for `take_screenshot`](/docs/api-references/mcp-tools/take_screenshot.md) ## `terminate_sandbox` Terminate a sandbox created by `create_sandbox`. Safe to call more than once; a missing or already-expired sandbox is not an error. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":false,"title":"Terminate Sandbox"} ``` [Input and output schemas for `terminate_sandbox`](/docs/api-references/mcp-tools/terminate_sandbox.md) ## `update_dashboard` Updates an existing dashboard. Provide the dashboard slug plus any fields to change: title, description, widgets, variables, or labels. Widget and variable lists replace the current values when provided. Returns the dashboard ID, slug, and URL. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Update Dashboard"} ``` [Input and output schemas for `update_dashboard`](/docs/api-references/mcp-tools/update_dashboard.md) ## `update_data_app` Replaces the source of a data app. The new source is bundled before save and may only import React, `@altertable/data-app`, and `@altertable/data-app-react`. It replaces the title, description and queries with those it declares. Returns the slug and URL. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Update Data App"} ``` [Input and output schemas for `update_data_app`](/docs/api-references/mcp-tools/update_data_app.md) ## `update_insight` Updates an existing insight. Provide the insight slug plus any fields to change. Definition fields (`funnel_definition`, `segmentation_definition`, `sql_definition`, etc.) and matching `*_parameters` apply to the current or updated `kind`. Returns the insight ID, slug, and URL. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Update Insight"} ``` [Input and output schemas for `update_insight`](/docs/api-references/mcp-tools/update_insight.md) ## `update_knowledge` Update a knowledge entry in the current environment. Only call this tool when the human explicitly asks to change a saved knowledge entry. The title and content must be either text the human provided directly, or text fetched verbatim from a source the human explicitly pointed at. Never paraphrase, summarize, synthesize, infer, translate, reformat, or generate either field yourself. Minimum scope: `mcp:analyst` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Update Knowledge"} ``` [Input and output schemas for `update_knowledge`](/docs/api-references/mcp-tools/update_knowledge.md) ## `update_task` Update a recurring task. Provide the task slug plus the fields to change: instructions, CRON schedule, event triggers, active state, title, target slugs, or the `script` and `timeout_seconds` of a `dlt_run` task. A task can carry a schedule and events at the same time; omitted fields are left unchanged. For a `dlt_run` task, catalog, schema and targets are fixed at creation; its script, timeout and the common task fields stay updateable. Minimum scope: `mcp:rw` ### Behavior hints ```json {"destructiveHint":true,"idempotentHint":false,"openWorldHint":false,"readOnlyHint":false,"title":"Update Task"} ``` [Input and output schemas for `update_task`](/docs/api-references/mcp-tools/update_task.md) ## `validate_data_app` Compiles and type-checks index.tsx and checks the queries it declares without saving. Valid when `errors` is empty, type errors included; `warnings` never fail validation. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":false,"readOnlyHint":true,"title":"Validate Data App"} ``` [Input and output schemas for `validate_data_app`](/docs/api-references/mcp-tools/validate_data_app.md) ## `validate_sql` Validates whether a SQL statement is valid DuckDB dialect. Before writing your first statement, load the `query-altertable` skill by reading `altertable://skills/query-altertable` with `read_resource`, unless it is already loaded. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Validate SQL"} ``` [Input and output schemas for `validate_sql`](/docs/api-references/mcp-tools/validate_sql.md) ## `web_search` Performs a web search and returns ranked results (title, URL, snippet, and age when known). Prefer other tools: use this only when you need to search for external/public information on the web. Minimum scope: `mcp:ro` ### Behavior hints ```json {"destructiveHint":false,"idempotentHint":true,"openWorldHint":true,"readOnlyHint":true,"title":"Web Search"} ``` [Input and output schemas for `web_search`](/docs/api-references/mcp-tools/web_search.md) --- url: https://altertable.ai/docs/api-references/opentelemetry.md --- # OpenTelemetry API ## Quick reference - **Base URL** https\://api.altertable.ai - **Protocol** OTLP/HTTP - **Authentication** [HTTP Basic Auth](#authentication) * [Export logs](#post-v1-logs) * [Export traces](#post-v1-traces) * [Export metrics](#post-v1-metrics) Send OTLP/HTTP logs, traces, and metrics with an OpenTelemetry SDK, collector, or HTTP client. Altertable writes accepted telemetry to the environment's `opentelemetry` catalog. For setup and verification, follow [Configure an exporter](/docs/opentelemetry/setup.md). The [deployed OpenAPI contract](https://api.altertable.ai/openapi/otlp.json) lists the operations; the [OTLP specification](https://opentelemetry.io/docs/specs/otlp/) defines the signal payloads. ## Authentication Use HTTP Basic Auth with [lakehouse credentials](/docs/administration/lakehouse-credentials.md) from the target environment. Every request requires `Authorization: Basic `. With curl, pass `--user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD"` to construct the header. Set these variables with the same environment’s lakehouse credentials before running the request examples: ```bash ALTERTABLE_USERNAME='{{LAKEHOUSE_USERNAME}}' ALTERTABLE_PASSWORD='{{LAKEHOUSE_PASSWORD}}' ``` ## Encoding and responses - Set `Content-Type` to `application/x-protobuf` for binary protobuf or `application/json` for OTLP JSON. - For compressed requests, gzip the body and send `Content-Encoding: gzip`. - The response uses the request's encoding and the corresponding OTLP `Export*ServiceResponse` message. A successful JSON response is currently `{}`. - Compatibility aliases `/otel/v1/logs`, `/otel/v1/traces`, and `/otel/v1/metrics` use the same handlers. Prefer `/v1/*` for new integrations. | Status | Meaning | | ------ | ------------------------------------------------------------------------ | | 200 | OTLP export accepted | | 400 | Invalid protobuf, JSON, or gzip payload | | 401 | Authentication required or credentials rejected | | 415 | Unsupported Content-Type; use application/x-protobuf or application/json | ## POST `/v1/logs` Export OpenTelemetry logs. **Authentication:** HTTP Basic Auth\ **Permission:** Write telemetry | Parameter | Type | Location | Required | Description | | ------------------ | ------------------------------------------ | -------- | -------- | ----------------------------------------------- | | `Content-Type` | application/x-protobuf \| application/json | header | Yes | Encoding of the OTLP export request. | | `Content-Encoding` | gzip | header | No | Set when the body is gzip-compressed. | | `body` | ExportLogsServiceRequest | body | Yes | OTLP logs payload in protobuf or JSON encoding. | Query accepted logs in `opentelemetry.main.logs`. See [Send logs](/docs/opentelemetry/logs.md) for setup and verification. The response is an `ExportLogsServiceResponse`, encoded as protobuf or JSON to match the request. The [response statuses](#encoding-and-responses) apply to this endpoint. ### Request ```curl curl https://api.altertable.ai/v1/logs \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H "Content-Type: application/x-protobuf" \ --data-binary @logs.pb ``` ### Response (200 OK (JSON request)) ```json {} ``` ## POST `/v1/traces` Export OpenTelemetry traces. **Authentication:** HTTP Basic Auth\ **Permission:** Write telemetry | Parameter | Type | Location | Required | Description | | ------------------ | ------------------------------------------ | -------- | -------- | ------------------------------------------------- | | `Content-Type` | application/x-protobuf \| application/json | header | Yes | Encoding of the OTLP export request. | | `Content-Encoding` | gzip | header | No | Set when the body is gzip-compressed. | | `body` | ExportTraceServiceRequest | body | Yes | OTLP traces payload in protobuf or JSON encoding. | Query accepted traces in `opentelemetry.main.spans`. See [Send traces](/docs/opentelemetry/traces.md) for setup and verification. The response is an `ExportTraceServiceResponse`, encoded as protobuf or JSON to match the request. The [response statuses](#encoding-and-responses) apply to this endpoint. ### Request ```curl curl https://api.altertable.ai/v1/traces \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H "Content-Type: application/x-protobuf" \ --data-binary @traces.pb ``` ### Response (200 OK (JSON request)) ```json {} ``` ## POST `/v1/metrics` Export OpenTelemetry metrics. **Authentication:** HTTP Basic Auth\ **Permission:** Write telemetry | Parameter | Type | Location | Required | Description | | ------------------ | ------------------------------------------ | -------- | -------- | -------------------------------------------------- | | `Content-Type` | application/x-protobuf \| application/json | header | Yes | Encoding of the OTLP export request. | | `Content-Encoding` | gzip | header | No | Set when the body is gzip-compressed. | | `body` | ExportMetricsServiceRequest | body | Yes | OTLP metrics payload in protobuf or JSON encoding. | Query accepted metrics in `opentelemetry.main.metrics`. This view joins the underlying `metric_points` and `metric_series` tables. Gauge, sum, histogram, exponential histogram, and summary points are supported. See [Send metrics](/docs/opentelemetry/metrics.md) for setup and verification. The response is an `ExportMetricsServiceResponse`, encoded as protobuf or JSON to match the request. The [response statuses](#encoding-and-responses) apply to this endpoint. ### Request ```curl curl https://api.altertable.ai/v1/metrics \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H "Content-Type: application/x-protobuf" \ --data-binary @metrics.pb ``` ```bash timestamp="$(date +%s)000000000" curl https://api.altertable.ai/v1/metrics \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H 'Content-Type: application/json' \ --data-binary @- < ai.altertable altertable-lakehouse-java 0.1.4 ``` ## Lakehouse environment variables JavaScript, Python, Ruby, and Go accept `ALTERTABLE_USERNAME` and `ALTERTABLE_PASSWORD`. Java and the CLI use `ALTERTABLE_LAKEHOUSE_USERNAME` and `ALTERTABLE_LAKEHOUSE_PASSWORD`. All of these clients also accept `ALTERTABLE_BASIC_AUTH_TOKEN` for a base64-encoded username/password pair. Follow the chosen client's README when relying on automatic environment configuration. The [HTTP query example](/docs/lakehouse/query/http-api.md#1-verify-authentication) uses `ALTERTABLE_USERNAME` and `ALTERTABLE_PASSWORD` with cURL's `--user` option, which constructs the Basic Auth header. ## Product Analytics SDKs Use [tracking setup](/docs/product-analytics/setup.md) for installation and verification: - [JavaScript and React](https://github.com/altertable-ai/altertable-js) - [Python](https://github.com/altertable-ai/altertable-python) - [Ruby](https://github.com/altertable-ai/altertable-ruby) - [Swift](https://github.com/altertable-ai/altertable-swift) - [Kotlin and Android](https://github.com/altertable-ai/altertable-kotlin) ## Data-app SDK Use the [local development guide](/docs/data-apps/local-development.md) for the CLI scaffold, preview, checks, and publishing commands. The [data-app repository](https://github.com/altertable-ai/data-app) documents the SDK. Lakehouse requests use [lakehouse credentials](/docs/administration/lakehouse-credentials.md). Product Analytics requests use a [Product Analytics API key](/docs/product-analytics/authentication.md). --- url: https://altertable.ai/docs/automation/cousteau.md --- # Cousteau Cousteau is Altertable’s autonomous product investigation agent. Connect the data you want it to investigate. Start with one source and add others as needed. The guides below explain how to send data, connect an existing database, or add business context. ## Set up an investigation Cousteau is enabled when an environment is created. It needs relevant data before it can investigate. Open **Cousteau** in that environment to check its status, focus areas, and findings. 1. Connect one source using the guides below and verify a known record with SQL. 2. Open Cousteau and create a focus area with a concrete title, such as **Checkout errors**. 3. Add guidelines describing the relevant tables, business definitions, and signals to investigate. Add shared [knowledge](/docs/agents/knowledge/entries.md) for context that should apply beyond this focus area. 4. Review the focus area's task and findings as investigations run. The global investigation is scheduled every six hours. Users with permission to manage bots can enable or disable Cousteau from its menu. Disabling pauses its tasks and triggers; enabling resumes the global task and active focus areas. Archive a focus area when it no longer needs monitoring. ## Verify an investigation Check the task's latest run and open a finding. Confirm the source tables, time range, and evidence against a direct SQL query. A **waiting for data** state means you should verify the connected source before expecting findings. A run without a finding is not proof that no issue exists. ## Troubleshooting - **Waiting for data:** query the source in the same environment and confirm that it contains usable records. - **No recent runs:** check whether Cousteau is disabled and inspect its task status. - **Findings miss business context:** add relevant knowledge and make the focus-area guidelines more specific. - **Enable or edit controls are unavailable:** ask an administrator to check your permissions. ## Connect data - [Product analytics](/docs/product-analytics/setup.md) - [OpenTelemetry](/docs/opentelemetry/setup.md) - [HTTP API](/docs/lakehouse/ingest/http-api.md) - [Object storage](/docs/lakehouse/ingest/object-storage.md) - [External catalogs](/docs/lakehouse/external-catalogs.md) - [Business context and knowledge](/docs/agents/knowledge/entries.md) ## Continue with your data Use [Ask Altertable](/docs/agents/ask.md) to investigate a question, or create a [task](/docs/automation/tasks.md) for a recurring check. --- url: https://altertable.ai/docs/automation.md --- # Automation Turn recurring questions into scheduled work. Tasks run SQL, code, or AI workflows on a cadence, and notifications deliver the results to the right people. For a recurring question, start with [Tasks](/docs/automation/tasks.md). Configure [Notifications](/docs/automation/notifications.md) when you need to change how findings are delivered. - [Cousteau](/docs/automation/cousteau.md): Connect data for Cousteau with guides for product analytics, OpenTelemetry, HTTP ingestion, object storage, external catalogs, and business context. - [Tasks](/docs/automation/tasks.md): Automate recurring work with Altertable Tasks. Set instructions, add context, and get notified when your data needs attention. - [Notifications](/docs/automation/notifications.md): Notifications surface agent findings and updates in Altertable, by email, or in Slack—scoped per environment. --- url: https://altertable.ai/docs/automation/notifications.md --- # Notifications Notifications bring important activity back to you: agent findings, workflow updates, and other signals for the **current environment**. ## Delivery channels Preferences are **per environment**. Pick how you want to be notified: | Channel | What it does | | --------- | --------------------------------------------------------- | | **Email** | Sends to your account email | | **Slack** | Posts to a channel after you connect your Slack workspace | Slack is optional. Connect Slack from **Integrations**, then choose the agent channels and environment. Set notification channels per environment under **Notification preferences**. ## Manage the inbox Open a notification to read its full finding and mark it as read automatically. From the notification menu, you can mark it read or unread, archive it, or restore it from the **Archived** view. ## Learn more - **[Ask Altertable](/docs/agents/ask.md)**: natural-language analysis and scheduled monitoring - **[Memories](/docs/agents/knowledge/memories.md)**: knowledge Ask Altertable builds from notification feedback - **[Dashboards](/docs/data-apps/dashboards.md)**: combine insights and monitor with tasks - **[Insights](/docs/data-apps/insights.md)**: visualizations and analyses Ask Altertable can review --- url: https://altertable.ai/docs/automation/tasks.md --- # Tasks **Tasks** automate work you want to repeat: monitor metrics, check data quality, or investigate changes. Run a task manually or set a schedule. Give the agent instructions and relevant context, such as an insight or dashboard. When it finds something worth your attention, it sends a [notification](/docs/automation/notifications.md). For a recurring metric check, start from the relevant insight. Use another context when the question is about that specific catalog, connection, segment, or semantic model. ## Create a task 1. Open an insight, dashboard, connection, database, segment, or semantic model you want the task to run on. 2. Start a new task or select one of the suggestions. 3. Write clear instructions. The quality of the results depends on this: include thresholds, what to watch for, what counts as noteworthy, and any context that helps narrow the focus. The more specific you are, the more relevant notifications you'll get. 4. Set a schedule that matches how often your data changes. 5. Save the task. It will run on its schedule and appear in your task list. ## Specialized tasks For workflows such as loading or transforming data, choose a specialized task: - **Code task**: run Python or Bash in an isolated environment. - **Synchronizer task**: copy tables from an external catalog, or [files from a bucket](/docs/lakehouse/ingest/object-storage.md), into a catalog on a schedule. - **dbt task**: [run a dbt project](/docs/lakehouse/modeling/dbt.md#run-dbt-in-a-task) from a connected repository to transform data in a catalog. - **dlt task**: [load source data into a catalog](/docs/lakehouse/ingest/integrations/dlt.md) with a Python pipeline. ## Learn more - **[Ask Altertable](/docs/agents/ask.md)**: natural-language analysis and monitoring - **[Notifications](/docs/automation/notifications.md)**: the findings that tasks produce - **[Memories](/docs/agents/knowledge/memories.md)**: knowledge agents carry across runs - **[Dashboards](/docs/data-apps/dashboards.md)**: attach tasks to monitor KPIs - **[Insights](/docs/data-apps/insights.md)**: the insights and analyses tasks run on --- url: https://altertable.ai/docs/data-apps/build.md --- # Build a data app Build an app that answers a question with your Altertable data. Start in the Altertable app for your first build. You can also use the [Ask Altertable in Slack](/docs/agents/ask.md#use-in-slack), or use a connected agent in [Claude](/docs/agents/setup/claude.md), [ChatGPT](/docs/agents/setup/chatgpt.md), or another MCP client. You need [connected or ingested data](/docs/lakehouse/ingest-or-connect.md) and [permission to manage data apps](/docs/administration/access-control.md). The resulting app is hosted in Altertable, where you can review and share it. ## 1. Describe the app Open **Data apps**, start a new app, and describe the audience, question, and controls you need. Include the intended date range and metric definitions when you know them. ```text Build a data app for our weekly signup review. Show signups over time, break them down by source, and add a date filter. ``` For a [connected MCP client](/docs/agents/setup.md), use [Analyst](/docs/agents/mcp.md#pin-a-scope) access. Send the same prompt and open the app URL returned by the agent. ## 2. Review the data Check the catalogs, tables, joins, filters, and calculations behind the app. Compare a displayed value with a direct [SQL query](/docs/lakehouse/query/sql-explorer.md). Check how the app handles an empty result and a different date range. ## 3. Refine the app Ask the agent for a specific change. Use annotations when you want to point to part of the app. For example, ask it to explain a metric, change a breakdown, or add a filter. When the results and interactions match your goal, [share the app](/docs/data-apps/share.md). --- url: https://altertable.ai/docs/data-apps/dashboards.md --- # Existing dashboards Use data apps for new interactive reporting. You can still view and edit existing dashboards. [Build a data app](/docs/data-apps/build.md) for new analyses. **Dashboards** are collections of [Insights](/docs/data-apps/insights.md), organized into a single view to monitor metrics, track KPIs, and share reports with your team. ## What are Dashboards? Dashboards combine multiple [Insights](/docs/data-apps/insights.md) (visualizations, metrics, tables) into a unified layout: - **Widgets**: add insights, text annotations, and grouped sections - **Grid layout**: arrange and resize widgets freely - **Variables**: filter all widgets at once with shared parameters ## Dashboard Variables Variables let you parameterize your entire dashboard: - **Shared filters**: apply date ranges or segments across all insights - **Dynamic values**: switch between different dimensions or metrics - **Variable mapping**: connect dashboard variables to individual insight parameters When you change a variable, all linked widgets update together. ## Tasks on dashboards Attach a [task](/docs/automation/tasks.md) to a dashboard to check it on a schedule and deliver findings through [notifications](/docs/automation/notifications.md). ## Move to a data app Describe the existing dashboard's metrics, filters, and audience to an agent. Ask it to inspect the underlying queries and build a data app with equivalent results. Verify the calculations, then [create and test a share link](/docs/data-apps/share.md) before distributing the new app. ```text Turn my weekly business review dashboard into a data app with the same metrics, filters, and layout. ``` ## Learn more - **[Insights](/docs/data-apps/insights.md)**: the building blocks of dashboards - **[Tasks](/docs/automation/tasks.md)**: scheduled monitoring on dashboards and other contexts - **[Notifications](/docs/automation/notifications.md)**: notifications from tasks on dashboards - **[Memories](/docs/agents/knowledge/memories.md)**: knowledge tasks build to improve monitoring over time --- url: https://altertable.ai/docs/data-apps.md --- # Data apps Data apps combine queries, charts, tables, and interactions in an application built with an agent. Describe the question and audience, then review and refine the result with your data. > **Beta:** Data apps are in beta. Their setup and behavior may change. Use them for new interactive reporting. Existing [dashboards](/docs/data-apps/dashboards.md) remain available to view and edit. For a saved query or chart, use [Insights](/docs/data-apps/insights.md). Start with [Build a data app](/docs/data-apps/build.md). You can build in Altertable, use the [Ask Altertable in Slack](/docs/agents/ask.md#use-in-slack), or use a connected agent in [Claude](/docs/agents/setup/claude.md), [ChatGPT](/docs/agents/setup/chatgpt.md), or another MCP client. - **Start here:** [Build a data app](/docs/data-apps/build.md): Describe a data app in Altertable, let the agent inspect relevant catalogs, and verify its queries, calculations, filters, and visible results. - [Share a data app](/docs/data-apps/share.md): Create a public share link for an Altertable data app, choose an expiration, verify the shared results, and revoke links when access is no longer needed. - [Develop locally](/docs/data-apps/local-development.md): Use the Altertable data-app CLI and SDK workflow when you need local source files, a development server, and control over publishing a data app. - [Insights](/docs/data-apps/insights.md): Insights are saved analyses over lakehouse data—semantic and SQL views you can share, embed, and monitor with scheduled tasks. --- url: https://altertable.ai/docs/data-apps/insights.md --- # Insights **Insights** are the primary way to create persistent visualizations and analyses in Altertable. They are saved analytics views that help you track key metrics, understand trends, and make data-driven decisions. ## What are Insights? Insights transform raw data into actionable visualizations by: - **Query logic**: query your lakehouse data using predefined logic - **Visualization types**: support charts, tables, and metrics - **Refresh**: refresh automatically or on-demand - **Sharing**: share with teams and embed in external tools > **Prerequisites:** > > Insights analyze data in your lakehouse. Start by [connecting external catalogs](/docs/lakehouse/external-catalogs.md) or [creating an Altertable catalog](/docs/lakehouse/catalogs.md). ## Insight Types Altertable offers semantic and SQL insight types for lakehouse data. For product-behavior analyses, see [Product Analytics query data](/docs/product-analytics/analyze.md). Start with a **Semantic** insight when a suitable semantic model exists. Use **SQL** when you need custom logic or raw tables. | Insight Type | Best For | Key Feature | When to Use | | --------------------------------------------------------------- | ---------------- | ---------------------- | -------------------------- | | [**Semantic**](/docs/data-apps/insights.md#1-semantic-insights) | Business metrics | No-SQL query builder | KPI dashboards and reports | | [**SQL**](/docs/data-apps/insights.md#2-sql-insights) | Custom analysis | Full query flexibility | Complex or ad-hoc queries | ### 1. Semantic Insights Semantic insights query [semantic models](/docs/agents/knowledge/semantic-layer.md) using business-friendly metrics and dimensions—no SQL required. Define models, measures, and dimensions in the [Semantic Layer](/docs/agents/knowledge/semantic-layer.md), then pick them in the insight builder to chart KPIs and reports. **Best for:** - Business metric reporting - KPI dashboards - Cross-functional analytics - Model-based queries **Example use cases:** - "What's our monthly recurring revenue by region?" - "Show customer acquisition cost by marketing channel" - "Track product adoption metrics over time" ![Semantic insight builder: choose a time grain, a business measure, and a chart type—preview updates from sample data](/screenshots/components-insights-semantic-line--docs-kpi-line-preview.png) *Semantic insight builder: choose a time grain, a business measure, and a chart type—preview updates from sample data* ### 2. SQL Insights Execute custom SQL queries using DuckDB dialect for complete flexibility in data analysis. **Best for:** - Complex custom queries - Ad-hoc analysis - Advanced calculations (window functions, CTEs) **Capabilities:** - Full DuckDB SQL dialect support - Cross-catalog federated queries - Advanced analytics functions (window functions, CTEs, recursive queries) - Configurable variables that viewers can change without editing the SQL - Complete query flexibility **Use cases:** - Custom time-series calculations with specific business rules - Complex multi-source joins (events + CRM + revenue data) - Advanced grouping analysis with custom time windows - Performance analysis with moving averages and percentiles ### Move insight definitions between environments Choose **Copy definition** from an insight menu to copy its portable JSON definition. Create an insight with **Import insight**, paste the definition, review the preview, and save it in the current environment. Definitions include the insight title, description, query, visualization, variables, and builder settings. They do not copy source data or grant access to catalogs, so verify that referenced catalogs and fields exist in the destination environment. ## Tasks on insights Attach a [task](/docs/automation/tasks.md) to an insight to check it on a schedule and deliver findings through [notifications](/docs/automation/notifications.md). ## Creating Insights 1. Navigate to **Insights** 2. Click **New insight** 3. Choose **Semantic** or **SQL** 4. Configure the query and chart 5. Save and optionally attach a [Task](/docs/automation/tasks.md) for monitoring ## Learn More - **[Dashboards](/docs/data-apps/dashboards.md)**: combine insights into shared layouts - **[Tasks](/docs/automation/tasks.md)**: scheduled monitoring on insights and other contexts - **[Notifications](/docs/automation/notifications.md)**: notifications from tasks on insights - **[Memories](/docs/agents/knowledge/memories.md)**: knowledge tasks build to improve monitoring over time - **[Semantic Layer](/docs/agents/knowledge/semantic-layer.md)**: define metrics, dimensions, and model structure for semantic insights - **[Product Analytics query data](/docs/product-analytics/analyze.md)**: behavioral analysis over tracked product events - **[Ask Altertable](/docs/agents/ask.md)**: ask natural-language questions about your data - **[Lakehouse](/docs/lakehouse.md)**: query and connect data to your lakehouse --- url: https://altertable.ai/docs/data-apps/local-development.md --- # Develop locally Start with [Build a data app](/docs/data-apps/build.md) for a hosted app. Use local development when you need source files, a local server, or control over publishing. ## Directory app workflow ### 1. Create a local app [Install the CLI and log in](/docs/lakehouse/clients/cli.md), then create a scaffold: ```bash altertable app create product-pulse ``` The scaffold checks the live connection. Confirm that the active profile points to the intended environment before continuing. ### 2. Run the development server ```bash cd product-pulse altertable app dev ``` Open the local URL printed by the command. Edit the scaffold's `src/main.tsx` and `App.tsx` source and verify a bounded query against a known table. Check loading, empty, and error states as well as a successful result. ### 3. Validate the app From the project directory, run: ```bash altertable app lint --lakehouse altertable app build ``` The lakehouse check validates queries against the connected environment. Compare at least one displayed metric with a direct SQL result before publishing. For all options, see the [app command reference](/docs/api-references/cli.md#altertable-app). ## Single-file preview and publishing The CLI also supports a separate single-file workflow. Author a self-contained React source file using the [data-app SDK](https://github.com/altertable-ai/data-app), then preview and validate that file: ```bash altertable app dev --file index.tsx altertable app lint --file index.tsx --lakehouse ``` Publishing creates a hosted app in the active environment. Review the source, title, target environment, and local preview before running: ```bash altertable app publish --title 'Product pulse' --file index.tsx ``` `app publish --file` uploads a single source file; it does not upload the directory scaffold or its build output. Use [hosted app building](/docs/data-apps/build.md) when you want Altertable to manage the app source and publishing. Follow the printed result URL and repeat the metric and interaction checks in the hosted app, then [configure sharing](/docs/data-apps/share.md). Use the [update command](/docs/api-references/cli.md#altertable-app-update) to change an existing app. ## Troubleshooting - **Scaffold connection check fails:** run `altertable profile show` and verify a query with the same profile. - **A query fails validation:** inspect the real catalog and column names; qualify tables with catalog and schema. - **The preview is empty:** query the source directly with the same filters and time range. - **Publishing fails:** check the entry file and the active environment, then inspect the command's error before retrying. Follow the [data-app repository](https://github.com/altertable-ai/data-app) for the CLI scaffold, SDK setup, development commands, and publishing workflow. Use its current instructions for the SDK version in your project. If you are working with an agent, ask it to use Altertable's local data-app workflow explicitly and verify the app locally before publishing. ## Build locally with your coding agent ```text Build an Altertable data app locally in this repository. Read https://altertable.ai/docs/data-apps/local-development.md and the current instructions in https://github.com/altertable-ai/data-app first. Inspect the project and use its package manager. Infer which question the app should answer and who will use it from the project context; ask only for missing details. Confirm the target Altertable environment, inspect available catalogs and schemas, and agree on the metric definitions and time range. Create or reuse the local scaffold and implement a small working app with bounded queries, useful filters, and clear loading, empty, and error states. Keep credentials out of browser code and source control. Verify the query results and calculations against actual data, then run the supported validation and build commands and inspect the app locally. Leave the source and local preview ready for review. Report how to run it, the data sources used, and any remaining issues. Do not publish, deploy, or share the app. ``` --- url: https://altertable.ai/docs/data-apps/share.md --- # Share a data app Share links let people view a data app without logging in to Altertable. The link exposes the app and its query results to anyone who has it. You need permission to manage data apps to create a link. ## Create a share link 1. Open the app and review its queries, visible fields, and results. Confirm that this data is suitable for the audience before creating a link. 2. Open **Share** and create a link. Choose an expiration date or leave the link without an expiration. 3. Copy the link and open it in a logged-out browser session. Check the data and interactions your audience can access. 4. Send the link to the intended audience. Return to **Share** to delete a link when you want to revoke access. To give someone access to the internal app through their Altertable account, manage [members and roles](/docs/administration/access-control.md). --- url: https://altertable.ai/docs/getting-started/connect-data.md --- # Connect an existing source Make an existing database, warehouse, or object-storage dataset queryable in Altertable. An external catalog reads data from its source without importing it into managed tables. You need connection credentials and permission to add a catalog. If the source restricts network access, configure [network access](/docs/administration/network.md) first. ## 1. Choose the source Open [Connect existing data](/docs/lakehouse/external-catalogs.md) and choose the guide for your database, warehouse, or table format. Follow that guide to create the catalog in the intended environment. For object storage, distinguish [querying files in place](/docs/lakehouse/query/object-storage.md) from [syncing files into managed tables](/docs/lakehouse/ingest/object-storage.md). ## 2. Inspect a table Open **Catalogs** and find the new catalog. Inspect its schemas and tables, then open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md). Replace the example catalog, schema, and table with names from your source: ```sql SELECT * FROM my_source.public.my_table LIMIT 10; ``` ## 3. Verify the result Compare the returned rows with the source. Confirm that you connected the intended environment and that the source credentials expose the tables you need. ## Do this with your agent ```text Read https://altertable.ai/docs/getting-started/connect-data.md. Help me connect an existing source to Altertable. Inspect project configuration without reading secret files and ask only for the source, target environment, or network constraints you cannot determine. Follow the matching supported connection guide and explain the required access and where the data stays. Confirm the catalog configuration before creating it. Then inspect the schemas and verify a bounded query against an agreed real table. Report the catalog, table, query, and any access or data gaps. ``` Next, [build a data app](/docs/data-apps/build.md) with the data you connected. --- url: https://altertable.ai/docs/getting-started/examples.md --- # Example workflows After [onboarding](/docs/getting-started.md), use these prompts to work with your data. The agent should inspect available schemas and confirm definitions before querying or creating outputs. ## Connect product usage to revenue Use [Product Analytics](/docs/product-analytics.md) alongside a connected billing or customer catalog. ```text Which product behaviors are associated with paying customers? ``` Check the join keys, the number of unmatched records, and whether each account appears at the intended grain. Continue with [SQL across catalogs](/docs/lakehouse/sql-engine.md#federated-queries). ## Investigate a customer error Use [OpenTelemetry logs and traces](/docs/opentelemetry.md) with customer or product context. ```text Help me investigate checkout errors affecting customers over the last 24 hours. ``` Verify the service, timestamps, and customer identifiers in the returned records. See [Query logs and traces](/docs/opentelemetry/query.md). ## Build a weekly review app Use a [data app](/docs/data-apps/build.md) for a repeatable view of your metrics. ```text Build a data app for our weekly business review. Show revenue, active customers, and product usage over time, with a date filter. ``` Compare displayed metrics with direct SQL results before [sharing the app](/docs/data-apps/share.md). ## Schedule a data check Use [Automation](/docs/automation.md) when a question should run repeatedly. ```text Check for missing or duplicate orders every morning and notify me when you find a problem. ``` Test the query or workflow against current data before scheduling it. See [Tasks](/docs/automation/tasks.md) and [Notifications](/docs/automation/notifications.md). --- url: https://altertable.ai/docs/getting-started/faq.md --- # FAQ and troubleshooting ## Which credentials do I need? [Lakehouse credentials](/docs/administration/lakehouse-credentials.md) authenticate SQL and ingestion. [Product Analytics API keys](/docs/product-analytics/authentication.md) send tracking data. MCP uses browser OAuth for one environment. Logging in to one interface does not configure the others. ## Login succeeds, but my query fails Run `altertable profile show` and confirm the organization and environment. Run `altertable query "SELECT 1 AS connected"` to separate authentication from table access. For missing credentials, run `altertable profile configure` and use the intended environment's lakehouse credentials. See [CLI troubleshooting](/docs/lakehouse/clients/cli.md#troubleshooting). ## There is no writable catalog List catalogs with `altertable catalog`. Create a managed destination with `altertable catalog create onboarding`, or ask an administrator for catalog access. External catalogs may be read-only. Continue with [first ingestion](/docs/getting-started/ingest-data.md). ## An append succeeded, but the table is empty Append processing is asynchronous. Wait a few seconds, then query for the exact record you sent. Check the environment, destination, and `altertable.main.quarantined_appends` for records rejected during insertion. See [ingestion troubleshooting](/docs/lakehouse/ingest/http-api.md#troubleshooting). ## Product Analytics has events, but my test is missing Query for the test event name and unique identity, then compare its timestamp and properties. A recent unrelated event does not verify your integration. Check the API key's environment and consent settings. See [tracking verification](/docs/product-analytics/setup.md#verification). ## My agent sees no tools or the wrong data Reconnect MCP, finish browser authorization, and select the correct environment. For `403 insufficient_scope`, match the endpoint to the token's access tier. See [MCP troubleshooting](/docs/agents/mcp.md#troubleshooting). ## Should I build a dashboard or a data app? Use [data apps](/docs/data-apps.md) for new interactive reporting. Existing dashboards remain available to view and edit. Use an insight for a saved query or chart. ## How much does Altertable cost? See [current plans and the pricing calculator](/pricing), the [pricing FAQ](/pricing#pricing-faq), and [documented technical limits](/docs/administration/limits.md). Storage, compute, and AI usage are separate resources. --- url: https://altertable.ai/docs/getting-started.md --- # Get started with Altertable Connect your data and run a first query. Follow a quickstart yourself, or use your agent to choose and configure a setup for your stack. You need an Altertable account and access to an [organization and environment](/docs/administration.md). If you do not have an account, [create one](https://altertable.ai/sign-up). ## Choose a starting point To try Altertable with a small record, start with **Ingest your first data**. If you already have a source or instrumentation goal, choose its quickstart below. - [Ingest your first data](/docs/getting-started/ingest-data.md): Append a small record to an Altertable table with the CLI, query it to verify ingestion, and continue with file uploads or an agent-led setup. - [Connect an existing source](/docs/getting-started/connect-data.md): Connect a database, warehouse, or object-storage catalog to Altertable, inspect its tables, and verify a query without copying data into managed tables. - [Track your first product event](/docs/getting-started/product-analytics.md): Enable Product Analytics, send a test event with an API key, and verify it in the lakehouse before instrumenting your application with an SDK. - [Send your first logs and traces](/docs/getting-started/opentelemetry.md): Enable OpenTelemetry in Altertable, configure an exporter for logs or traces, and verify recent telemetry from your service with SQL. If you are unsure which route fits, [compare ingestion and external catalogs](/docs/lakehouse/ingest-or-connect.md). ## Set up with your agent Your agent can inspect your project, ask about your stack, and help connect your data. If it can read documentation, start with this prompt: ```text Fetch https://altertable.ai/docs/getting-started/prompt.md and help me set up Altertable for my stack. Inspect the available project and connection context, and ask only for information you cannot discover. ``` The prompt guides the agent through MCP setup, adds the CLI when needed, and verifies a first answer from your data. Complete browser authorization when prompted. For clients without terminal access, the agent can guide you through the app or provide commands to run locally. **Set up the tools yourself** Follow [Connect your agent](/docs/agents/setup.md) for your client. Install the [CLI](/docs/lakehouse/clients/cli.md) when you need terminal commands or local files. Once your data is connected, try an [example workflow](/docs/getting-started/examples.md). For setup problems and common questions, see [FAQ and troubleshooting](/docs/getting-started/faq.md). --- url: https://altertable.ai/docs/getting-started/ingest-data.md --- # Ingest your first data Create a writable catalog, append one record, and read it back. A new environment already has lakehouse credentials, a managed storage bucket, and a read-only Altertable system catalog. The initial `sandbox` environment also has read-only Sample Data. You do not need to provision credentials or storage for this quickstart; you do need a writable catalog for your own data. > **Before using the Altertable CLI:** > > Install the CLI if you haven’t already: > > ```bash > curl -fsSL https://install.altertable.ai | sh > altertable login > ``` > > If `altertable` isn’t found after installation, open a new terminal. Run `altertable login` to log in to your organization and environment. Follow the [CLI setup guide](/docs/lakehouse/clients/cli.md#2-log-in) to configure credentials and verify access. ## 1. Choose a destination After login, confirm the environment and create a writable catalog: ```bash altertable profile show altertable catalog create onboarding ``` Use `onboarding` in the commands below. If that name already exists, run `altertable catalog` and choose a writable catalog you own, or create one with a different name. Replace `onboarding` consistently in both commands. ## 2. Append a record ```bash altertable append '{"event":"onboarding_test","source":"quickstart"}' --to onboarding.main.onboarding_events ``` Append creates a missing table and infers its columns. Processing is asynchronous, so allow a few seconds before querying. ## 3. Read it back ```bash altertable query "SELECT event, source FROM onboarding.main.onboarding_events WHERE event = 'onboarding_test' LIMIT 10" ``` Confirm that the result contains `onboarding_test` and `quickstart`. If the record is missing, inspect the append result and [ingestion troubleshooting](/docs/lakehouse/ingest/http-api.md#troubleshooting). ## Do this with your agent ```text Read https://altertable.ai/docs/getting-started/ingest-data.md. Help me load a small first dataset into Altertable. Inspect available catalogs and ask only which source data or target environment you cannot determine. Show the exact destination and write operation for review before writing; do not overwrite existing data. Verify a known record with a bounded query, compare its fields with the source, and report the rows loaded and any rejected records. Diagnose an empty result rather than treating it as success. ``` Next, [load your own records or files](/docs/lakehouse/ingest/http-api.md). --- url: https://altertable.ai/docs/getting-started/opentelemetry.md --- # Send your first logs and traces Connect an instrumented service to Altertable and verify a log or trace. Altertable accepts OTLP/HTTP logs, traces, and metrics in protobuf or JSON. For metrics, follow [Send metrics](/docs/opentelemetry/metrics.md). ## 1. Enable OpenTelemetry In **Catalogs**, choose **New Catalog** and enable **OpenTelemetry** in the intended environment. Copy that environment's [lakehouse credentials](/docs/administration/lakehouse-credentials.md). ## 2. Configure one signal If your service already uses an OpenTelemetry SDK or collector, follow [Send logs](/docs/opentelemetry/logs.md) or [Send traces](/docs/opentelemetry/traces.md). Configure the signal endpoint and the [Basic Auth header](/docs/opentelemetry/setup.md#1-configure-authentication), then generate activity in the service. If your service is not instrumented, start with [SDK and collector setup](/docs/opentelemetry/setup.md#sdk-and-collector-setup). You can also use the agent prompt below to configure instrumentation for your application. ## 3. Verify the data In [SQL Explorer](/docs/lakehouse/query/sql-explorer.md), run the query for the signal you sent: ```sql SELECT * FROM opentelemetry.main.logs ORDER BY timestamp DESC LIMIT 10; ``` ```sql SELECT * FROM opentelemetry.main.spans ORDER BY start_time DESC LIMIT 10; ``` Confirm that the returned service, timestamps, and attributes match the activity you generated. Continue with [querying and correlating telemetry](/docs/opentelemetry/query.md). ## Do this with your agent ```text Read https://altertable.ai/docs/opentelemetry/setup.md. Configure this service to send logs or traces to Altertable. Inspect existing instrumentation and ask only for the signal or target environment you cannot determine. Reuse the existing SDK or collector and preserve other exporters. Keep lakehouse credentials in server-side secret configuration. Send one controlled test signal and query it by service, test identifier, and a bounded time window. Confirm the timestamps and attributes; do not treat a successful export request as proof that the data arrived. ``` --- url: https://altertable.ai/docs/getting-started/product-analytics.md --- # Track your first product event Send one event and verify it before instrumenting your application. ## 1. Enable Product Analytics In **Catalogs**, choose **New Catalog** and enable **Product Analytics** in your environment. Create or copy a [Product Analytics API key](/docs/product-analytics/authentication.md). Product Analytics uses its own API key. MCP authorization and lakehouse credentials do not replace it. ## 2. Send an event Set the API key for the target environment. The key selects the environment: ```bash ALTERTABLE_API_KEY='{{API_KEY}}' curl https://api.altertable.ai/track \ -H "X-API-Key: $ALTERTABLE_API_KEY" \ -H 'Content-Type: application/json' \ -d '{"event":"Onboarding Test","properties":{},"distinct_id":"onboarding_test_user"}' ``` ## 3. Find the event Run this in [SQL Explorer](/docs/lakehouse/query/sql-explorer.md): ```sql SELECT event, distinct_id, timestamp FROM product_analytics.main.events WHERE event = 'Onboarding Test' ORDER BY timestamp DESC LIMIT 10; ``` Confirm that the event belongs to `onboarding_test_user` and has a recent timestamp. ## Do this with your agent ```text Read https://altertable.ai/docs/product-analytics/setup.md. Set up Product Analytics for this application. Inspect the framework, existing analytics, authentication, and consent handling. Ask which completed action I want to measure and confirm the target environment. Reuse a supported SDK and propose one event with a stable test identity. Keep the environment's Product Analytics key outside source control. Verify the event with a bounded query by its name and test identity, including its properties and timestamp. ``` Next, [install an SDK in your application](/docs/product-analytics/setup.md) to send real events. --- url: https://altertable.ai/docs/getting-started/prompt.md --- # Onboarding prompt Copy this prompt into your agent. It checks your existing setup and completes the steps needed for your goal. ```text Help me set up Altertable and answer a first question with my data. Reuse anything already configured. 1. Ask what I want to answer or build. Inspect the available project and agent configuration to identify my client, application stack, and data sources. Ask one question at a time, only for details you cannot discover. Do not read secret files or print credentials. 2. If the Altertable connection is missing, follow the matching client guide at https://altertable.ai/docs/agents/setup.md. Preserve existing skills, plugins, and MCP servers. Merge configuration, ask before changing user-scoped settings, and use the normal browser authorization flow. Keep credentials out of project files. If a fresh session is required, give me a short handoff to continue. 3. Call initialize before any other Altertable tool. Confirm the organization and environment, check that installed skills are discoverable, and list the available catalogs. If I only want to connect my agent, report the verified connection and stop here. 4. Inspect the relevant catalogs and schemas. If the data I need is missing, read https://altertable.ai/docs/lakehouse/ingest-or-connect.md and recommend the shortest supported route. Explain what will be copied, what will stay in its source, and how fresh the data needs to be. 5. Add the CLI only when the workflow needs terminal commands, local files, or scripts. Follow https://altertable.ai/docs/lakehouse/clients/cli.md, preserve existing profiles, and verify the target environment. Use --agent for structured output. MCP and CLI authenticate separately; verify both when using both. 6. Explain the proposed changes and get my approval before creating resources, changing application code, or sending data. Connect one source or ingest a small, relevant dataset. Do not overwrite data, deploy, or publish anything. If you lack terminal or project access, guide me through the app or provide commands for my shell. 7. Run a bounded query and verify known rows, fields, and timestamps against the source. Diagnose missing data instead of treating an empty result as success. Answer my original question with the verified data, stating the source, filters, time range, and limitations. Summarize what changed, what you verified, and anything I still need to do. ``` --- url: https://altertable.ai/docs/index.md --- # Altertable docs Altertable is an **AI-native data platform** for agents, apps, and humans. At its core is a **context lakehouse built on DuckDB**, bringing data, business definitions, and knowledge together. Ingest records and files through SQL, APIs, or existing pipelines. Capture product analytics events with SDKs, or send logs, traces, and metrics through OpenTelemetry. Federate queries across existing databases, warehouses, and S3 buckets, leaving the data where it lives. Query and join ingested and external data with the same SQL engine. Add business context through metrics, semantic models, documentation, and knowledge from your team so agents and humans can interpret the data. Use that shared context to investigate questions, build interactive data apps, and automate recurring work. - Ingest files, records, product events, logs, traces, and metrics into lakehouse tables. - Connect databases, warehouses, and object storage through external catalogs. Both paths use DuckDB SQL with business context and shared permissions. Agents, SQL clients, and data apps use this same data layer. ## Get started [Choose a quickstart](/docs/getting-started.md) to connect or ingest data, track product events, or send telemetry. You can follow the steps yourself or use an agent to guide the setup. - [Ingest your first data](/docs/getting-started/ingest-data.md): Append a small record to an Altertable table with the CLI, query it to verify ingestion, and continue with file uploads or an agent-led setup. - [Connect an existing source](/docs/getting-started/connect-data.md): Connect a database, warehouse, or object-storage catalog to Altertable, inspect its tables, and verify a query without copying data into managed tables. - [Track a product event](/docs/getting-started/product-analytics.md): Enable Product Analytics, send a test event with an API key, and verify it in the lakehouse before instrumenting your application with an SDK. - [Send logs and traces](/docs/getting-started/opentelemetry.md): Enable OpenTelemetry in Altertable, configure an exporter for logs or traces, and verify recent telemetry from your service with SQL. ## Explore Altertable - [Lakehouse](/docs/lakehouse.md): Understand managed tables, external catalogs, DuckDB SQL, compute, and storage, then choose how to connect or ingest data into Altertable. - [Agents](/docs/agents.md): Connect your AI client to Altertable, ask questions about data, and add business context through skills, semantic models, repositories, and memories. - [Data apps](/docs/data-apps.md): Build interactive data apps with an agent, query your Altertable catalogs, review the results, and share the app with your intended audience. - [Product Analytics](/docs/product-analytics.md): Product behavior data in the same runtime as revenue, operations, analytics, and agents—one lakehouse for product and business questions. - [OpenTelemetry](/docs/opentelemetry.md): Ingest OpenTelemetry logs, traces, and metrics into Altertable, then query observability data alongside product and business data. - [Automation](/docs/automation.md): Schedule recurring SQL, code, and AI work, then deliver the resulting findings to the right people in the right environment. - [Administration](/docs/administration.md): Learn how Altertable organizations and environments separate account administration from isolated data workspaces. - [Reference](/docs/api-references.md): Find Altertable API references for SQL queries, ingestion, OpenTelemetry, product analytics, authentication, and MCP tools. --- url: https://altertable.ai/docs/lakehouse/catalogs.md --- # Altertable catalogs Altertable catalogs are hosted lakehouse databases you create and manage on the Altertable platform. They are the default place to store new data when you want Altertable to handle the catalog, storage layout, and query path for you. Unlike [external catalogs](/docs/lakehouse/external-catalogs.md), which connect existing databases, warehouses, or buckets, Altertable catalogs are managed by Altertable from the start. They use Altertable [Workers](/docs/lakehouse/compute/workers.md) for compute and [Buckets](/docs/lakehouse/storage/buckets.md) for durable object storage. Each table in an Altertable catalog is backed by Parquet files written to an underlying bucket. Because Altertable controls the full path from SQL execution to file layout, Altertable catalogs are the most optimized catalog type available on the platform. ## When to use Altertable catalogs Use an Altertable catalog when you want to: - Create new lakehouse tables without provisioning your own database. - Store durable data in open Parquet files. - Let Altertable optimize file layout, caching, and query execution together. - Query managed tables alongside external catalogs in the same SQL engine. ## Create a catalog 1. Open **Catalogs** and choose **New catalog**. 2. Choose **Default** to create a managed Altertable catalog. 3. Enter a name and choose a [bucket](/docs/lakehouse/storage/buckets.md) for its storage. 4. Choose **Create catalog**. Continue with [Ingest your first data](/docs/getting-started/ingest-data.md) to write a record and query it back. Use the catalog's name in fully qualified table references, such as `my_catalog.main.events`. ## Learn more - **[Workers](/docs/lakehouse/compute/workers.md)**: Workers that execute SQL and cache hot data - **[Buckets](/docs/lakehouse/storage/buckets.md)**: Buckets that store catalog files - **[External Catalogs](/docs/lakehouse/external-catalogs.md)**: Connect existing databases, warehouses, and buckets - **[Endorsement](/docs/lakehouse/endorsement.md)**: Steer which catalogs and tables agents use --- url: https://altertable.ai/docs/lakehouse/clients/arrow-flight-sql.md --- # Arrow Flight SQL ## Goal Connect an Arrow Flight SQL client to Altertable over TLS, run a query, and confirm that the client can stream Arrow record batches. ## Prerequisites - Your [lakehouse username and password](/docs/administration/lakehouse-credentials.md). - An Arrow Flight SQL-compatible client. - Access to at least one table you can query. ## Steps ### 1. Configure the endpoint | Hostname | Port | | ---------------------- | ------------- | | `flight.altertable.ai` | 443 (TLS/SSL) | [Arrow Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html) provides the highest-performance interface for streaming large analytical datasets between Altertable and your tools. Built on Apache Arrow's columnar format, it's ideal for: - **Analytics tools** like dbt, Airbyte, and custom data pipelines - **Large result sets** that benefit from efficient columnar data transfer - **High-throughput workloads** requiring minimal serialization overhead ### 2. Configure authentication Use your Altertable credentials with any Arrow Flight SQL client: - **Username**: Your lakehouse username - **Password**: Your lakehouse password ### 3. Run your first query ### Arrow Flight SQL ```python # Altertable exposes a native Arrow Flight SQL endpoint. # Connect any Arrow Flight SQL-compatible client — no extra setup needed. # Python (altertable-flightsql-python) from altertable_flightsql import Client # Connect to flight.altertable.ai:443 w/ TLS/SSL with Client(username="{{LAKEHOUSE_USERNAME}}", password="{{LAKEHOUSE_PASSWORD}}") as client: # Execute a query reader = client.query("SELECT * FROM users WHERE age > 18") # Process results for batch in reader: df = batch.data.to_pandas() print(df) ``` ### 4. Use the Python client Altertable provides an official Python client library ([`altertable-flightsql`](https://github.com/altertable-ai/altertable-flightsql-python)) that handles authentication, connection management, queries, prepared statements, transactions, and metadata operations. Install the package: ```bash pip install altertable-flightsql ``` #### Prepared statements Prepare a statement once and execute it multiple times with different parameters: ```python with client.prepare("SELECT * FROM users WHERE id = $id") as stmt: result = stmt.query(parameters={"id": 1}) for batch in result: print(batch.data.to_pandas()) ``` #### Transactions Execute multiple statements atomically: ```python with client.begin_transaction(): client.execute("INSERT INTO users ...") client.execute("UPDATE accounts ...") ``` #### Metadata queries Discover catalogs, schemas, and tables: ```python catalogs = client.get_catalogs() schemas = client.get_schemas(catalog="my_db") tables = client.get_tables(catalog="my_db", schema_pattern="main") ``` For more details, examples, and advanced usage, see the [GitHub repository](https://github.com/altertable-ai/altertable-flightsql-python). ### 5. Connect another client Any Arrow Flight SQL-compatible client can connect to `flight.altertable.ai:443` with TLS. **Python ADBC** ```python import adbc_driver_flightsql.dbapi as flight_sql conn = flight_sql.connect( uri="grpc+tls://flight.altertable.ai:443", db_kwargs={ "username": "{{LAKEHOUSE_USERNAME}}", "password": "{{LAKEHOUSE_PASSWORD}}" } ) cursor = conn.cursor() cursor.execute("SELECT * FROM your_database.your_schema.your_table LIMIT 10") results = cursor.fetchall() ``` ## Verification Run a query that does not depend on an existing table: ```python from altertable_flightsql import Client with Client( username="{{LAKEHOUSE_USERNAME}}", password="{{LAKEHOUSE_PASSWORD}}", ) as client: for batch in client.query("SELECT 1 AS connected"): print(batch.data.to_pandas()) ``` The result should contain one `connected` column with the value `1`. ## Troubleshooting - **Authentication fails:** copy the lakehouse username and password from the same environment again. - **The TLS connection fails:** connect to `flight.altertable.ai:443` and keep TLS enabled. - **A table cannot be found:** use `get_catalogs()`, `get_schemas()`, and `get_tables()` to confirm its fully qualified name. - **A generic client cannot connect:** confirm that it supports Arrow Flight SQL rather than only the Arrow in-memory format. ## Next steps - [Query with the HTTP API](/docs/lakehouse/query/http-api.md) when HTTPS and JSONL fit the integration better. - [Connect a BI tool](/docs/lakehouse/clients.md) through Arrow Flight SQL or the Postgres adapter. - See the [Python client repository](https://github.com/altertable-ai/altertable-flightsql-python) for advanced examples. --- url: https://altertable.ai/docs/lakehouse/clients/cli.md --- # Altertable CLI Install the CLI, log in, and run a first query. You need an Altertable account with access to an [organization and environment](/docs/administration.md). ## Set up with your agent ```text Set up the Altertable CLI in this environment. Follow https://altertable.ai/docs/lakehouse/clients/cli.md. Detect my operating system, shell, and any existing CLI installation. Install or reuse the supported CLI and preserve existing profiles. Guide me through browser login, then confirm the intended organization and environment with the active profile. Verify lakehouse credentials are available without printing them. Run SELECT 1 AS connected and confirm the returned value is 1. Use --agent when the agent consumes structured CLI output. Report the installed version, active profile, verification result, and any remaining setup. Do not ingest data or change remote resources during this setup. ``` ## 1. Install the CLI Run the installer in your terminal: ```bash curl -fsSL https://install.altertable.ai | sh ``` **Other installation methods** Install from npm: ```bash npm install -g @altertable/cli ``` You can also download a binary for your operating system from [GitHub Releases](https://github.com/altertable-ai/altertable-cli/releases). ## 2. Log in ```bash altertable login ``` Complete browser login and choose your organization and environment. The CLI creates or reuses a local profile with those settings. The first lakehouse command automatically provisions temporary credentials for that profile. You do not need to copy credentials for this login-based workflow. For environment-variable configuration, supply `ALTERTABLE_BASIC_AUTH_TOKEN` or `ALTERTABLE_LAKEHOUSE_USERNAME` and `ALTERTABLE_LAKEHOUSE_PASSWORD` explicitly. Automatic provisioning applies to stored profiles with management access. See [profile configuration](/docs/api-references/cli.md#altertable-profile-configure) for manually configured profiles. ## 3. Verify your first query Check that the active profile points to the intended environment, then run a query that needs no existing table: ```bash altertable profile show altertable query "SELECT 1 AS connected" ``` The query should return a `connected` column with the value `1`. Next, [ingest your first data](/docs/getting-started/ingest-data.md) and read it back. **More CLI workflows** - [Query options, SQL files, and output formats](/docs/api-references/cli.md#altertable-query) - [Append records](/docs/api-references/cli.md#altertable-append), [upload files](/docs/api-references/cli.md#altertable-upload), and [upsert rows](/docs/api-references/cli.md#altertable-upsert) - [Manage profiles](/docs/api-references/cli.md#altertable-profile) and [log in as a service account](/docs/api-references/cli.md#altertable-login) - [Call the management API](/docs/api-references/cli.md#altertable-api) - [Develop data apps](/docs/api-references/cli.md#altertable-app) - [Use the CLI with agents](/docs/agents/cli.md) For all commands and global flags, see the [CLI reference](/docs/api-references/cli.md). ## Troubleshooting - **Credentials expired:** run `altertable login` again. - **Wrong environment:** [switch to the correct profile](/docs/api-references/cli.md#altertable-profile-switch). - **Login works but queries fail:** check the profile's [lakehouse credentials](/docs/administration/lakehouse-credentials.md). --- url: https://altertable.ai/docs/lakehouse/clients/grafana.md --- # Grafana availability The Altertable Grafana data source is not generally available. [Contact us](/contact-us) for access and build instructions. For telemetry already in Altertable, use the app's logs and traces explorers or [query with SQL](/docs/opentelemetry/query.md). For other lakehouse data, use [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) or [connect a supported SQL client](/docs/lakehouse/clients.md). --- url: https://altertable.ai/docs/lakehouse/clients/hex.md --- # Hex [Hex](https://hex.tech/) is a collaborative data workspace that combines SQL, Python, R, and no-code tools in a single platform. It enables data teams to explore, analyze, and share insights through interactive notebooks and dashboards. ## Connection Setup Hex connects to Altertable using our [Postgres adapter](/docs/lakehouse/clients/postgres.md), which works with Hex's built-in PostgreSQL database support. ### Configuration Steps 1. In Hex, navigate to **Data** → **Warehouse** → **Add data connection** 2. Select **Postgres** as the connection type 3. Configure the connection with the following details: | Field | Value | | ------------ | ------------------------------------- | | Display name | `Altertable` (or your preferred name) | | Host | `postgres.altertable.ai` | | Port | `443` | | Database | Your Altertable catalog name | | Username | Your lakehouse username | | Password | Your lakehouse password | | SSL/TLS | ✓ **Required** | 4. Click **Save** to save the connection ## Verify the connection Run `SELECT 1 AS connected` in a Hex SQL cell. Then query a table from your catalog and compare the returned rows with the source. Queries use [DuckDB SQL](/docs/lakehouse/sql-engine.md), even though the connection uses the Postgres protocol. ## Notes - **SQL Dialect**: While the connection uses the Postgres protocol, queries are executed using **DuckDB SQL dialect**. PostgreSQL-specific functions may not be available. - **SSL/TLS Required**: Make sure SSL/TLS is enabled in the connection settings - **Catalog Name**: Enter your Altertable catalog name in the **Database** field --- url: https://altertable.ai/docs/lakehouse/clients.md --- # SQL and BI clients Start with the [CLI](/docs/lakehouse/clients/cli.md) for a first terminal query. Choose a protocol adapter when connecting an existing SQL client or BI tool. The PostgreSQL adapter provides protocol compatibility. Queries still execute with [DuckDB SQL](/docs/lakehouse/sql-engine.md); PostgreSQL protocol support does not imply that every PostgreSQL function or extension is available. - [Altertable CLI](/docs/lakehouse/clients/cli.md): Install the Altertable CLI, log in to your environment, and verify your first query. Find advanced commands and options in the CLI reference. - [Connect a Postgres client](/docs/lakehouse/clients/postgres.md): Connect to Altertable with the Postgres wire protocol so SQL clients and BI tools can query governed lakehouse data. - [Arrow Flight SQL](/docs/lakehouse/clients/arrow-flight-sql.md): High-performance streaming analytics with Arrow Flight SQL. Built on Apache Arrow, ideal for dbt, Airbyte, and large datasets. - [Hex](/docs/lakehouse/clients/hex.md): Connect Hex to Altertable for collaborative analysis with SQL, Python, and R. Access all connected data sources via federated queries. - [Metabase](/docs/lakehouse/clients/metabase.md): Connect Metabase to Altertable with Arrow Flight SQL or the Postgres adapter to query governed lakehouse data in BI dashboards. --- url: https://altertable.ai/docs/lakehouse/clients/metabase.md --- # Metabase [Metabase](https://www.metabase.com/) is an open source business intelligence tool for visualizing and exploring data. ## Connection Methods You can connect Metabase to Altertable using two methods: ### Method 1: Arrow Flight SQL (Recommended) Arrow Flight SQL is the preferred connection method for Altertable because it provides native columnar transport. Metabase does not support Arrow Flight SQL by default, so you need a community driver. #### Installation Install the [metabase-flightsql-driver](https://github.com/J0hnG4lt/metabase-flightsql-driver) to add Arrow Flight SQL support to Metabase: 1. Download the driver JAR file from the GitHub repository 2. Place it in your Metabase plugins directory 3. Restart Metabase #### Configuration 1. In Metabase, navigate to **Admin settings** → **Databases** 2. Click **Add database** 3. Select **Arrow Flight SQL** as the database type 4. Configure the connection with the following details: | Field | Value | | ------------- | ------------------------------------- | | Display name | `Altertable` (or your preferred name) | | Host | `flight.altertable.ai` | | Port | `443` | | Database name | Your Altertable catalog name | | Username | Your lakehouse username | | Password | Your lakehouse password | | Use TLS/SSL | ✓ **Enabled** | 5. Click **Save** ### Method 2: Postgres Adapter (Alternative) If you prefer not to install the Arrow Flight SQL driver, you can connect Metabase to Altertable through our [Postgres adapter](/docs/lakehouse/clients/postgres.md), which works with Metabase's built-in PostgreSQL database support. #### Configuration 1. In Metabase, navigate to **Admin settings** → **Databases** 2. Click **Add database** 3. Select **PostgreSQL** as the database type 4. Configure the connection with the following details: | Field | Value | | ----------------------------- | ------------------------------------- | | Display name | `Altertable` (or your preferred name) | | Host | `postgres.altertable.ai` | | Port | `443` | | Database name | Your Altertable catalog name | | Username | Your lakehouse username | | Password | Your lakehouse password | | Use a secure connection (SSL) | ✓ **Enabled** | 5. Click **Save** ### Notes - **SQL Dialect**: While the connection uses the Postgres protocol, queries are executed using **DuckDB SQL dialect**. PostgreSQL-specific functions may not be available. - **SSL/TLS Required**: Make sure to enable SSL/TLS in the connection settings - **Catalog Name**: Enter your Altertable catalog name in the **Database name** field --- url: https://altertable.ai/docs/lakehouse/clients/postgres.md --- # Connect a Postgres client | Hostname | Port | | ------------------------ | ------------- | | `postgres.altertable.ai` | 443 (TLS/SSL) | The Postgres adapter lets you connect to Altertable through the PostgreSQL wire protocol. It works with BI tools and SQL clients that already speak Postgres. This adapter is ideal for: - **BI tools** like Hex, Looker, Tableau, Power BI, and Metabase - **SQL clients** such as DBeaver and pgAdmin ## Authentication Use your lakehouse username and lakehouse password with TLS enabled. ## SQL Dialect While the connection uses the Postgres protocol, queries are executed using **DuckDB SQL dialect**. This means that Postgres-specific functions (e.g., `string_agg`, `generate_series`) may not work. ## Quick Start ### Postgres ```sql -- Altertable speaks the Postgres wire protocol natively. -- No SDK needed — any Postgres client works out of the box. -- psql psql "postgresql://{{LAKEHOUSE_USERNAME}}:{{LAKEHOUSE_PASSWORD}}@postgres.altertable.ai:443/my_db" ``` ## Connection examples **psql** ```bash PGSSLMODE=require psql -h postgres.altertable.ai -p 443 -U {{LAKEHOUSE_USERNAME}} your_database ``` **Python (psycopg2)** ```python import psycopg2 conn = psycopg2.connect( host="postgres.altertable.ai", port=443, database="your_database", user="{{LAKEHOUSE_USERNAME}}", password="{{LAKEHOUSE_PASSWORD}}", sslmode="require" ) ``` ## TLS negotiation The Postgres adapter supports standard PostgreSQL SSL negotiation on port 443. If a connection times out before authentication, check whether your network performs TLS inspection on port 443. Some configurations, including Cloudflare Gateway TLS inspection, may reject PostgreSQL's plaintext SSLRequest preamble. - Preferred: bypass TLS inspection for `postgres.altertable.ai`. - For clients using libpq 17 or newer, add `sslnegotiation=direct` alongside `sslmode=require`. - Clients without TLS-direct support, including Node `pg`, require the network bypass. --- url: https://altertable.ai/docs/lakehouse/compute.md --- # Compute and performance Workers execute SQL independently of object storage. Choose query compute for the workload, then use query plans and measured performance to decide whether to change compute or table layout. - [Workers](/docs/lakehouse/compute/workers.md): Workers are the DuckDB-based compute units that execute queries, cache data, and connect Altertable to your sources. - [Choose compute for a query](/docs/lakehouse/compute/sizing.md): Choose automatic or fixed compute sizing for lakehouse queries, understand available Worker tiers, and account for session reuse constraints. - [Performance](/docs/lakehouse/compute/performance.md): Performance characteristics for Workers, cache behavior, and how we benchmark query latency and cost on the Altertable lakehouse. --- url: https://altertable.ai/docs/lakehouse/compute/performance.md --- # Performance We regularly test our architecture, infrastructure, and query engine at scale to ensure Altertable delivers consistent performance as your data grows. ## Benchmarking One benchmark we use is TPC-H at scale factor 100. It provides a repeatable workload for query performance comparisons. Benchmark numbers are directional: real-world performance depends on data shape, storage layout, query mix, and cache warmth. ### Query Time Average TPCH Benchmark (sf=100) results - lower is better | Database | Size | Query time | | ---------- | ------- | ---------- | | Altertable | | <1s | | Snowflake | X-Large | 1s | | ClickHouse | | 2s | | Databricks | | 3s | | BigQuery | | 3s | | Fabric | | 3s | | Snowflake | X-Small | 4s | | Redshift | | 4s | ## Cold and cached queries A cold query may need to download data from object storage to the [Worker](/docs/lakehouse/compute/workers.md) before executing it. Later queries can reuse the Worker’s local cache, reducing storage reads and latency. Compare queries with similar cache conditions when measuring performance. ## Choose the next optimization Start with the slow query and its scan size: - [Inspect the query](/docs/api-references/lakehouse.md#post-explain) to estimate rows, files, and bytes scanned. - [Choose compute](/docs/lakehouse/compute/sizing.md) for the workload. - [Optimize table layout](/docs/lakehouse/features/optimize-table.md) when file layout or compaction limits performance. --- url: https://altertable.ai/docs/lakehouse/compute/sizing.md --- # Choose compute for a query Without per-query sizing, every statement can run on the same compute profile—whether it scans ten rows or ten billion. Pass `"compute_size": "AUTO"` (the default in SQL Explorer) and Altertable picks a size **per query** from the SQL itself. A narrow `LIMIT 100` over a large table can run on `XS`; a full-table aggregate may need `XL`. ```json { "statement": "SELECT id, email FROM users WHERE created_at > now() - interval '7 days' LIMIT 100", "compute_size": "AUTO" } ``` ## How `AUTO` works Before execution, Altertable runs `EXPLAIN` on the statement and walks the physical plan to estimate how much parallelism the query needs. It then selects the smallest tier that fits: | Size | Threads | Typical fit | | ----- | ------- | ------------------------------------------------------ | | `XS` | 2 | Point lookups, small scans, queries with early `LIMIT` | | `S` | 4 | Moderate scans | | `M` | 8 | Larger scans or joins | | `L` | 16 | Heavy joins or aggregations | | `XL` | 32 | Full-table scans and large shuffles | | `2XL` | 64 | Large parallel workloads | | `3XL` | 96 | Large parallel workloads | | `4XL` | 128 | Large parallel workloads | **Constraints:** - `AUTO` cannot be combined with an explicit `session_id`. Reuse sessions when you need temporary state; pick a fixed size when you need both session reuse and predictable compute. Requested sizes cannot exceed the environment’s provider maximum. Actual allocation can be lower when compute allowances are constrained. ## Fixed sizes Override inference when you know the workload or need repeatable performance: ```json { "statement": "SELECT ...", "compute_size": "M" } ``` Use a fixed tier for long-running batch jobs, dashboards that should always warm the same worker profile, or queries where you have already benchmarked the right size. --- url: https://altertable.ai/docs/lakehouse/compute/workers.md --- # Workers **Workers** are the unit of compute in Altertable. They execute DuckDB SQL, read and write lakehouse files, connect to data sources, cache hot data locally, and stream query results back to your application. Workers are stateless. You can scale compute without moving storage, and idle compute can disappear while the catalog remains durable in object storage. ## Managed Workers Managed Workers run on Altertable infrastructure and are the default for new environments. You can create environments on [Hetzner](https://www.hetzner.com/) in **Falkenstein, Germany (`fsn1`)** or on **AWS in Ireland (`eu-west-1`)**. Google Cloud environments are planned. Request access from the environment creation screen if you need GCP for residency or procurement requirements. When inbound traffic is restricted, allow [Altertable's egress IP addresses](/docs/administration/network.md#egress-ip-addresses) so managed Workers can connect. Self-hosted Workers use your own network instead. ## Allocate compute Each environment receives compute units from your organization's plan. Allocate at least 2 compute units per environment, in increments of 2. You can rebalance the allocation from the environments list as workloads change, but the total allocation cannot exceed the compute units available on the plan. This environment allocation sets the compute capacity available to its workloads. Individual HTTP queries can still request a fixed Worker size or use [`AUTO`](/docs/lakehouse/compute/sizing.md) within that capacity. ## Self-hosted Workers Self-hosted Workers are designed for teams that need compute inside their own cloud. A Worker can run from a container image, so queries run close to your buckets and databases instead of moving large datasets across cloud boundaries. That keeps data egress predictable and helps avoid cross-cloud transfer fees. ## Learn more - **[Performance](/docs/lakehouse/compute/performance.md)**: Benchmarking, cache behavior, and query latency - **[Network](/docs/administration/network.md)**: Egress IP addresses and SSH tunnels for reaching restricted catalogs - **[Encryption](/docs/administration/encryption.md)**: How credentials and data are encrypted in transit and at rest --- url: https://altertable.ai/docs/lakehouse/endorsement.md --- # Endorsement Endorsement tells AI agents which tables to reach for first. Each table has one of three statuses, set from the **Endorsement** menu on a table or catalog: | Status | Effect on agents | | -------- | -------------------------------------------------------------------- | | Verified | Preferred when several tables could answer. | | Draft | The default. Fully usable, with no preference either way. | | Excluded | Omitted from default discovery. Used only when the request names it. | Statuses are set per environment, so a table can be verified in production and draft in staging. ## Excluded data stays reachable Excluding a table keeps it out of what agents discover on their own. The table is not deleted, and when a request names it, the agent can still query it. Semantic metrics stay available even when they are built on an excluded table. ## Catalog level An [Altertable catalog](/docs/lakehouse/catalogs.md) or [external catalog](/docs/lakehouse/external-catalogs.md) has its own endorsement status. Tables discovered later inherit it. Setting a catalog's status overwrites the status of its existing tables, so review table statuses before you change a catalog that mixes them. Excluded catalogs are left out of the catalog lists agents see. They stay usable by name. ## Endorsement is not a permission Endorsement steers agents. It does not control who can read data. A user or service account with access to a catalog can still query an excluded table. To restrict access, assign [catalog roles](/docs/administration/access-control.md). ## Learn more - **[Catalogs](/docs/lakehouse/catalogs.md)**: Create hosted catalogs - **[Connect existing data](/docs/lakehouse/external-catalogs.md)**: Connect external catalogs - **[Access control](/docs/administration/access-control.md)**: Restrict who can read data --- url: https://altertable.ai/docs/lakehouse/external-catalogs/apache-polaris.md --- # Apache Polaris [Apache Polaris](https://polaris.apache.org/) is an open-source Iceberg REST catalog. Connect it to Altertable when Polaris manages your Iceberg table metadata and you want to query those tables alongside the rest of your lakehouse. ## Connect Apache Polaris 1. Open **Catalogs** in the Altertable app. 2. Click **New catalog**. 3. Select **Apache Polaris**. 4. Enter the catalog URI, such as `https://polaris.example.com/api/catalog`, the warehouse (the name of the Polaris catalog), and the default namespace for unqualified table names. Tables in other namespaces stay queryable. 5. Choose an authentication method and enter its credentials. 6. Save the catalog, then query the Iceberg tables with SQL. ## Authentication - **Client credentials**: the client ID and client secret of a Polaris principal. Altertable requests tokens from `/v1/oauth/tokens` with the `PRINCIPAL_ROLE:ALL` scope unless you set a different token endpoint or scope. - **Bearer token**: a static access token. Altertable sends it as is, so replace it in the catalog settings before it expires. Altertable reads table data with the storage credentials Polaris vends for each table, so the catalog must have credential vending configured. Grant the principal role only the catalog privileges Altertable needs, such as `TABLE_READ_DATA` for read-only access. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/aws-glue.md --- # AWS Glue [AWS Glue Data Catalog](https://docs.aws.amazon.com/glue/latest/dg/components-overview.html) can act as the catalog for Iceberg tables stored in AWS. Connect Glue to Altertable when Glue is the source of truth for table metadata and Altertable should query those Iceberg tables alongside other lakehouse data. ## Prerequisites Prepare the AWS account ID, region, and IAM role ARN. The role must allow reading Glue catalog metadata and the underlying Iceberg objects. Configure its trust relationship for the principal that will assume it; obtain the required trust details for your Altertable environment before saving the connection. A read-only policy must cover: - `glue:GetDatabase`, `glue:GetDatabases`, `glue:GetTable`, and `glue:GetTables` on the catalog, intended databases, and tables. - `s3:ListBucket` on the data bucket, scoped to the relevant prefixes. - `s3:GetObject` on those prefixes, including Iceberg metadata, manifests, and data files. - `kms:Decrypt` on the key when those objects use customer-managed KMS encryption. If AWS Lake Formation governs the tables, configure its permissions as well as IAM. Use the AWS [Glue IAM guide](https://docs.aws.amazon.com/glue/latest/dg/security_iam_service-with-iam.html) to scope the role and its trust policy. ## Connect AWS Glue 1. Create or choose an IAM role with the Glue and storage permissions required for the tables Altertable should query. 2. Open **Catalogs** in the Altertable app. 3. Click **New catalog**. 4. Select **AWS Glue**. 5. Enter the AWS account ID, region, and role ARN. 6. Save the catalog, then query the Glue-backed tables with SQL. Scope the IAM role to the catalogs, databases, tables, and storage locations Altertable needs. If table data is encrypted with AWS KMS, include the required decrypt permissions. ## Verify the connection Open the saved catalog and check that the intended database and tables appear. In [SQL Explorer](/docs/lakehouse/query/sql-explorer.md), replace the example names and read a known table: ```sql SELECT * FROM my_catalog.my_database.my_table LIMIT 10; ``` Compare a known record with its source and inspect the returned column types. Metadata discovery alone does not verify S3 access. ## Troubleshooting - **Role cannot be assumed:** check the role ARN and trust policy for the configured environment. - **Database or table is missing:** check the region, Glue IAM resources, and Lake Formation grants. - **Tables appear but data cannot be read:** check S3 prefixes, object permissions, and KMS decrypt access. - **Schema changed upstream:** inspect the source table metadata and recheck the catalog before changing queries. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/bigquery.md --- # BigQuery Add your BigQuery project as an external catalog in Altertable to query and analyze it alongside your other data sources. ## Prerequisites Before connecting BigQuery to Altertable, you'll need to configure your Google Cloud project: ### 1. Enable the BigQuery API Enable the BigQuery API within your Google Cloud project. ### 2. Check Organization Policies Ensure your organization doesn't have the `constraints/iam.disableServiceAccountKeyCreation` policy enforced: ```bash # Check at project level gcloud resource-manager org-policies describe constraints/iam.disableServiceAccountKeyCreation --project=PROJECT_ID # Or check at organization level gcloud resource-manager org-policies describe constraints/iam.disableServiceAccountKeyCreation --organization=ORG_ID ``` If enforced, you can disable it with: ```bash gcloud resource-manager org-policies disable-enforce constraints/iam.disableServiceAccountKeyCreation --project=PROJECT_ID ``` ### 3. Create a Service Account Create a service account ([learn how to create and download a service account key](https://docs.cloud.google.com/iam/docs/keys-create-delete#creating)) and grant it roles for the access you need. #### Read-only access (default) Grant these 3 roles: - **BigQuery Data Viewer** (`roles/bigquery.dataViewer`): Read data and metadata. Grant it on the datasets Altertable needs to read. - **BigQuery Job User** (`roles/bigquery.jobUser`): Run jobs, including queries. Grant it on the project used to run queries. - **BigQuery Read Session User** (`roles/bigquery.readSessionUser`): Create and use read sessions through the BigQuery Storage Read API. Grant it on the project used to run queries. #### Read-write access Grant these 3 roles when Altertable also needs to write data: - **BigQuery Data Editor** (`roles/bigquery.dataEditor`): Read and write data, and create, update, and delete tables. Grant it on the datasets Altertable needs to read and write, or on the project if it also needs to create datasets. - **BigQuery Job User** (`roles/bigquery.jobUser`): Run jobs, including queries. Grant it on the project used to run queries. - **BigQuery Read Session User** (`roles/bigquery.readSessionUser`): Create and use read sessions through the BigQuery Storage Read API. Grant it on the project used to run queries. See [Google Cloud's BigQuery IAM roles](https://docs.cloud.google.com/bigquery/docs/access-control) for the full permissions of each role. **If you cannot grant the roles**, ensure your admin user has the `orgpolicy.policyAdmin` role: ```bash gcloud organizations add-iam-policy-binding ORG_ID \ --member="user:YOUR_ADMIN_USER_EMAIL" \ --role="roles/orgpolicy.policyAdmin" ``` **Grant missing roles** with: ```bash gcloud projects add-iam-policy-binding PROJECT_ID \ --member="serviceAccount:SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com" \ --role="THE_MISSING_ROLE" ``` **Validate the roles** with: ```bash gcloud projects get-iam-policy PROJECT_ID \ --flatten="bindings[].members" \ --filter="bindings.members:serviceAccount:SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com" \ --format="table(bindings.role)" ``` ## Connecting to Altertable To add a new BigQuery external catalog: 1. Navigate to the **Catalogs** section in the dashboard 2. Click **New catalog** 3. Select **BigQuery** as the engine 4. Fill in the connection details: - **Name**: A descriptive name for your catalog - **Service Account Credentials**: Upload your service account JSON key file 5. Click **Save** ![New BigQuery connection form](/screenshots/components-connectionform--big-query-empty.png) *New BigQuery connection form* Once added, you can start querying your BigQuery data through Altertable's SQL engine. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/bucket-tables.md --- # Bucket Tables Bucket Tables expose files from a connected [bucket](/docs/lakehouse/storage/buckets.md) as queryable external tables. Use them when Parquet, CSV, or JSON files already exist in object storage and you want to query them without copying the data into an Altertable catalog. Each table maps a logical table name to a path in the bucket. Queries read the files at that path through the lakehouse SQL engine. ## When to use Bucket Tables Use Bucket Tables for: - Data exports that already land in object storage. - Shared Parquet datasets owned by another pipeline. - CSV or JSON files that need to be joined with other lakehouse sources. - Hive-style partitioned paths where partition keys should be available as columns. If the files are an Iceberg dataset, use [Iceberg Tables](/docs/lakehouse/external-catalogs/iceberg-tables.md) instead so Altertable can read Iceberg metadata. ## Connect Bucket Tables 1. Open **Catalogs** in the Altertable app. 2. Click **New catalog**. 3. Select **Bucket Tables**. 4. Choose the connected bucket that contains the files. 5. Select the file format: Parquet, CSV, or JSON. 6. Add one or more table mappings with a table name and path. 7. Save the catalog, then query the mapped tables with SQL. ## Table mappings Use stable, descriptive table names and point each one at the file or prefix Altertable should read: | Field | Description | Example | | ---------- | --------------------------------------------------- | ----------------- | | Table name | The table name exposed in the external catalog | `events` | | Path | The file path or prefix inside the connected bucket | `exports/events/` | After the catalog is connected, query tables with fully qualified names such as `bucket_exports.main.events`. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/duckdb.md --- # Connect a DuckDB database DuckDB external catalogs expose an existing `.duckdb` database file that already lives in a connected bucket. Use this when another process produces a DuckDB database and you want to query it from Altertable without importing it into an Altertable catalog. ## Requirements - A [bucket](/docs/lakehouse/storage/buckets.md) connected to the environment - A DuckDB database file stored in that bucket - A path ending in `.duckdb` DuckDB catalogs are read-only in Altertable. ## Connect a DuckDB Catalog 1. Open **Catalogs** in the target environment. 2. Click **New catalog** and choose **DuckDB**. 3. Select the bucket that contains the DuckDB file. 4. Enter the path to the `.duckdb` file, such as `exports/reporting.duckdb`. 5. Test and create the catalog. After the catalog is created, its schemas and tables appear in the catalog browser and can be queried with SQL. ## Query Example ```sql SELECT customer_id, total_revenue FROM reporting_duckdb.main.customer_revenue ORDER BY total_revenue DESC LIMIT 25; ``` You can join DuckDB catalog tables with Altertable catalogs and other external catalogs in the same query. ## When to Use an Altertable Catalog Instead Use an [Altertable catalog](/docs/lakehouse/catalogs.md) when Altertable should own the storage layout, snapshots, writes, and retention. Use a DuckDB external catalog when the `.duckdb` file is produced elsewhere and Altertable only needs read access. For other DuckDB workflows, see the [DuckDB integration hub](/docs/lakehouse/integrations/duckdb.md). --- url: https://altertable.ai/docs/lakehouse/external-catalogs/hugging-face.md --- # Hugging Face > **Beta:** > > The Hugging Face external catalog is available in beta. Its syntax and behavior may change as the implementation evolves. Hugging Face external catalogs expose datasets hosted on the [Hugging Face Hub](https://huggingface.co/datasets) as tables. Files are read in place from the Hub: nothing is copied into Altertable. ## Requirements - One or more dataset repositories on the Hugging Face Hub, stored as Parquet, CSV or JSON files - For private or gated datasets, a Hugging Face [access token](https://huggingface.co/settings/tokens) with read access to those datasets Hugging Face catalogs are read-only in Altertable. ## Connect a Hugging Face Catalog 1. Open **Catalogs** in the target environment. 2. Click **New catalog** and choose **Hugging Face**. 3. Paste an access token, or leave it empty for public datasets. 4. Select the file format of the datasets. 5. Map each table name to a dataset location, written as `owner/dataset[@revision][/path]`. 6. Test and create the catalog. | Location | Reads | | ----------------------------- | ------------------------------------------------- | | `acme/train-records` | Every file of the default branch | | `acme/train-records/data` | Every file under `data/` on the default branch | | `acme/train-records@v3/data` | Every file under `data/` at the `v3` tag | | `acme/train-records@~parquet` | The Parquet conversion maintained by Hugging Face | The token is only sent to the dataset repositories mapped in the catalog. ## Query Example ```sql SELECT language, stage, label, count(*) AS n FROM hf.main.train_records GROUP BY ALL ORDER BY ALL; ``` ## Query Other Revisions Every mapped table also gets a `_at(rev)` table function that reads the same dataset at any branch, tag or commit: ```sql SELECT block, label, count(*) AS n FROM hf.main.train_mix_at('v2') GROUP BY ALL; ``` Pin to a commit hash for fully reproducible results, since branches and tags can move. ## Performance Each query reads the dataset files from the Hub, so large scans are bound by download speed and by [Hugging Face rate limits](https://huggingface.co/docs/hub/rate-limits), and queries you run again download the same files again. For a dataset you query repeatedly, copy it into an Altertable catalog once and keep the copy in sync instead: 1. Open the Altertable catalog that should hold the copy — not this Hugging Face catalog, which is read-only. 2. On its **Ingest** tab, click **New synchronizer** in **Synchronizers**. 3. Choose **External catalog** as the source type, select this Hugging Face catalog, and pick the schema and tables to copy. 4. Choose the destination schema and a schedule. Each run replaces the copied tables with the latest data from the Hub, so queries read the local copy instead of downloading the dataset again. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/iceberg-tables.md --- # Iceberg Tables Iceberg Tables expose existing Apache Iceberg datasets from a connected [bucket](/docs/lakehouse/storage/buckets.md). Use them when another system already owns the Iceberg table metadata and data files, and Altertable should query those tables without copying them. Each mapping gives Altertable a logical table name and the Iceberg table path in the bucket. Altertable reads the Iceberg metadata to resolve snapshots, manifests, and data files. ## When to use Iceberg Tables Use Iceberg Tables when: - The dataset is already stored as Apache Iceberg in object storage. - You know the bucket and path for each table you want to expose. - Another engine or pipeline remains responsible for writing the Iceberg tables. Use [Bucket Tables](/docs/lakehouse/external-catalogs/bucket-tables.md) for plain Parquet, CSV, or JSON files that are not managed by Iceberg metadata. ## Connect Iceberg Tables 1. Open **Catalogs** in the Altertable app. 2. Click **New catalog**. 3. Select **Iceberg Tables**. 4. Choose the connected bucket that stores the Iceberg datasets. 5. Add one or more table mappings with a table name and Iceberg table path. 6. Save the catalog, then query the mapped tables with SQL. ## Table mappings | Field | Description | Example | | ---------- | ----------------------------------------------------- | ------------------- | | Table name | The table name exposed in Altertable | `orders` | | Path | The Iceberg table root or metadata path in the bucket | `warehouse/orders/` | After the catalog is connected, query tables with fully qualified names such as `iceberg_external.main.orders`. --- url: https://altertable.ai/docs/lakehouse/external-catalogs.md --- # Connect existing data External catalogs connect your existing systems to Altertable, making them queryable alongside your [Altertable catalogs](/docs/lakehouse/catalogs.md) and other data sources. If inbound traffic is restricted, allow [Altertable's egress IP addresses](/docs/administration/network.md#egress-ip-addresses) so managed Workers can reach the catalog. For network-restricted database-style catalogs, Altertable can also connect through an **SSH tunnel**. Configure the bastion host while creating the catalog, then copy the generated public key from the catalog details page into the SSH server's `authorized_keys`. After connecting, open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) and verify a small query using a fully qualified `catalog.schema.table` name. - [Connect a Postgres database](/docs/lakehouse/external-catalogs/postgres.md): Add Postgres as an external catalog in Altertable for unified analytics. Query across Postgres and other sources with the federated query engine. - [Supabase](/docs/lakehouse/external-catalogs/supabase.md): Add Supabase as an external catalog in Altertable for unified analytics using Transaction Pooler. Query and analyze data across Supabase and other sources. - [MySQL](/docs/lakehouse/external-catalogs/mysql.md): Add MySQL as an external catalog for federated analytics. Query MySQL alongside other sources through Altertable's SQL engine. - [MariaDB](/docs/lakehouse/external-catalogs/mariadb.md): Add MariaDB as an external catalog in Altertable for unified analytics. Query across MariaDB and other sources with federated query engine. - [MongoDB](/docs/lakehouse/external-catalogs/mongodb.md): Add MongoDB as an external catalog in Altertable. Query Atlas or self-hosted collections as SQL tables alongside your other data sources. - [BigQuery](/docs/lakehouse/external-catalogs/bigquery.md): Add BigQuery as an external catalog in Altertable for unified analytics. Configure service account credentials and required roles to query BigQuery data. - [Snowflake](/docs/lakehouse/external-catalogs/snowflake.md): Add Snowflake as an external catalog in Altertable so teams can query warehouse data with lakehouse analytics. - [Redshift](/docs/lakehouse/external-catalogs/redshift.md): Add Amazon Redshift as an external catalog in Altertable for unified analytics. Query and analyze data across Redshift and other sources with federated queries. - [Connect a DuckDB database](/docs/lakehouse/external-catalogs/duckdb.md): Connect an existing DuckDB database file from object storage as an external catalog for federated queries in Altertable. - [Hugging Face](/docs/lakehouse/external-catalogs/hugging-face.md): Query public and private datasets hosted on the Hugging Face Hub as an external catalog, pinned to any branch, tag or commit. - [Bucket Tables](/docs/lakehouse/external-catalogs/bucket-tables.md): Map files in a connected bucket to external Altertable tables and query object storage data from the lakehouse. - [Iceberg Tables](/docs/lakehouse/external-catalogs/iceberg-tables.md): Map existing Iceberg tables in connected object storage to external tables in Altertable for unified lakehouse queries. - [R2 Data Catalog](/docs/lakehouse/external-catalogs/r2-data-catalog.md): Connect Cloudflare R2 Data Catalog to Altertable as an Iceberg REST catalog for open, governed lakehouse analytics. - [S3 Tables](/docs/lakehouse/external-catalogs/s3-tables.md): Connect Amazon S3 Tables to Altertable as an Iceberg REST catalog for managed, open-table lakehouse analytics. - [AWS Glue](/docs/lakehouse/external-catalogs/aws-glue.md): Connect AWS Glue Data Catalog to Altertable so teams can query Iceberg tables through a shared lakehouse foundation. - [Apache Polaris](/docs/lakehouse/external-catalogs/apache-polaris.md): Connect an Apache Polaris catalog to Altertable and query its Iceberg tables through the Iceberg REST API. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/mariadb.md --- # MariaDB Add your MariaDB database as an external catalog in Altertable to query and analyze it alongside your other data sources. Before connecting, make sure Altertable can reach your source. Configure [network access](/docs/administration/network.md) if it requires an IP allowlist or SSH tunnel. Use credentials for the source database, rather than Altertable lakehouse credentials. ## Connect the catalog 1. Open **Catalogs** and choose **New catalog**. 2. Select **MariaDB** as the engine. 3. Fill in the connection details: - **Host**: The hostname or IP address of your MariaDB server - **Port**: The port number (default: 3306) - **Database**: The name of the database to connect to - **Schema** (optional): The name of the schema to connect to - **Username**: Your MariaDB username - **Password**: Your MariaDB password 4. Choose **Connect** to create the catalog. ![New MariaDB connection form](/screenshots/components-connectionform--maria-db-empty.png) *New MariaDB connection form* ## Verify the connection Open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) and inspect the new catalog's schemas and tables. Replace the example identifiers with names from your source: ```sql SELECT * FROM my_source.my_schema.my_table LIMIT 10; ``` Compare the returned rows with the source. You can then [join tables across catalogs](/docs/lakehouse/sql-engine.md#federated-queries). --- url: https://altertable.ai/docs/lakehouse/external-catalogs/mongodb.md --- # MongoDB Add a MongoDB database as an external catalog in Altertable to query its collections with SQL alongside your other data sources. Each collection in the connected database appears as a table. ## Prerequisites ### Allow Altertable egress IPs If inbound traffic is restricted—including **MongoDB Atlas** IP access lists—allow [Altertable's egress IP addresses](/docs/administration/network.md#egress-ip-addresses) so Workers can connect. For Atlas: 1. Open the Atlas project and go to **Network Access** 2. Click **Add IP Address** 3. Add each [egress IP address](/docs/administration/network.md#egress-ip-addresses) (Atlas accepts the address or `address/32`) 4. Confirm the entries are active before you test the catalog in Altertable You can also connect a self-hosted MongoDB server through an [SSH tunnel](/docs/lakehouse/external-catalogs.md). ### Create a database user Use a MongoDB database user, not your Atlas organization login. In Atlas, create the user under **Database Access** with read access to the database you will query. ## Connecting to Altertable To add a new MongoDB external catalog: 1. Navigate to the **Catalogs** section in the dashboard 2. Click **New catalog** 3. Select **MongoDB** as the engine 4. Fill in the connection details: - **Connection string** (optional): Paste a `mongodb://` or `mongodb+srv://` URL to fill Host, username, SRV, and related options. Enter the password separately. Atlas connection strings often omit the database name, so you still need to set **Database**. - **Host**: The hostname. For Atlas, use the hostname from the connection string, such as `cluster0.xxxxx.mongodb.net`—without `mongodb+srv://` or credentials. - **Port**: Default `27017`. Leave it empty when **SRV connection** is enabled. - **Database**: The MongoDB database whose collections you want to query (for example, `sample_mflix`). Collections in that database appear as tables. - **Username**: The database user - **Password**: The database user password - **SRV connection**: Use a `mongodb+srv` connection and discover hosts through DNS - **TLS**: Encrypt the connection with TLS. Atlas SRV connections already use TLS, so you typically leave this off unless you need a custom CA. - **Authentication source** (optional): Database where the user is defined, often `admin` - **CA certificate** (optional): Custom CA used to verify the MongoDB server 5. Click **Save** > **Atlas: enable SRV connection:** > > Atlas clusters advertise hosts through DNS SRV records (`mongodb+srv`). Enable **SRV connection**. If you leave it off, Altertable can still connect, but collections often fail to appear in the schema. ![New MongoDB connection form](/screenshots/app-components-connections-connection-form--mongo-empty.png) *New MongoDB connection form* Once added, collections from the connected database show up as tables in the catalog browser. You can query them with DuckDB SQL: ```sql SELECT * FROM mongo.sample_mflix.comments LIMIT 100; ``` If the catalog is named `mongo` and the database is `sample_mflix`, the fully qualified table name is `mongo.sample_mflix.comments`. ## Atlas mapping | Atlas | Altertable | | ----------------------------------------------- | -------------------------------------------- | | Cluster hostname (`cluster0.xxxxx.mongodb.net`) | **Host** | | Database (`sample_mflix`) | **Database** (schema in the catalog browser) | | Collection (`comments`) | Table | | Database user | **Username** and **Password** | --- url: https://altertable.ai/docs/lakehouse/external-catalogs/mysql.md --- # MySQL Add your MySQL database as an external catalog in Altertable to query and analyze it alongside your other data sources. Before connecting, make sure Altertable can reach your source. Configure [network access](/docs/administration/network.md) if it requires an IP allowlist or SSH tunnel. Use credentials for the source database, rather than Altertable lakehouse credentials. ## Connect the catalog 1. Open **Catalogs** and choose **New catalog**. 2. Select **MySQL** as the engine. 3. Fill in the connection details: - **Host**: The hostname or IP address of your MySQL server - **Port**: The port number (default: 3306) - **Database**: The name of the database to connect to - **Schema** (optional): The name of the schema to connect to - **Username**: Your MySQL username - **Password**: Your MySQL password 4. Choose **Connect** to create the catalog. ![New MySQL connection form](/screenshots/components-connectionform--my-sql-empty.png) *New MySQL connection form* ## Verify the connection Open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) and inspect the new catalog's schemas and tables. Replace the example identifiers with names from your source: ```sql SELECT * FROM my_source.my_schema.my_table LIMIT 10; ``` Compare the returned rows with the source. You can then [join tables across catalogs](/docs/lakehouse/sql-engine.md#federated-queries). --- url: https://altertable.ai/docs/lakehouse/external-catalogs/postgres.md --- # Connect a Postgres database Add your Postgres database as an external catalog in Altertable to query and analyze it alongside your other data sources. Before connecting, make sure Altertable can reach your source. Configure [network access](/docs/administration/network.md) if it requires an IP allowlist or SSH tunnel. Use credentials for the source database, rather than Altertable lakehouse credentials. ## Connect the catalog 1. Open **Catalogs** and choose **New catalog**. 2. Select **PostgreSQL** as the engine. 3. Fill in the connection details: - **Host**: The hostname or IP address of your Postgres server - **Port**: The port number (default: 5432) - **Database**: The name of the database to connect to - **Schema** (optional): The name of the schema to connect to - **Username**: Your Postgres username - **Password**: Your Postgres password 4. Choose **Connect** to create the catalog. ![New PostgreSQL connection form](/screenshots/components-connectionform--postgres-empty.png) *New PostgreSQL connection form* ## Verify the connection Open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) and inspect the new catalog's schemas and tables. Replace the example identifiers with names from your source: ```sql SELECT * FROM my_source.my_schema.my_table LIMIT 10; ``` Compare the returned rows with the source. You can then [join tables across catalogs](/docs/lakehouse/sql-engine.md#federated-queries). --- url: https://altertable.ai/docs/lakehouse/external-catalogs/r2-data-catalog.md --- # R2 Data Catalog [Cloudflare R2 Data Catalog](https://developers.cloudflare.com/r2-data-catalog/) exposes Iceberg tables through an Iceberg REST catalog. Connect it to Altertable when your Iceberg metadata is managed by Cloudflare and you want to query those tables alongside the rest of your lakehouse. ## Prerequisites Enable the catalog for the intended R2 bucket, then copy its **Warehouse** and **Catalog URI** from Cloudflare. The URI has the form `https://catalog.cloudflarestorage.com//`; use the exact value Cloudflare provides. Create a token with both catalog and storage read permissions. An object-only token cannot list or load Iceberg catalog metadata. For a read-only connection, use Cloudflare's Admin Read only permissions and scope access to the intended resources. See [R2 token permissions](https://developers.cloudflare.com/r2/api/tokens/) for current permission groups and scopes. ## Connect R2 Data Catalog 1. Open **Catalogs** in the Altertable app. 2. Click **New catalog**. 3. Select **R2 Data Catalog**. 4. Enter the warehouse, catalog URI, and token. 5. Save the catalog. 6. Query the exposed Iceberg tables with SQL. Use a token scoped to the warehouse Altertable should read. If permissions change in Cloudflare, update or rotate the token in the catalog settings. ## Verify the connection Open the saved catalog and confirm that the expected namespaces and Iceberg tables appear. In [SQL Explorer](/docs/lakehouse/query/sql-explorer.md), replace the names with a real table: ```sql SELECT * FROM my_catalog.my_namespace.my_table LIMIT 10; ``` Compare a known row with the source. This checks both catalog metadata access and the ability to read the underlying objects. ## Troubleshooting - **Unauthorized:** check the token's catalog and storage permissions, expiry, and resource scope. - **Warehouse not found:** copy the warehouse and URI from the same bucket's catalog settings. - **Tables appear but queries fail:** check the token's object-read access to the table's data and metadata files. - **A rotated token stops working:** update the saved connection, then rerun the known-table query. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/redshift.md --- # Redshift Add your Redshift warehouse as an external catalog in Altertable to query and analyze it alongside your other data sources. Before connecting, make sure Altertable can reach your source. Configure [network access](/docs/administration/network.md) if it requires an IP allowlist or SSH tunnel. Use credentials for the source database, rather than Altertable lakehouse credentials. ## Connect the catalog 1. Open **Catalogs** and choose **New catalog**. 2. Select **Redshift** as the engine. 3. Fill in the connection details: - **Host**: The hostname or IP address of your Redshift server - **Port**: The port number (default: 5439) - **Database**: The name of the database to connect to - **Schema** (optional): The name of the schema to connect to - **Username**: Your Redshift username - **Password**: Your Redshift password 4. Choose **Connect** to create the catalog. ![New Redshift connection form](/screenshots/components-connectionform--redshift-empty.png) *New Redshift connection form* ## Verify the connection Open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) and inspect the new catalog's schemas and tables. Replace the example identifiers with names from your source: ```sql SELECT * FROM my_source.my_schema.my_table LIMIT 10; ``` Compare the returned rows with the source. You can then [join tables across catalogs](/docs/lakehouse/sql-engine.md#federated-queries). --- url: https://altertable.ai/docs/lakehouse/external-catalogs/s3-tables.md --- # S3 Tables [Amazon S3 Tables](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables.html) provides an Iceberg-backed table bucket managed by AWS. Connect S3 Tables to Altertable when your Iceberg tables live in an S3 table bucket and should be queryable from the Altertable lakehouse. ## Prerequisites You need a table bucket ARN, its AWS region, and credentials that can discover namespaces and tables and read their metadata and data. Use a dedicated principal with access to the intended table bucket. For read-only access, attach a policy like this to that principal. Replace the region, account ID, and bucket name in both resources: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3tables:GetTableBucket", "s3tables:GetNamespace", "s3tables:ListNamespaces", "s3tables:ListTables" ], "Resource": "arn:aws:s3tables:us-east-1:123456789012:bucket/my-table-bucket" }, { "Effect": "Allow", "Action": [ "s3tables:GetTable", "s3tables:GetTableMetadataLocation", "s3tables:GetTableData" ], "Resource": "arn:aws:s3tables:us-east-1:123456789012:bucket/my-table-bucket/table/*" } ] } ``` Restrict table resources further when only specific tables should be readable. Cross-account access also requires an appropriate table bucket or table resource policy. Encrypted tables may need KMS permissions. See AWS's [S3 Tables access management](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-setting-up.html) and [action/resource reference](https://docs.aws.amazon.com/service-authorization/latest/reference/list_s3tables.html). ## Connect S3 Tables 1. Open **Catalogs** in the Altertable app. 2. Click **New catalog**. 3. Select **S3 Tables**. 4. Enter the table bucket ARN, region, access key ID, and secret access key. 5. Save the catalog. 6. Query the exposed tables with SQL. Use least-privilege AWS credentials that grant Altertable only the table bucket access it needs. ## Verify the connection Open the saved catalog and confirm that the expected namespaces and tables appear. In [SQL Explorer](/docs/lakehouse/query/sql-explorer.md), query one known table, replacing all three parts with the names shown in Altertable: ```sql SELECT * FROM my_catalog.my_namespace.my_table LIMIT 10; ``` Compare a known row and column types with the source. Listing a table does not prove that its data files are readable. ## Troubleshooting - **Cannot list namespaces or tables:** check the region, bucket ARN, and bucket-level IAM actions. - **Tables appear but queries fail:** check table-level data/metadata permissions, resource policies, and any KMS decrypt grant. - **The expected table is missing:** confirm its namespace and that the principal can read that table's ARN. - **Credentials were rotated:** update the saved catalog connection and rerun the known-row check. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/snowflake.md --- # Snowflake Add your Snowflake warehouse as an external catalog in Altertable to query and analyze it alongside your other data sources. ## Prerequisites ### Setup Snowflake Network Policy Before connecting Snowflake to Altertable, you'll need to configure your Snowflake network policy to allow connections from Altertable. Update your Snowflake network policy to include [Altertable's egress IP addresses](/docs/administration/network.md#egress-ip-addresses) in the allowed list. #### Example: Creating a Network Policy You can create a network rule in and network policy to allow Altertable's IP address: ```sql -- Create a network rule for Altertable's IP CREATE NETWORK RULE altertable_network_rule MODE = INGRESS TYPE = IPV4 VALUE_LIST = ('46.224.237.164', '46.4.253.101', '94.130.221.56', '188.40.149.154', '138.199.242.155', '138.199.241.17'); -- Create a network policy that uses the network rule CREATE NETWORK POLICY altertable_network_policy ALLOWED_NETWORK_RULE_LIST = ('altertable_network_rule'); -- Apply the policy to your user -- WARNING: This action may revoke the user's access to the Snowflake dashboard. -- Using a dedicated user is recommended. ALTER USER "" SET NETWORK_POLICY = altertable_network_policy; -- Or add the rule to an existing policy ALTER NETWORK POLICY your_existing_policy ADD ALLOWED_NETWORK_RULE_LIST = ('altertable_network_rule'); ``` This ensures that Altertable can establish connections to your Snowflake account. Refer to Snowflake's [network policy documentation](https://docs.snowflake.com/en/user-guide/network-policies) for more details on managing network policies. ### Create an Access Token > **Recommendation: use an access token:** > > A programmatic access token isn't required—Altertable can authenticate with your password—but using one is recommended because it lets you: > > - Avoid sharing your Snowflake account password > - Bypass multi-factor authentication (TOTP) > - Revoke Altertable's access at any time without changing your password Generate one for the user and role you want Altertable to connect with: ```sql ALTER USER "" ADD PROGRAMMATIC ACCESS TOKEN altertable_token ROLE_RESTRICTION = '' DAYS_TO_EXPIRY = 365; ``` ## Connecting to Altertable To add a new Snowflake external catalog: 1. Navigate to the **Catalogs** section in the dashboard 2. Click **New catalog** 3. Select **Snowflake** as the engine 4. Fill in the connection details: - **Snowflake Account URL**: Your Snowflake account URL ([learn how to find your account URL](https://docs.snowflake.com/en/user-guide/admin-account-identifier)) - **Snowflake Warehouse**: The warehouse to use for queries ([learn more about warehouses](https://docs.snowflake.com/en/user-guide/warehouses-overview)) - **Username**: Your Snowflake username - **Access Token**: Your Snowflake access token - **Database**: The name of the database to connect to - **Schema** (optional): The name of the schema to connect to 5. Click **Save** ![New Snowflake connection form](/screenshots/components-connectionform--snowflake-empty.png) *New Snowflake connection form* Once added, you can start querying your Snowflake data through Altertable's SQL engine. --- url: https://altertable.ai/docs/lakehouse/external-catalogs/supabase.md --- # Supabase Add your Supabase database as an external catalog in Altertable to query and analyze it alongside your other data sources. Before connecting, make sure Altertable can reach your source. Configure [network access](/docs/administration/network.md) if it requires an IP allowlist or SSH tunnel. Use credentials for the source database, rather than Altertable lakehouse credentials. ## Connect the catalog 1. Open **Catalogs** and choose **New catalog**. 2. Select **Supabase** as the engine. 3. Fill in the connection details: - **Host**: The hostname or IP address of your Supabase server - **Port**: The port number (default: 5432) - **Database**: The name of the database to connect to - **Schema** (optional): The name of the schema to connect to - **Username**: Your Supabase username - **Password**: Your Supabase password 4. Choose **Connect** to create the catalog. ![New Supabase connection form](/screenshots/app-components-connections-connection-form--supabase-empty.png) *New Supabase connection form* ## Verify the connection Open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) and inspect the new catalog's schemas and tables. Replace the example identifiers with names from your source: ```sql SELECT * FROM my_source.my_schema.my_table LIMIT 10; ``` Compare the returned rows with the source. You can then [join tables across catalogs](/docs/lakehouse/sql-engine.md#federated-queries). ## Caveats You will need to use **Transaction Pooler** connection method. Direct connection and session pooler methods don't work with Altertable for now. --- url: https://altertable.ai/docs/lakehouse/features/data-change-feed.md --- # Data Change Feed Snapshots record every committed change to an Altertable catalog. [Time travel](/docs/lakehouse/features/time-travel.md) reads **state** — what a table looked like at a snapshot or timestamp. The data change feed reads **deltas** — which rows were inserted, updated, or deleted between two bounds. Both features share the same snapshot timeline. Each committed [transaction](/docs/lakehouse/features/transactions.md) creates one snapshot, and bounds can be snapshot IDs or timestamps in either API. Run the statements below through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md) against an [Altertable catalog](/docs/lakehouse/catalogs.md) you own. ## Time travel vs change feed Use time travel when you need a full table image at a point in time: ```sql SELECT sku, label FROM my_catalog.main.product_skus AT (VERSION => 2); ``` Use the change feed when you need the row-level mutations that happened in or between snapshots: ```sql FROM my_catalog.table_changes('product_skus', 2, 4) ORDER BY snapshot_id; ``` | Goal | Use | | ----------------------------------------------- | --------------------------------------------------------------------------------- | | Reproduce a report as it ran last Tuesday | [Time travel](/docs/lakehouse/features/time-travel.md) with `AT (TIMESTAMP => …)` | | Sync only new and changed rows to a warehouse | `table_insertions` / `table_changes` between snapshot bounds | | Investigate who removed a SKU and when | `table_deletions` or `table_changes` for the suspect snapshot range | | Compare catalog state before and after a deploy | Time travel at two snapshot versions | ## Finding snapshot bounds Pass snapshot IDs or timestamps to `table_changes` and its helpers. List available snapshots with `snapshots()`, or read the latest ID with `current_snapshot()` — the same helpers described in [Time travel](/docs/lakehouse/features/time-travel.md#snapshots). ```sql SELECT snapshot_id, snapshot_time, changes, author, commit_message FROM my_catalog.snapshots() ORDER BY snapshot_id; ``` Commit messages from `set_commit_message` are especially useful here: they label snapshots in the change feed the same way they appear in the snapshot history. ## Example workflow The workflow below uses the same `product_skus` table referenced in the time travel docs. Each comment marks the snapshot created by that statement: ```sql -- Snapshot 1 CREATE TABLE my_catalog.main.product_skus (sku VARCHAR, label VARCHAR); -- Snapshot 2 INSERT INTO my_catalog.main.product_skus VALUES ('SKU-001', 'Oak shelf'), ('SKU-002', 'Maple bench'); -- Snapshot 3 DELETE FROM my_catalog.main.product_skus WHERE sku = 'SKU-001'; -- Snapshot 4 UPDATE my_catalog.main.product_skus SET label = concat(label, ' (v2)'); ``` At snapshot 2 the table contains two rows — the same result as `AT (VERSION => 2)` in [Time travel](/docs/lakehouse/features/time-travel.md). Snapshots 3 and 4 each add one kind of delta. ### Changes from a single snapshot Inspect everything that happened in snapshot 2: ```sql FROM my_catalog.table_changes('product_skus', 2, 2); ``` | snapshot\_id | rowid | change\_type | sku | label | | ------------ | ----- | ------------ | ------- | ----------- | | 2 | 0 | insert | SKU-001 | Oak shelf | | 2 | 1 | insert | SKU-002 | Maple bench | ### Changes across multiple snapshots Follow the delete in snapshot 3 and the update in snapshot 4: ```sql FROM my_catalog.table_changes('product_skus', 3, 4) ORDER BY snapshot_id; ``` | snapshot\_id | rowid | change\_type | sku | label | | ------------ | ----- | ----------------- | ------- | ---------------- | | 3 | 0 | delete | SKU-001 | Oak shelf | | 4 | 1 | update\_preimage | SKU-002 | Maple bench | | 4 | 1 | update\_postimage | SKU-002 | Maple bench (v2) | ### Changes over a time range Bounds accept timestamps as well as snapshot IDs: ```sql FROM my_catalog.table_changes('product_skus', now() - INTERVAL '1 week', now()); ``` ## table\_changes `table_changes` takes a table name and two inclusive bounds: a start snapshot and an end snapshot. Bounds can be snapshot IDs or timestamps. The result uses the table schema as of the end snapshot, plus three extra columns: | Column | Description | | ------------- | ------------------------------------------------------------ | | `snapshot_id` | The snapshot that made the change | | `rowid` | The row identifier of the changed row | | `change_type` | `insert`, `update_preimage`, `update_postimage`, or `delete` | Updates appear as two rows: `update_preimage` is the row before the update; `update_postimage` is the row after the update. When the table schema changes between the start and end bounds, results use the schema as of the end snapshot. If a column is dropped, it is omitted from the entire result. If a column is added, changes before the addition show the column's default value. ## table\_deletions `table_deletions` returns only rows deleted between two snapshots. It takes the same arguments as `table_changes`: ```sql FROM my_catalog.table_deletions('product_skus', 3, 3); ``` | snapshot\_id | rowid | sku | label | | ------------ | ----- | ------- | --------- | | 3 | 0 | SKU-001 | Oak shelf | The result includes `snapshot_id` and `rowid`, followed by the table's data columns. There is no `change_type` column because every row is a deletion. ## table\_insertions `table_insertions` returns only rows inserted between two snapshots: ```sql FROM my_catalog.table_insertions('product_skus', 2, 2); ``` | snapshot\_id | rowid | sku | label | | ------------ | ----- | ------- | ----------- | | 2 | 0 | SKU-001 | Oak shelf | | 2 | 1 | SKU-002 | Maple bench | ## Compaction and retention Compaction that expires snapshots limits both time travel and the change feed. If deleted rows are removed during compaction, `table_deletions` can no longer return them even though you know they existed. Set [snapshot retention](/docs/lakehouse/features/time-travel.md#snapshot-retention) with care when downstream pipelines depend on historical change feeds or long-running `AT` queries. ## Learn more - **[Time Travel](/docs/lakehouse/features/time-travel.md)**: Query table state at a snapshot and list snapshot history - **[Transactions](/docs/lakehouse/features/transactions.md)**: Each committed transaction creates one snapshot - **[SQL Engine](/docs/lakehouse/sql-engine.md)**: DuckDB dialect and federated queries --- url: https://altertable.ai/docs/lakehouse/features/full-text-search.md --- # Full-text search Full-text search indexes text columns so queries can find records by words, phrases, and approximate text matches without scanning every value as plain strings. Use it for log messages, trace messages, support tickets, documents, event properties, or any table where users search by text rather than exact IDs. The built-in `opentelemetry` catalog uses full-text indexes on message fields so logs and spans can be searched quickly. ## Create an index Create a full-text index on the column you search most often: ```sql CREATE INDEX logs_message_fts ON opentelemetry.main.logs USING TNVYX(message); ``` Index names must be unique in the table. Build indexes on columns with meaningful free-form text; low-cardinality fields such as status codes are usually better handled with normal filters. ## Query with filters Combine text search with time and service filters so the engine can narrow the scan before ranking or returning matches: ```sql SELECT timestamp, service_name, severity_text, message FROM opentelemetry.main.logs WHERE timestamp >= now() - INTERVAL '1 day' AND service_name = 'api' AND message @@ 'payment timeout' ORDER BY timestamp DESC LIMIT 100; ``` ## When to use it Use full-text search when exact equality, `LIKE`, or JSON field filters are too narrow or too slow for the question. Keep structured predicates in the query whenever possible; they make text search more selective and easier to reason about. ## Learn more - **[OpenTelemetry](/docs/opentelemetry.md)**: logs and traces stored in the lakehouse. - **[SQL Explorer](/docs/lakehouse/query/sql-explorer.md)**: test search queries interactively. - **[SQL engine](/docs/lakehouse/sql-engine.md)**: DuckDB SQL support in Altertable. --- url: https://altertable.ai/docs/lakehouse/features.md --- # SQL capabilities Altertable catalog tables support capabilities that shape how data is written, queried, and maintained over time. Use these features when you need predictable scan performance, historical reads, reusable logic, or incremental downstream processing. Partitioning and sorted tables improve write layout and query pruning. Time travel and views help you inspect history and encapsulate logic. Data change feeds, macros, and optimize-table support operational pipelines and housekeeping. - [Partitioning](/docs/lakehouse/features/partitioning.md): Partition Altertable catalog tables by column or transform to improve write layout and query pruning. - [Sorted tables](/docs/lakehouse/features/sorted-tables.md): Configure a physical sort order on Altertable catalog tables to improve min/max statistics and query pruning. - [Transactions](/docs/lakehouse/features/transactions.md): Run ACID transactions against Altertable catalogs with snapshot isolation and all-or-nothing semantics. - [Time travel](/docs/lakehouse/features/time-travel.md): Query Altertable catalog tables as of earlier snapshots using version or timestamp bounds for audits and reproducible reports. - [Views](/docs/lakehouse/features/views.md): Create, store, and query SQL views in Altertable catalog metadata so teams can reuse governed lakehouse logic. - [Macros](/docs/lakehouse/features/macros.md): Define reusable scalar and table macros in Altertable catalog metadata with standard CREATE MACRO syntax. - [Data change feed](/docs/lakehouse/features/data-change-feed.md): Query inserts, updates, and deletes between catalog snapshots using table\_changes and related functions. - [Full-text search](/docs/lakehouse/features/full-text-search.md): Search text across Altertable tables with SQL-native full-text search for faster discovery across lakehouse data. - [Optimize table](/docs/lakehouse/features/optimize-table.md): Optimize Altertable tables for better storage layout, faster queries, and improved lakehouse performance at scale. - [MATCH\_RECOGNIZE](/docs/lakehouse/features/match-recognize.md): Find ordered row patterns such as funnels, journeys, and state transitions with SQL:2016-style MATCH\_RECOGNIZE. - [SESSIONIZE](/docs/lakehouse/features/sessionize.md): Assign ordered events to gap-based sessions and calculate session-level measures with DuckDB SQL in Altertable. - [JOURNEYS](/docs/lakehouse/features/journeys.md): Discover observed paths from a start event to an end event, including drop-offs, with the JOURNEYS operator in Altertable. --- url: https://altertable.ai/docs/lakehouse/features/journeys.md --- # JOURNEYS > **Beta:** > > `JOURNEYS` is available in beta. Its syntax and behavior may change as the implementation evolves. `JOURNEYS` discovers observed paths over an identity partition. Use it when you need the routes from a start event to an end event, the sequences after a start, or the sequences before an end — including users who never convert. Run these statements through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md). ## First example Suppose an event table contains one row for each user action: ```sql WITH events(user_id, ts, event_id, event) AS ( VALUES (1, TIMESTAMP '2024-01-01 10:00:00', 1, 'Homepage'), (1, TIMESTAMP '2024-01-01 10:01:00', 2, 'Signup'), (1, TIMESTAMP '2024-01-01 10:02:00', 3, 'Docs'), (1, TIMESTAMP '2024-01-01 10:03:00', 4, 'Checkout'), (1, TIMESTAMP '2024-01-01 10:04:00', 5, 'Purchase'), (2, TIMESTAMP '2024-01-01 10:00:00', 1, 'Signup'), (2, TIMESTAMP '2024-01-01 10:01:00', 2, 'Docs'), (2, TIMESTAMP '2024-01-01 10:02:00', 3, 'Checkout'), (2, TIMESTAMP '2024-01-01 10:03:00', 4, 'Purchase'), (3, TIMESTAMP '2024-01-01 10:00:00', 1, 'Pricing'), (3, TIMESTAMP '2024-01-01 10:01:00', 2, 'Signup'), (3, TIMESTAMP '2024-01-01 10:02:00', 3, 'Create project'), (3, TIMESTAMP '2024-01-01 10:03:00', 4, 'Purchase'), (4, TIMESTAMP '2024-01-01 10:00:00', 1, 'Docs'), (4, TIMESTAMP '2024-01-01 10:01:00', 2, 'Signup'), (4, TIMESTAMP '2024-01-01 10:02:00', 3, 'Docs'), (4, TIMESTAMP '2024-01-01 10:03:00', 4, 'Docs') ) SELECT path, converted, truncated, count AS journey_count, avg_duration FROM events JOURNEYS ( PARTITION BY user_id ORDER BY ts, event_id STEP event PATHS BETWEEN (event = 'Signup') AND (event = 'Purchase') MEASURED AS UNIQUES WITHIN INTERVAL 1 DAY MAX STEPS 9 COLLAPSE REPEATS ) ORDER BY journey_count DESC, path; ``` The query starts at each user's first signup and follows events to the earliest purchase within one day. Events before signup are excluded. User 4 signed up and never purchased, so that path stays in the result as an incomplete journey. Consecutive documentation visits collapse into one step. ```text path converted truncated journey_count [Signup, Docs, Checkout, Purchase] true false 2 [Signup, Create project, Purchase] true false 1 [Signup, Docs] false false 1 ``` - `PARTITION BY user_id` creates an independent sequence for each user. - `ORDER BY ts, event_id` defines event order, with `event_id` breaking timestamp ties. - `STEP event` is the value stored in each path position. - `PATHS BETWEEN` keeps events from signup through purchase, or through the end of the window when purchase is missing. - `MEASURED AS UNIQUES` counts one path per user. - `COLLAPSE REPEATS` merges consecutive identical steps. ## Operator syntax The supported `FROM`-clause shape is: ```sql FROM input_relation [input_alias] JOURNEYS ( PARTITION BY identity_column [, ...] ORDER BY time_column [, tie_breaker_column ...] STEP expression [EXPAND BY property WHEN (predicate)] PATHS STARTING WITH (predicate) | PATHS ENDING WITH (predicate) | PATHS BETWEEN (start_predicate) AND (end_predicate) [MEASURED AS UNIQUES | EVENT TOTALS] [WITHIN interval_expression | WITHIN SESSION session_column] [MAX STEPS n] [COLLAPSE REPEATS] [EXCLUDE WHEN (predicate)]... [REQUIRE (predicate)]... ) [journeys_alias] ``` `PARTITION BY`, `ORDER BY`, `STEP`, and one `PATHS` mode are required. `MEASURED AS` defaults to `UNIQUES`. `MAX STEPS` defaults to `9`. The source relation must appear immediately before `JOURNEYS`. The result can be aliased after the clause: ```sql SELECT j.path, j.count FROM events AS e JOURNEYS ( PARTITION BY user_id ORDER BY ts, event_id STEP event PATHS STARTING WITH (event = 'Signup') ) AS j; ``` Common SQL constructs such as CTEs, subqueries, joins, outer filters, grouping, and aliases can surround the operator. ## Choose a path mode ### `PATHS BETWEEN` Start at a start event and scan forward for an end event. This is the signup-to-purchase shape: ```sql PATHS BETWEEN (event = 'Signup') AND (event = 'Purchase') ``` `converted` is `true` when an end event occurs inside the window, and `false` when the path drops off. `JOURNEYS` uses the first start event. It does not skip that start to pick a later start that would convert. With `UNIQUES`, the first start is the only start. With `EVENT TOTALS`, each unused start begins a new instance. After a conversion, the next instance resumes after the end event. After a drop-off, it resumes after the start event. ### `PATHS STARTING WITH` Start at a start event and keep the following visible steps. `converted` is `NULL` because there is no end condition: ```sql PATHS STARTING WITH (event = 'Signup') ``` `UNIQUES` uses the first start. `EVENT TOTALS` starts a path at every start event. Those paths may overlap in the source rows. ### `PATHS ENDING WITH` End at an end event and look backward for the visible steps that precede it. `converted` is `NULL`: ```sql PATHS ENDING WITH (event = 'Purchase') ``` `UNIQUES` uses the last end event. `EVENT TOTALS` ends a path at every end event, looking backward within the window. ## Count uniques or every occurrence `MEASURED AS UNIQUES` emits one instance per identity. `MEASURED AS EVENT TOTALS` emits every selected start or end: ```sql MEASURED AS UNIQUES MEASURED AS EVENT TOTALS ``` Use `UNIQUES` when you want one path per user. Use `EVENT TOTALS` when the same user can start or complete the journey more than once and each attempt should count. ## Result columns `JOURNEYS` aggregates identical paths. The output does not keep one row per user. | Column | Description | | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | | `path` | `LIST` of steps, or `LIST` after `EXPAND BY`. Retained visible steps after exclusion, collapse, and truncation | | `converted` | `BOOLEAN`. `true` or `false` for `PATHS BETWEEN`; `NULL` for starting-only and ending-only modes | | `truncated` | `BOOLEAN`. `true` when the visible path exceeded `MAX STEPS` | | `count` | `BIGINT`. Identities for `UNIQUES`, or selected occurrences for `EVENT TOTALS` | | `min_duration` / `avg_duration` / `max_duration` | `INTERVAL`. Duration of the raw span from the first row to the last row, including excluded events | Duration is `timestamp(end_row) - timestamp(start_row)` of that raw span. A drop-off uses the last row still inside the window. ## Define the event streams ### `PARTITION BY` `PARTITION BY` is required. It splits the input into independent identities: ```sql PARTITION BY user_id PARTITION BY user_id, app_id ``` A path never crosses a partition boundary. Partition expressions must be input columns. ### `ORDER BY` `ORDER BY` is required. The first column is the time key: ```sql ORDER BY ts, event_id ``` It must be an ascending input column with a `DATE`, `TIMESTAMP`, or `TIMESTAMPTZ` type. Additional columns are tie-breakers. Add a stable tie-breaker when multiple events can share a timestamp. ### `STEP` `STEP` is the value stored in each path position. It can be any scalar expression over the input row: ```sql STEP event STEP concat(event, ':', page) ``` A null step is omitted from the visible path. The row can still sit in the raw time span when other predicates match the original row. ### `EXPAND BY` `EXPAND BY` splits a step by a property when a predicate is true: ```sql STEP event EXPAND BY plan WHEN (event = 'Checkout') ``` Matching events become `STRUCT(event, property)`. Non-matching events keep a null `property`. Different property values are different steps, so Checkout on `pro` and Checkout on `free` are distinct paths. A null `WHEN` predicate is false. A true predicate with a null property still emits a visible expanded step. Anchor, exclusion, and requirement predicates evaluate against the original input row, not the expanded struct. ## Bound a path ### `WITHIN` `WITHIN` requires the path to finish inside a clock window: ```sql WITHIN INTERVAL 1 DAY ``` The interval is inclusive: an end event at exactly `start_time + interval` still converts. One unit later does not. The expression must be a non-null, strictly positive, foldable DuckDB `INTERVAL`. Without `WITHIN` or `WITHIN SESSION`, a BETWEEN drop-off runs to the end of the partition. ### `WITHIN SESSION` `WITHIN SESSION` keeps the path inside one session value instead of a clock window: ```sql WITHIN SESSION session_id ``` The start and end must share the same session column value. A null session value is a session break. You cannot combine `WITHIN` and `WITHIN SESSION`. Assign session IDs first with [`SESSIONIZE`](/docs/lakehouse/features/sessionize.md) when the source table does not already have them. ### `MAX STEPS` `MAX STEPS` limits the retained visible path. The default is `9`: ```sql MAX STEPS 9 ``` Paths of 9 visible steps or fewer are untruncated. Longer paths keep the first 9 steps and set `truncated = true`. Conversion and duration still use the full raw span. ### `COLLAPSE REPEATS` `COLLAPSE REPEATS` merges consecutive identical visible steps after expansion and exclusion: ```sql COLLAPSE REPEATS ``` Repeated documentation visits become one `Docs` step. After `EXPAND BY`, the same event with different property values does not collapse. ## Filter events on a path ### `EXCLUDE WHEN` `EXCLUDE WHEN` hides matching rows from the visible path. They remain in the raw time span and still consume window time: ```sql EXCLUDE WHEN (event = 'Heartbeat') ``` An excluded start or end still bounds the span. You can repeat the clause. ### `REQUIRE` `REQUIRE` keeps a candidate only when every required predicate appears in the raw span: ```sql REQUIRE (event = 'Checkout') ``` If a requirement is missing: - `PATHS BETWEEN` with `UNIQUES` emits a drop-off. - `PATHS BETWEEN` with `EVENT TOTALS` skips that start and resumes after it. - Starting-only and ending-only modes skip the candidate. You can repeat the clause. ## Practical patterns ### Split Checkout by plan Use `EXPAND BY` when the same event should become different steps for different property values: ```sql SELECT path, converted, count FROM events JOURNEYS ( PARTITION BY user_id ORDER BY ts, event_id STEP event EXPAND BY plan WHEN (event = 'Checkout') PATHS BETWEEN (event = 'Signup') AND (event = 'Purchase') MEASURED AS EVENT TOTALS WITHIN INTERVAL 1 DAY COLLAPSE REPEATS ) ORDER BY count DESC, path; ``` ### Paths after every start `EVENT TOTALS` counts every start event, including overlapping attempts by the same user: ```sql SELECT path, count FROM events JOURNEYS ( PARTITION BY user_id ORDER BY ts, event_id STEP event PATHS STARTING WITH (event = 'Signup') MEASURED AS EVENT TOTALS WITHIN INTERVAL 1 DAY MAX STEPS 9 ) ORDER BY count DESC, path; ``` ## Learn more - **[Event Paths in DuckDB](/blog/2026-09-17-journeys-in-duckdb)**: why `JOURNEYS` exists and how it compares with SQL joins. - **[MATCH\_RECOGNIZE](/docs/lakehouse/features/match-recognize.md)**: find ordered row patterns with an explicit regular-expression-like pattern. - **[SESSIONIZE](/docs/lakehouse/features/sessionize.md)**: assign events to gap-based sessions. - **[SQL Explorer](/docs/lakehouse/query/sql-explorer.md)**: run and refine path queries interactively. - **[SQL engine](/docs/lakehouse/sql-engine.md)**: learn about Altertable's DuckDB SQL dialect. --- url: https://altertable.ai/docs/lakehouse/features/macros.md --- # Macros Altertable catalogs support scalar and table macros created with standard `CREATE MACRO` syntax. Macros are stored in catalog metadata and can be referenced from queries, view definitions, and sort-order expressions. Macros participate in [time travel](/docs/lakehouse/features/time-travel.md) — querying a catalog at an earlier snapshot reflects the macros that existed at that point. Run the statements below through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md) against an [Altertable catalog](/docs/lakehouse/catalogs.md) you own. ## Scalar macros Scalar macros return a single value: ```sql CREATE MACRO margin_pct(price, cost) AS (price - cost) / price * 100; SELECT margin_pct(49.99, 31.50); ``` ## Table macros Table macros return a table: ```sql CREATE MACRO high_stock_skus(min_qty) AS TABLE SELECT sku, warehouse_id, on_hand FROM my_catalog.main.inventory_levels WHERE on_hand >= min_qty; SELECT * FROM high_stock_skus(500); ``` ## Default parameters Macros can define default values for parameters: ```sql CREATE MACRO apply_discount(price, pct := 10) AS price * (1 - pct / 100.0); SELECT apply_discount(120.00); ``` ## Typed parameters Macros can specify types for their parameters: ```sql CREATE MACRO pallet_count(units INTEGER, units_per_pallet INTEGER) AS ceil(units / units_per_pallet); ``` ## Drop macros Drop scalar macros with `DROP MACRO` and table macros with `DROP MACRO TABLE`: ```sql DROP MACRO margin_pct; DROP MACRO TABLE high_stock_skus; ``` ## When to use macros Use macros to centralize repeated expressions — date bucketing, unit conversions, or shared filter logic — so queries stay readable and changes propagate from one definition. For sort orders and views that depend on a macro, define the macro before referencing it in `SET SORTED BY` or `CREATE VIEW`. ## Learn more - **[Sorted Tables](/docs/lakehouse/features/sorted-tables.md)**: Use macros in expression-based sort keys - **[Views](/docs/lakehouse/features/views.md)**: Encapsulate queries behind stable names - **[SQL Engine](/docs/lakehouse/sql-engine.md)**: DuckDB dialect and federated queries --- url: https://altertable.ai/docs/lakehouse/features/match-recognize.md --- # MATCH\_RECOGNIZE `MATCH_RECOGNIZE` finds sequences of rows that match a regular-expression-like pattern. Use it for funnels, customer journeys, state transitions, sessions, anomaly shapes, and other ordered event-stream problems that are difficult to express with self-joins and window functions. Run these statements through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md). ## First example Suppose an event table contains one row for each user action: ```sql WITH events(user_id, ts, event) AS ( VALUES (1, TIMESTAMP '2024-01-01 10:00:00', 'view'), (1, TIMESTAMP '2024-01-01 10:01:00', 'click'), (1, TIMESTAMP '2024-01-01 10:02:00', 'view'), (2, TIMESTAMP '2024-01-01 10:00:00', 'view') ) SELECT * FROM events MATCH_RECOGNIZE ( PARTITION BY user_id ORDER BY ts MEASURES A.ts AS view_ts, B.ts AS click_ts PATTERN (A B) DEFINE A AS event = 'view', B AS event = 'click' ) ORDER BY user_id, view_ts; ``` This query finds a `view` immediately followed by a `click` for each user: - `PARTITION BY user_id` creates an independent sequence for each user. - `ORDER BY ts` defines the order in which rows are consumed. - `A` and `B` are pattern variables that describe row roles. - `DEFINE` specifies which rows qualify for each role. - `PATTERN (A B)` requires two consecutive rows: an `A` row followed by a `B` row. - `MEASURES` defines the columns in each result row. Matches are non-overlapping. After a match is emitted, matching resumes after the last consumed row. Rows that cannot start a complete match are skipped. ## Operator syntax The supported `FROM`-clause shape is: ```sql FROM input_relation [input_alias] MATCH_RECOGNIZE ( [PARTITION BY partition_column [, ...]] [ORDER BY input_column [ASC | DESC] [NULLS FIRST | NULLS LAST] [, ...]] [MEASURES measure_expression AS output_name [, ...]] [ONE ROW PER MATCH | ALL ROWS PER MATCH [SHOW EMPTY MATCHES | OMIT EMPTY MATCHES | WITH UNMATCHED ROWS]] [AFTER MATCH SKIP PAST LAST ROW | TO NEXT ROW | TO FIRST variable | TO LAST variable] PATTERN (row_pattern) [WITHIN interval_expression] [SUBSET subset_name = (variable [, ...]) [, ...]] [DEFINE pattern_variable AS boolean_expression [, ...]] ) [match_alias] ``` `PATTERN` is required. `ORDER BY`, `MEASURES`, `PARTITION BY`, `SUBSET`, and `DEFINE` are optional, but the query must produce at least one output column. The source relation must appear immediately before `MATCH_RECOGNIZE`. The result can be aliased after the clause: ```sql SELECT m.user_id, m.start_ts FROM events AS e MATCH_RECOGNIZE ( PARTITION BY user_id ORDER BY ts MEASURES A.ts AS start_ts PATTERN (A) DEFINE A AS event = 'view' ) AS m; ``` Common SQL constructs such as CTEs, subqueries, joins, outer filters, grouping, and aliases can surround the operator. ## Define the row sequence ### `PARTITION BY` `PARTITION BY` splits the input into independent streams. A pattern never crosses a partition boundary: ```sql PARTITION BY user_id, channel ``` The partition columns appear in the output before the measure columns. Without `PARTITION BY`, the entire input is treated as one partition. ### `ORDER BY` `ORDER BY` defines the sequence that the pattern consumes: ```sql ORDER BY ts, sequence_number ``` It supports input columns and `ASC`, `DESC`, `NULLS FIRST`, and `NULLS LAST`. If the primary ordering column can tie, add a stable secondary column so matching is deterministic. When `ORDER BY` is omitted, rows keep their input order within each partition. Add an outer `ORDER BY` when result order matters, especially when the query uses multiple partitions or multiple threads. ### `DEFINE` `DEFINE` maps pattern variables to Boolean predicates: ```sql DEFINE A AS event = 'view', B AS event = 'click', C AS event NOT IN ('view', 'click') ``` The predicate is evaluated against the current input row. A `NULL` predicate does not match; only `TRUE` qualifies. An undefined pattern variable matches every row, but explicitly defining every variable makes the query intent clearer. ## Write patterns Pattern variables represent row roles. The pattern language supports: - `A` — one row matching variable `A` - `A B` — concatenation - `A | B` — alternation - `(A B)` — grouping - `A*`, `A+`, `A?` — greedy repetition - `A{n}`, `A{n,}`, `A{n,m}` — bounded repetition - `A*?`, `A+?`, `A??`, and similar forms — reluctant repetition - `PERMUTE(A, B, …)` — any ordering of the listed variables - `^` and `$` — partition-start and partition-end anchors - `{- A -}` — exclusion from `ALL ROWS PER MATCH` output Patterns can be composed: ```sql PATTERN ((A B) | (A C* B) | A) ``` Alternatives are evaluated from left to right. In the example above, a complete `A B` match is preferred, then `A C* B`, then the single-row `A` fallback. Repetition is greedy by default. ## Return match results ### `MEASURES` `MEASURES` defines the output columns. Every expression needs an alias: ```sql MEASURES A.ts AS start_ts, B.ts AS end_ts, B.ts - A.ts AS elapsed ``` In a measure, `A.column` returns the value from the last row assigned to `A`. An unqualified column such as `ts` returns the value from the last row of the whole match. Pattern expressions also support navigation and pattern functions, including: - `PREV`, `NEXT`, `FIRST`, and `LAST` - `CLASSIFIER()` - `MATCH_NUMBER()` - Pattern aggregates such as `COUNT`, `SUM`, `AVG`, `MIN`, and `MAX` - Top-level `RUNNING` and `FINAL` prefixes, such as `RUNNING LAST(value)` or `FINAL COUNT(*)` If a variable did not participate in the selected alternative, referencing it produces `NULL`. ### Rows per match `ONE ROW PER MATCH` is the default and emits one summary row for each successful match: ```sql ONE ROW PER MATCH ``` `ALL ROWS PER MATCH` emits one row for each matched input row. It supports modifiers for empty and unmatched rows: ```sql ALL ROWS PER MATCH SHOW EMPTY MATCHES ALL ROWS PER MATCH OMIT EMPTY MATCHES ALL ROWS PER MATCH WITH UNMATCHED ROWS ``` With `ONE ROW PER MATCH`, the output contains partition columns followed by measure columns. With `ALL ROWS PER MATCH`, the output also includes order columns and remaining input columns. ### `AFTER MATCH SKIP` Choose where matching resumes after a match: ```sql AFTER MATCH SKIP PAST LAST ROW AFTER MATCH SKIP TO NEXT ROW AFTER MATCH SKIP TO FIRST variable AFTER MATCH SKIP TO LAST variable ``` `PAST LAST ROW` is the default. A skip target that does not advance the search, or that refers to a missing variable, raises an error. ### `SUBSET` Create a union variable that refers to multiple pattern variables: ```sql SUBSET milestone = (A, B) ``` Union variables can be used in `MEASURES`, `DEFINE`, and `AFTER MATCH SKIP TO FIRST` or `TO LAST`. ## Bound a match by time Add `WITHIN` after `PATTERN` to require that a match completes within an interval: ```sql PATTERN (A B C) WITHIN INTERVAL 30 SECOND ``` The interval is inclusive: the last timestamp can equal the first timestamp plus the interval. `WITHIN` requires an ascending temporal first `ORDER BY` key, such as `DATE`, `TIMESTAMP`, or `TIMESTAMPTZ`. The interval must be a non-null, strictly positive, foldable DuckDB `INTERVAL`. ## Practical patterns ### Funnel with filler events Use a repeated variable when unrelated events may occur between two milestones: ```sql SELECT * FROM events MATCH_RECOGNIZE ( PARTITION BY user_id ORDER BY ts MEASURES A.ts AS view_ts, B.ts AS click_ts PATTERN (A C* B) DEFINE A AS event = 'view', B AS event = 'click', C AS event NOT IN ('view', 'click') ) ORDER BY user_id, view_ts; ``` `C*` consumes zero or more filler rows. It cannot consume rows that do not match `C`, and the trailing `B` still needs to complete the pattern. ### Preferred path with a fallback Use an alternation when a longer match should win, but a shorter match should still produce a result: ```sql SELECT * FROM events MATCH_RECOGNIZE ( PARTITION BY user_id ORDER BY ts MEASURES A.ts AS start_ts, B.ts AS click_ts PATTERN ((A B) | A) DEFINE A AS event = 'view', B AS event = 'click' ) ORDER BY user_id, start_ts; ``` When a `view` is followed by a `click`, `(A B)` wins and consumes both rows. When no click follows, the query falls back to `A`, producing a row with `click_ts = NULL`. ## Pattern recognition in windows Pattern recognition is also supported in named or inline window structures. A pattern window produces one result row per input row: ```sql SELECT user_id, value OVER w, label OVER w FROM events WINDOW w AS ( PARTITION BY user_id ORDER BY ts MEASURES RUNNING LAST(value) AS value, CLASSIFIER() AS label ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING AFTER MATCH SKIP PAST LAST ROW SEEK PATTERN (A B+) DEFINE B AS value < PREV(B.value) ); ``` Window pattern frames must start at `CURRENT ROW`. The frame can end at `CURRENT ROW`, a constant `n FOLLOWING`, or `UNBOUNDED FOLLOWING`. `INITIAL` and `SEEK`, `AFTER MATCH SKIP`, and `SUBSET` are supported in pattern windows. Anchors and `MATCH_NUMBER()` are not supported in windows. Ordinary window functions over the matched frame currently cover `SUM`, `COUNT`, `AVG`, `MIN`, and `MAX`. ## Learn more - **[JOURNEYS](/docs/lakehouse/features/journeys.md)**: discover observed paths without writing a pattern. - **[SESSIONIZE](/docs/lakehouse/features/sessionize.md)**: assign events to gap-based sessions. - **[SQL Explorer](/docs/lakehouse/query/sql-explorer.md)**: run and refine pattern queries interactively. - **[SQL engine](/docs/lakehouse/sql-engine.md)**: learn about Altertable's DuckDB SQL dialect. --- url: https://altertable.ai/docs/lakehouse/features/optimize-table.md --- # Optimize table Optimize table maintenance rewrites one table's files so future queries scan fewer, better-organized Parquet files. Use it after large ingests, many small appends, or update-heavy workloads that leave data spread across many files. Optimization is housekeeping for an [Altertable catalog](/docs/lakehouse/catalogs.md) table. It does not change the logical rows in the table; it changes the physical layout used by the query engine. ## SQL example Run `OPTIMIZE TABLE` from SQL after a large ingest or a period of frequent small writes: ```sql OPTIMIZE TABLE my_catalog.main.events; ``` The command returns optimization work by tier, such as `cold`, `warm`, and `hot`, including the number of files considered and rewritten. ## What optimization does For the target table, optimization can: - Compact many small files into larger files. - Apply the table's current [sort order](/docs/lakehouse/features/sorted-tables.md) when rewriting data. - Improve file-level statistics used for pruning. - Reduce metadata and planning overhead for frequently queried tables. ## When to run it Optimize tables that receive frequent small writes or show slower query planning over time. It is most useful for large tables that are queried repeatedly by dashboards, AI agents, scheduled tasks, or external BI tools. Skip optimization for tiny tables or tables that are about to be fully replaced. The rewrite costs compute now to improve repeated reads later. ## Learn more - **[SQL Explorer](/docs/lakehouse/query/sql-explorer.md)**: run maintenance SQL interactively. - **[Partitioning](/docs/lakehouse/features/partitioning.md)**: split data by partition keys. - **[Sorted tables](/docs/lakehouse/features/sorted-tables.md)**: configure physical sort order. - **[Performance](/docs/lakehouse/compute/performance.md)**: understand cache behavior and benchmarking. --- url: https://altertable.ai/docs/lakehouse/features/partitioning.md --- # Partitioning Altertable catalog tables can be partitioned by a user-defined set of partition keys. When partition keys are set, new data is written into separate data files grouped by those keys. During query planning, Altertable uses file-level zone maps — per-column min/max statistics stored in catalog metadata — to decide which files to scan. For order-preserving partition transforms (`identity`, `year`, `month`, `day`, `hour`), partitioned writes produce tight zone maps on the partition column, so file-level pruning narrows scans to the matching partition files. Partition keys only affect **new** data written to a table. Data written before a layout change keeps the partitioning keys that were active at write time. That lets the partition layout evolve over time without rewriting historical files. Partition key values are stored in catalog metadata. They do not need to appear inside the data files or in object storage paths. Run the statements below through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md) against an [Altertable catalog](/docs/lakehouse/catalogs.md) you own. ## Set partition keys Partition new writes on a column: ```sql ALTER TABLE my_catalog.main.shipments SET PARTITIONED BY (destination_region); ``` Partition on time transforms: ```sql ALTER TABLE my_catalog.main.shipments SET PARTITIONED BY (year(departed_at), month(departed_at)); ``` Distribute rows into a fixed number of buckets using an Iceberg-compatible Murmur3 hash: ```sql ALTER TABLE my_catalog.main.shipments SET PARTITIONED BY (bucket(8, order_id)); ``` Combine bucket partitioning with other transforms: ```sql ALTER TABLE my_catalog.main.shipments SET PARTITIONED BY (bucket(8, order_id), month(departed_at)); ``` ## Remove partition keys Stop partitioning new writes: ```sql ALTER TABLE my_catalog.main.shipments RESET PARTITIONED BY; ``` Existing files keep their prior layout until you rewrite or compact them through maintenance operations. ## Supported transforms | Transform | Expression | | --------- | ------------------------ | | identity | `column_name` | | bucket | `bucket(N, column_name)` | | year | `year(timestamp)` | | month | `month(timestamp)` | | day | `day(timestamp)` | | hour | `hour(timestamp)` | ## When to partition Partition when queries repeatedly filter on the same columns — especially high-cardinality IDs combined with time, or date/time columns used in range filters. Partitioning trades more files and metadata for smaller, targeted scans. Skip partitioning when filters are unpredictable, tables stay small, or write throughput matters more than read pruning. You can add or change partition keys later; only new data follows the updated layout. ## Learn more - **[Altertable Catalogs](/docs/lakehouse.md)**: Catalogs for storing and querying your data - **[SQL Engine](/docs/lakehouse/sql-engine.md)**: DuckDB dialect and federated queries - **[Buckets](/docs/lakehouse/storage/buckets.md)**: Object storage that backs catalog data files --- url: https://altertable.ai/docs/lakehouse/features/sessionize.md --- # SESSIONIZE > **Beta:** > > `SESSIONIZE` is available in beta. Its syntax and behavior may change as the implementation evolves. `SESSIONIZE` assigns ordered rows to sessions separated by a period of inactivity. Use it to label events with a session ID or to calculate one result row per session without building the usual `lag`, cumulative-sum, and grouping pipeline. Run these statements through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md). ## First example Suppose an event table contains one row for each user action: ```sql WITH events(user_id, ts, event) AS ( VALUES (1, TIMESTAMP '2024-01-01 10:00:00', 'view'), (1, TIMESTAMP '2024-01-01 10:10:00', 'click'), (1, TIMESTAMP '2024-01-01 10:50:00', 'purchase'), (2, TIMESTAMP '2024-01-01 10:00:00', 'view') ) SELECT user_id, ts, event, session_start, session_end, session_id FROM events SESSIONIZE ( PARTITION BY user_id ORDER BY ts GAP INTERVAL 30 MINUTE ) ORDER BY user_id, ts; ``` The first two events for user `1` belong to session `1` because they are 10 minutes apart. The 40-minute gap before `purchase` starts session `2`. Session IDs restart at `1` for each user. - `PARTITION BY user_id` creates an independent event stream for each user. - `ORDER BY ts` defines event order. - `GAP INTERVAL 30 MINUTE` starts a new session after more than 30 minutes of inactivity. - `session_start` and `session_end` contain the first and last time-key values in the session. - `session_id` is a 1-based number within each partition. ## Operator syntax The supported `FROM`-clause shape is: ```sql FROM input_relation [input_alias] SESSIONIZE ( [PARTITION BY partition_column [, ...]] ORDER BY time_column [, tie_breaker_column ...] GAP interval_expression [MEASURES measure_expression AS output_name [, ...]] [ONE ROW PER SESSION] ) [session_alias] ``` `ORDER BY` and `GAP` are required. `PARTITION BY` is optional. `MEASURES` can only be used with `ONE ROW PER SESSION`. The source relation must appear immediately before `SESSIONIZE`. The result can be aliased after the clause: ```sql SELECT s.user_id, s.session_id FROM events AS e SESSIONIZE ( PARTITION BY user_id ORDER BY ts GAP INTERVAL 30 MINUTE ) AS s; ``` Common SQL constructs such as CTEs, subqueries, joins, outer filters, grouping, and aliases can surround the operator. ## How sessions are formed Within each partition, `SESSIONIZE` sorts rows by the `ORDER BY` columns and compares each time-key value with the preceding value. A new session starts when: ```text current_time > previous_time + GAP ``` Equality stays in the current session. With a 30-minute gap, events exactly 30 minutes apart belong to the same session; an event 30 minutes and one second later starts a new one. The comparison uses consecutive events, not the session start. A session can therefore last longer than the gap as long as each event arrives within the gap after the previous event. ## Choose the output mode ### Preserve every input row Row-preserving mode is the default. It returns the input columns in their original order, followed by `session_start`, `session_end`, and `session_id`: ```sql SELECT * FROM events SESSIONIZE ( PARTITION BY user_id ORDER BY ts GAP INTERVAL 30 MINUTE ); ``` Every row in the same session receives the same metadata. `session_start` and `session_end` are the first and last time-key values, not calculated window boundaries such as `session_end + GAP`. Use this mode when downstream logic still needs individual events: ```sql WITH labeled_events AS ( SELECT * FROM events SESSIONIZE ( PARTITION BY user_id ORDER BY ts, event_id GAP INTERVAL 30 MINUTE ) ) SELECT * FROM labeled_events WHERE session_id = 1 ORDER BY user_id, ts, event_id; ``` ### Return one row per session Add `ONE ROW PER SESSION` to emit session metadata without preserving the input rows: ```sql SELECT * FROM events SESSIONIZE ( PARTITION BY user_id ORDER BY ts GAP INTERVAL 30 MINUTE ONE ROW PER SESSION ) ORDER BY user_id, session_id; ``` The output contains, in order: 1. The `PARTITION BY` columns, when present. 2. `session_start`, `session_end`, and `session_id`. 3. The explicitly declared `MEASURES`, when present. No event count is added automatically. Add `count(*)` as a measure when you need one. ## Calculate session measures `MEASURES` calculates values across all rows in a session. Every measure needs an `AS` alias, and the clause requires `ONE ROW PER SESSION`: ```sql WITH events(user_id, ts, event, amount) AS ( VALUES (1, TIMESTAMP '2024-01-01 10:00:00', 'view', 0), (1, TIMESTAMP '2024-01-01 10:10:00', 'purchase', 49), (1, TIMESTAMP '2024-01-01 10:50:00', 'view', 0) ) SELECT * FROM events SESSIONIZE ( PARTITION BY user_id ORDER BY ts GAP INTERVAL 30 MINUTE MEASURES count(*) AS event_count, sum(amount) AS revenue, FIRST(event) AS first_event, event AS last_event, list(event) AS events ONE ROW PER SESSION ) ORDER BY user_id, session_id; ``` Measure expressions support: - Aggregates such as `count`, `sum`, `avg`, `min`, `max`, `list`, and `count(DISTINCT ...)`. - `FIRST(expression)` and `LAST(expression)` for values from the first and last rows. - A bare input column, which returns its value from the last row of the session. - `PARTITION BY` columns, which are constant throughout a session. - Constants, scalar functions, and arithmetic composed from these values. Measures use whole-session semantics. Pattern-only expressions from `MATCH_RECOGNIZE`, including pattern variables, `CLASSIFIER()`, `MATCH_NUMBER()`, `PREV`, `NEXT`, `RUNNING`, and `FINAL`, are not supported. Aggregate `FILTER` clauses and aggregate-local `ORDER BY` clauses are not supported in `SESSIONIZE` measures. When you need them, preserve the sessionized rows and aggregate by the partition columns and `session_id` in an outer query. ## Define the event streams ### `PARTITION BY` `PARTITION BY` splits the input into independent event streams: ```sql PARTITION BY user_id, device_id ``` Sessions never cross a partition boundary. Without `PARTITION BY`, the entire input is treated as one stream. Partition expressions must be input columns. In summary mode, partition columns appear first in the result and can be referenced directly in measures. ### `ORDER BY` The first `ORDER BY` column is the time key: ```sql ORDER BY ts, event_id ``` It must be an ascending input column with a `DATE`, `TIMESTAMP`, or `TIMESTAMPTZ` type. Supported timestamp precisions include seconds, milliseconds, microseconds, and nanoseconds. Additional columns are tie-breakers. Add a stable tie-breaker when multiple events can have the same timestamp and their relative order matters. ### `GAP` `GAP` sets the maximum inactivity period within a session: ```sql GAP INTERVAL 30 MINUTE ``` The expression must be a non-null, strictly positive, foldable DuckDB `INTERVAL`. Dynamic per-row gaps are not supported. ## Output names and ordering The metadata names `session_start`, `session_end`, and `session_id` are reserved. In row-preserving mode, an input column cannot use one of these names. In summary mode, partition columns and measure aliases cannot use them, and measure aliases cannot duplicate partition columns or each other. Result order is not guaranteed. Add an outer `ORDER BY` whenever downstream behavior depends on row order: ```sql SELECT * FROM events SESSIONIZE ( PARTITION BY user_id ORDER BY ts, event_id GAP INTERVAL 30 MINUTE ) ORDER BY user_id, session_id, ts, event_id; ``` ## Learn more - **[JOURNEYS](/docs/lakehouse/features/journeys.md)**: discover observed paths through event sequences. - **[MATCH\_RECOGNIZE](/docs/lakehouse/features/match-recognize.md)**: find more complex ordered event patterns. - **[SQL Explorer](/docs/lakehouse/query/sql-explorer.md)**: run and refine session queries interactively. - **[SQL engine](/docs/lakehouse/sql-engine.md)**: learn about Altertable's DuckDB SQL dialect. --- url: https://altertable.ai/docs/lakehouse/features/sorted-tables.md --- # Sorted Tables Altertable catalog tables can be configured with a sort order. When a sort order is defined, data is physically sorted by the specified columns whenever it is written out as data files — during `INSERT`, file compaction, and inlined data flushing. Sorting data before writing improves the effectiveness of min/max statistics at query time, which lets the query engine skip data files whose value ranges do not overlap with a query's filter predicates. Run the statements below through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md) against an [Altertable catalog](/docs/lakehouse/catalogs.md) you own. ## Set a sort order Set the sort order for a table: ```sql ALTER TABLE my_catalog.main.sensor_readings SET SORTED BY (recorded_at ASC); ``` Multiple sort keys are supported: ```sql ALTER TABLE my_catalog.main.sensor_readings SET SORTED BY (recorded_at ASC, sensor_id DESC); ``` `ASC` and `DESC` control the sort direction. `NULLS FIRST` and `NULLS LAST` are also supported to control null ordering: ```sql ALTER TABLE my_catalog.main.sensor_readings SET SORTED BY (recorded_at ASC NULLS LAST); ``` ### Expression-based sort keys Arbitrary expressions are supported in `SET SORTED BY`, not just column references. This includes function calls, casts, and catalog macros. Sort by the hour extracted from a timestamp: ```sql ALTER TABLE my_catalog.main.sensor_readings SET SORTED BY (date_trunc('hour', recorded_at) ASC); ``` Sort by a macro: ```sql CREATE MACRO reading_day(t) AS date_trunc('day', t); ALTER TABLE my_catalog.main.sensor_readings SET SORTED BY (reading_day(recorded_at) ASC); ``` Expressions are validated when `SET SORTED BY` runs — an error is returned if any referenced columns or functions cannot be resolved. ## Remove a sort order Remove the sort order from a table: ```sql ALTER TABLE my_catalog.main.sensor_readings RESET SORTED BY; ``` After resetting, subsequent compactions and flushes write data without sorting. ## When to sort Sort when queries repeatedly filter or range-scan on the same columns — especially timestamps, status codes, or IDs used in `WHERE` and `ORDER BY` clauses. Sorting trades write-time cost for tighter file statistics and fewer files scanned at read time. Skip sorting when write throughput is the bottleneck, filters are unpredictable, or tables stay small enough that full scans are cheap. You can add, change, or remove a sort order later; compaction and flush apply the current order going forward. ## Learn more - **[Altertable Catalogs](/docs/lakehouse.md)**: Catalogs for storing and querying your data - **[Partitioning](/docs/lakehouse/features/partitioning.md)**: Split writes by partition key for file-level pruning - **[SQL Engine](/docs/lakehouse/sql-engine.md)**: DuckDB dialect and federated queries --- url: https://altertable.ai/docs/lakehouse/features/time-travel.md --- # Time Travel Every catalog snapshot represents a consistent state of the database. Altertable keeps a record of historic snapshots and their changesets until compaction removes snapshots that fall outside [retention](#snapshot-retention). Time travel lets you query table state as of any recorded snapshot. Specify the snapshot with a version ID or a timestamp. Use the `snapshots` function to list valid snapshots for a catalog. Run the statements below through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md) against an [Altertable catalog](/docs/lakehouse/catalogs.md) you own. ## Query at a snapshot Query a table at a specific snapshot version: ```sql SELECT sku, label FROM my_catalog.main.product_skus AT (VERSION => 3); ``` Query a table as it was at a point in time: ```sql SELECT sku, label FROM my_catalog.main.product_skus AT (TIMESTAMP => now() - INTERVAL '1 week'); ``` Time travel applies to individual table references in a query. Join current and historical data by mixing plain and `AT` table references in the same statement. To inspect row-level inserts, updates, and deletes instead of full table state, use the [data change feed](/docs/lakehouse/features/data-change-feed.md). ## Snapshots Snapshots are commits to a catalog. Each snapshot applies a set of changes — creating tables, inserting or deleting rows, altering schemas — that move the catalog to a new consistent state. Every write goes through a snapshot; changes cannot land outside a transaction. ### List snapshots Query all snapshots and their changesets: ```sql SELECT * FROM my_catalog.snapshots(); ``` After creating a table and inserting a row, the history might look like this: | snapshot\_id | snapshot\_time | schema\_version | changes | author | commit\_message | commit\_extra\_info | | ------------ | ---------------------- | --------------- | -------------------------------------- | ------ | --------------- | ------------------- | | 0 | 2026-04-10 12:57:02+02 | 0 | `{schemas_created=[main]}` | NULL | NULL | NULL | | 1 | 2026-04-10 12:57:02+02 | 1 | `{tables_created=[main.product_skus]}` | NULL | NULL | NULL | | 2 | 2026-04-10 12:57:02+02 | 1 | `{inlined_insert=[1]}` | NULL | NULL | NULL | ### Current snapshot helpers Read the latest snapshot ID: ```sql FROM my_catalog.current_snapshot(); ``` When multiple connections write to the same catalog, read the latest snapshot committed on the current connection: ```sql FROM my_catalog.last_committed_snapshot(); ``` A new connection returns `NULL` from `last_committed_snapshot()` until it commits its own transaction. ### Commit messages Attach author, message, and optional metadata to a snapshot inside a transaction: ```sql BEGIN; INSERT INTO my_catalog.main.product_skus VALUES ('SKU-003', 'Walnut desk'); CALL my_catalog.set_commit_message( 'alex', 'Add walnut desk SKU', extra_info => '{"ticket": "CAT-42"}' ); COMMIT; ``` Those fields appear on the corresponding row in `snapshots()`: | snapshot\_id | author | commit\_message | commit\_extra\_info | | ------------ | ------ | ------------------- | ---------------------- | | 3 | alex | Add walnut desk SKU | `{"ticket": "CAT-42"}` | Use commit messages when auditing who changed a catalog and why. ## Snapshot retention The **Snapshot retention** setting on an Altertable catalog controls how long old snapshots are kept. Retention applies in the catalog UI when you create or edit a catalog. | Value | Behavior | | -------- | --------------------------------------------- | | Blank | Retain snapshots indefinitely | | `0` | Disable snapshot retention after cleanup runs | | `N` days | Retain snapshots newer than `N` days | Set a retention period when storage cost or compliance requires old table versions to expire. Leave it blank when long-running time travel and historical auditability matter more than storage cleanup. Expired snapshots limit how far back you can query with `AT` and how much history the [data change feed](/docs/lakehouse/features/data-change-feed.md) can return. ## Learn more - **[Transactions](/docs/lakehouse/features/transactions.md)**: Each committed transaction creates one snapshot - **[Data Change Feed](/docs/lakehouse/features/data-change-feed.md)**: Read row-level changes between snapshots - **[Altertable Catalogs](/docs/lakehouse.md)**: Catalogs for storing and querying your data --- url: https://altertable.ai/docs/lakehouse/features/transactions.md --- # Transactions Altertable catalogs provide full ACID support with snapshot isolation for all reads and writes. Every operation — including DDL statements such as `CREATE TABLE` or `ALTER TABLE` — runs inside a transaction with all-or-nothing semantics. A transaction can compose multiple changes that succeed or fail together. Run the statements below through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md) against an [Altertable catalog](/docs/lakehouse/catalogs.md) you own. ## Basic transaction syntax Start a transaction, apply changes, then commit or roll back: ```sql BEGIN TRANSACTION; INSERT INTO my_catalog.main.purchase_orders (po_id, status, amount) VALUES ('PO-8821', 'draft', 4200.00); UPDATE my_catalog.main.purchase_orders SET status = 'approved' WHERE po_id = 'PO-8821'; COMMIT; ``` To discard uncommitted work: ```sql ROLLBACK; ``` `ABORT` has the same behavior as `ROLLBACK`. ## Snapshots Each committed transaction — a `BEGIN` … `COMMIT` block — creates one catalog snapshot. Snapshots power [time travel](/docs/lakehouse/features/time-travel.md) and the [data change feed](/docs/lakehouse/features/data-change-feed.md). Use transactions when you need a consistent point-in-time view across multiple statements, or when a batch of writes must land atomically. ## Concurrent writes When multiple transactions modify the same table concurrently, Altertable applies a default retry mechanism for conflicting writes. For long-running batches, keep transactions as short as practical to reduce contention and snapshot buildup. ## Learn more - **[Time Travel](/docs/lakehouse/features/time-travel.md)**: Query historical snapshots and configure retention - **[Data Change Feed](/docs/lakehouse/features/data-change-feed.md)**: Read inserts, updates, and deletes between snapshots - **[SQL Engine](/docs/lakehouse/sql-engine.md)**: DuckDB dialect and federated queries --- url: https://altertable.ai/docs/lakehouse/features/views.md --- # Views Altertable catalog views are defined with standard `CREATE VIEW` syntax and stored in catalog metadata. They provide a stable name over a query so dashboards, agents, and downstream SQL can reference a logical table without repeating the underlying statement. Run the statements below through Altertable's [SQL engine](/docs/lakehouse/sql-engine.md) against an [Altertable catalog](/docs/lakehouse/catalogs.md) you own. ## Create a view ```sql CREATE VIEW my_catalog.main.open_purchase_orders AS SELECT po_id, vendor_id, amount, submitted_at FROM my_catalog.main.purchase_orders WHERE status IN ('draft', 'submitted'); ``` Views can reference other views and tables in the same catalog. Altertable resolves the view definition at query time using the current catalog state. ## Drop a view ```sql DROP VIEW my_catalog.main.open_purchase_orders; ``` ## When to use views Use views to encapsulate common filters, joins, or column projections that many queries share. They keep consumer SQL stable when underlying table layouts evolve, as long as the view definition is updated to preserve the expected columns. Views do not store data — they are query definitions only. For materialized results that persist between queries, write to a table instead. ## Learn more - **[Altertable Catalogs](/docs/lakehouse.md)**: Catalogs for storing and querying your data - **[Macros](/docs/lakehouse/features/macros.md)**: Reusable scalar and table expressions - **[SQL Engine](/docs/lakehouse/sql-engine.md)**: DuckDB dialect and federated queries --- url: https://altertable.ai/docs/lakehouse.md --- # Lakehouse Store data in managed tables, query existing sources through external catalogs, and join both with DuckDB SQL. Agents, the app, APIs, and SQL clients use the same catalog namespace and access permissions. ## How it fits together - **Catalogs** organize schemas and tables. Managed catalogs hold data ingested into Altertable; external catalogs expose data in other systems. - **Workers** execute queries and cache data. Use managed Workers or deploy them in your infrastructure. - **Buckets** store managed table files. Use managed storage or a connected bucket. Start with [Ingest or connect?](/docs/lakehouse/ingest-or-connect.md), then [query a table in SQL Explorer](/docs/lakehouse/query/sql-explorer.md). SQL clients → Altertable Workers → object storage (Parquet files). The Worker pool scales independently of object storage. Each Worker has its own local SSD cache for hot data. Workers read and write managed table files and query existing databases, warehouses, and buckets through external catalogs. ## Explore the lakehouse - [Catalogs](/docs/lakehouse/catalogs.md): Create hosted, optimized lakehouse catalogs on Altertable, backed by Parquet files in object storage. - [Endorsement](/docs/lakehouse/endorsement.md): Mark tables and catalogs as verified, draft, or excluded to steer which data AI agents prefer and discover. - [Ingest or connect?](/docs/lakehouse/ingest-or-connect.md): Choose how to bring data to Altertable by comparing managed ingestion, live external catalogs, object storage, product events, and telemetry. - [Integration directory](/docs/lakehouse/integrations.md): Find every guide for your integration in one place, including DuckDB, object storage, Iceberg catalogs, SDKs, SQL clients, and ingestion pipelines. - [Ingest data](/docs/lakehouse/ingest.md): Load records, files, DataFrames, and scheduled bucket imports into managed lakehouse tables using APIs, SDKs, the CLI, or existing pipelines. - [Connect existing data](/docs/lakehouse/external-catalogs.md): Connect databases, warehouses, and file-backed buckets to Altertable as external catalogs for federated queries. - [Explore and query](/docs/lakehouse/query.md): Explore tables in SQL Explorer or query from applications, DuckDB, and DataFrames using the same lakehouse namespace and access permissions. - [SQL engine](/docs/lakehouse/sql-engine.md): Altertable uses DuckDB SQL for all queries, including federated queries across multiple data sources. - [SQL and BI clients](/docs/lakehouse/clients.md): Connect terminal workflows, SQL clients, and BI tools to Altertable using the CLI, PostgreSQL-compatible protocol, or Arrow Flight SQL. - [Model and transform data](/docs/lakehouse/modeling.md): Organize lakehouse transformations with SQL, dbt, and medallion architecture, and connect project metadata to your agents as business context. - [SQL capabilities](/docs/lakehouse/features.md): Table capabilities for partitioning, time travel, views, search, and incremental processing in Altertable catalogs. - [Compute and performance](/docs/lakehouse/compute.md): Understand how Workers execute lakehouse SQL, choose compute for a query, and improve performance with caching and table layout. - [Storage](/docs/lakehouse/storage.md): Configure object storage for managed lakehouse tables and distinguish storage buckets from external catalogs and scheduled bucket imports. - [Lakehouse credentials](/docs/administration/lakehouse-credentials.md): Altertable generates secure credentials for each environment. Access them to connect via the HTTP API, Arrow Flight SQL, or Postgres adapter. - [Network](/docs/administration/network.md): How Altertable workers reach your catalogs. Allow these egress IP addresses, or connect through an SSH tunnel when the database is not publicly reachable. --- url: https://altertable.ai/docs/lakehouse/ingest-or-connect.md --- # Ingest or connect? **Ingest data** to copy records or files into Altertable-managed tables. **Connect an external catalog** to query data where it already lives. You can use both and join their tables with SQL. ## Choose a route | Your data | Route | Where the queried data lives | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | | Existing database or warehouse | [External catalog](/docs/lakehouse/external-catalogs.md) | In the source system | | Files, Iceberg datasets, or managed table catalogs in object storage | [Query object storage](/docs/lakehouse/query/object-storage.md) | In your bucket or table catalog | | Local CSV, JSON, or Parquet files | [Upload files](/docs/lakehouse/ingest/http-api.md#3-upload-files) | In managed lakehouse tables | | New files arriving in a bucket | [Bucket synchronizer](/docs/lakehouse/ingest/object-storage.md) | Copied into managed tables on a schedule | | Application records or backend events | [Append or upsert](/docs/lakehouse/ingest/http-api.md) | In managed lakehouse tables | | DataFrames or local DuckDB results | [DataFrame ingestion](/docs/lakehouse/ingest/pandas-polars-ibis.md) or [DuckDB ingestion](/docs/lakehouse/ingest/duckdb.md) | In managed lakehouse tables | | Product behavior and user identities | [Product Analytics SDK or API](/docs/product-analytics/setup.md) | In the Product Analytics catalog | | Application logs and traces | [OpenTelemetry SDK or collector](/docs/opentelemetry/setup.md) | In the OpenTelemetry catalog | | Existing data pipelines | [Pipeline integration](/docs/lakehouse/ingest/integrations.md) | Depends on the integration | Connecting a source does not schedule a copy of its data. Query freshness and performance depend on the source. A bucket synchronizer copies new files into managed tables on its configured schedule. ## Choose the ingestion operation - **Append** for new JSON records. Altertable can create missing tables and add columns. - **Upload** to create a table from a file or append file rows. Use overwrite only when you intend to replace the table. - **Upsert** to insert new rows and update matching rows by a primary key. See the [ingestion guide](/docs/lakehouse/ingest/http-api.md) for working examples and the [HTTP API reference](/docs/api-references/lakehouse.md) for request limits and exact behavior. ## Ask your agent to choose ```text Read https://altertable.ai/docs/lakehouse/ingest-or-connect.md. Help me choose how to bring my data to Altertable. Inspect available project and catalog context, then ask where the data lives, whether it must stay there, and how fresh it needs to be if those details are missing. Recommend a supported ingestion or external-catalog route. Explain copying, data residency, freshness, and access requirements. Stop at a concrete setup plan; do not create a connection or ingest data yet. ``` --- url: https://altertable.ai/docs/lakehouse/ingest/duckdb.md --- # Ingest from DuckDB The [DuckDB Altertable extension](https://github.com/altertable-ai/duckdb-altertable) lets DuckDB write to attached Altertable catalogs. Use it when your data already lives in DuckDB, local files, or a DuckDB-compatible source and you want to load it into Altertable with SQL. The extension supports: - **Attached databases** for ingesting into Altertable catalogs with standard SQL - **Direct table access** through DuckDB's catalog - **Remote writes** for bulk loads, appends, and schema updates ## Installation Install and load the community extension from DuckDB: ```sql INSTALL altertable FROM community; LOAD altertable; ``` ## Attach a database Use your lakehouse username, lakehouse password, and catalog in the connection string: ```sql ATTACH 'user={{LAKEHOUSE_USERNAME}} password={{LAKEHOUSE_PASSWORD}} catalog=my_catalog' AS remote_db (TYPE ALTERTABLE); ``` ### Connection parameters | Parameter | Description | Example | | ---------- | ------------------------- | ------------------------ | | `user` | Lakehouse username | `{{LAKEHOUSE_USERNAME}}` | | `password` | Lakehouse password | `{{LAKEHOUSE_PASSWORD}}` | | `catalog` | Remote Altertable catalog | `analytics` | Or use DuckDB secrets: ```sql CREATE SECRET my_altertable ( TYPE altertable, USER '{{LAKEHOUSE_USERNAME}}', PASSWORD '{{LAKEHOUSE_PASSWORD}}', CATALOG 'my_catalog' ); ATTACH '' AS remote_db (TYPE altertable, SECRET my_altertable); ``` To disconnect the attached catalog: ```sql DETACH remote_db; ``` ## Create a remote table Create tables through the attached Altertable database: ```sql CREATE TABLE remote_db.main.orders ( order_id INTEGER, amount DOUBLE, status VARCHAR ); ``` ## Load local data Create an Altertable table from a local DuckDB table: ```sql CREATE TABLE remote_db.main.orders AS SELECT * FROM local_db.main.orders; ``` You can also load files that DuckDB can read: ```sql CREATE TABLE remote_db.main.orders AS SELECT * FROM read_parquet('orders.parquet'); ``` ## Append rows Append new records from a query: ```sql INSERT INTO remote_db.main.orders SELECT order_id, amount, status FROM read_csv('new-orders.csv'); ``` Append individual rows when you need a quick manual load: ```sql INSERT INTO remote_db.main.orders VALUES (1001, 42.50, 'open'), (1002, 19.99, 'paid'); ``` ## Update table definitions Use `ALTER TABLE` when incoming data needs a new column: ```sql ALTER TABLE remote_db.main.orders ADD COLUMN note VARCHAR; ``` For Python DataFrame workflows, see [Ingest with Pandas, Polars, and Ibis](/docs/lakehouse/ingest/pandas-polars-ibis.md). For querying attached Altertable tables from DuckDB, see [Query data with DuckDB](/docs/lakehouse/query/duckdb.md). For more examples and source builds, see the [DuckDB Altertable extension repository](https://github.com/altertable-ai/duckdb-altertable). For other DuckDB workflows, see the [DuckDB integration hub](/docs/lakehouse/integrations/duckdb.md). --- url: https://altertable.ai/docs/lakehouse/ingest/http-api.md --- # Load records and files Use append for new JSON records, upload for files, and upsert for rows that should update existing primary keys. You need a writable catalog and [lakehouse credentials](/docs/administration/lakehouse-credentials.md). The steps below use the CLI. For application code, use the SDK or HTTP examples in step 2. Exact limits and request fields live in the [HTTP API reference](/docs/api-references/lakehouse.md). ## Set up with your agent ```text Help me load records or files into Altertable. Read https://altertable.ai/docs/lakehouse/ingest/http-api.md first. Inspect the source format and any existing ingestion code. Confirm the destination environment, catalog, schema, and table. Choose append for new JSON records, upload for files, or upsert for updates with a stable primary key. Ask only for information you cannot inspect. Use the CLI for a one-off load or a supported SDK for application code. Keep lakehouse credentials in local or server-side secret configuration. Explain the chosen operation and obtain confirmation before writing data; do not overwrite an existing table. Load a small representative sample, wait for insertion to finish, and query the destination. Compare the returned fields and values with the sample and inspect rejected or quarantined records. An accepted request is not sufficient verification. Summarize the result and show how to repeat the load. ``` > **Before using the Altertable CLI:** > > Install the CLI if you haven’t already: > > ```bash > curl -fsSL https://install.altertable.ai | sh > altertable login > ``` > > If `altertable` isn’t found after installation, open a new terminal. Run `altertable login` to log in to your organization and environment. Follow the [CLI setup guide](/docs/lakehouse/clients/cli.md#2-log-in) to configure credentials and verify access. ## Steps ### 1. Choose an ingest mode | Operation | Use it for | | --------- | --------------------------------------------------------------------- | | Append | Add new JSON records, with automatic table creation and new columns | | Upload | Create a table from a CSV, JSON, or Parquet file, or append file rows | | Upsert | Update matching primary keys and insert new rows from a file | Use [bucket synchronization](/docs/lakehouse/ingest/object-storage.md) for scheduled file imports from object storage. ### 2. Append JSON records From the CLI, append a test record to a new table. Replace `my_catalog` with your destination catalog: ```bash altertable append '{"event":"onboarding_test","source":"cli"}' --to my_catalog.main.events ``` Append acknowledges accepted records before insertion finishes. Allow a few seconds, then query the table to verify that they arrived. An acknowledgement alone does not verify insertion. If a row fails insertion after retries, Altertable stores it in `altertable.main.quarantined_appends`. Inspect the error and original payload before correcting and resending it. **Append from application code** Use your [lakehouse credentials](/docs/administration/lakehouse-credentials.md#verification) to set `ALTERTABLE_USERNAME` and `ALTERTABLE_PASSWORD` before running cURL. Choose a [published SDK](/docs/api-references/sdks.md#lakehouse-sdks) before initializing a client: ### cURL ```bash curl -X POST "https://api.altertable.ai/append?catalog=my_catalog&schema=main&table=events" \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H "Content-Type: application/json" \ -d '{"timestamp":"2026-01-01","event":"click","value":42}' ``` ### JavaScript ```typescript import { AltertableLakehouseClient } from '@altertable/lakehouse'; const client = new AltertableLakehouseClient({ username: '{{LAKEHOUSE_USERNAME}}', password: '{{LAKEHOUSE_PASSWORD}}', }); await client.append({ catalog: 'my_catalog', schema: 'main', table: 'events', body: { Single: { timestamp: '2026-01-01', event: 'click', value: 42 }, }, }); ``` ### Python ```python from altertable_lakehouse import Client client = Client( username="{{LAKEHOUSE_USERNAME}}", password="{{LAKEHOUSE_PASSWORD}}", ) client.append( catalog="my_catalog", schema="main", table="events", data={"timestamp": "2026-01-01", "event": "click", "value": 42}, ) ``` ### Ruby ```ruby require 'altertable/lakehouse' client = Altertable::Lakehouse::Client.new( username: '{{LAKEHOUSE_USERNAME}}', password: '{{LAKEHOUSE_PASSWORD}}' ) client.append( catalog: 'my_catalog', schema: 'main', table: 'events', payload: { timestamp: '2026-01-01', event: 'click', value: 42 } ) ``` ### Go ```go import ( "context" "log" altertable "github.com/altertable-ai/altertable-lakehouse-go" ) client, err := altertable.NewClient(altertable.Config{ Username: "{{LAKEHOUSE_USERNAME}}", Password: "{{LAKEHOUSE_PASSWORD}}", }) if err != nil { log.Fatal(err) } _, err = client.Append(context.Background(), altertable.AppendParams{ Catalog: "my_catalog", Schema: "main", Table: "events", }, altertable.AppendRequest{ Single: altertable.AppendPayload{ "timestamp": "2026-01-01", "event": "click", "value": 42, }, }) if err != nil { log.Fatal(err) } ``` ### 3. Upload files To create a new table from a file: ```bash altertable upload users.csv --to my_catalog.main.users --mode create ``` To add file rows to an existing table, use `--mode append`. Use `--mode overwrite` only when you intend to replace the table with the uploaded data. See [upload behavior](/docs/api-references/lakehouse.md#post-upload) for supported formats and limits. ### 4. Upsert by primary key If `users.csv` contains a stable `id` column, update matching rows and insert new ones with: ```bash altertable upsert users.csv --to my_catalog.main.users --key id ``` Check that the key is populated and identifies the rows you intend to update. See [upsert behavior](/docs/api-references/lakehouse.md#post-upsert) for HTTP requests and supported formats. ## Verification After uploading or upserting the file, inspect the destination: ```bash altertable query "SELECT * FROM my_catalog.main.users LIMIT 10" ``` Compare the returned values with the file, including any rows you expected to update. For append, query its destination table after processing has completed. ## Troubleshooting - **Unauthorized request:** check the destination environment and lakehouse credentials. - **A new append row is missing:** wait for processing, then inspect `altertable.main.quarantined_appends` for failures. - **The file exceeds the request limit:** consult the [endpoint limits](/docs/api-references/lakehouse.md#post-upload) and split the data when needed. - **Upsert changes unexpected rows:** check the primary key column and its values before sending another request. --- url: https://altertable.ai/docs/lakehouse/ingest.md --- # Ingest data Load records and files into managed lakehouse tables. If the data should stay in an existing database, warehouse, or bucket, [connect it as an external catalog](/docs/lakehouse/external-catalogs.md). Start by [appending a record and reading it back](/docs/getting-started/ingest-data.md) with the CLI. When you are ready to load your own data, follow [Load records and files](/docs/lakehouse/ingest/http-api.md). That guide also provides HTTP and SDK examples for application code. Product events belong in [Product Analytics](/docs/product-analytics/setup.md). Logs, traces, and metrics use [OpenTelemetry](/docs/opentelemetry/setup.md). - [Load records and files](/docs/lakehouse/ingest/http-api.md): Append JSON records, upload files, or upsert by primary key into Altertable, then verify the data with SQL using the CLI, SDKs, or HTTP API. - [Import from object storage](/docs/lakehouse/ingest/object-storage.md): Import files from object storage into Altertable tables with bucket synchronizers for scheduled, queryable ingestion. - [Ingest from DuckDB](/docs/lakehouse/ingest/duckdb.md): Ingest data from DuckDB into Altertable with the community extension for local files, tables, and SQL workflows. - [Ingest from Pandas, Polars, and Ibis](/docs/lakehouse/ingest/pandas-polars-ibis.md): Load Pandas, Polars, and Ibis tables into Altertable using DuckDB's Python client and the Altertable community extension. - [Pipeline integrations](/docs/lakehouse/ingest/integrations.md): Connect ETL tools, event platforms, and data workflows to load records into your governed Altertable lakehouse. - [Altertable CLI](/docs/lakehouse/clients/cli.md): Install the Altertable CLI, log in to your environment, and verify your first query. Find advanced commands and options in the CLI reference. --- url: https://altertable.ai/docs/lakehouse/ingest/integrations/airbyte.md --- # Airbyte [Airbyte](https://airbyte.com/) is an open-source data integration platform that can sync data from hundreds of sources into Altertable. Use our custom Altertable destination connector to load data directly into your Altertable lakehouse. ## Goal Install the custom Altertable destination in Airbyte, connect it to a lakehouse catalog, and verify a source sync with SQL. ## Prerequisites - An Airbyte instance that allows custom Docker connectors. - Your lakehouse username and password from [API keys & credentials](/docs/administration/lakehouse-credentials.md) in your organization settings. - The target [Altertable catalog](/docs/lakehouse/catalogs.md) and schema names. - An Airbyte source ready to sync. ## Steps ### 1. Add the Docker connector The Altertable destination connector is available as a Docker image and can be added as a custom connector in your Airbyte instance. `ghcr.io/altertable-ai/airbyte-destination-altertable` 1. In your Airbyte instance, go to **Settings** → **Destinations**. 2. Choose **New connector** → **Add a new Docker connector**. 3. Enter `Altertable` as the display name. 4. Enter `ghcr.io/altertable-ai/airbyte-destination-altertable` as the Docker repository name. 5. Set the Docker image tag to `latest`, or use a pinned release tag. ### 2. Configure the destination In the Airbyte destination settings, enter the lakehouse credentials below, along with the catalog name and schema where you want to sync data. ```bash LAKEHOUSE_USERNAME="{{LAKEHOUSE_USERNAME}}" LAKEHOUSE_PASSWORD="{{LAKEHOUSE_PASSWORD}}" ``` To use custom lakehouse credentials, create them in **Continue in Altertable:** [Open API keys & credentials](/app/settings/api-keys-and-credentials) in your organization settings. ### 3. Create and run a connection Select the Altertable destination when creating or editing an Airbyte connection. Choose a source and run the first sync. ## Verification After Airbyte reports that the sync finished, query one of the synced tables with its fully qualified name: ```sql SELECT * FROM your_catalog.your_schema.your_table LIMIT 10; ``` Confirm that the returned columns and rows match the selected Airbyte stream. ## Troubleshooting - **Airbyte cannot pull the connector:** confirm that the Airbyte host can access `ghcr.io` and that the repository and tag are correct. - **The destination check fails:** copy the lakehouse username and password again from the target environment. - **The sync writes to the wrong location:** check the catalog and schema configured on the destination. - **The sync completes but no rows appear:** confirm that the source stream is selected and query the table name created by that stream. ## Next steps The connector source code is available at [github.com/altertable-ai/airbyte-destination-altertable](https://github.com/altertable-ai/airbyte-destination-altertable) - [Query with SQL Explorer](/docs/lakehouse/query/sql-explorer.md) - [Query with the HTTP API](/docs/lakehouse/query/http-api.md) - [Manage lakehouse credentials](/docs/administration/lakehouse-credentials.md) --- url: https://altertable.ai/docs/lakehouse/ingest/integrations/dlt.md --- # dlt A **dlt task** loads data from an external source into an Altertable catalog. [Ask Altertable](/docs/agents/ask.md) can prepare the pipeline and schedule recurring loads for you. You need a writable [catalog](/docs/lakehouse/catalogs.md) and [permission](/docs/administration/access-control.md) to create and edit tasks, and to manage catalogs in the environment. ## Create a task with Agent Open **Agent** in the sidebar. Describe the source, what to load, the destination catalog and schema, and the schedule. For example: ```text Create a dlt task to load Attio contacts into the raw schema of my lakehouse catalog every day at 09:00 UTC. Inspect the supported source and destination schema, and confirm the destination table and incremental behavior. Show the pipeline and schedule for review before saving. Use Secrets needed for credentials; do not ask me to paste them in chat. Validate the generated pipeline, report the rows loaded and any errors, and confirm whether the schedule is active. ``` Replace the source and destination with yours, then: 1. Confirm the proposed task and destination when the agent asks. 2. If the source requires credentials, enter them in the **Secrets needed** fields and select **Save securely**. Values are encrypted and kept out of the agent conversation. Do not paste credentials into a chat message. 3. Wait while the agent generates and tests the pipeline. This can take several minutes. The task starts paused during setup. After successful validation, the agent saves the pipeline and activates its schedule. If validation is blocked, it explains what is needed and leaves the task paused. Validation uses a temporary schema. Actual task runs load the destination you chose. ## Check and manage runs Open the task from **Tasks**. For an active task, select **Run now** from its menu to start a load immediately. Open **Run history**, select a run, and check **Run output** for its result or error. Verify the loaded tables in your catalog. Change **Schedule** in the task's **Settings**. Schedule times use UTC. Use **Pause** to stop scheduled runs and **Resume** to restart them. If a run fails, share its error with [Ask Altertable](/docs/agents/ask.md) and ask it to repair the task. ## Use an existing pipeline For a pipeline you already maintain, open **Tasks → New task** and choose **dlt task** under **Set up manually**. Select the destination **Catalog** and **Schema**, then choose a source: - **Inline script**: choose **Python** and paste your pipeline. - **Repository**: select a repository, optionally set **Git ref** and **Working directory**, and enter the **Command** to run after checkout. For example, use `python import.py` with `jobs` as the working directory for `jobs/import.py`. If the source needs credentials, add them under **Advanced options → Environment variables**. Values are encrypted and cannot be viewed after saving. Keep **Schedule** set to **Manually** for a first run, then select **Create task**. The runtime includes [dlt-altertable](https://github.com/altertable-ai/dlt-altertable) and supplies destination credentials. See the adapter documentation for pipeline configuration and running dlt outside Altertable. --- url: https://altertable.ai/docs/lakehouse/ingest/integrations/fivetran.md --- # Fivetran Use [Fivetran](https://www.fivetran.com/) to load source data into Altertable through its standard **PostgreSQL destination**. Connect the destination to Altertable's [Postgres adapter](/docs/lakehouse/clients/postgres.md) using your lakehouse credentials. ## Goal Configure a PostgreSQL destination in Fivetran, point it at an Altertable catalog, and verify a source sync with SQL. ## Prerequisites - A Fivetran account with permission to create or manage destinations. - Your [lakehouse username and password](/docs/administration/lakehouse-credentials.md). - A writable [Altertable catalog](/docs/lakehouse/catalogs.md) for the synced data. - A source ready to connect in Fivetran. ## Steps ### 1. Add a PostgreSQL destination In Fivetran, open **Destinations**, choose **Add destination**, and name it `Altertable`. Select **PostgreSQL** as the destination type and use the generic PostgreSQL setup with a direct connection. ### 2. Configure the connection Enter the following connection details: | Field | Value | | -------- | ---------------------------- | | Host | `postgres.altertable.ai` | | Port | `443` | | User | Your lakehouse username | | Password | Your lakehouse password | | Database | Your Altertable catalog name | | TLS/SSL | **Required** | The database field selects the catalog where Fivetran writes data. Use credentials from the Altertable environment that owns that catalog. Altertable manages lakehouse credentials, so use those credentials in place of the PostgreSQL user creation steps in Fivetran's generic guide. Complete the remaining destination settings for your Fivetran account, then choose **Save & Test**. See Fivetran's [PostgreSQL destination setup guide](https://fivetran.com/docs/destinations/postgresql/setup-guide) for its current setup form and connection tests. ### 3. Run a source sync Create a source connection in Fivetran and select the `Altertable` destination. Configure the destination schema for that connection, select the tables to sync, and start an initial sync. Altertable exposes the PostgreSQL wire protocol but executes queries with DuckDB. Validate both the destination tests and a sync with your source before scheduling recurring loads, including updates and deletes if your source uses them. ## Verification After the initial sync completes, open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md). Replace the example identifiers with the catalog, schema, and table configured in Fivetran: ```sql SELECT * FROM my_catalog.my_schema.my_table LIMIT 10; ``` Compare the sample rows and destination row count with the completed Fivetran sync. If the source includes updates or deletes, verify those changes before enabling recurring syncs. ## Next steps - [Query with SQL Explorer](/docs/lakehouse/query/sql-explorer.md) - [Postgres adapter](/docs/lakehouse/clients/postgres.md) - [Manage lakehouse credentials](/docs/administration/lakehouse-credentials.md) --- url: https://altertable.ai/docs/lakehouse/ingest/integrations.md --- # Pipeline integrations Use Altertable integrations to load data from pipelines and event platforms into your [lakehouse](/docs/lakehouse.md). - [Airbyte](/docs/lakehouse/ingest/integrations/airbyte.md): Use Airbyte to sync data from hundreds of sources into your Altertable lakehouse. Configure the custom Altertable destination connector with Docker integration. - [Fivetran](/docs/lakehouse/ingest/integrations/fivetran.md): Configure Fivetran's standard PostgreSQL destination to load source data into your Altertable lakehouse through the Postgres adapter. - [Segment](/docs/lakehouse/ingest/integrations/segment.md): Ingest Segment events into Altertable so product behavior can be queried with business, operational, and lakehouse data. - [PostHog](/docs/lakehouse/ingest/integrations/posthog.md): Forward events from an existing PostHog JavaScript integration into Altertable Product Analytics without changing tracking calls. - [dlt](/docs/lakehouse/ingest/integrations/dlt.md): Ask Altertable to create a dlt pipeline, provide source credentials securely when prompted, and schedule data loads into your Altertable catalog. --- url: https://altertable.ai/docs/lakehouse/ingest/integrations/posthog.md --- # PostHog If your application already tracks events with PostHog, the Altertable JavaScript SDK can forward those captured events into the `product_analytics` catalog. Your existing PostHog tracking calls can stay in place. ## Connect PostHog Enable [Product Analytics](/docs/product-analytics.md) and copy its [API key](/docs/product-analytics/authentication.md). Initialize Altertable before PostHog: ```typescript import posthog from 'posthog-js'; import { altertable } from '@altertable/altertable-js'; altertable.init('{{API_KEY}}', { adapters: { posthog }, }); posthog.init('POSTHOG_KEY'); ``` While the adapter is configured, Altertable's own `track`, `page`, `identify`, `alias`, `updateTraits`, and `reset` calls are ignored, and its automatic capture stays off. Altertable still applies its tracking consent setting. Forwarded events do not pass through `transformEvent`. PostHog events need a distinct ID and timestamp. Cookieless PostHog events are not supported. For events tracked directly with Altertable, see [Track](/docs/product-analytics/track.md). --- url: https://altertable.ai/docs/lakehouse/ingest/integrations/segment.md --- # Segment [Segment](https://www.twilio.com/en-us/segment) sends product events into Altertable when Segment is already the event router for your application. Altertable becomes an analytical destination where raw behavior can be queried beside revenue, support, operations, and product data. After events land in Altertable, you can analyze them with SQL, dashboards, product analytics visualizations, AI agents, and downstream tools connected to the lakehouse. ## When to use Segment Use this path when: - Your application already tracks events through Segment. - You want Altertable to receive a copy of those events without changing application instrumentation. - Product behavior needs to be joined with other lakehouse catalogs. If you are not already using Segment, send events directly through Altertable's product analytics SDKs or ingestion APIs instead. ## Setup outline 1. Enable [Product Analytics](/docs/product-analytics.md) for the target Altertable environment. 2. Create or choose a Segment source for your application. 3. Add Altertable as a destination or webhook target in Segment. 4. Configure the destination with the credentials and endpoint for your Altertable environment. 5. Send a test event, then verify that events appear in the built-in `product_analytics` catalog. ## Learn more - **[Product Analytics](/docs/product-analytics.md)**: understand where events land and how they are modeled. - **[Track](/docs/product-analytics/track.md)**: send events directly from your application. - **[Ingest with HTTP API](/docs/lakehouse/ingest/http-api.md)**: write records or files without Segment. --- url: https://altertable.ai/docs/lakehouse/ingest/object-storage.md --- # Import from object storage Use a bucket synchronizer when files land in Amazon S3, Cloudflare R2, Google Cloud Storage, or another S3-compatible bucket and you want Altertable to load them into a catalog. A bucket synchronizer runs on a schedule. Each run imports new files from the configured bucket paths into destination tables, and skips files that were already loaded. ## When to use a bucket synchronizer Use this path for: - Parquet, CSV, or JSON files written to stable bucket paths. - File drops produced by ELT jobs, reverse ETL jobs, or application exports. - Append-style datasets where new files arrive over time. - Data that should be copied into Altertable-managed tables for SQL, dashboards, and AI workflows. If you only need to query files in place without copying them, use [Bucket Tables](/docs/lakehouse/external-catalogs/bucket-tables.md), [Iceberg Tables](/docs/lakehouse/external-catalogs/iceberg-tables.md), or [S3 Tables](/docs/lakehouse/external-catalogs/s3-tables.md). ## Set up a bucket synchronizer 1. Connect the [bucket](/docs/lakehouse/storage/buckets.md) that contains the files. 2. Open the destination catalog in the Altertable app. 3. In **Synchronizers**, click **Add synchronizer**. 4. Choose **Bucket** as the source type. 5. Select the bucket and file format. 6. Choose the target schema. 7. Map each bucket path to the table Altertable should load. 8. Save the synchronizer and configure its schedule. The file format applies to every path in the synchronizer. Use separate synchronizers when different paths use different formats. ## Query synced tables After the synchronizer runs, query the destination tables with fully qualified names such as `analytics.main.events` from the SQL explorer, HTTP query API, DuckDB extension, BI adapters, MCP tools, and AI workflows. ## Keep paths stable Bucket synchronizers work best when upstream jobs write new files to predictable paths. Keep partitioning consistent, avoid rewriting files after they have been loaded, and grant Altertable least-privilege access to the bucket paths it needs. To send records or ad hoc files directly to Altertable, use [ingest with HTTP API](/docs/lakehouse/ingest/http-api.md) or [ingest with DuckDB](/docs/lakehouse/ingest/duckdb.md). --- url: https://altertable.ai/docs/lakehouse/ingest/pandas-polars-ibis.md --- # Ingest from Pandas, Polars, and Ibis Use DuckDB's Python client and the [DuckDB Altertable extension](https://github.com/altertable-ai/duckdb-altertable) to load local data into Altertable. DuckDB exposes registered DataFrames and Ibis tables as SQL relations, and the extension sends the resulting `CREATE TABLE AS SELECT` or `INSERT ... SELECT` operation to the attached Altertable catalog. The Altertable community extension requires DuckDB 1.5.4 or later. ## Connect DuckDB to Altertable Load the community extension and attach an Altertable catalog: ```python import os import duckdb con = duckdb.connect() con.execute("INSTALL altertable FROM community") con.execute("LOAD altertable") con.execute( f"ATTACH 'user={os.environ['LAKEHOUSE_USERNAME']} " f"password={os.environ['LAKEHOUSE_PASSWORD']} " f"catalog={os.environ['LAKEHOUSE_CATALOG']}' " "AS remote (TYPE ALTERTABLE)" ) ``` The connection uses the `LAKEHOUSE_USERNAME`, `LAKEHOUSE_PASSWORD`, and `LAKEHOUSE_CATALOG` environment variables. For other connection options and DuckDB secrets, see [Ingest with DuckDB](/docs/lakehouse/ingest/duckdb.md). ## Load a Pandas DataFrame ```bash pip install 'duckdb==1.5.4' pandas ``` Register a DataFrame with a SQL name, then use it as the source of a remote table: ```python import pandas as pd events = pd.DataFrame( { "event_id": [1, 2, 3], "customer_id": ["customer-1", "customer-2", "customer-1"], "event_name": ["signed_up", "signed_up", "activated"], } ) con.register("events_df", events) con.execute(""" CREATE TABLE remote.main.events AS SELECT * FROM events_df """) ``` To append another DataFrame to the existing table, use `INSERT INTO`. `BY NAME` matches columns by name instead of relying on their order: ```python new_events = pd.DataFrame( { "customer_id": ["customer-3"], "event_name": ["signed_up"], "event_id": [4], } ) con.register("new_events_df", new_events) con.execute(""" INSERT INTO remote.main.events BY NAME SELECT * FROM new_events_df """) ``` ## Load a Polars DataFrame ```bash pip install 'duckdb==1.5.4' polars pyarrow ``` `pyarrow` is required for DuckDB's Polars integration. Polars DataFrames use the same DuckDB relation interface: ```python import polars as pl events = pl.DataFrame( { "event_id": [1, 2, 3], "customer_id": ["customer-1", "customer-2", "customer-1"], "event_name": ["signed_up", "signed_up", "activated"], } ) con.register("events_df", events) con.execute(""" CREATE TABLE remote.main.events AS SELECT * FROM events_df """) ``` Append Polars data to an existing table in the same way: ```python new_events = pl.DataFrame( { "event_id": [4], "customer_id": ["customer-3"], "event_name": ["signed_up"], } ) con.register("new_events_df", new_events) con.execute(""" INSERT INTO remote.main.events BY NAME SELECT * FROM new_events_df """) ``` ## Load with Ibis ```bash pip install 'ibis-framework[duckdb]' 'duckdb==1.5.4' ``` [Ibis](https://ibis-project.org/) compiles dataframe expressions to SQL and uses the same DuckDB attach workflow underneath: ```python import os import ibis con = ibis.duckdb.connect() con.raw_sql("INSTALL altertable FROM community") con.raw_sql("LOAD altertable") con.raw_sql( f"ATTACH 'user={os.environ['LAKEHOUSE_USERNAME']} " f"password={os.environ['LAKEHOUSE_PASSWORD']} " f"catalog={os.environ['LAKEHOUSE_CATALOG']}' " "AS remote (TYPE ALTERTABLE)" ) ``` Build a local table with `memtable`, then create a remote table from it: ```python events = ibis.memtable( { "event_id": [1, 2, 3], "customer_id": ["customer-1", "customer-2", "customer-1"], "event_name": ["signed_up", "signed_up", "activated"], } ) con.create_table("events", obj=events, database="remote.main", overwrite=True) ``` `database="remote.main"` targets the attached Altertable catalog and its `main` schema. Append rows with `insert`: ```python new_events = ibis.memtable( { "customer_id": ["customer-3"], "event_name": ["signed_up"], "event_id": [4], } ) con.insert("events", new_events, database="remote.main") ``` When column order differs from the remote table, use `INSERT INTO ... BY NAME` through `raw_sql`: ```python new_events = ibis.memtable( { "customer_id": ["customer-3"], "event_name": ["signed_up"], "event_id": [4], } ) con.create_table("new_events_df", obj=new_events, overwrite=True) con.raw_sql(""" INSERT INTO remote.main.events BY NAME SELECT * FROM new_events_df """) ``` For large tables, load data in batches so each transaction stays within your available memory and transaction limits. If you need to define the remote schema explicitly before loading, create the table with `CREATE TABLE` and then use `INSERT INTO ... BY NAME`. --- url: https://altertable.ai/docs/lakehouse/integrations/amazon-s3.md --- # Amazon S3 integration Query S3 files in place, import them into managed tables, or connect Iceberg metadata. | Workflow | Guide | | ------------------------------------------------------------- | ---------------------------------------------------------------------- | | Read source files without importing them into managed tables. | [Query files in place](/docs/lakehouse/query/object-storage.md) | | Copy source files into managed tables. | [Import files on a schedule](/docs/lakehouse/ingest/object-storage.md) | | Read an AWS-managed Iceberg table bucket. | [Connect S3 Tables](/docs/lakehouse/external-catalogs/s3-tables.md) | | Use Glue metadata for Iceberg tables in S3. | [Connect AWS Glue](/docs/lakehouse/external-catalogs/aws-glue.md) | | Use your S3 bucket as storage for Altertable-managed tables. | [Store managed catalog files](/docs/lakehouse/storage/buckets.md) | --- url: https://altertable.ai/docs/lakehouse/integrations/cloudflare-r2.md --- # Cloudflare R2 integration Query R2 files, import them, or connect R2 Data Catalog. | Workflow | Guide | | ------------------------------------------------------------- | ------------------------------------------------------------------------------- | | Read existing Iceberg tables through the catalog interface. | [Connect R2 Data Catalog](/docs/lakehouse/external-catalogs/r2-data-catalog.md) | | Read existing files with SQL. | [Query files in place](/docs/lakehouse/query/object-storage.md) | | Copy files into managed tables on a schedule. | [Import source files](/docs/lakehouse/ingest/object-storage.md) | | Use a connected R2 bucket for Altertable-managed table files. | [Store managed catalog files](/docs/lakehouse/storage/buckets.md) | --- url: https://altertable.ai/docs/lakehouse/integrations/duckdb.md --- # DuckDB integration Query Altertable from DuckDB, ingest local results, or connect an existing DuckDB database. | Workflow | Guide | | ------------------------------------------------------------------ | ------------------------------------------------------------------------ | | Use local DuckDB as a client for Altertable catalogs. | [Query Altertable from DuckDB](/docs/lakehouse/query/duckdb.md) | | Copy results from local DuckDB into a writable Altertable catalog. | [Ingest from DuckDB](/docs/lakehouse/ingest/duckdb.md) | | Query an existing database through an external catalog. | [Connect a DuckDB database](/docs/lakehouse/external-catalogs/duckdb.md) | | Use DuckDB syntax and join data across catalogs. | [DuckDB SQL engine](/docs/lakehouse/sql-engine.md) | --- url: https://altertable.ai/docs/lakehouse/integrations.md --- # Integration directory Choose a technology to find its connection, ingestion, and query guides. The task-based guides remain available in the lakehouse navigation. ## Technologies with multiple workflows - [DuckDB integration](/docs/lakehouse/integrations/duckdb.md): Choose how DuckDB works with Altertable: query managed tables, ingest local results, connect an existing database, or write DuckDB SQL. - [Amazon S3 integration](/docs/lakehouse/integrations/amazon-s3.md): Choose how to use Amazon S3 with Altertable: query existing files, import files, connect S3 Tables or Glue, or store managed catalog data. - [Cloudflare R2 integration](/docs/lakehouse/integrations/cloudflare-r2.md): Choose how Cloudflare R2 works with Altertable: connect its Iceberg catalog, query or import files, or provide storage for managed catalogs. For other object storage, compare [querying files](/docs/lakehouse/query/object-storage.md), [importing files](/docs/lakehouse/ingest/object-storage.md), and [managed catalog storage](/docs/lakehouse/storage/buckets.md). For other Iceberg catalogs, see [Iceberg tables](/docs/lakehouse/external-catalogs/iceberg-tables.md) and [Apache Polaris](/docs/lakehouse/external-catalogs/apache-polaris.md). ## Databases and warehouses Use [the external catalog directory](/docs/lakehouse/external-catalogs.md) for Snowflake, BigQuery, Postgres, MySQL, MariaDB, MongoDB, Redshift, Supabase, and other sources. Each guide lists the connection fields and source-specific prerequisites. ## Application code and dataframes - [SDK installation and availability](/docs/api-references/sdks.md#lakehouse-sdks). - [Query from application code](/docs/lakehouse/query/http-api.md) and [load records or files](/docs/lakehouse/ingest/http-api.md). - [Query with Pandas, Polars, or Ibis](/docs/lakehouse/query/pandas-polars-ibis.md) and [ingest from dataframes](/docs/lakehouse/ingest/pandas-polars-ibis.md). ## SQL clients, BI, and pipelines - [SQL and BI clients](/docs/lakehouse/clients.md): CLI, Postgres, Arrow Flight SQL, Metabase, and Hex. - [Pipeline integrations](/docs/lakehouse/ingest/integrations.md): Airbyte, dlt, Fivetran, PostHog, and Segment. - [dbt transformations](/docs/lakehouse/modeling/dbt.md). --- url: https://altertable.ai/docs/lakehouse/modeling/dbt.md --- # dbt [dbt](https://www.getdbt.com/) helps data teams model and transform analytical data. Use the Altertable adapter to run dbt models directly in your Altertable lakehouse. You can also [connect a dbt repository](#sync-project-knowledge) to make project definitions, lineage, and semantic metadata available to Altertable and its agents. ## Installation For a worked example of organizing raw data, transformations, and marts, follow the [medallion architecture guide](/docs/lakehouse/modeling/medallion-architecture.md). The Altertable adapter for dbt is available on PyPI and currently requires Python `3.10+` with `dbt-core >=1.8,<2.0`. ```bash pip install dbt-altertable ``` If you manage Python dependencies with `uv`, you can install it with: ```bash uv add dbt-altertable ``` ## Configuration Add an Altertable profile to your `profiles.yml` file. By default, dbt looks for profiles in `~/.dbt/profiles.yml`, but you can also place it in your project directory or specify a custom location using the `--profiles-dir` flag. Learn more about [dbt profiles](https://docs.getdbt.com/docs/local/profiles.yml) in the dbt documentation. ### Profile Example ```yaml your_project_name: target: dev outputs: dev: type: altertable username: {{LAKEHOUSE_USERNAME}} password: {{LAKEHOUSE_PASSWORD}} database: your_catalog_name schema: your_schema_name compute_size: XL ``` ### Configuration Parameters - **type**: Must be set to `altertable` (required) - **username**: Your lakehouse username (required) — find this in the [Credentials](/docs/administration/lakehouse-credentials.md) section of the dashboard - **password**: Your lakehouse password (required) — find this in the [Credentials](/docs/administration/lakehouse-credentials.md) section of the dashboard - **database**: The name of your [Altertable catalog](/docs/lakehouse/catalogs.md) where models will be materialized (required) - **schema**: The schema within your catalog where models will be created (required) - **compute\_size**: Optional compute size for your workload. Omit it to use the server default. ## Usage Once configured, you can run dbt commands against Altertable: ```bash # Test your connection dbt debug # Run your models dbt run # Test your models dbt test # Generate and serve documentation dbt docs generate dbt docs serve ``` Learn more about [dbt commands](https://docs.getdbt.com/reference/dbt-commands) in the dbt documentation. ## Sync project knowledge Connect your dbt project as a [knowledge repository](/docs/agents/knowledge/entries.md). Altertable detects `dbt_project.yml` at the repository root or in a subdirectory and turns project source files into searchable knowledge entries. A repository can contain multiple dbt projects. In **Knowledge**, select **New knowledge**, choose **Repository**, and connect your GitHub or GitLab account. Choose the repository and branch, then select **Add repository**. Leave the branch empty to use the repository's default branch. ### Which files need to be committed? For Git-based sync, commit your dbt project source files to the connected branch. Include the artifacts below for lineage, model enrichment, and generated metric definitions. [Contact us](mailto:contact@altertable.ai) if you need to sync artifacts from a bucket. | File | Purpose | | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- | | `dbt_project.yml` and project source files | Detect the project and extract knowledge from SQL, YAML, and documentation. | | `target/manifest.json` | Add model lineage and enrich existing semantic models with dbt metadata. | | [`target/semantic_manifest.json`](https://docs.getdbt.com/reference/artifacts/sl-manifest) | Make generated metric and semantic definitions available as knowledge. | Repository sync does not run dbt to generate artifacts. Without a valid `manifest.json`, Altertable still extracts knowledge from the source files, but skips manifest-based lineage and semantic model enrichment. ### Generate and refresh the artifacts From the directory containing `dbt_project.yml`, install project dependencies and [parse the project](https://docs.getdbt.com/reference/commands/parse). Use the target whose catalog and schema names match the data you want to describe in Altertable. Replace `prod` with that target's name in your dbt profile: ```bash dbt deps dbt parse --target prod git add -f target/manifest.json target/semantic_manifest.json git commit -m "docs(dbt): refresh project artifacts" git push ``` Commit both artifacts under `target/` relative to `dbt_project.yml`. After pushing, select **Sync now** from the repository menu. Regenerate and commit the artifacts when your dbt project changes. ### Enrich existing semantic models With a valid `manifest.json`, Altertable can update existing [semantic models](/docs/agents/knowledge/semantic-layer.md) with dbt descriptions, column metadata, supported measures and dimensions, and relationships derived from entities. It matches each dbt model to an existing Altertable model by catalog, schema, and table name, using the dbt alias when configured. This enrichment does not materialize tables or create missing semantic models. Metric definitions remain available as knowledge even when this enrichment cannot import them as native measures. For example, filtered simple metrics are not imported as unfiltered Altertable measures. ## Run dbt in a task A **dbt task** runs a project from a [connected repository](#sync-project-knowledge) in an isolated environment and writes its models to an Altertable catalog. Altertable supplies the dbt profile, so the task needs no `profiles.yml` or credentials. Creating one requires [permission](/docs/administration/access-control.md) to manage catalogs. Open **Tasks → New task** and choose **dbt task** under **Set up manually**: 1. Under **Project**, select the **Repository**. Set **Project directory** when `dbt_project.yml` is not at the repository root, and optionally a **Git ref**. 2. Under **Command**, choose `build`, `run`, or `source freshness`, and enter a **Selector** such as `tag:daily`. 3. Under **Destination**, choose the **Catalog** and **Schema** to write models to. 4. Set a **Schedule**, then select **Create dbt run**. ## SQL Dialect Altertable uses DuckDB as its query engine, so write models with DuckDB-compatible functions, syntax, and data types. The adapter builds on `dbt-duckdb` and automatically dispatches `duckdb__` macros from your project and installed packages. Learn more about [DuckDB SQL](https://duckdb.org/docs/sql/introduction) in the DuckDB documentation. ## MetricFlow The adapter can run dbt Semantic Layer and MetricFlow workloads by rendering MetricFlow SQL with DuckDB-compatible syntax. Keep your semantic models in dbt as usual, then run MetricFlow against the same Altertable profile. This works best when your metrics use DuckDB-compatible functions and data types. If a metric relies on warehouse-specific SQL, adapt the expression to DuckDB SQL before running it on Altertable. ## Seeds By default, `dbt seed` inserts rows in batches so seeds work reliably across local files and remote execution. When you want a direct DuckDB `COPY` path for a local seed file, set the seed config `fast: true`. ```yaml seeds: your_project_name: +fast: true ``` Use the default batched path unless you know the seed file is accessible from the process running dbt. ## Incremental models The adapter supports `delete+insert`, `merge`, and sequential `microbatch` incremental strategies. With `delete+insert`, configure a stable `unique_key` so each run deletes matching destination rows before inserting the updated batch: ```sql {{ config( materialized='incremental', incremental_strategy='delete+insert', unique_key='id' ) }} SELECT * FROM {{ source('app', 'orders') }} {% if is_incremental() %} WHERE updated_at > (SELECT max(updated_at) FROM {{ this }}) {% endif %} ``` Each dbt connection uses an isolated Flight session, so concurrent model work does not share temporary session state. For more information about incremental strategies and adapter configuration, see the [dbt documentation](https://docs.getdbt.com/docs/build/incremental-models) and the [Altertable dbt adapter](https://github.com/altertable-ai/dbt-altertable). ## Persisting descriptions The adapter supports dbt's [`persist_docs`](https://docs.getdbt.com/reference/resource-configs/persist_docs) setting for model and column descriptions. Enable it in `dbt_project.yml` or on a model: ```yaml models: your_project_name: +persist_docs: relation: true columns: true ``` After `dbt run`, dbt writes model and column descriptions to Altertable with DuckDB `COMMENT ON TABLE` and `COMMENT ON COLUMN` statements. View column comments are skipped gracefully when DuckDB cannot apply them. ## Source Code The adapter source code is available at [github.com/altertable-ai/dbt-altertable](https://github.com/altertable-ai/dbt-altertable) --- url: https://altertable.ai/docs/lakehouse/modeling.md --- # Model and transform data Use [views and macros](/docs/lakehouse/features.md) to reuse SQL logic. Use [dbt](/docs/lakehouse/modeling/dbt.md) to run versioned models and tests against Altertable. Connecting a dbt repository as [business context](/docs/agents/knowledge.md) makes definitions and metadata available to agents. Repository sync does not execute the models. - [Medallion architecture with dbt](/docs/lakehouse/modeling/medallion-architecture.md): Turn raw data accessed through Altertable federation into tested, business-ready tables with dbt, reusable business rules, and marts your teams can rely on. - [dbt](/docs/lakehouse/modeling/dbt.md): Run dbt models on Altertable and sync project knowledge, lineage, and semantic metadata from a connected repository. --- url: https://altertable.ai/docs/lakehouse/modeling/medallion-architecture.md --- # Medallion architecture with dbt Turn raw orders into tested tables your teams can use to report on sales. With Altertable federation, dbt reads the orders where they already live, without copying them first. Your dbt models then clean them into staging tables in your lakehouse, apply shared business rules, and publish a mart, all running through [dbt-altertable](https://github.com/altertable-ai/dbt-altertable). This guide walks through an orders pipeline from a connected database to a completed-orders mart, then uses that mart to calculate daily sales. We recommend following dbt's [project structure](https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview?version=2): organize models into `staging`, `intermediate`, and `marts`. Medallion names describe the data's role: | Layer | dbt equivalent | Responsibility | | ------ | ------------------------------- | --------------------------------------------------------------------------------------------------- | | Bronze | Sources | Read raw records from external catalogs or ingested tables. | | Silver | Staging and intermediate models | Clean, type, and deduplicate raw records into one row per entity, then apply shared business rules. | | Gold | Marts | Publish tested, business-ready tables your dashboards and teams query directly. | The pipeline starts at Bronze. Connect an [external catalog](/docs/lakehouse/external-catalogs.md) and read its raw tables through `source()`, without first copying them into a separate raw schema in Altertable. Already-ingested raw tables work too. Downstream models use `ref()` to declare their dependencies. ## Materialize at the staging layer Use federation for source access and staging tables for repeated analytical work: External catalog app\_postgres.public.orders (bronze) → Altertable catalog analytics: staging tables (silver) → intermediate views (silver) → marts (gold). dbt reads the external source through federation and materializes staging; downstream models read the stored data. Each staging build reads the source through Altertable, cleans and types its records, and stores the normalized rows in a managed table. Intermediate models and marts reuse those prepared rows without repeating the cleanup or querying the external source again. The staging refresh still puts load on the source; choose its frequency and scope accordingly. Organize your dbt models into three folders: `staging` for cleaning source records, `intermediate` for reusable transformations, and `marts` for datasets ready for analysis. This guide uses SQL models. For the staging models, set `+materialized: table` in `dbt_project.yml`, as shown below. This tells dbt to run each staging query and save its results as a table in Altertable. The next dbt build refreshes that table. Using `view` would save only the query definition, so reading it would query the external source again. Federation exposes the data the source retains; it does not create an immutable bronze history. If you need replay after source updates or deletes, retain raw history upstream or [ingest an archival copy](/docs/lakehouse/ingest.md) before transforming it. Rebuilding a staging table replaces its prior contents. ## Before you start - Create a writable [Altertable catalog](/docs/lakehouse/catalogs.md). This example uses `analytics`; substitute your catalog name throughout. - Install and configure the [dbt adapter](/docs/lakehouse/modeling/dbt.md) with your [lakehouse credentials](/docs/administration/lakehouse-credentials.md). - Check the adapter's [README](https://github.com/altertable-ai/dbt-altertable#readme) for current Python and dbt version requirements and supported features. The dbt best-practice links describe project organization. - Connect your source database as an [external catalog](/docs/lakehouse/external-catalogs.md). This example calls it `app_postgres` and expects `app_postgres.public.orders` with `id`, `ordered_at`, `amount_cents`, and `status`, with one row per order and amounts in a single currency. Substitute your external catalog and schema names. Keep raw values and any source identifiers or ingestion timestamps you need to replay and audit data. This example reads a source with one row per order. If your source contains duplicates or multiple versions of each order, deduplicate in staging using the source's change timestamps and delete markers before applying the completed-order rule. ## Organize the project Create these files in your dbt project: ```text dbt_project.yml models/ staging/ app/ _sources.yml _models.yml stg_app__orders.sql intermediate/ int_orders__completed.sql marts/ sales/ orders.sql _models.yml ``` In `dbt_project.yml`, configure the project and folder defaults: ```yaml name: orders_analytics profile: orders_analytics models: orders_analytics: staging: +schema: staging +materialized: table intermediate: +schema: intermediate +materialized: view marts: +schema: marts +materialized: table ``` The key under `models` must match the project's `name`. ## Map folders to Altertable schemas The adapter's `database` field identifies an Altertable catalog. `schema` identifies the target namespace inside it. Set up two targets in `~/.dbt/profiles.yml`, supplying credentials through environment variables: ```yaml orders_analytics: target: dev outputs: dev: type: altertable username: "{{ env_var('ALTERTABLE_LAKEHOUSE_USERNAME') }}" password: "{{ env_var('ALTERTABLE_LAKEHOUSE_PASSWORD') }}" database: analytics schema: alice_dev prod: type: altertable username: "{{ env_var('ALTERTABLE_LAKEHOUSE_USERNAME') }}" password: "{{ env_var('ALTERTABLE_LAKEHOUSE_PASSWORD') }}" database: analytics schema: analytics_prod ``` Replace `alice_dev` with a developer-specific name. This example uses one Altertable environment and catalog, with separate output schemas. dbt target names do not create or select Altertable environments; use the appropriate environment's credentials and catalog when separating environments. Following dbt's [folder-to-schema mapping](https://docs.getdbt.com/best-practices/how-we-structure/6-mapping-folders-to-warehouse?version=2), keep its default `_` naming: | Folder | Development schema | Production schema | | -------------- | ------------------------ | ----------------------------- | | `staging` | `alice_dev_staging` | `analytics_prod_staging` | | `intermediate` | `alice_dev_intermediate` | `analytics_prod_intermediate` | | `marts` | `alice_dev_marts` | `analytics_prod_marts` | All these output schemas live in `analytics`. Both targets read `app_postgres.public.orders` through federation; neither writes models into the external catalog. Avoid replacing dbt's schema-generation macro just to remove the prefix: shared output names can let a development run overwrite production models. You can use `+schema: silver` and `+schema: gold` if your team requires medallion names. Assigning `silver` to both staging and intermediate puts them in the same schema. Keeping folder names makes those transformation steps easier to distinguish. ## Declare the bronze source In `models/staging/app/_sources.yml`, point dbt at the table exposed by the external catalog. This source's `database` differs from the output catalog in your profile: ```yaml version: 2 sources: - name: app database: app_postgres schema: public tables: - name: orders description: Source orders with amounts in cents, in a single currency. columns: - name: id tests: - not_null ``` Declaring a source does not connect the external catalog or create a table. Configure the connection in Altertable first and make sure your dbt credentials can read it. dbt connects to Altertable; Altertable accesses the external system using that catalog's connection. If you already ingest raw orders into `analytics.raw.orders`, use `database: analytics` and `schema: raw` in this source definition instead. The transformation models stay the same. ## Build the silver models In `models/staging/app/stg_app__orders.sql`, normalize the source fields with DuckDB-compatible SQL: ```sql select cast(id as bigint) as order_id, cast(ordered_at as timestamp) as ordered_at, cast(amount_cents as decimal(18, 2)) / 100.0 as amount, lower(trim(status)) as status from {{ source('app', 'orders') }} ``` With the folder configuration above, dbt reads `app_postgres.public.orders` and materializes this model as `analytics.alice_dev_staging.stg_app__orders` in development. Its `source()` reference crosses catalogs; downstream `ref()` references resolve to models in `analytics`. In `models/staging/app/_models.yml`, test that Silver has one row per order. Bronze records may contain duplicates; the uniqueness check belongs on the prepared staging model: ```yaml version: 2 models: - name: stg_app__orders description: Cleaned orders with one row per order. columns: - name: order_id tests: - not_null - unique ``` Choose a timestamp convention that matches your source. This example assumes `ordered_at` is already expressed in the timezone you use for daily reporting. In `models/intermediate/int_orders__completed.sql`, isolate the completed-order rule: ```sql select order_id, cast(ordered_at as date) as order_date, amount from {{ ref('stg_app__orders') }} where status = 'completed' ``` This step gives other models a shared definition of a completed order. Add intermediate models when they isolate useful logic; a small pipeline can go directly from staging to a mart. ## Publish a gold mart In `models/marts/sales/orders.sql`, publish one row per completed order. Following dbt's [mart guidance](https://docs.getdbt.com/best-practices/how-we-structure/4-marts), keep the mart at the order grain so consumers can calculate daily sales or group by other reporting periods: ```sql select order_id, order_date, amount from {{ ref('int_orders__completed') }} ``` In `models/marts/sales/_models.yml`, document the grain and test it: ```yaml version: 2 models: - name: orders description: One row per completed order, with amounts in the source currency. columns: - name: order_id tests: - not_null - unique - name: order_date description: Calendar date of the completed order. tests: - not_null - name: amount tests: - not_null ``` These are dbt data tests that query the data, not database constraints. Add business-specific checks for cancellations, refunds, currencies, and late-arriving updates before treating this as a financial reporting model. ## Build and verify From the project directory, verify the connection, test the raw input, and build the models: ```bash dbt debug --target dev dbt test --select source:app.orders --target dev dbt build --target dev ``` Source tests query the external system through federation too. Proceed only if they pass. After a successful build, inspect the materialized staging table in Altertable: ```sql select * from analytics.alice_dev_staging.stg_app__orders limit 10; ``` Then calculate daily sales from the development mart. This query aggregates the order-level data into daily metrics: ```sql select order_date, count(*) as completed_orders, sum(amount) as completed_order_amount from analytics.alice_dev_marts.orders group by order_date order by order_date desc; ``` After reviewing the results, run the same source checks and build against production: ```bash dbt test --select source:app.orders --target prod dbt build --target prod ``` Point production queries, insights, and dashboards at `analytics.analytics_prod_marts.orders`. The daily sales query returns dates with completed orders; add a date spine if consumers need rows for days with no orders. ## Schedule dbt in Altertable Altertable can run this project for you on a schedule with a **dbt task**. Each run executes a dbt command from a connected repository and writes models into your chosen Altertable catalog. 1. Commit the dbt project to a [connected knowledge repository](/docs/agents/knowledge/entries.md#dbt-repositories). 2. Open **Tasks**, open the **New task** menu, and choose **dbt task**. 3. Select your **Repository**. Set **Project directory** if `dbt_project.yml` is in a subdirectory; otherwise leave it empty. Set **Git ref** to the branch, tag, or commit you want to run, or leave it empty for the default branch. 4. Configure the command and destination: | Field | Value for this guide | Purpose | | ----------- | -------------------- | ------------------------------------------------------- | | dbt command | `build` | Build the models and run their tests. | | Selector | `*` | Include all models in this example project. | | Target | `prod` | Expose `prod` as `target.name` in the project. | | Catalog | `analytics` | Store the resulting tables and views in Altertable. | | Schema | `analytics_prod` | Set the base schema used by dbt's custom schema naming. | With the project's `+schema` settings, the outputs go into `analytics_prod_staging`, `analytics_prod_intermediate`, and `analytics_prod_marts`. The **Target** name and destination are separate settings: entering `prod` does not select a catalog or schema for you. 5. Choose **Hourly**, **Daily**, **Weekly**, or **Custom cron** in **Schedule**. Schedule times are in UTC. For example, `0 6 * * *` runs every day at 06:00 UTC. Choose a time after your upstream data is ready. 6. Create the task. You can also leave it on a manual schedule, run it from its task page, and inspect the run history before setting up recurring runs. Each scheduled build refreshes the staging tables from the federated source, then builds the dependent intermediate models and marts. For a larger project, narrow **Selector** to the models or tags you want to refresh. The task also supports `run` for model execution without tests and **source freshness** for checking configured source freshness thresholds. Repository knowledge sync and dbt execution are separate: [syncing project metadata](/docs/lakehouse/modeling/dbt.md#sync-project-knowledge) makes definitions and lineage available to Altertable; the dbt task executes the transformations. See [Tasks](/docs/automation/tasks.md) for more about scheduled work. --- url: https://altertable.ai/docs/lakehouse/query/duckdb.md --- # Query Altertable from DuckDB The [DuckDB Altertable extension](https://github.com/altertable-ai/duckdb-altertable) connects DuckDB to Altertable. Use it when you want to query governed Altertable data from DuckDB or join it with local files. The extension supports: - **Attached databases** for querying Altertable catalogs with standard SQL - **Direct table access** through DuckDB's catalog - **Raw query execution** for remote SQL statements ## Installation Install and load the community extension from DuckDB: ```sql INSTALL altertable FROM community; LOAD altertable; ``` ## Attach a database Use your lakehouse username, lakehouse password, and catalog in the connection string: ```sql ATTACH 'user={{LAKEHOUSE_USERNAME}} password={{LAKEHOUSE_PASSWORD}} catalog=my_catalog' AS remote_db (TYPE ALTERTABLE); ``` ### Connection parameters | Parameter | Description | Example | | ---------- | ------------------------- | ------------------------ | | `user` | Lakehouse username | `{{LAKEHOUSE_USERNAME}}` | | `password` | Lakehouse password | `{{LAKEHOUSE_PASSWORD}}` | | `catalog` | Remote Altertable catalog | `analytics` | Or use DuckDB secrets: ```sql CREATE SECRET my_altertable ( TYPE altertable, USER '{{LAKEHOUSE_USERNAME}}', PASSWORD '{{LAKEHOUSE_PASSWORD}}', CATALOG 'my_catalog' ); ATTACH '' AS remote_db (TYPE altertable, SECRET my_altertable); ``` To disconnect the attached catalog: ```sql DETACH remote_db; ``` ## Query a remote table Query tables through the attached Altertable database: ```sql SELECT * FROM remote_db.main.orders; ``` ## Filter results Run `SELECT` queries against attached Altertable tables: ```sql SELECT order_id, amount, status FROM remote_db.main.orders WHERE status = 'open'; ``` ## Export to local files Export attached tables to local files: ```sql COPY remote_db.main.orders TO 'orders.csv'; COPY remote_db.main.orders TO 'orders.parquet'; COPY remote_db.main.orders TO 'orders.json'; ``` For Python DataFrame workflows, see [Query with Pandas, Polars, and Ibis](/docs/lakehouse/query/pandas-polars-ibis.md). For ingesting data from DuckDB, see [Ingest data with DuckDB](/docs/lakehouse/ingest/duckdb.md). For more examples and source builds, see the [DuckDB Altertable extension repository](https://github.com/altertable-ai/duckdb-altertable). For other DuckDB workflows, see the [DuckDB integration hub](/docs/lakehouse/integrations/duckdb.md). --- url: https://altertable.ai/docs/lakehouse/query/http-api.md --- # Query from an application Run SQL over HTTPS with an Altertable SDK or HTTP client. You need [lakehouse credentials](/docs/administration/lakehouse-credentials.md) for the environment that owns your catalogs. For terminal queries, use the [CLI](/docs/lakehouse/clients/cli.md). For all request fields, response formats, and errors, use the [HTTP API reference](/docs/api-references/lakehouse.md#post-query). ## Set up with your coding agent ```text Add an Altertable query to this application. Read https://altertable.ai/docs/lakehouse/query/http-api.md and the linked SDK or HTTP reference first. Inspect the application runtime and existing data-access code. Ask what result the application needs and confirm the target environment. Choose an SDK supported by this runtime, or use the HTTP API. Keep lakehouse credentials on the server in the project's secret configuration. Verify authentication with SELECT 1 AS connected, then inspect the real catalog and table schemas. Use a bounded query and bind parameters for user-supplied values. Integrate result handling and errors using the application's existing patterns. Handle the API's streamed response correctly. Run the relevant checks and verify a real result without logging credentials or sensitive rows. Summarize the code changes and required configuration. ``` ## 1. Verify authentication Set your credentials locally, then send a query that needs no existing table: ```bash export ALTERTABLE_USERNAME='{{LAKEHOUSE_USERNAME}}' export ALTERTABLE_PASSWORD='{{LAKEHOUSE_PASSWORD}}' curl https://api.altertable.ai/query \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H 'Content-Type: application/json' \ -d '{"statement":"SELECT 1 AS connected"}' ``` Confirm that the response contains a `connected` column and a row with `1`. If authentication fails, check the credentials and target environment before querying application data. ## 2. Query your data See [lakehouse SDK installation and availability](/docs/api-references/sdks.md#lakehouse-sdks) before choosing a library. Rust and PHP are not yet published; use HTTP from those runtimes. The cURL example below reuses `ALTERTABLE_USERNAME` and `ALTERTABLE_PASSWORD` from step 1. Replace the example catalog and table with names from your environment. The SDK examples show client initialization and result handling: ### cURL ```bash curl -X POST https://api.altertable.ai/query \ --user "$ALTERTABLE_USERNAME:$ALTERTABLE_PASSWORD" \ -H 'Content-Type: application/json' \ -d '{"statement": "SELECT * FROM my_catalog.main.events ORDER BY timestamp DESC LIMIT 100"}' ``` ### JavaScript ```typescript import { AltertableLakehouseClient } from '@altertable/lakehouse'; const client = new AltertableLakehouseClient({ username: '{{LAKEHOUSE_USERNAME}}', password: '{{LAKEHOUSE_PASSWORD}}', }); const result = await client.queryAll({ statement: ` SELECT date_trunc('day', timestamp) AS day, event, COUNT(*) AS events, ROUND(AVG(value), 2) AS avg_value FROM my_catalog.main.events WHERE timestamp >= NOW() - INTERVAL '30 days' GROUP BY 1, 2 ORDER BY day DESC, events DESC `, }); for (const row of result.rows) { console.log(row); } ``` ### Python ```python from altertable_lakehouse import Client from altertable_lakehouse.models import QueryRequest client = Client( username="{{LAKEHOUSE_USERNAME}}", password="{{LAKEHOUSE_PASSWORD}}", ) req = QueryRequest(statement=""" SELECT date_trunc('day', timestamp) AS day, event, COUNT(*) AS events, ROUND(AVG(value), 2) AS avg_value FROM my_catalog.main.events WHERE timestamp >= NOW() - INTERVAL '30 days' GROUP BY 1, 2 ORDER BY day DESC, events DESC """) result = client.query_all(req) for row in result.rows: print(row) ``` ### Ruby ```ruby require 'altertable/lakehouse' client = Altertable::Lakehouse::Client.new( username: '{{LAKEHOUSE_USERNAME}}', password: '{{LAKEHOUSE_PASSWORD}}' ) client.query_all( statement: <<~SQL, params: { "event" => "checkout_completed" } SELECT date_trunc('day', timestamp) AS day, event, COUNT(*) AS events, ROUND(AVG(value), 2) AS avg_value FROM my_catalog.main.events WHERE timestamp >= NOW() - INTERVAL '30 days' AND event = $event GROUP BY 1, 2 ORDER BY day DESC, events DESC SQL )[:rows].each do |row| p row end ``` ### Go ```go import ( "context" "fmt" "log" altertable "github.com/altertable-ai/altertable-lakehouse-go" ) client, err := altertable.NewClient(altertable.Config{ Username: "{{LAKEHOUSE_USERNAME}}", Password: "{{LAKEHOUSE_PASSWORD}}", }) if err != nil { log.Fatal(err) } result, err := client.QueryAll(context.Background(), altertable.QueryRequest{ Statement: ` SELECT date_trunc('day', timestamp) AS day, event, COUNT(*) AS events, ROUND(AVG(value), 2) AS avg_value FROM my_catalog.main.events WHERE timestamp >= NOW() - INTERVAL '30 days' GROUP BY 1, 2 ORDER BY day DESC, events DESC `, }) if err != nil { log.Fatal(err) } for _, row := range result.Rows { fmt.Println(row) } ``` ### Altertable CLI ```bash # Query lakehouse data altertable query " SELECT date_trunc('day', timestamp) AS day, event, COUNT(*) AS events, ROUND(AVG(value), 2) AS avg_value FROM my_catalog.main.events WHERE timestamp >= NOW() - INTERVAL '30 days' GROUP BY 1, 2 ORDER BY day DESC, events DESC " ``` Use fully qualified table names and bound exploratory queries with `LIMIT`. In application code, store credentials in your runtime's secret configuration. Use bind parameters for values supplied by users. Named parameters use `$name`; positional parameters use `$1`, `$2`, and so on. Send an object or array in `params` rather than interpolating values into SQL: ```json { "statement": "SELECT * FROM my_catalog.main.users WHERE id = $id", "params": { "id": 123 } } ``` ## 3. Consume results The default HTTP response is a JSONL stream containing query metadata, a column-name array, and result rows. Process it line by line rather than parsing the whole response as one JSON document. An SDK can handle the stream for you. If your consumer expects a standard format, request CSV, row-oriented JSONL, or Parquet. The [response format reference](/docs/api-references/lakehouse.md#post-query) defines the exact envelopes and options. ## Tune execution when needed - [Choose compute](/docs/lakehouse/compute/sizing.md) for workload sizing and session constraints. - [Transpile SQL](/docs/lakehouse/sql-engine.md#transpile-another-sql-dialect) when porting queries from another dialect. - [Inspect or cancel a query](/docs/api-references/lakehouse.md#get-query-query-uuid) through the query lifecycle endpoints. ## Troubleshooting - **Unauthorized request:** use lakehouse credentials from the intended environment. - **Missing table:** qualify the table with its catalog and schema, then check catalog permissions. - **Unexpected response shape:** check whether you requested the default envelope or a standard output format. - **No available Worker:** inspect compute availability and retry with backoff. See [compute sizing](/docs/lakehouse/compute/sizing.md). --- url: https://altertable.ai/docs/lakehouse/query.md --- # Explore and query Use [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) to inspect tables and run a first query. Use [Ask Altertable](/docs/agents/ask.md) when you want to explore a question with an agent. Queries run with [DuckDB SQL](/docs/lakehouse/sql-engine.md). Qualify tables as `catalog.schema.table` to join managed tables with connected sources. For application code, use the [HTTP API or SDK](/docs/lakehouse/query/http-api.md). For terminal workflows, use the [CLI](/docs/lakehouse/clients/cli.md). To connect a BI tool, choose a [SQL client interface](/docs/lakehouse/clients.md). - [SQL Explorer](/docs/lakehouse/query/sql-explorer.md): Explore Altertable data with SQL, browse catalogs, run DuckDB-powered queries, and inspect results from your lakehouse. - [Query from an application](/docs/lakehouse/query/http-api.md): Run SQL from application code with an Altertable SDK or HTTP request, consume streamed results, and verify authentication before querying tables. - [Query object storage](/docs/lakehouse/query/object-storage.md): Query files and Iceberg datasets in S3-compatible object storage from Altertable without moving data into another system. - [Query Altertable from DuckDB](/docs/lakehouse/query/duckdb.md): Query Altertable databases from DuckDB with the community extension and work with lakehouse data from local SQL workflows. - [Query with Pandas, Polars, and Ibis](/docs/lakehouse/query/pandas-polars-ibis.md): Query Altertable tables with Pandas, Polars, or Ibis through the DuckDB extension or the Python lakehouse client's Ibis backend. --- url: https://altertable.ai/docs/lakehouse/query/object-storage.md --- # Query object storage Altertable can query data that already lives in Amazon S3, Cloudflare R2, Google Cloud Storage, or another S3-compatible bucket. Map the files once, then use the same SQL engine and query interfaces you use for Altertable catalogs. Object storage queries run in place. Altertable reads mapped files from the connected bucket. ## Goal Map an object-storage dataset, query it with a fully qualified table name, and join it with another Altertable catalog. ## Prerequisites - An S3-compatible bucket containing Parquet, CSV, JSON, Iceberg, or S3 Tables data. - Permission for Altertable to read the intended bucket paths and metadata. - A [mapped external catalog](/docs/lakehouse/external-catalogs.md), or the connection details needed to create one. ## Steps ### 1. Choose a table type | Source | Best fit | Learn more | | --------------------------- | ------------------------------------------------ | --------------------------------------------------------------------- | | Parquet, CSV, or JSON files | Query file paths and prefixes as external tables | [Bucket Tables](/docs/lakehouse/external-catalogs/bucket-tables.md) | | Apache Iceberg datasets | Query datasets through Iceberg table metadata | [Iceberg Tables](/docs/lakehouse/external-catalogs/iceberg-tables.md) | | Amazon S3 table buckets | Query AWS-managed Iceberg table buckets | [S3 Tables](/docs/lakehouse/external-catalogs/s3-tables.md) | Follow the guide for the selected source to map its files or metadata as a table. ### 2. Query a mapped table After you map object storage, query the exposed tables with fully qualified names: ```sql SELECT event_name, count(*) AS event_count FROM bucket_exports.main.events WHERE event_date >= DATE '2026-01-01' GROUP BY event_name ORDER BY event_count DESC; ``` ### 3. Join another catalog Mapped object-storage tables can be joined with Altertable-managed catalogs and other external catalogs: ```sql SELECT customers.segment, count(*) AS events FROM bucket_exports.main.events JOIN analytics.main.customers ON events.customer_id = customers.customer_id GROUP BY customers.segment; ``` ### 4. Explore in SQL Explorer Start in [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) to inspect results and refine your query. **Use another query interface** - Use the [HTTP API](/docs/lakehouse/query/http-api.md) from application code. - Use [DuckDB](/docs/lakehouse/query/duckdb.md) for local analysis and file joins. - Use the [Postgres adapter](/docs/lakehouse/clients/postgres.md) or [Arrow Flight SQL](/docs/lakehouse/clients/arrow-flight-sql.md) when your BI tool needs a database protocol. - Use [Ask Altertable](/docs/agents/ask.md) to investigate in conversation. ### 5. Improve query performance > **Recommendation: use columnar formats:** > > For large object-storage datasets, prefer columnar formats such as Parquet or Iceberg, keep partitions predictable, and filter on partition columns when possible. Altertable can query CSV and JSON files, but columnar formats usually scan less data and return results faster. ## Verification Run a small count against the mapped table: ```sql SELECT count(*) AS row_count FROM bucket_exports.main.events; ``` Confirm that `row_count` is consistent with the files or Iceberg dataset mapped to the table. ## Troubleshooting - **The table is not found:** qualify it with the mapped catalog and schema, then confirm the exposed table name. - **Files are missing:** check that the mapped path or prefix includes the expected objects. - **The bucket cannot be read:** verify the object-storage credentials and permissions for the mapped paths and metadata. - **Queries scan too much data:** prefer Parquet or Iceberg, keep partitions predictable, and filter on partition columns. - **A join returns no rows:** inspect the join-key types and values in each catalog before joining them. **Continue in Altertable:** [Create catalog](/app/lakehouse/catalogs/new) ## Next steps - [Map bucket tables](/docs/lakehouse/external-catalogs/bucket-tables.md) - [Map Iceberg tables](/docs/lakehouse/external-catalogs/iceberg-tables.md) - [Map Amazon S3 Tables](/docs/lakehouse/external-catalogs/s3-tables.md) - [Query with SQL Explorer](/docs/lakehouse/query/sql-explorer.md) --- url: https://altertable.ai/docs/lakehouse/query/pandas-polars-ibis.md --- # Query with Pandas, Polars, and Ibis Use DuckDB's Python client and the [DuckDB Altertable extension](https://github.com/altertable-ai/duckdb-altertable) to query Altertable tables with Pandas, Polars, or [Ibis](https://ibis-project.org/). DuckDB executes SQL against the attached Altertable catalog, then returns results in the dataframe format you choose. The Altertable community extension requires DuckDB 1.5.4 or later. For Ibis queries over HTTP, use the [Python lakehouse client](#query-with-ibis-over-http) instead. ## Connect DuckDB to Altertable Load the community extension and attach an Altertable catalog: ```python import os import duckdb con = duckdb.connect() con.execute("INSTALL altertable FROM community") con.execute("LOAD altertable") con.execute( f"ATTACH 'user={os.environ['LAKEHOUSE_USERNAME']} " f"password={os.environ['LAKEHOUSE_PASSWORD']} " f"catalog={os.environ['LAKEHOUSE_CATALOG']}' " "AS remote (TYPE ALTERTABLE)" ) ``` The connection uses the `LAKEHOUSE_USERNAME`, `LAKEHOUSE_PASSWORD`, and `LAKEHOUSE_CATALOG` environment variables. For other connection options and DuckDB secrets, see [Query with DuckDB](/docs/lakehouse/query/duckdb.md). ## Query into Pandas ```bash pip install 'duckdb==1.5.4' pandas ``` Call `.df()` on the result of a DuckDB query to return a Pandas DataFrame: ```python events = con.sql(""" SELECT event_id, customer_id, event_name FROM remote.main.events WHERE event_name = 'activated' ORDER BY event_id """).df() print(events) ``` ## Query into Polars ```bash pip install 'duckdb==1.5.4' polars pyarrow ``` `pyarrow` is required for DuckDB's Polars integration. Call `.pl()` to return a Polars DataFrame: ```python events = con.sql(""" SELECT event_id, customer_id, event_name FROM remote.main.events WHERE event_name = 'activated' ORDER BY event_id """).pl() print(events) ``` ## Join remote tables with local DataFrames Register a local DataFrame with DuckDB to use it in the same SQL query as an attached Altertable table: ```python import polars as pl customers = pl.DataFrame( { "customer_id": ["customer-1", "customer-2"], "segment": ["self_serve", "enterprise"], } ) con.register("customers_df", customers) result = con.sql(""" SELECT c.customer_id, c.segment, COUNT(*) AS event_count FROM customers_df AS c JOIN remote.main.events AS e ON e.customer_id = c.customer_id GROUP BY c.customer_id, c.segment ORDER BY event_count DESC """).pl() ``` The same registration pattern works with Pandas DataFrames. ## Query with Ibis ```bash pip install 'ibis-framework[duckdb]' 'duckdb==1.5.4' ``` Install `pandas` or `polars pyarrow` as well if you plan to call `.to_pandas()` or `.to_polars()`. Create an Ibis connection with the same attach workflow: ```python import os import ibis con = ibis.duckdb.connect() con.raw_sql("INSTALL altertable FROM community") con.raw_sql("LOAD altertable") con.raw_sql( f"ATTACH 'user={os.environ['LAKEHOUSE_USERNAME']} " f"password={os.environ['LAKEHOUSE_PASSWORD']} " f"catalog={os.environ['LAKEHOUSE_CATALOG']}' " "AS remote (TYPE ALTERTABLE)" ) ``` Reference a remote table, filter and sort with Ibis expressions, then materialize the result: ```python events = con.table("events", database="remote.main") activated = events.filter(events.event_name == "activated").order_by("event_id") activated.to_pandas() ``` Return a Polars DataFrame instead: ```python activated.to_polars() ``` Use `sql()` when you prefer to write SQL directly: ```python events = con.sql(""" SELECT event_id, customer_id, event_name FROM remote.main.events WHERE event_name = 'activated' ORDER BY event_id """) events.to_pandas() ``` Join a local `memtable` to an attached Altertable table in the same expression: ```python customers = ibis.memtable( { "customer_id": ["customer-1", "customer-2"], "segment": ["self_serve", "enterprise"], } ) events = con.table("events", database="remote.main") result = ( customers.join(events, "customer_id") .group_by("customer_id", "segment") .aggregate(event_count=events.event_id.count()) .order_by(ibis.desc("event_count")) ) result.to_polars() ``` For the SQL-only workflow, including attachment options and direct table access, see [Query with DuckDB](/docs/lakehouse/query/duckdb.md). ## Query with Ibis over HTTP The Python lakehouse client also provides an optional Ibis backend. Install the Ibis extra: ```bash pip install 'altertable-lakehouse[ibis]' ``` Connect directly to Altertable without loading the DuckDB extension: ```python import os import ibis con = ibis.altertable.connect( username=os.environ["LAKEHOUSE_USERNAME"], password=os.environ["LAKEHOUSE_PASSWORD"], catalog=os.environ["LAKEHOUSE_CATALOG"], database="main", ) try: events = con.table("events") counts = events.group_by("event_name").aggregate(event_count=events.count()) print(counts.execute()) finally: con.disconnect() ``` `catalog` selects the Altertable catalog; `database` selects its schema. Expressions execute remotely through the HTTP query API, and results are buffered in memory. Filter and aggregate before calling `execute()` to limit the result size. This backend is for reading data. Use the lakehouse client's upload and ingestion methods for writes; local memtable joins and Python UDFs are not supported. For local DataFrame joins, use the DuckDB workflow above. --- url: https://altertable.ai/docs/lakehouse/query/sql-explorer.md --- # Query with SQL Explorer SQL Explorer is the in-app workspace for writing and running ad-hoc SQL against Altertable. Use it when you want to inspect tables, test joins, debug a query, or turn an exploration into a saved analysis. Queries use the same [DuckDB SQL dialect](/docs/lakehouse/sql-engine.md) as the rest of the lakehouse. You can reference Altertable catalogs and external catalogs in one statement. Select **Ask Agent** in SQL Explorer when you want help drafting or refining a query without leaving the workspace. Review the generated SQL before running it, especially when it writes data. ## Example Join product usage to account data from another catalog: ```sql SELECT accounts.name, COUNT(*) AS sessions FROM product_analytics.analytics.web_sessions sessions JOIN crm.public.accounts accounts ON sessions.account_id = accounts.id WHERE sessions.started_at >= now() - INTERVAL '7 days' GROUP BY accounts.name ORDER BY sessions DESC LIMIT 20; ``` ## When to use it Use SQL Explorer for fast iteration and one-off checks. Use [SQL insights](/docs/data-apps/insights.md#2-sql-insights) when the query should become a saved visualization, and use the [HTTP API](/docs/lakehouse/query/http-api.md) when application code needs to run queries programmatically. ## Learn more - **[SQL engine](/docs/lakehouse/sql-engine.md)**: DuckDB dialect and cross-catalog queries. - **[Insights](/docs/data-apps/insights.md)**: save SQL as reusable analyses. - **[HTTP API](/docs/lakehouse/query/http-api.md)**: run SQL from scripts and applications. --- url: https://altertable.ai/docs/lakehouse/sql-engine.md --- # SQL engine ## Dialect Altertable uses the **DuckDB SQL dialect** for all queries. If you are familiar with PostgreSQL or SQLite, most core syntax will be familiar. Example query when Product Analytics is enabled for the environment: ```sql SELECT distinct_id, COUNT(*) AS sessions FROM product_analytics.analytics.web_sessions WHERE started_at >= date_trunc('week', now()) GROUP BY distinct_id ORDER BY sessions DESC LIMIT 10; ``` For complete syntax reference, see the [DuckDB SQL Documentation](https://duckdb.org/docs/sql/introduction). ## Data types Altertable supports the same data types as DuckDB, including: | Category | Types | | -------- | ------------------------------------------------------ | | Numeric | `INTEGER`, `BIGINT`, `DOUBLE`, `DECIMAL`, `HUGEINT` | | Text | `VARCHAR`, `TEXT` | | Temporal | `DATE`, `TIMESTAMP`, `TIMESTAMPTZ`, `TIME`, `INTERVAL` | | Binary | `BLOB`, `BITSTRING` | | Boolean | `BOOLEAN` | | Nested | `ARRAY`, `LIST`, `MAP`, `STRUCT`, `UNION` | | Special | `UUID`, `JSON` | For a complete list of supported data types and their specifications, see the [DuckDB Data Types Overview](https://duckdb.org/docs/stable/sql/data_types/overview). ## Federated queries Connect multiple data sources, from BigQuery to Postgres catalogs, in the same SQL namespace: ```sql SELECT u.email, b.total_revenue FROM pg_production.users u JOIN bigquery.revenue b ON u.id = b.user_id; ``` The engine handles cross-catalog federation automatically, optimizing query execution across all data sources in your lakehouse. ## Transpile another SQL dialect Altertable executes [DuckDB SQL](/docs/lakehouse/sql-engine.md). Most of the lakehouse—SQL Explorer, Postgres adapter, dashboards—assumes you write DuckDB syntax directly. The `dialect` parameter is different: it lets you send SQL written for another engine and have Altertable transpile it to DuckDB before execution. That is useful when you are porting saved queries from another platform, running BI-generated SQL, or prototyping a migration without rewriting every function by hand. ```json { "statement": "SELECT DATEDIFF(day, start_date, end_date) AS days FROM my_catalog.main.orders", "dialect": "snowflake" } ``` Altertable transpiles the statement, then runs the DuckDB version on your catalog. ### How it works 1. You send `statement` in the source dialect and set `dialect` to that engine. 2. Altertable transpiles the SQL to DuckDB using [polyglot-sql](https://github.com/tobilg/polyglot). 3. The transpiled statement runs on Workers like any other query. Omit `dialect` (or write DuckDB SQL and leave it unset) when you already target DuckDB. Transpilation is optional, not the default path. Dialect names are case-insensitive. Some engines accept aliases—for example, `tsql`, `mssql`, and `sqlserver` all map to SQL Server. ### Common source dialects Transpilation coverage depends on the source dialect and the functions in your SQL. These are the engines teams most often port from: | `dialect` value | Source engine | | --------------- | -------------------- | | `snowflake` | Snowflake | | `databricks` | Databricks | | `bigquery` | Google BigQuery | | `spark` | Apache Spark SQL | | `trino` | Trino | | `presto` | Presto | | `redshift` | Amazon Redshift | | `postgresql` | PostgreSQL | | `mysql` | MySQL | | `tsql` | Microsoft SQL Server | | `clickhouse` | ClickHouse | | `athena` | Amazon Athena | Many other dialects are supported, including `hive`, `oracle`, `sqlite`, `teradata`, and `fabric`. Unknown dialect names return `400 Bad Request`. > **Transpilation limits:** > > Transpilation handles common syntax and function differences, but it is not a full semantic rewrite. Warehouse-specific UDFs, session variables, staged files, and engine-only table functions may fail to transpile or behave differently after conversion. > > When transpilation fails, the API returns `400 Bad Request` with an error such as `Failed to transpile SQL to DuckDB`. Fix the statement, simplify it, or rewrite the incompatible parts in DuckDB SQL. > > For production workloads, treat transpilation as a migration shortcut: validate results against the source engine, then commit to DuckDB-native SQL where performance and clarity matter. Check the [query API reference](/docs/api-references/lakehouse.md#post-query) for the complete request shape and all supported parameters. --- url: https://altertable.ai/docs/lakehouse/storage/buckets.md --- # Buckets **Buckets** are the unit of storage in Altertable. They store the files behind lakehouse catalogs and file-backed external catalogs. Buckets are object storage locations, so storage scales independently from compute. Workers can come and go while Buckets keep catalog data durable, portable, and ready for future queries. ## Supported providers Altertable can connect to object storage from: - **[Cloudflare R2](https://developers.cloudflare.com/r2/)** - **[Amazon S3](https://aws.amazon.com/s3/)** - **[Google Cloud Storage](https://cloud.google.com/storage)** - **[S3-compatible providers](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)** ## Managed Buckets Managed Buckets run on Altertable infrastructure and are the default for new environments. Today, managed Buckets are Cloudflare R2 buckets available in **Europe (Germany)**. We can provision managed Buckets on **AWS** or **Google Cloud** on demand for teams with cloud, residency, or procurement requirements. ## Bring your own bucket Bring your own bucket is designed for teams that already have object storage in their cloud provider. Connect an existing bucket from AWS, Google Cloud, Cloudflare, or another S3-compatible provider so lakehouse data stays in the account, region, and governance boundary you already operate. ## Connect your bucket You need permission to manage buckets in the target environment and access credentials for an existing storage bucket. Grant the credentials read, list, and write access to that bucket; include the metadata and object operations required by your provider. Keep this separate from a read-only external catalog connection. 1. Open **Buckets** in the target environment and start a new bucket connection. 2. Select the provider and enter the bucket name, access key ID, and secret access key. 3. Set the region and endpoint when required by the provider. For S3-compatible storage, use its S3 endpoint. 4. Test the connection and resolve any reported credential or bucket error before saving. 5. Select the connected bucket when creating a managed [catalog](/docs/lakehouse/catalogs.md). For Google Cloud Storage, use the provider's S3-compatible interoperability credentials. Keep storage credentials in the connection settings rather than application code. ## Verify storage Create a small table in a managed catalog that uses the connected bucket, append a known record, and query it back. Follow the [first ingestion example](/docs/getting-started/ingest-data.md), replacing its catalog with the one backed by your bucket. Confirm that the files appear in the chosen provider and bucket. The connection test checks access to the bucket; a successful write and read verifies the managed table workflow. Do not rename or delete managed table objects directly in the storage provider. ## Troubleshooting - **Bucket test fails:** check the bucket name, endpoint, region, and access key permissions. - **Reads work but ingestion fails:** confirm write permissions and that the catalog uses this bucket. - **Access stopped after rotation:** update the bucket's saved credentials and repeat the read/write check. - **You only want to query existing files:** use [object-storage queries](/docs/lakehouse/query/object-storage.md) or an [external catalog](/docs/lakehouse/external-catalogs.md) instead of creating managed storage. ## Learn more - **[Altertable Catalogs](/docs/lakehouse/catalogs.md)**: Catalogs for storing and querying your data - **[External Catalogs](/docs/lakehouse/external-catalogs.md)**: Connect databases, warehouses, and file-backed catalogs - **[Encryption](/docs/administration/encryption.md)**: How Altertable protects credentials and stored data --- url: https://altertable.ai/docs/lakehouse/storage.md --- # Storage Managed lakehouse tables store their files in [Buckets](/docs/lakehouse/storage/buckets.md). You can use managed storage or connect a bucket in your infrastructure. A storage bucket, a [bucket synchronizer](/docs/lakehouse/ingest/object-storage.md), and an [external catalog](/docs/lakehouse/external-catalogs.md) serve different purposes: storing managed tables, importing files, and querying existing data in place. - [Buckets](/docs/lakehouse/storage/buckets.md): Buckets are the object storage units that keep Altertable lakehouse data durable, portable, and scalable. --- url: https://altertable.ai/docs/opentelemetry/data-model.md --- # OpenTelemetry data model When you enable OpenTelemetry, Altertable creates the catalog tables for you and exposes OTLP/HTTP endpoints that make ingestion easier. Accepted traces, logs, and metrics are written to normal SQL tables: | Table | Description | | ---------------------------- | ------------------------------------------------------------------------------------ | | `opentelemetry.main.spans` | Trace spans with service, timing, status, attributes, events, and links | | `opentelemetry.main.metrics` | Metric points joined with metric names, types, services, and series attributes | | `opentelemetry.main.logs` | Log records with severity, body, trace/span IDs, attributes, and resource attributes | Query `opentelemetry.main.metrics` for metric values with their series attributes. The view joins the underlying `metric_points` and `metric_series` tables. Gauge and sum points expose `value`; histogram and summary points expose fields such as `count` and `sum`. --- url: https://altertable.ai/docs/opentelemetry.md --- # OpenTelemetry Send OpenTelemetry logs, traces, and metrics to the `opentelemetry` catalog, investigate them in the app, and join them with product or business data using SQL. Once enabled, telemetry data lives in the same lakehouse namespace as your other catalogs. You can federate it with product analytics, operational data, and business context, then consume it the same way with SQL, data apps, insights, or agents. That means observability questions do not have to stay isolated from business and product data. You can investigate application behavior alongside revenue, customer activity, feature usage, and any other source already available in Altertable. ## Enable OpenTelemetry Open **Catalogs**, choose **New Catalog**, and enable **OpenTelemetry** for the target environment. After the catalog is enabled, configure your OpenTelemetry SDK or collector to export logs, traces, and metrics to Altertable. ## Data model See the [catalog tables and views](/docs/opentelemetry/data-model.md) for SQL access to the data. ## Setup and analysis - [Configure an exporter](/docs/opentelemetry/setup.md): Authenticate OpenTelemetry exporters and send logs, traces, and telemetry signals into Altertable for SQL analysis. - [Logs](/docs/opentelemetry/logs.md): Send OpenTelemetry logs to Altertable through the OTLP HTTP logs endpoint and analyze application telemetry with SQL. - [Traces](/docs/opentelemetry/traces.md): Send OpenTelemetry traces to Altertable through the OTLP HTTP traces endpoint and analyze spans with SQL workflows. - [Send metrics](/docs/opentelemetry/metrics.md): Export OpenTelemetry metrics to Altertable with an SDK or collector, then verify gauge, sum, and histogram data with SQL. - [Query telemetry](/docs/opentelemetry/query.md): Query OpenTelemetry logs, traces, and metrics with SQL, correlate services and requests, and join telemetry with product or business data. - [OpenTelemetry data model](/docs/opentelemetry/data-model.md): Look up the OpenTelemetry logs, spans, and metrics in Altertable, understand accepted signals, and query telemetry alongside other lakehouse catalogs. --- url: https://altertable.ai/docs/opentelemetry/logs.md --- # Logs Send OpenTelemetry logs to Altertable when you want application log records to be queryable beside product, operational, and business data. Accepted log records are written to `opentelemetry.main.logs`. ## SDK and collector setup Follow [Configure an exporter](/docs/opentelemetry/setup.md) for instrumentation, authentication, and OTLP/HTTP setup. See [`POST /v1/logs`](/docs/api-references/opentelemetry.md#post-v1-logs) for the request body and responses. ## Query logs Logs are normal SQL rows once they are ingested: ```sql SELECT timestamp, severity_text, body, trace_id, span_id FROM opentelemetry.main.logs ORDER BY timestamp DESC LIMIT 100; ``` You can also explore logs in the Altertable Explore tab with Lucene-syntax search. Continue with [Query telemetry](/docs/opentelemetry/query.md) to investigate and correlate signals. --- url: https://altertable.ai/docs/opentelemetry/metrics.md --- # Send metrics Send metrics with an OpenTelemetry SDK or collector, then query them alongside logs, traces, and business data. ## Configure the exporter [Configure authentication](/docs/opentelemetry/setup.md#1-configure-authentication) with credentials from the environment that owns your OpenTelemetry catalog. Set the metrics exporter endpoint to `https://api.altertable.ai/v1/metrics` and use OTLP/HTTP with protobuf or JSON encoding. Use your language's [OpenTelemetry metrics SDK](https://opentelemetry.io/docs/languages/) or an existing collector to export gauge, sum, histogram, exponential histogram, or summary data. Set the `service.name` resource attribute so you can find the service in SQL. The [API reference](/docs/api-references/opentelemetry.md#post-v1-metrics) defines headers, payloads, and response statuses. ## Verify the data Generate activity in the service, allow an export interval to pass, then run: ```sql SELECT timestamp, service_name, metric_name, metric_type, value, count, sum FROM opentelemetry.main.metrics ORDER BY timestamp DESC LIMIT 100; ``` Confirm that the service, metric names, and timestamps match the activity you generated. `value` applies to gauge and sum points; use `count` and `sum` for histogram and summary points. For the underlying storage tables, see the [data model](/docs/opentelemetry/data-model.md). --- url: https://altertable.ai/docs/opentelemetry/query.md --- # Query telemetry Query OpenTelemetry logs, traces, and metrics with SQL-backed tools in Altertable. Use the signal guides for individual records, then correlate signals or join them to business data with SQL. In Altertable, the Logs explorer can group matching records into a segmentation chart over time. The Traces explorer can limit results to root spans, open a trace waterfall, and correlate a trace with its logs. Use SQL when you need to join telemetry to product, customer, or operational catalogs. ## Query a signal - [Logs](/docs/opentelemetry/logs.md#query-logs) contain log bodies, severity, and trace/span identifiers. - [Traces](/docs/opentelemetry/traces.md#query-traces) contain request spans, timings, and service attributes. - [Metrics](/docs/opentelemetry/metrics.md#verify-the-data) contain gauge, sum, histogram, and summary points. ## Correlate logs and traces Join a log to its span using both `trace_id` and `span_id`: ```sql SELECT logs.timestamp, logs.severity_text, logs.body, spans.service_name, spans.name AS span_name FROM opentelemetry.main.logs AS logs JOIN opentelemetry.main.spans AS spans ON logs.trace_id = spans.trace_id AND logs.span_id = spans.span_id WHERE logs.timestamp >= now() - INTERVAL '1 hour' ORDER BY logs.timestamp DESC LIMIT 100; ``` This query returns logs with matching spans. Logs without both identifiers or a matching span are not included. When joining telemetry to business data, also check customer or account identifiers and the time range. ## Investigate with an agent ```text Investigate errors from the last hour. Which services are affected, and what might be causing them? ``` For table schemas and accepted signals, see the [data model](/docs/opentelemetry/data-model.md). For the status of the Grafana integration, see [Grafana availability](/docs/lakehouse/clients/grafana.md). --- url: https://altertable.ai/docs/opentelemetry/setup.md --- # Configure an exporter Configure OpenTelemetry SDKs or collectors to send OTLP/HTTP data to Altertable. Logs, traces, and metrics are written into the built-in `opentelemetry` catalog for the target environment. ## Set up with your coding agent Open this prompt from the repository for the service you want to instrument. ```text Set up OpenTelemetry export to Altertable for this service. Read https://altertable.ai/docs/opentelemetry/setup.md and the official documentation for the installed SDK or collector first. Inspect the runtime, existing instrumentation, exporters, and deployment configuration. Determine which signal I need first: logs, traces, or metrics. Ask only if the signal or target environment is unclear. Reuse existing instrumentation and preserve other telemetry destinations. Configure OTLP over HTTP, the correct signal endpoint and encoding, batching, and service resource attributes. Keep lakehouse credentials in server-side secret configuration; do not print or commit them. Confirm the target Altertable environment. Run the project's checks and send one non-sensitive test signal. Inspect the actual schema in the opentelemetry catalog and query the matching logs, spans, or metrics. Verify the service, timestamp, and test payload rather than relying on an exporter acknowledgement. Report the changes and any deployment configuration I must complete. ``` ## Goal Authenticate an OpenTelemetry exporter, send logs, traces, or metrics to Altertable, and verify the accepted telemetry with SQL. ## Prerequisites - Enable [OpenTelemetry](/docs/opentelemetry.md) for the target environment. - Have access to the target environment’s [lakehouse credentials](/docs/administration/lakehouse-credentials.md). - Configure an OpenTelemetry SDK or collector that supports OTLP over HTTP with protobuf or JSON encoding. ## SDK and collector setup Use the [OpenTelemetry SDK for your application language](https://opentelemetry.io/docs/languages/) or an existing collector with an OTLP/HTTP exporter. Configure instrumentation, batching, and resource attributes in the SDK or collector, then use the credentials and endpoints below. ## Steps ### 1. Configure authentication OpenTelemetry uses HTTP Basic Auth with your environment’s lakehouse credentials. Copy this header into your SDK or collector configuration: ```http Authorization: Basic {{BASE64_TOKEN}} ``` The token is already Base64-encoded. If your exporter has separate header name and value fields, use `Authorization` as the name and `Basic ` as the value. If the exporter provides separate username and password fields for Basic Auth, use the lakehouse credentials from **API keys & credentials** in organization settings. Use credentials from the environment that owns the OpenTelemetry catalog. **Continue in Altertable:** [Open API keys & credentials](/app/settings/api-keys-and-credentials) To build the header yourself, Base64-encode `username:password` and prefix the result with `Basic` followed by a space. The [OpenTelemetry API reference](/docs/api-references/opentelemetry.md#authentication) defines the authentication header and request format. ### 2. Choose a signal - [Send logs](/docs/opentelemetry/logs.md) to `POST https://api.altertable.ai/v1/logs`. - [Send metrics](/docs/opentelemetry/metrics.md) to `POST https://api.altertable.ai/v1/metrics`. - [Send traces](/docs/opentelemetry/traces.md) to `POST https://api.altertable.ai/v1/traces`. ### 3. Export telemetry Configure your SDK or collector with the selected endpoint, the content type matching its encoding (`application/x-protobuf` or `application/json`), and the Basic Auth header. Start the instrumented application or collector to send a batch. ## Verification Open [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) and choose the signal you sent: ### Logs ```sql SELECT * FROM opentelemetry.main.logs ORDER BY timestamp DESC LIMIT 10; ``` ### Traces ```sql SELECT * FROM opentelemetry.main.spans ORDER BY start_time DESC LIMIT 10; ``` ### Metrics ```sql SELECT * FROM opentelemetry.main.metrics ORDER BY timestamp DESC LIMIT 10; ``` Find your service and check that the timestamp and telemetry fields match the test signal you sent. If it is missing, use the troubleshooting checks below. ## Troubleshooting - **The exporter receives an authentication error:** rebuild the Basic Auth header with credentials from the same environment as the OpenTelemetry catalog. - **No rows appear:** confirm that the exporter uses OTLP/HTTP, the correct signal endpoint, and a content type matching its payload. - **Telemetry is in the wrong environment:** update the exporter credentials to those of the intended environment. ## Next steps - [Logs](/docs/opentelemetry/logs.md): Send OpenTelemetry logs to Altertable through the OTLP HTTP logs endpoint and analyze application telemetry with SQL. - [Traces](/docs/opentelemetry/traces.md): Send OpenTelemetry traces to Altertable through the OTLP HTTP traces endpoint and analyze spans with SQL workflows. - [Send metrics](/docs/opentelemetry/metrics.md): Export OpenTelemetry metrics to Altertable with an SDK or collector, then verify gauge, sum, and histogram data with SQL. - [OpenTelemetry API](/docs/api-references/opentelemetry.md): OTLP/HTTP reference for logs, traces, and metrics, including authentication, encodings, compression, and response statuses. --- url: https://altertable.ai/docs/opentelemetry/traces.md --- # Traces Send OpenTelemetry traces to Altertable when you want spans, service timing, status, attributes, events, and links to be queryable beside product, operational, and business data. Accepted spans are written to `opentelemetry.main.spans`. ## SDK and collector setup Follow [Configure an exporter](/docs/opentelemetry/setup.md) for instrumentation, authentication, and OTLP/HTTP setup. See [`POST /v1/traces`](/docs/api-references/opentelemetry.md#post-v1-traces) for the request body and responses. ## Query traces Spans are normal SQL rows once they are ingested: ```sql SELECT trace_id, span_id, service_name, name, start_time, duration_ns / 1e6 AS duration_ms, status_code FROM opentelemetry.main.spans ORDER BY start_time DESC LIMIT 100; ``` You can also explore traces in the Altertable Explore tab with Lucene-syntax search. ## Advanced: trace ID locality > **Time-prefixed trace IDs:** > > Keep your SDK’s default ID generator for initial setup. When tuning large trace datasets, consider time-prefixed IDs (AWS X-Ray style): > > - **Trace ID** (16 bytes): 4-byte unix seconds prefix + 12 random bytes > - **Span ID** (8 bytes): 4-byte unix seconds prefix + 4 random bytes > > That trades a little entropy for time locality. Spans created in the same second share an ID prefix, so co-ingested rows cluster together and lakehouse [sorted tables](/docs/lakehouse/features/sorted-tables.md) get tighter locality and better metadata pruning. > > Register a custom ID generator in your OpenTelemetry SDK when your language supports it. Continue with [Query telemetry](/docs/opentelemetry/query.md) to investigate and correlate signals. --- url: https://altertable.ai/docs/product-analytics/alias.md --- # Alias Users Use `alias` to explicitly link two existing identifiers to the same user profile. Most teams do **not** need `alias()` for normal sign-up or login flows. In the common case, [`identify()`](/docs/product-analytics/identify.md) is enough. ## When to call it Use `alias()` when you are intentionally merging identifiers, such as: - a legacy ID and a new ID format - an external system identifier and your internal user ID - two known profiles that should resolve to one person ## SDK examples When transitioning to a new ID format, alias the new ID to preserve historical data: ### TypeScript ```typescript // after a Stripe payment altertable.alias(`stripe:${stripeCustomerId}`); // After HubSpot sync altertable.alias(`hubspot:${hubspotContactId}`); ``` ### React ```tsx function LinkLegacyIdButton() { const { alias } = useAltertable(); return ; } ``` ### Python ```python # link a CRM identifier client.alias( distinct_id="user_123", new_user_id="stripe:cus_abc123" ) ``` ### Ruby ```ruby # link a CRM identifier Altertable.alias('user_123', 'stripe:cus_abc123') ``` ### Swift ```swift client.identify(userId: "12345") client.alias(newUserId: "user_12345") ``` ### Kotlin ```kotlin Altertable.shared?.identify(userId = "12345") Altertable.shared?.alias(newUserId = "user_12345") ``` ## API reference For HTTP request details, response shape, and error codes, see [`POST /alias`](/docs/api-references/product-analytics.md#post-alias). ## Best practices - Use aliasing sparingly and only for meaningful identity merges. - Pick one canonical `distinct_id` and alias secondary IDs to it. - Add source prefixes to alias IDs, such as `stripe:`, `crm:`, or `legacy:`. - Link aliases directly to the primary user ID, not to each other. --- url: https://altertable.ai/docs/product-analytics/analyze.md --- # Analyze product behavior Product Analytics data is queryable as lakehouse tables and through specialized insight types for common behavioral questions. Use these views when you want to segment users, measure retention, or inspect conversion through a funnel. ## Optimize large insights The insight builder can trade some precision for faster queries on large Product Analytics datasets: - **Sampling** reads a repeatable subset of data. **Default** samples up to 1,000,000 rows; **Aggressive** samples up to 100,000 rows. Keep sampling **Off** when exact results matter. - **Approximate counts** estimates distinct counts instead of calculating every unique value exactly. These settings are saved with the insight. Sampled or approximate results are labeled in charts and tables so viewers can see when the result is not exact. --- url: https://altertable.ai/docs/product-analytics/authentication.md --- # Product Analytics authentication Product Analytics uses public API keys. Use the key for the organization and environment you want to send data to. ## Get an API key 1. Open the **Product Analytics** catalog. 2. Open the **Details** tab. 3. In **Credentials**, copy an existing key, or create one if you manage API keys for the organization. **Continue in Altertable:** [Manage API keys](/app/lakehouse/catalogs/product_analytics?tab=details) ## SDK authentication examples Initialize your SDK once with the API key for the target environment: ### TypeScript ```typescript import { altertable } from '@altertable/altertable-js'; altertable.init('{{API_KEY}}'); ``` ### React ```tsx import { altertable } from '@altertable/altertable-js'; import { AltertableProvider } from '@altertable/altertable-react'; altertable.init('{{API_KEY}}'); export function App() { return ( {/* Your app */} ); } ``` ### Python ```python from altertable import Altertable client = Altertable("{{API_KEY}}") ``` ### Ruby ```ruby require 'altertable' Altertable.init('{{API_KEY}}') ``` ### Swift ```swift import Altertable let client = Altertable(apiKey: "{{API_KEY}}") ``` ### Kotlin ```kotlin import ai.altertable.sdk.Altertable Altertable.setup { apiKey = "{{API_KEY}}" } ``` > **SDK environment selection:** > > The API key selects the target environment. Use separate keys for staging and production, and omit the `environment` parameter from new integrations. In JavaScript and React SDKs v1.4.0 and later, the deprecated `environment` option is ignored. ## Direct API authentication For direct HTTP requests, use the same API key as your SDKs. See [API authentication](/docs/api-references/product-analytics.md#authentication) for supported headers and query parameters. > **Security guidance:** > > - API keys are ingestion keys for product events and user identification. They cannot read historical analytics data. Identity requests can update user traits or merge identities. > - Prefer sending keys in headers. Use the query parameter form only when a header is not practical. ## Next steps - **[Track events](/docs/product-analytics/track.md)**: Start sending product events - **[Identify users](/docs/product-analytics/identify.md)**: Set up user identification --- url: https://altertable.ai/docs/product-analytics/data-model.md --- # Product Analytics data model When you enable Product Analytics, Altertable creates the catalog tables for you and exposes API endpoints that make ingestion easier. As your application sends events and identity updates through the [Product Analytics API](/docs/api-references/product-analytics.md), Altertable writes the resulting data into these tables and views: | Table | Description | | ------------------------------------- | ------------------------------------------------ | | `main.events` | Raw events (everything `/track` receives) | | `main.identities` | Raw identities (everything `/identify` receives) | | `main.identity_distinct_ids` | Raw distinct IDs mapping | | `main.identity_distinct_id_overrides` | Raw distinct ID override mappings | | `analytics.events` | Identity-resolved events | | `analytics.identities` | Identity-resolved user profiles | | `analytics.web_sessions` | Aggregated session data | | `analytics.web_pageviews` | Page-level analytics | --- url: https://altertable.ai/docs/product-analytics/funnel.md --- # Funnel insights Measure how users progress through ordered events, such as **Product Viewed → Added to Cart → Purchase Completed**. Use a funnel to find where users drop off between steps. You need [tracked events](/docs/product-analytics/track.md) for each step and a consistent [user identity](/docs/product-analytics/identify.md) across them. ## Create a funnel 1. Open **Insights** and create an insight. 2. Choose **Funnel** as the insight type. 3. Add at least two event steps in the order you want to measure. 4. Add event-property or user-trait filters where needed, and set the date range. Use **Completed within** to limit how long users have to complete the funnel. 5. Inspect the chart, open **Results** for the underlying rows, and use **SQL** to review the generated query. 6. Save the insight when the steps, filters, and results match the journey you want to measure. ![Funnel insight showing step-by-step conversion in the chart preview](/screenshots/charts-funnelchart--single-series.png) *Funnel insight showing step-by-step conversion in the chart preview* ## Verify the definition Check that each event represents the intended action, that the same user identifier is available across steps, and that the date range and conversion window match your question. Review the number of users at each step before interpreting the conversion rate. For return behavior after an initial event, use [Retention](/docs/product-analytics/retention.md). For an event metric over time, use [Segmentation](/docs/product-analytics/segmentation.md). To investigate a result directly, [query product data with SQL](/docs/product-analytics/query.md) using the [Product Analytics data model](/docs/product-analytics/data-model.md). --- url: https://altertable.ai/docs/product-analytics/identify.md --- # Identify Users Use `identify` when you know who a user is. Identification connects anonymous or device-level activity to a stable user ID and stores traits such as plan, role, or account tier. In practice, `identify()` is the moment when analytics move from "someone did this" to "this user did this". ## When to call it Call `identify()` as soon as your application establishes or restores authenticated user state: - after sign-up - after login - on application load when the user is already authenticated Client-side SDKs can link the anonymous activity that happened before login to the stable user ID. Server-side SDKs and direct API clients should pass the `distinct_id` and traits explicitly. ## SDK examples Update user or account traits when they change. Keep traits focused on fields you expect to analyze, filter, or join against: ### TypeScript ```typescript altertable.updateTraits({ plan: 'enterprise', onboarding_completed: true }); ``` ### React ```tsx function UpgradeButton() { const { updateTraits } = useAltertable(); return ( ); } ``` ### Python ```python # No updateTraits() helper; identify the user again with updated traits. client.identify(distinct_id="user_123", options={"traits": {"plan": "enterprise"}}) ``` ### Ruby ```ruby # No updateTraits helper; identify the user again with updated traits. Altertable.identify('user_123', traits: { plan: 'enterprise' }) ``` ### Swift ```swift client.updateTraits(["plan": "enterprise"]) ``` ### Kotlin ```kotlin Altertable.shared?.updateTraits(mapOf("plan" to "enterprise")) ``` On client-side SDKs, call `reset()` on logout so later activity is not attributed to the previous user: ### TypeScript ```typescript // Reset identity, keep device ID altertable.reset(); // Reset all IDs including device ID altertable.reset({ resetDeviceId: true }); ``` ### React ```tsx function LogoutButton() { const { reset } = useAltertable(); return ; } ``` ### Python ```python # The Python SDK has no reset() method; it does not retain a client-side identity. # Pass the next user's distinct_id explicitly on subsequent calls. ``` ### Ruby ```ruby # The Ruby SDK has no reset method; it does not retain a client-side identity. # Pass the next user's ID explicitly on subsequent calls. ``` ### Swift ```swift client.reset() ``` ### Kotlin ```kotlin // Reset identity, keep device ID Altertable.shared?.reset() // Reset all IDs including device ID Altertable.shared?.reset(resetDeviceId = true) ``` ## API reference For HTTP request details, response shape, and error codes, see [`POST /identify`](/docs/api-references/product-analytics.md#post-identify). ## Best practices - Use stable, immutable IDs from your own user system. - Avoid mutable identifiers such as email addresses as `distinct_id`. - Keep traits current as account state changes. - Avoid secrets and regulated sensitive data in traits. --- url: https://altertable.ai/docs/product-analytics.md --- # Product Analytics Track product events and user identities, then analyze segmentation, conversion, and retention in the built-in `product_analytics` catalog. Once enabled, product behavior data lives in the same lakehouse namespace as your other catalogs. You can federate it with operational, billing, support, and warehouse data, then consume it the same way with SQL, data apps, insights, or agents. That means product questions do not live in a separate analytics silo. You can connect feature usage to revenue, support activity, billing state, and any other source already available in Altertable. ## Enable Product Analytics Open **Catalogs**, choose **New Catalog**, and enable **Product Analytics** for the target environment. After the catalog is enabled, authenticate an SDK or direct API client with a Product Analytics API key, identify users when they are known, and track the product events you want to analyze. ## Analyze product behavior Open **Insights** to turn tracked events into analysis: Segmentation compares metrics and breakdowns over time. Funnels measure conversion through sequential events. Retention shows whether cohorts return after a starting event. These builders are available from the insight type control; the Product Analytics section of the navigation is the place to explore tracked events, identities, and segments. ## Data model See the [catalog tables and views](/docs/product-analytics/data-model.md) for SQL access to the data. ## Setup and analysis - [Set up tracking](/docs/product-analytics/setup.md): Authenticate clients, track events, identify users, and alias identities into the built-in `product_analytics` catalog. - [Product Analytics authentication](/docs/product-analytics/authentication.md): Authenticate Product Analytics SDKs and API requests with public API keys for the target staging or production environment. - [Track](/docs/product-analytics/track.md): Track product events from SDKs or API clients and make behavioral data queryable in your Altertable lakehouse. - [Identify](/docs/product-analytics/identify.md): Identify users in Altertable Product Analytics so events connect to stable profiles, traits, and business context. - [Alias](/docs/product-analytics/alias.md): Alias product analytics identifiers in Altertable to merge user profiles when identity systems or user IDs change. - [Analyze product behavior](/docs/product-analytics/analyze.md): Analyze product behavior in Altertable with funnels, retention, segmentation, dashboards, and SQL over lakehouse data. - [Segmentation](/docs/product-analytics/segmentation.md): Plot product event metrics over time, compare breakdowns by event properties or user traits, and verify the results in a segmentation insight. - [Funnel](/docs/product-analytics/funnel.md): Measure conversion through ordered event steps, inspect where users drop off, and verify a funnel insight against its SQL results. - [Retention](/docs/product-analytics/retention.md): Measure how cohorts that perform a starting event come back and perform a returning event over time. - [Query product data with SQL](/docs/product-analytics/query.md): Query identity-resolved product events with DuckDB SQL and join product behavior to connected business data using the Product Analytics catalog. - [Product Analytics data model](/docs/product-analytics/data-model.md): Look up the raw and identity-resolved Product Analytics tables for events, users, web sessions, and pageviews in the built-in lakehouse catalog. --- url: https://altertable.ai/docs/product-analytics/query.md --- # Query product data with SQL Use SQL when you need a custom analysis or want to join product behavior with another catalog. Use the [identity-resolved views](/docs/product-analytics/data-model.md) for analysis that should apply identity mappings. ## Inspect available data Before writing a query, inspect the table schema in [SQL Explorer](/docs/lakehouse/query/sql-explorer.md) or ask your agent to describe it. Start with a bounded query: ```sql SELECT * FROM product_analytics.analytics.events LIMIT 10; ``` Use `product_analytics.main.events` when you need the raw records received by the tracking endpoint. Choose the view based on whether you want raw or identity-resolved data. ## Analyze with an agent ```text Analyze signup conversion over the last month. Where do users drop off? ``` For visual analyses, use [segmentation](/docs/product-analytics/segmentation.md), [funnels](/docs/product-analytics/funnel.md), or [retention](/docs/product-analytics/retention.md). For an interactive application, [build a data app](/docs/data-apps/build.md). --- url: https://altertable.ai/docs/product-analytics/retention.md --- # Retention Insights **Retention** insights plot classic cohort retention: users or other entities who did a **starting event** in a period, then whether they performed a **returning event** again in later time buckets. Use them to see whether new signups, activated users, or any cohort keep engaging over days or weeks. **Best for:** - Onboarding and activation ("did they come back after signup?") - Habit and repeat usage ("after first purchase, when do they return?") - Comparing cohorts side by side with **Segmented by** filters **Example questions:** - "What share of signups used the product again on day 1, 7, and 30?" - "After a user's first key action, how does return behavior differ by plan or channel?" ## Creating Retention Insights 1. Open **Insights** and create or edit a chart. 2. In the insight type control, choose **Retention**. The control is labeled "Retention insight" when selected. 3. Configure **Starting event**: the action that puts users into the cohort, for example sign up or first purchase. 4. Configure **Returning event**: the action that counts as "retained", often the same event as the starting event or a deeper engagement event. 5. Set **Measured as**: - **Return on or after**: a user counts as retained in a bucket if they performed the returning event on that day or any later bucket in the window. - **Return on**: retained only if they performed the returning event in that specific bucket. 6. Set **Calculated by**: - **24-hour windows**: periods are aligned in 24-hour windows from the cohort start. - **Strict calendar day**: periods follow calendar days, which can differ from rolling 24-hour logic. 7. Optionally add **Segmented by** segments to compare multiple cohort curves on the same chart. 8. Use **Query variables** when your chart defines them, such as date range or interval, so the preview matches the period you care about. The preview shows a cohort retention visualization. Use the **Results** tab for the table and **SQL** to inspect the generated queries. ![Retention insight preview: cohort retention curve over day offsets](/screenshots/charts-retentionchart--single-cohort.png) *Retention insight preview: cohort retention curve over day offsets* ## How this differs from segmentation [Segmentation insights](/docs/product-analytics/segmentation.md) compare metrics and breakdowns over time. **Retention** insights are optimized for **cohort × offset** curves from exactly two events plus retention-specific measurement rules. Use segmentation when you are comparing groups on a single metric; use retention when the question is explicitly "of people who did X, what percentage did Y again after N days?" ## Practical notes - You must select **both** starting and returning events before the chart can run; the UI prompts you until both are set. - Retention uses the same event-step builder as funnels, so you can filter each side of the definition by properties, traits, and segments. - Numbers may not match other analytics products exactly, because **Measured as**, **Calculated by**, timezone, and event definitions all affect cohort math. Trends and relative comparisons should still be reliable when definitions are held constant. ## Related - [Funnel insights](/docs/product-analytics/funnel.md): sequential steps and conversion, not cohort return over offsets - [Segmentation insights](/docs/product-analytics/segmentation.md): time series and breakdowns - [Product analytics: track](/docs/product-analytics/track.md): instrumenting the events retention depends on --- url: https://altertable.ai/docs/product-analytics/segmentation.md --- # Segmentation insights Plot an event metric over time and compare it by property or user attribute. For example, count purchases by region or compare feature usage across subscription plans. ## Create a segmentation insight 1. Open **Insights** and create an insight. 2. Choose **Segmentation** as the insight type. 3. Select an event and aggregation, such as event count, unique users, sum, or average. 4. Add filters and an optional breakdown by an event property or user trait. 5. Set the time range and interval. Open **Results** to inspect the rows and **SQL** to review the generated query. 6. Save the insight when the metric and breakdown match your question. ![Segmentation insight line chart comparing a metric over time](/screenshots/charts-segmentationchart-linechart--single-series.png) *Segmentation insight line chart comparing a metric over time* ## Choose a breakdown - Use an **event property**, such as `page_url` or `endpoint`, to compare the context of each action. - Use a **user trait**, such as plan or region, to compare groups of identified users. [Identify users](/docs/product-analytics/identify.md) to attach these traits. Check missing property values and the selected aggregation before comparing groups. Use [SQL](/docs/product-analytics/query.md) and the [data model](/docs/product-analytics/data-model.md) when you need to inspect the source records. For conversion through ordered steps, use [Funnel](/docs/product-analytics/funnel.md). For the percentage of a cohort that returns after a starting event, use [Retention](/docs/product-analytics/retention.md). --- url: https://altertable.ai/docs/product-analytics/setup.md --- # Set up tracking Send product events and identity updates to Altertable with SDKs or direct HTTP requests. The same payload model writes to the built-in `product_analytics` catalog, so client, server, mobile, and batch sources can be analyzed together. ## Set up with your coding agent Open this prompt in your coding agent from your application repository. It will inspect your stack and guide the implementation. ```text Set up Altertable Product Analytics in this repository. Read https://altertable.ai/docs/product-analytics/setup.md and the linked official SDK documentation first. Inspect my framework, runtime, package manager, authentication, and existing analytics. Choose the matching SDK. Ask which user journey I want to measure and propose a small event plan before changing code. Guide me through enabling Product Analytics and configuring its environment-specific API key without committing secrets. MCP and lakehouse credentials do not replace this key. Reuse or initialize one shared client. Track completed actions, identify signed-in users, reset identity at sign-out, and respect consent. Check which helpers the SDK supports and avoid sensitive properties. Run checks. Send one named verification event with a unique test identity in the chosen environment. Query product_analytics.main.events by that event name and identity, and confirm the properties and timestamp. Do not use the newest row as proof. Use my Altertable connection if available. Summarize changed files, verification results, and remaining setup. ``` Prefer to configure it yourself? Follow the steps below. ## Goal Choose an SDK, connect it with a Product Analytics API key, and send events and identity updates to the built-in `product_analytics` catalog. ## Prerequisites - Enable [Product Analytics](/docs/product-analytics.md) for the target environment. - Copy a Product Analytics [API key](/docs/product-analytics/authentication.md). - Choose a client-side or server-side SDK based on where your application sends events. ## Steps ### 1. Install an SDK Choose the SDK for your application language: #### TypeScript / JavaScript [TypeScript / JavaScript SDK repository](https://github.com/altertable-ai/altertable-js/) npm ```bash npm install @altertable/altertable-js ``` pnpm ```bash pnpm add @altertable/altertable-js ``` Yarn ```bash yarn add @altertable/altertable-js ``` Bun ```bash bun add @altertable/altertable-js ``` #### React [React SDK repository](https://github.com/altertable-ai/altertable-js/) npm ```bash npm install @altertable/altertable-js @altertable/altertable-react ``` pnpm ```bash pnpm add @altertable/altertable-js @altertable/altertable-react ``` Yarn ```bash yarn add @altertable/altertable-js @altertable/altertable-react ``` Bun ```bash bun add @altertable/altertable-js @altertable/altertable-react ``` #### Python [Python SDK repository](https://github.com/altertable-ai/altertable-python) pip ```bash pip install altertable ``` Poetry ```bash poetry add altertable ``` #### Ruby [Ruby SDK repository](https://github.com/altertable-ai/altertable-ruby) gem ```bash gem install altertable ``` #### Swift [Swift SDK repository](https://github.com/altertable-ai/altertable-swift) Swift Package Manager ```swift .package(url: "https://github.com/altertable-ai/altertable-swift.git", from: "0.1.0") ``` #### Kotlin / Android [Kotlin / Android SDK repository](https://github.com/altertable-ai/altertable-kotlin) Gradle (JVM) ```kotlin implementation("ai.altertable.sdk:altertable-kotlin:0.1.0") ``` Gradle (Android) ```kotlin implementation("ai.altertable.sdk:altertable-android:0.1.0") ``` For Swift, add the package declaration to `Package.swift` dependencies and link the `Altertable` product to your target. For Kotlin, add the JVM dependency to `build.gradle.kts`; Android projects use the Android dependency. Use Maven Central in your Gradle repositories. See the [SDK repositories](/docs/api-references/sdks.md#product-analytics-sdks) for platform requirements. ### 2. Initialize the client and send data Use these examples as the starting point for your application code: ### TypeScript ```typescript import { altertable } from '@altertable/altertable-js'; // Initialize with your API key altertable.init('{{API_KEY}}'); // Track a user event altertable.track('checkout_completed', { revenue: 49.99, plan: 'pro', currency: 'USD', }); // Identify a user altertable.identify('user_abc123', { email: 'alice@acme.com', plan: 'pro', }); ``` ### React ```tsx import { altertable } from '@altertable/altertable-js'; import { AltertableProvider, useAltertable, } from '@altertable/altertable-react'; // Initialize the core SDK altertable.init('{{API_KEY}}'); function App() { return ( ); } function CheckoutPage() { const { track, identify } = useAltertable(); function handleCheckout() { track('checkout_completed', { revenue: 49.99, plan: 'pro', currency: 'USD', }); } function handleLogin(userId) { identify(userId, { email: 'alice@acme.com', plan: 'pro', }); } return (
); } ``` ### Python ```python from altertable import Altertable client = Altertable("{{API_KEY}}") # Track a user event client.track( event="checkout_completed", distinct_id="user_abc123", options={ "properties": { "revenue": 49.99, "plan": "pro", "currency": "USD", } } ) # Identify a user client.identify( distinct_id="user_abc123", options={ "traits": { "email": "alice@acme.com", "plan": "pro", } } ) ``` ### Ruby ```ruby require 'altertable' Altertable.init('{{API_KEY}}') # Track a user event Altertable.track('checkout_completed', 'user_abc123', properties: { revenue: 49.99, plan: 'pro', currency: 'USD' }) # Identify a user Altertable.identify('user_abc123', traits: { email: 'alice@acme.com', plan: 'pro' }) ``` ### Swift ```swift import Altertable let client = Altertable(apiKey: "{{API_KEY}}") // Track a user event client.track( event: "checkout_completed", properties: [ "revenue": 49.99, "plan": "pro", "currency": "USD" ] ) // Identify a user client.identify( userId: "user_abc123", traits: [ "email": "alice@acme.com", "plan": "pro" ] ) ``` ### Kotlin ```kotlin import ai.altertable.sdk.Altertable // Initialize with your API key Altertable.setup { apiKey = "{{API_KEY}}" } // Track a user event Altertable.shared?.track( event = "checkout_completed", properties = mapOf( "revenue" to 49.99, "plan" to "pro", "currency" to "USD" ) ) // Identify a user Altertable.shared?.identify( userId = "user_abc123", traits = mapOf( "email" to "alice@acme.com", "plan" to "pro" ) ) ``` After initialization: 1. Track events with event properties. 2. Identify users and attach traits. 3. Optionally alias related identities. ### 3. Choose client-side or server-side behavior Use a client-side SDK for events from a browser or mobile application. Use a server-side SDK for events from backend services or jobs, and pass a stable `distinct_id` to associate each event with its user. Automatic page capture, session management, consent helpers, and offline queuing vary by SDK. Check the repository for your SDK for the features available in your runtime. ### 4. Configure page, screen, and consent helpers If your SDK provides page or screen helpers, configure them for the views you want to track: ### TypeScript ```typescript // Disable auto-capture if you need manual control altertable.init('{{API_KEY}}', { autoCapture: false }); // Then track page views manually altertable.page('https://example.com/products'); ``` ### React ```tsx import { altertable } from '@altertable/altertable-js'; import { useAltertable } from '@altertable/altertable-react'; altertable.init('{{API_KEY}}', { autoCapture: false }); function ProductPage() { const { page } = useAltertable(); return ( ); } ``` ### Python ```python # Automatic page and screen tracking is not supported in the Python SDK. # Track meaningful page or screen views as regular events with client.track(). ``` ### Ruby ```ruby # Automatic page and screen tracking is not supported in the Ruby SDK. # Track meaningful page or screen views as regular events with Altertable.track. ``` ### Swift ```swift Text("Welcome") .screenView(name: "Home") ``` ### Kotlin ```kotlin Column(modifier = Modifier.screenView(name = "Home")) { // Screen content } ``` Where supported, use consent helpers to control whether the SDK sends data: ### TypeScript ```typescript // JavaScript — initialize with consent pending altertable.init('{{API_KEY}}', { trackingConsent: 'pending' }); // Later, when the user grants consent altertable.configure({ trackingConsent: 'granted' }); ``` ### React ```tsx import { altertable } from '@altertable/altertable-js'; import { useAltertable } from '@altertable/altertable-react'; altertable.init('{{API_KEY}}', { trackingConsent: 'pending' }); function ConsentButton() { const { configure } = useAltertable(); return ( ); } ``` ### Python ```python # The Python SDK has no built-in tracking consent state. # Check consent in your application before calling client.track() or client.identify(). ``` ### Ruby ```ruby # The Ruby SDK has no built-in tracking consent state. # Check consent in your application before calling Altertable.track or Altertable.identify. ``` ### Swift ```swift // Swift — initialize with consent pending let config = AltertableConfig(trackingConsent: .pending) let client = Altertable(apiKey: "{{API_KEY}}", config: config) // Later, when the user grants consent client.configure { config in config.trackingConsent = .granted } ``` ### Kotlin ```kotlin // Kotlin — initialize with consent pending Altertable.setup { apiKey = "{{API_KEY}}" tracking { consent = TrackingConsent.PENDING } } // Later, when the user grants consent Altertable.shared?.configure { tracking { consent = TrackingConsent.GRANTED } } ``` ## Verification Send a verification event named `docs_verification` with a unique test identity. Replace `docs_test_` below with the identity you sent. Use a new identity for each run: ```sql SELECT * FROM product_analytics.main.events WHERE event = 'docs_verification' AND distinct_id = 'docs_test_' ORDER BY timestamp DESC LIMIT 10; ``` Confirm that the event name, distinct ID, timestamp, and properties match the payload your application sent. A different recent event does not prove this test succeeded. Allow for ingestion delay and retry the same filtered query before sending another event. ## Troubleshooting - **No events appear:** confirm that Product Analytics is enabled in the same environment as the API key. - **Requests are rejected:** check the [authentication methods](/docs/product-analytics/authentication.md) and replace an incorrect or expired API key. - **Users remain anonymous:** call `identify` when a stable user ID becomes available, and pass `distinct_id` explicitly from server-side SDKs. - **Automatic page or screen events are missing:** confirm that your SDK supports page or screen capture, then enable its helper. **Continue in Altertable:** [Set up ingestion](/app/lakehouse/catalogs/product_analytics?tab=ingest) ## Next steps - [Product Analytics authentication](/docs/product-analytics/authentication.md): Authenticate Product Analytics SDKs and API requests with public API keys for the target staging or production environment. - [Track](/docs/product-analytics/track.md): Track product events from SDKs or API clients and make behavioral data queryable in your Altertable lakehouse. - [Identify](/docs/product-analytics/identify.md): Identify users in Altertable Product Analytics so events connect to stable profiles, traits, and business context. - [Alias](/docs/product-analytics/alias.md): Alias product analytics identifiers in Altertable to merge user profiles when identity systems or user IDs change. --- url: https://altertable.ai/docs/product-analytics/track.md --- # Track Events Use `track` to record product actions such as signups, purchases, feature usage, page views, and activation milestones. Track a small set of meaningful events first. A few stable event names with useful properties are easier to query than a large stream of low-signal UI clicks. ## When to call it Use `track` when something happened in your product and you want it available for segmentation, retention, funnel analysis, SQL, or AI agents. Good first events often describe: - activation moments - conversion steps - recurring feature usage - billing or account lifecycle changes ## SDK examples Use `track()` where the product action happens: ### TypeScript ```typescript altertable.track('checkout_completed', { revenue: 49.99, plan: 'pro', currency: 'USD', }); ``` ### React ```tsx function CheckoutButton() { const { track } = useAltertable(); function handleCheckout() { track('checkout_completed', { revenue: 49.99, plan: 'pro', currency: 'USD', }); } return ; } ``` ### Python ```python client.track( event="checkout_completed", distinct_id="user_abc123", options={ "properties": { "revenue": 49.99, "plan": "pro", "currency": "USD", } }, ) ``` ### Ruby ```ruby Altertable.track('checkout_completed', 'user_abc123', properties: { revenue: 49.99, plan: 'pro', currency: 'USD' }) ``` ### Swift ```swift client.track( event: "checkout_completed", properties: [ "revenue": 49.99, "plan": "pro", "currency": "USD" ] ) ``` ### Kotlin ```kotlin Altertable.shared?.track( event = "checkout_completed", properties = mapOf( "revenue" to 49.99, "plan" to "pro", "currency" to "USD" ) ) ``` `distinct_id` ties an event to a person, device, or session. Client-side SDKs can manage anonymous IDs automatically. Server-side SDKs and direct API clients should pass a stable ID on each call. Call [`identify()`](/docs/product-analytics/identify.md) when you know who the user is, so anonymous and authenticated activity can be analyzed together. ## Batch events Batching reduces request overhead when you track several events. JavaScript, React, Swift, and Kotlin batch queued calls automatically. Ruby accepts an array in one `track` call. Python sends one request per call; for an explicit batch, send an array to [`POST /track`](/docs/api-references/product-analytics.md#post-track). ### TypeScript ```typescript altertable.track('signup', { plan: 'pro' }); altertable.track('login'); ``` ### React ```tsx function SignupButton() { const { track } = useAltertable(); function handleSignup() { track('signup', { plan: 'pro' }); track('welcome_seen'); } return ; } ``` ### Python ```python # The Python SDK has no bulk track method. client.track("signup", "user_1", options={"properties": {"plan": "pro"}}) client.track("login", "user_2") ``` ### Ruby ```ruby Altertable.track([ { event: 'signup', distinct_id: 'user_1', properties: { plan: 'pro' } }, { event: 'login', distinct_id: 'user_2' } ]) ``` ### Swift ```swift client.track(event: "signup", properties: ["plan": "pro"]) client.track(event: "login") ``` ### Kotlin ```kotlin Altertable.shared?.track(event = "signup", properties = mapOf("plan" to "pro")) Altertable.shared?.track(event = "login") ``` ## Transform events Where supported, add application context before events are sent. These examples label the product area from the page URL or screen name and record the surface that produced the event: ### TypeScript ```typescript import { altertable } from '@altertable/altertable-js'; altertable.init('{{API_KEY}}', { transformEvent(event) { const url = event.properties.$url; let pathname = globalThis.location?.pathname ?? '/'; if (typeof url === 'string') { try { pathname = new URL(url, globalThis.location?.origin).pathname; } catch { pathname = globalThis.location?.pathname ?? '/'; } } const title = globalThis.document?.title; const properties = { ...event.properties, surface: 'website' }; if (pathname === '/shop' || pathname.startsWith('/shop/')) { properties.product = 'shop'; } if (title) { properties.title = title; } return { ...event, properties }; }, }); ``` ### React ```tsx import { altertable } from '@altertable/altertable-js'; import { AltertableProvider } from '@altertable/altertable-react'; altertable.init('{{API_KEY}}', { transformEvent(event) { const url = event.properties.$url; let pathname = globalThis.location?.pathname ?? '/'; if (typeof url === 'string') { try { pathname = new URL(url, globalThis.location?.origin).pathname; } catch { pathname = globalThis.location?.pathname ?? '/'; } } const title = globalThis.document?.title; const properties = { ...event.properties, surface: 'website' }; if (pathname === '/shop' || pathname.startsWith('/shop/')) { properties.product = 'shop'; } if (title) { properties.title = title; } return { ...event, properties }; }, }); function App() { return ( ); } ``` ### Python ```python # The Python SDK does not support transforming events before delivery. # Set or filter properties in your application before calling client.track(). ``` ### Ruby ```ruby # The Ruby SDK does not support transforming events before delivery. # Set or filter properties in your application before calling Altertable.track. ``` ### Swift ```swift // The Swift SDK does not support transforming events before delivery. // Set or filter properties in your app before calling client.track(). ``` ### Kotlin ```kotlin import ai.altertable.sdk.Altertable import ai.altertable.sdk.Event Altertable.setup { apiKey = "{{API_KEY}}" beforeSend = listOf { event -> if (event is Event.Track) { val screen = event.properties["\$screen_name"] as? String val product = if (screen?.startsWith("Shop") == true) mapOf("product" to "shop") else emptyMap() event.copy(properties = event.properties + product + ("surface" to "app")) } else { event } } } ``` In JavaScript and React, `transformEvent` runs on track and page events after the SDK adds context and identifiers. It does not run on identify, alias, or events forwarded from [PostHog](/docs/lakehouse/ingest/integrations/posthog.md). Configure the shared JavaScript client before rendering `` in React. ## API reference For HTTP request details, batching behavior, response shape, and error codes, see [`POST /track`](/docs/api-references/product-analytics.md#post-track). ## Best practices - Use clear event names and keep them stable over time. - Track meaningful product actions, not every low-level interaction. - Include only properties you plan to filter, group, or query. - Avoid secrets and regulated sensitive data in event properties.