Build an Atlas connector
Extend the @futurity/atlas-connector SDK, scaffold with bun create atlas-connector, prove it with atlas-conform, and connect it as a source.
A connector is a small HTTP service you host that answers Atlas's queries over
data Atlas cannot read first-hand: a business system that only speaks its own
API, a database behind a VPN, a warehouse with no first-party driver. Atlas
discovers its tables, measures its keys, and plans queries over it next to every
other source in the same Atlas. The
@futurity/atlas-connector
SDK owns the wire protocol, the server, bearer auth, timeouts, NDJSON streaming,
and the error envelope. You write the part only you can: describing your data
and fetching it.
One connector, many tenants
A connector is written once and hosted once, by whoever knows the upstream system. Every client of that integrator connects the same hosted connector into their own Atlas, typing their own upstream credentials into the connect form. Those credentials travel on every request Atlas sends, and the connector holds nothing between calls. The host never stores a tenant's secret, an added tenant is a form someone fills in, and a leaked connector process has no customer secret to leak.
integrator hosts one connector
https://connector.example.com
^
credentials A on | credentials B on
every request | every request
|
+------------------------+ | +------------------------+
| Northwind Trading |-------+-------| Harbor Logistics |
| their Atlas, their | | their Atlas, their |
| source, their secrets | | source, their secrets |
+------------------------+ +------------------------+
| |
v v
accounts, contacts, deals vessels, ports, voyages, incidentsTwo things make this work. The connector's public document declares a
credentialSchema, the exact inputs Atlas asks the person connecting to fill
in. Then every authenticated request body carries a credentials object with
those values, and that object is the only place your code reads them from.
Never read upstream credentials from the environment
The only secret in a connector's environment is the bearer token Atlas
presents. Database URLs, API keys, and app secrets belong to the tenant and
arrive as req.credentials on each call. A connector that reads them from
process.env serves exactly one tenant and cannot be connected twice.
Step zero: credentialSchema and check()
The first thing an author writes is the answer to two questions: what does a tenant have to type to reach their own instance, and what is the cheapest upstream call that proves they typed it right.
The Lark Base connector in the SDK's examples/lark asks for an app id, an app
secret, and the token of the base to read. type: "password" masks the input
in the connect form, type: "textarea" gives it the multi-line box a pasted key
needs, and required: false marks a field a tenant may leave blank:
// per tenant: the app credentials and the base they open
credentialSchema: [
{
key: "appId",
label: "App ID",
type: "text",
required: true,
placeholder: "cli_XXXXXXXXXXXXXXXX",
help: "Lark Developer Console → your app → **Credentials & Basic Info**, the field labelled **App ID**. Your apps are listed at [open.larksuite.com/app](https://open.larksuite.com/app).",
},
{
key: "appSecret",
label: "App secret",
type: "password",
required: true,
help: "The **App Secret** on that same **Credentials & Basic Info** page of the [Lark Developer Console](https://open.larksuite.com/app). The app also needs the `bitable:app:readonly` permission.",
},
{
key: "appToken",
label: "Base app token",
type: "text",
required: true,
placeholder: "bascnXXXXXXXXXXXXXXXXXXXXXX",
help: "The id in the base's URL, `https://<tenant>.larksuite.com/base/<app_token>`. Add the app to that base as a collaborator first, or it cannot read the tables.",
},
],help is short markdown, and it is the part a tenant actually reads: name the
exact page in the vendor's console the value is copied from and link the
vendor's own doc for it, rather than restating the label. The connect form
renders it above the input, with placeholder shown inside the empty field.
A field left out of required altogether is required: the flag only ever
loosens. An optional field a tenant leaves blank arrives with its key absent
from credentials, so read it as missing rather than as an empty string.
check(req) receives those values on req.credentials, plus the host's own
req.timeoutMs, and throws if they are wrong. Its message is the one place a
connector speaks to the tenant directly: Atlas shows it verbatim next to
Test connection, so write it for the person who typed the credentials.
async check(req: CheckRequest): Promise<void> {
await clientFor(req.credentials).checkAccess(makeDeadline(req.timeoutMs));
}Lark's checkAccess mints a tenant token (which proves the app id and secret)
and reads the base's table list with page_size: "1" (which proves the app
token), so either half being wrong fails here with Lark's own message.
For a SQL source SqlConnector supplies both halves: the default
credentialSchema is one masked databaseUrl input, and check runs
SELECT 1 on the pool opened from it.
The class, by plane
You extend AtlasConnector. Its methods fall into four planes, and the class
file is laid out in that order. Every method receives the parsed wire request,
including credentials and timeoutMs; serve() has already checked the
bearer, validated the body, and started the deadline.
Identity
| member | wire | Atlas uses it for | cost | default | override when |
|---|---|---|---|---|---|
slug | in the document | naming the connector; must match the document's slug | none | abstract | always: you declare it |
capability() | GET /.well-known/futurity/atlas.json | which pushdowns it may use and which credentials to ask for; read on connect and refresh | none | abstract (derived for SQL) | always on the REST path |
check(req) | POST /check | Test connection, and every conformance run | one round trip | abstract (SELECT 1 SQL) | always on the REST path |
The document is fetched unauthenticated and read on every connect and rediscovery, so build it from constants.
Query
| method | wire | Atlas uses it for | cost | default | override when |
|---|---|---|---|---|---|
query(req) | POST /query | every row Atlas reads: previews, exports, local joins, host-side counts | one scan under the request's filter | abstract | always |
count(req) | POST /count | pagination and preview sizing | one counting call | absent | the source counts under a filter |
aggregate(req) | POST /aggregate | group-by pushdown Atlas tries before folding rows itself | one grouped call | absent | the source groups server-side |
query is the only route that moves rows, and it is always a stream. It returns
AsyncIterable<SourceRow[]>; each batch frames as an NDJSON line, with the
heartbeats and the terminator around it. Yield batches of at most 5,000 rows; a
larger batch is re-chunked on the wire.
count and aggregate are mounted only when you override them, and only what
the document's endpoints lists is ever posted. Anything you leave out, Atlas
answers itself over query inside its own reach, so a connector that pushes
nothing is still correct, only slower.
Two laws hold on this plane.
The filter law. Every row you answer must satisfy every filter you declared
an operator for. A filter naming a field that does not exist is refused with a
422, never answered with unfiltered rows. A filter you simply do not push is
different: answer the superset and Atlas judges the rest itself, because
operators is what it plans against. Which of the two you did is what the
stream's first line says, served.filters true only when every predicate held.
The kit's
assertKnownFields(req, fieldNames) throws the 422 for an unknown field; call
it in query and count before any fetch. SqlConnector refuses an unknown
column on its own.
The aggregate rule. A wrong number is never legal. aggregate returns rows
or undefined, and undefined becomes a 204 that tells Atlas to fold the rows
itself. Add "aggregate" to the document's endpoints only when you override
the method; serve() warns at boot when the two disagree.
Discovery
| method | wire | Atlas uses it for | cost | default | override when |
|---|---|---|---|---|---|
discover(req) | POST /discovery | the tables, fields, keys, and foreign keys a source is built from; called at setup and on rediscovery | whatever metadata costs; may be slow | abstract | always: only you know the shape upstream |
The answer is { tables, warnings? }. Each table names its primaryKey, its
foreignKeys, and its fields with an Atlas type, nullable, unique, a few
samples, and a sourceDescription. Descriptions travel into Atlas's
understanding of the source, a declared key that is not really unique poisons
joins, and a foreign key you forget is a join Atlas never plans, so this is the
method that deserves the most care.
Size and measurement
Atlas needs one number before it plans anything: how big an entity is. It asks
size once, in one request, and compares the answer against what it reaches in
a single query. Past that it refuses, naming the rows and the routes, rather
than half-answering.
The other two measurements are lazy. Atlas takes them when a question needs a
fact your discovery did not declare, and it takes them itself over query when
you do not serve the route.
| method | wire | Atlas uses it for | override when |
|---|---|---|---|
size(req) | POST /size | the reach check before any pull, and the row count a tenant sees | the source knows its size; exact: false when it only estimates |
cardinality(req) | POST /cardinality | non-null and distinct counts per column; key promotion picks join keys from them | the source has COUNT DISTINCT; answer null per column it cannot count |
linkHitRate(req) | POST /linkHitRate | the orphan rate of a candidate foreign key; keeps or drops the join hop | the source can LEFT JOIN |
An entity's grain is derived, never asked for: one row per business object means
a column whose distinct count equals its non-null count equals size.
Every route here is mounted only when you override it, and Atlas posts only what
the document's endpoints lists. Leave size out and Atlas sizes the entity
with a bounded pull of its own; leave the other two out and it measures them the
same way. cardinality is judged against the row count size carries, so a
connector that serves cardinality without size gets sized by that pull and
keeps whatever columns it did answer, the pull filling in only the ones it left
out. Conformance has no row count to judge the grain against there and skips
it, so serve the pair.
Two paths: SQL and REST
A SQL database: extend SqlConnector
When the data lives in a SQL engine the SDK writes the SQL. You declare a
catalog and provide three things: how to open a pool from a tenant's
credentials, how to close one, and how to run one parameterized statement. The
reference connector Brightline, a CRM over Postgres in examples/brightline-crm,
is exactly that:
protected override async openPool(credentials: Credentials): Promise<SQL> {
if (!credentials.databaseUrl) throw new Error("databaseUrl is required");
return new SQL(pinnedUrl(credentials.databaseUrl));
}
protected override async closePool(pool: SQL): Promise<void> {
await pool.close();
}
// the only path SQL text reaches pg on; values never travel inside the statement string
async run(pool: SQL, sql: string, params: unknown[]): Promise<Row[]> {
return (await pool.unsafe(sql, params)) as Row[];
}SqlConnector keeps one pool per credential set, keyed by a hash of the
credentials so key order never matters, and holds at most 16; the least
recently used pool is closed off the request path when a seventeenth tenant
arrives, and a pool a request is still reading from stays open until that
request returns. From catalog and run it derives every protocol method:
check, discovery, the query stream, counts, sizes, column cardinality, link hit
rates, and GROUP BY pushdown. The capability document is computed from
the catalog and the flavor, so it cannot advertise an operator the builders
cannot render, and credentialSchema defaults to the single databaseUrl
input.
The catalog is a typed literal. Each col(name, wire, type, opts?) names the
column, how its storage crosses the wire (int, decimal, text, boolean,
date, datetime, text_array), and the Atlas type discovery reports:
{
name: "deals",
description: "Opportunities. company_id/owner_id are enforced pg FKs.",
primaryKey: ["id"],
foreignKeys: [
{ field: "company_id", targetTable: "companies", targetField: "id" },
{
field: "primary_contact_id",
targetTable: "contacts",
targetField: "id",
},
{ field: "owner_id", targetTable: "owners", targetField: "id" },
],
columns: [
col("id", "int", "number"),
col("company_id", "int", "number"),
col("primary_contact_id", "int", "number", { nullable: true }),
col("owner_id", "int", "number"),
col("name", "text", "string"),
col("stage", "text", "string"),
col("amount", "decimal", "decimal", { nullable: true }),
col("currency", "text", "string"),
col("expected_close", "date", "date", { nullable: true }),
col("closed_at", "datetime", "datetime", { nullable: true }),
col("created_at", "datetime", "datetime"),
],
},Set unique: true only for a real database UNIQUE or primary-key constraint,
and set keysEnforced = true on the class only when every declared key
is one. Three seams exist for when the defaults run out: flavor (Postgres is
the shipped dialect), credentialSchema (override when the driver takes
separate parts instead of one URL), and streamBatches, which lets a driver
with real cursors replace the limit/offset paging query otherwise uses.
Brightline overrides streamBatches with a DECLARE ... CURSOR and
FETCH FORWARD 5000.
A REST or ERP API: extend AtlasConnector
When the source is an API there is no SQL to derive from, so you extend
AtlasConnector directly and write three methods: check, query, and
discover, plus size wherever the API knows how big a table is. Everything
else is optional, and what you leave out Atlas answers itself. The document is
authored by hand, and every flag in it is earned: an advertised operator your
query silently drops corrupts answers downstream.
The work concentrates in one decision: what pushes down to the API and what
runs in memory. Most business APIs filter on a few fields and no more. Push
those, then run the full filter set through applyFilters, the in-memory twin
of the SDK's SQL where-builder. Both routes count as honoring an operator.
Lark's scan does this, and refuses the field it cannot evaluate first:
// pushes the pushable slice of and[]; rows carry exactly the needed columns
private async *scan(client: LarkClient, req: QueryShape, deadline: Deadline): AsyncIterable<SourceRow[]> {
if (req.joins && req.joins.length > 0) throw unsupported("joins are not supported; atlas joins locally");
const { meta, table } = await this.resolveTable(client, req.table, deadline);
const fieldsByName = await this.fields(client, meta, table.table_id, deadline);
assertKnownFields(req, [RECORD_ID, ...fieldsByName.keys()]);
const columns = neededColumns(req);
const realFields = [...columns].filter((column) => fieldsByName.has(column));
const batches = client.searchAll(table.table_id, deadline, {
// field_names must name real fields; record_id rides along on every record anyway
fieldNames: realFields.length > 0 ? realFields : undefined,
conditions: pushdownConditions(req.and, fieldsByName),
});
for await (const records of batches) {
yield records.map((record) => toRow(record, columns, fieldsByName));
}
}
// scan batches with the FULL filter set re-applied locally; batches may come out empty
private async *scanFiltered(client: LarkClient, req: QueryShape, deadline: Deadline): AsyncIterable<SourceRow[]> {
for await (const batch of this.scan(client, req, deadline)) {
yield applyFilters(batch, { and: req.and, or: req.or }, req.fieldTypes);
}
}Lark's document advertises only the operators the API really pushes, and Atlas
judges the rest over the rows it answers. It declares join: false so Atlas
joins hops locally, and endpoints: ["size"] because every search page already
carries the table's total while a base has no server-side group-by, no distinct
count, and no join to measure a link with.
The rows you return are SourceRow values, Record<string, string | number | boolean | null>,
and the wire spellings are exact: decimals and integers past 2^53 cross as
digit-exact strings, datetimes as ISO-8601 UTC, arrays and JSON as JSON text
inside a string, missing as null. Atlas validates every answer and refuses a
malformed one rather than coercing it.
From scaffold to connected source
Scaffold it
bun create atlas-connector stamps a working project for either path:
$ bun create atlas-connector erp-bridge --kind rest --port 4200
Scaffolded 'erp-bridge' (rest) at /home/you/erp-bridge
Next steps:
cd erp-bridge
cp .env.example .env # set ATLAS_CONNECTOR_TOKEN to a 32+ char secret
bun install
bun run start # serves on :4200
Then fill in the YOUR CODE HERE methods in src/connector.ts; earn each flag in src/capability.ts, and point atlas-conform at it to grade the result.Run it bare and it prompts for the name, the kind (a sql database or a rest or erp api), and the port. The sql template is a working Postgres connector
as stamped: openPool, closePool, and run over Bun's SQL, and the only
thing to fill in is src/catalog.ts. The rest template stamps check,
query, count, and discover as YOUR CODE HERE stubs, each with its
one-line contract, calls assertKnownFields in query and count already,
and ships a deliberately narrow src/capability.ts. Both pin the SDK at the
version the CLI shipped with.
src/index.ts reads ATLAS_CONNECTOR_TOKEN and the port and calls
serve(new MyConnector(), { token, port }), which refuses to boot on a token
under 32 characters or a capability document that does not parse.
Run it locally
Start it and read the document, then prove a credential set against /check
with the same body shape Atlas will send:
curl http://localhost:4100/.well-known/futurity/atlas.json
curl -X POST http://localhost:4100/check \
-H "authorization: Bearer $ATLAS_CONNECTOR_TOKEN" -H 'content-type: application/json' \
-d '{"credentials":{"databaseUrl":"postgres://user:pass@localhost:5432/mydb"},"timeoutMs":5000}'A good set answers {"ok":true}. A bad one answers HTTP 400 with
{"error":{"code":"check_failed","message":"..."}}, where the message is
whatever your check threw.
Prove it with atlas-conform
atlas-conform is the conformance program packaged as a CLI, so you can grade a
connector before anyone connects it. Atlas runs the same program in live mode
after every connect, so a verdict here is the verdict that lands on the source.
The runner ships from Futurity rather than a public registry: ask your Futurity
contact for it, or connect the source in a test organization and read the report
Atlas records, which carries the same check ids.
atlas-conform --url https://connector.example.com --token "$ATLAS_CONNECTOR_TOKEN" \
--mode live --credentials '{"databaseUrl":"postgres://..."}'It exits 0 on pass, 1 when any check fails, and 2 when it could not run at
all (the document would not fetch, or the fixture corpus is not loaded).
--credentials takes inline JSON or @path/to/file.json; a connector whose
document declares an empty credentialSchema needs neither. --token can be
replaced by the ATLAS_CONNECTOR_TOKEN environment variable, and --report out.json writes the full report.
There are three modes. fixture (the default) runs phases 0 to 3 and compares
your answers against a DuckDB oracle over a known corpus you load into your
source first with atlas-conform load --url <dsn>. live runs phases 0, 1, 3
and 4 read-only against your real data, and is what Atlas runs. both runs
everything. --phases 0,1,3 narrows a run, and --row-budget (default
200,000) keeps the live streaming checks off tables larger than that.
| phase | id prefix | what it proves |
|---|---|---|
| 0 | P0.x | the document is served unauthenticated, parses, does not redirect; every route rejects an empty body with a 400 envelope |
| 1 | P1.x | no token, a wrong token, and a query-string token all 401 without leaking; the right one 200s; /check answers { ok: true } |
| 2 | P2.x | fixture agreement: discovery types, every operator, sort, paging, counts, streams, aggregates, measurement-to-join agreement, wire type fidelity |
| 3 | P3.x | behaviour: deadlines honored, unknown entity 404, malformed body 400, unknown filter field 400 or 422, stream bounds and heartbeats, joins refused rather than silently dropped |
| 4 | L1 to L10 | live self-consistency: an exact size equals the streamed rows, filters narrow, measured counts stay ordered, declared keys are unique, pages compose |
Each line of the report is one verdict: a pass, a fail with the detail and the
request that produced it, or a skip. Skips are not failures: a check skips when
your document does not declare the thing it tests (sort: "none", no
aggregate endpoint, join: false) or when the data cannot exercise it.
Deploy it
Any host that runs Bun and gives you a public HTTPS address works. Atlas calls
the connector from its own servers and refuses hosts that resolve onto a private
network, so localhost is only for atlas-conform. The connector's environment
holds exactly two things: the port (the examples read PORT, then
CONNECTOR_PORT, then default to 4100) and ATLAS_CONNECTOR_TOKEN. Nothing
about any tenant's upstream goes there.
Redeploy after changing secrets
Changing a variable in a host's dashboard does not always reach a running
container. After you set or rotate ATLAS_CONNECTOR_TOKEN, force a fresh
deploy, then re-read the well-known document and re-run /check before
trusting it.
Connect it in Atlas
In Select Data Sources, the External Connector card appears when you hold the Manage Atlas permission (the Atlas External Connectors feature is on unless an administrator switched it off for your organization). The form takes a name, the connector's base URL, and the bearer token you issued. The base URL is any https prefix, path included, so a deployment hosting several connectors gives each one its own path.
The Token field is the connector's own bearer token, the value its host set
as ATLAS_CONNECTOR_TOKEN. It proves to the connector that the caller is
Atlas, so it is issued by whoever hosts the deployment and is shared by every
client connecting to that deployment. It is not one of the credentials that
open your own upstream data; those are the fields the form renders once the
document is read. If an integrator hosts the connector for you, they are who
you get this token from. If you host it yourself, you mint it (openssl rand -hex 24 gives 48 characters) and set it as ATLAS_CONNECTOR_TOKEN; nothing is
issued by Futurity.
Fill in the base URL and the token, then click Read connector. Atlas
fetches the public document and checks the token against your /check; a
bearer the connector refuses fails the read with "The connector rejected the
token". On success it shows the slug it found, and renders one input per
credentialSchema entry: masking the password ones, giving a textarea one
a multi-line box, marking anything the connector declared optional, and showing
each field's help above it. Changing the URL forgets the document and the
values typed against it. Changing the token forgets the document, since the read
is what judges the token, and keeps what you typed.

Test connection is enabled once the document is read, the token is 32
characters or more, and every required credential input has a value. It posts to your
/check with those credentials. A rejected set shows your check message
verbatim; a rejected bearer shows "The connector rejected the token".

A wrong App secret above reaches the form as
lark tenant_access_token failed: code=10014 app secret invalid, which is
Lark's own wording carried through check() untouched. Write your message the
same way: name what was rejected and what the upstream said about it.

Click Add source. Atlas caches the document, marks the source
verifying, and a background worker runs atlas-conform in live mode with
the credentials you typed. The verdict lands on the source's status chip as
conformant or failed conformance; a failed run keeps the first ten
failures on the source, each with its check id, and the chip's tooltip carries
the detail of the first. The circular-arrows button on a connected connector
row, labelled Rebind, reopens the same form with a Reconnect button: it
re-reads the document, takes a fresh credential set, and runs conformance again,
which is how you pick up a connector that changed the entities or fields it
exposes.
From there the connector is discovered and queried like any other kind. Discovery asks each table for its size and reads one page of rows; anything Atlas has to measure itself is a WAN round trip to your service, so it takes those lazily, when a question needs the fact.
The same connector, twice
Multitenancy is not a claim the document makes; it is observable. The SDK's
examples/lark/seed fills two Lark bases with two unrelated datasets: tenant A,
Northwind Trading, a trading company with accounts (30 rows), contacts
(80) and deals (60); and tenant B, Harbor Logistics, a shipping company
with vessels (12), ports (15), voyages (90) and incidents (20). The two
share no table names on purpose, so a leak between tenants would be visible at
a glance.
One deployed connector, with no Lark credentials in its environment, is connected into Atlas twice: once with Northwind's app credentials and base token, once with Harbor's. Both pass Test connection, and swapping Northwind's secret for a wrong one fails it.

Discovery for Northwind lists accounts, contacts, and deals and nothing
else. Discovery for Harbor lists vessels, ports, voyages, and
incidents. A query on each returns its own row counts, and asking the
Northwind source for a deals table works while asking Harbor for one answers
404 unknown_entity. The process serving both holds a metadata cache keyed by
the whole credential set, so a tenant who cannot open a base never reads its
tables out of another tenant's entry.
Atlas names an entity per discovered table, so the two catalogs stay separate all the way into the entity list:

Ask Corint how many Northwind deals are won and it plans over the Deal entity, counts through the connector, and answers 11, the number seeded into that base and no other:

Debugging and common pitfalls
Most connector problems surface the same way: atlas-conform fails a check,
or a live query returns the wrong rows. These are the ones you are most likely
to hit.
Reject filters on fields you do not have
The single most common bug on the AtlasConnector path. applyFilters
treats an unknown filter field as a missing value: it silently matches nothing
instead of erroring, and the empty answer looks like a correct one. The
protocol requires a filter on a field the table does not have to be refused.
SqlConnector does this for you; the hand-written path must call the kit:
import { assertKnownFields } from "@futurity/atlas-connector";
async *query(req: NativeQueryRequest): AsyncIterable<SourceRow[]> {
// a filter you cannot answer must 422 HERE: a row that skipped a filter reads as a row that matched it
assertKnownFields(req, fieldsOf(req.table));This passes every hand test
Your own filters always name real fields, so the bug only shows up as
conformance check P3.4c, "an unknown filter field → 400/422", reporting
got no-error. Add the guard to query and count before you write your
first fetch.
Read a conformance failure
A failed run in Atlas stores the first ten failures on the source; a CLI run
prints every verdict and can write the whole report with --report. Each
failure names the check id, its title, and what it saw:
{
"mode": "live",
"summary": { "pass": 31, "fail": 1, "skip": 2 },
"failures": [
{
"id": "P3.4c",
"title": "an unknown filter field → 400/422",
"detail": "got no-error"
}
]
}Each id maps to one protocol obligation from the phase table above, and
detail states the gap in plain terms. Fix that one check and re-run; the CLI
report also carries the exact request body that produced a failure.
When a live call fails and Atlas shows only internal
The wire is sanitized on purpose. Anything your method throws that is not a
ConnectorError reaches Atlas as
{"error":{"code":"internal","message":"internal error"}}, never your stack.
serve() logs the real error to the connector's own stderr first, prefixed
with the slug, so read your host's logs for the cause. Most first failures are
the upstream rejecting a credential, a wrong base URL, or a response shape that
differs from what you parsed. Hit the upstream API directly with curl and
compare its real response to what your code assumes.
To answer with a specific status instead, throw one of the SDK's constructors:
badRequest, unauthorized, unknownEntity, unsupported, or timeout.
Lark, for example, maps its API's not-found code to unknownEntity, which is
what makes an unknown table a 404 rather than a 500.
Wire reference
Every route is relative to the base URL a source is registered with, which is
any https prefix. A connector mounted at https://atlas.example.com/lark-base
serves its document at
https://atlas.example.com/lark-base/.well-known/futurity/atlas.json and its
routes under the same prefix, so one host can carry several connectors.
Every route except the well-known document is a POST with a JSON body and
the header Authorization: Bearer <token>. Every authenticated body carries
credentials, a flat object of strings, and timeoutMs, a positive integer
the server enforces with a 408. Every JSON answer is a wrapped object, never a
bare array. Values in rows are string | number | boolean | null.
GET /.well-known/futurity/atlas.json
Unauthenticated. At most 64 KiB, content-type: application/json, no
redirect. Unknown top-level fields are stripped by the reader.
{
"protocolVersion": 1,
"slug": "lark-base",
"capabilities": {
"operators": [
"eq",
"neq",
"gt",
"gte",
"lt",
"lte",
"in",
"nin",
"contains",
"includes",
"startswith",
"isnull",
"notnull"
],
"dateBucket": false,
"sort": "multi",
"offset": true,
"join": false,
"keysEnforced": true,
"limits": { "pageSizeMax": 500, "rowsPerTableMax": 20000, "concurrency": 1 }
},
"credentialSchema": [
{
"key": "appId",
"label": "App ID",
"type": "text",
"required": true,
"placeholder": "cli_XXXXXXXXXXXXXXXX",
"help": "Lark Developer Console → your app → **Credentials & Basic Info**, the field labelled **App ID**. Your apps are listed at [open.larksuite.com/app](https://open.larksuite.com/app)."
},
{
"key": "appSecret",
"label": "App secret",
"type": "password",
"required": true,
"help": "The **App Secret** on that same **Credentials & Basic Info** page of the [Lark Developer Console](https://open.larksuite.com/app). The app also needs the `bitable:app:readonly` permission."
},
{
"key": "appToken",
"label": "Base app token",
"type": "text",
"required": true,
"placeholder": "bascnXXXXXXXXXXXXXXXXXXXXXX",
"help": "The id in the base's URL, `https://<tenant>.larksuite.com/base/<app_token>`. Add the app to that base as a collaborator first, or it cannot read the tables."
}
],
"endpoints": []
}| field | type | notes |
|---|---|---|
protocolVersion | literal 1 | any other value fails the parse |
slug | string | ^[a-z][a-z0-9-]{2,39}$; must not collide with a built-in source kind |
dialect | string, optional | present only for a dialect-mode connector that accepts compiled SQL; absent for native mode |
capabilities.operators | array of Op, possibly empty | the filter operators query and count push upstream; [] means Atlas judges every filter itself |
capabilities.dateBucket | boolean | whether aggregate accepts a grain on a group-by |
capabilities.sort | "none", "single", "multi" | how many sort keys query honors |
capabilities.offset | boolean | whether query honors offset |
capabilities.join | boolean | whether query executes joins; false means Atlas joins hops locally |
capabilities.keysEnforced | boolean | every declared primary key and unique field is a real source constraint |
capabilities.limits | object | the vendor's own ceilings: pageSizeMax, rowsPerTableMax, concurrency (1 to 16), offsetMax; Atlas plans its pulls against them |
credentialSchema | array of credential fields | each is { key, label, type, required } plus optional placeholder and help; empty only when the source needs no per-tenant secret |
credentialSchema[].type | "text", "password", "textarea" | password masks the input; textarea is the multi-line box a pasted key needs |
credentialSchema[].required | boolean, defaults to true | a required field gates Test connection and submit; a blank optional one is absent from credentials |
credentialSchema[].placeholder | string, optional | example value shown inside the empty input |
credentialSchema[].help | string, optional | short markdown above the input, naming where the value is found |
endpoints | array of optional route names | "size", "count", "aggregate", "cardinality", "linkHitRate"; the SDK fills it from the methods you override |
Filters, sorts, and joins
A Filter is one of three shapes, each strict:
{ "field": "stage", "op": "eq", "value": "won" }
{ "field": "stage", "op": "in", "values": ["won", "lost"] }
{ "field": "closed_at", "op": "isnull" }Value ops are eq, neq, gt, gte, lt, lte, contains, includes,
startswith and carry a scalar value. Member ops are in and nin and
carry an array values. Nullary ops are isnull and notnull. and is a
conjunction; or is an array of groups in disjunctive normal form, and the
whole or block is one more conjunct.
A sort is { "field": "amount", "dir": "asc" } with an optional
"collate": true on a compiled sort. A join hop is
{ "fromTable", "toTable", "fromField", "toField", "fields": [{ "field", "as", "type" }] }.
fieldTypes is an optional record of field name to Atlas type
(string, number, decimal, boolean, date, datetime, json,
array, reference) and is how applyFilters knows to compare digits rather
than bytes.
POST /check
Request: { "credentials": {...}, "timeoutMs": 5000 }.
Answer: { "ok": true }. A rejected credential set answers HTTP 400 with
{ "error": { "code": "check_failed", "message": "<what check threw>" } }.
POST /discovery
Request: { "credentials": {...}, "timeoutMs": 15000 }.
Answer: { "tables": [...], "warnings": ["..."] } where warnings is
optional and each table is:
{
"name": "deals",
"sourceDescription": "lark base table",
"rowCount": 60,
"storesRows": true,
"primaryKey": ["record_id"],
"foreignKeys": [
{
"field": "account",
"targetTable": "accounts",
"targetField": "record_id"
}
],
"fields": [
{
"name": "record_id",
"sourceColumn": "record_id",
"type": "string",
"nullable": false,
"unique": true,
"samples": ["recuA1", "recuA2"],
"sourceDescription": "lark record id (system primary key)",
"stats": {
"nullPercent": 0,
"distinctCount": 60,
"min": "recuA1",
"max": "recuZ9"
},
"filterable": true,
"groupable": true,
"aggregatable": false
}
]
}rowCount, stats (with optional nullPercent, min, max and a required
distinctCount), filterable, groupable, and aggregatable are optional;
everything else is required.
POST /query
Request, native mode:
{
"table": "deals",
"and": [{ "field": "stage", "op": "eq", "value": "won" }],
"sort": [{ "field": "amount", "dir": "desc" }],
"limit": 100,
"offset": 0,
"fields": ["record_id", "name", "amount"],
"fieldTypes": { "amount": "number" },
"credentials": {},
"timeoutMs": 30000
}table, and, sort, fields, credentials, and timeoutMs are
required; or, limit, offset, joins, and fieldTypes are optional, and
the body also carries idleTimeoutMs and maxTimeoutMs. Atlas sends an idle
timeout of 30 seconds, three missed heartbeats. In dialect mode the query fields
are replaced by { "sql": "...", "params": [...] }.
Answer: content-type: application/x-ndjson, one JSON object per line:
{ "served": { "filters": true, "sort": true, "window": false } }
{ "rows": [ { "record_id": "recuA1" } ] }
{ "ping": 1 }
{ "end": 1 }
{ "error": { "code": "timeout", "message": "no rows within idle deadline" } }served is the first line, written before any rows, and it names which parts
of the request this answer already applied. filters means every and and
or predicate held, with Atlas's semantics. sort means the rows arrive in
the order the request asked for. window means offset and limit were
applied, with no row beyond them on the stream. A stream that omits the line
served none of the three, which is always safe: Atlas finishes what is left
over the rows it receives. window is read only when filters and sort are
served with it, because a page cut from the wrong rows in the wrong order is
the wrong page; ndjsonStream clamps it for you, and counts a request that
asked for no order as sorted. This line is where the filter law is answered:
push every predicate and say filters: true, or answer the superset and say
filters: false.
A rows line carries between 1 and 5,000 rows. ping is written after 10
seconds without a batch. end is written once, after the producer completes.
error is terminal with no end. A stream that closes with neither end nor
error is truncated and the reader fails it.
POST /count
Request: { "table", "and", "or"?, "fieldTypes"?, "credentials", "timeoutMs" },
the filter subset of /query. Mounted only when count() is overridden.
Answer: { "count": 11 }.
POST /aggregate
Request:
{
"table": "deals",
"and": [],
"or": [],
"groupBy": [{ "field": "closed_at", "as": "month", "grain": "month" }],
"measures": [
{ "fn": "sum", "field": "amount", "as": "total" },
{ "fn": "count", "as": "n" }
],
"stringFields": ["stage"],
"joins": [],
"fieldTypes": { "amount": "decimal" },
"limit": 1001,
"credentials": {},
"timeoutMs": 30000
}fn is one of count, sum, min, max, count_distinct; avg is never
pushed. grain is one of year, quarter, month, day and is sent only
to a dateBucket: true connector. limit is the group budget plus one: Atlas
sends 1,001 so that a 1,001st group proves overflow, and it discards an answer
that long rather than reporting a truncated set, so refuse rather than
truncate. The number never varies with the question: a top-N is cut from the
groups Atlas gets back, never asked for on this route.
Answer: { "rows": [ { "month": "2026-03-01", "total": "48210.50", "n": 7 } ] },
or HTTP 204 with no body to decline this aggregate.
POST /size
Request: { "table": "deals", "credentials": {}, "timeoutMs": 15000 }.
Answer: { "size": { "rows": 60, "exact": true } }, exact: false when the
source only estimates, or { "size": null } when it cannot say at all. The null
sits inside the wrapper, never as a bare body.
POST /cardinality
Request: { "table": "contacts", "columns": ["email", "company_id"], "credentials": {}, "timeoutMs": 15000 },
with at least one column.
Answer:
{
"columns": {
"email": { "nonNull": 80, "distinct": 79 },
"company_id": null
}
}A column you cannot count distinctly answers null, and Atlas measures that one
itself over query. The counts are judged against size, so a source that
serves this route should serve that one too.
POST /linkHitRate
Request: { "fromTable": "contacts", "fromColumn": "company_id", "toTable": "accounts", "toColumn": "record_id", "credentials": {}, "timeoutMs": 15000 }.
Answer:
{ "fromNonNull": 78, "orphanCount": 1, "orphanRate": 0.01282, "orphanSamples": ["recuMissing"] },
with at most 20 samples.
The error envelope
Every non-2xx JSON body is exactly { "error": { "code": "...", "message": "..." } }.
| status | code | when |
|---|---|---|
| 400 | bad_request | the body did not parse against the route's schema |
| 400 | check_failed | /check only: the credentials were rejected; the message is the author's |
| 401 | unauthorized | a missing or wrong bearer |
| 404 | unknown_entity | a table or column the source does not have |
| 408 | timeout | the request's own timeoutMs elapsed |
| 422 | unsupported | a legal Atlas request the document never advertised, or a filter field the connector cannot evaluate |
| 500 | internal | anything else; the message is always internal error and the real error is logged server-side |
Limits
| limit | value |
|---|---|
| capability document | 64 KiB |
| JSON answer body | 32 MiB |
| one NDJSON line | 16 MiB |
| rows per stream batch | 5,000 |
| heartbeat interval | 10 s |
| Atlas's stream idle timeout | 30 s |
| bearer token | 32+ chars |