datasets

Choose which tables or objects a sync replicates, and how their records land in the destination. Each dataset selects one source table or object and sets an ingestion mode: append, replace, or merge. A dataset exists only as a child of a sync, so most subcommands require --sync.

The subcommands form one workflow. available lists what a connection could replicate. list shows what the sync replicates today. upsert changes that selection. fields inspects the columns of a single table or object.

REST API: Syncs

celigo datasets <subcommand> [args] [flags]

Supports all global flags.

---

Subcommands

Subcommand	Purpose
list	List the datasets configured on a sync — the tables and objects it replicates.
get <datasetId>	Fetch one dataset, including its column selections (dataElements).
upsert	Batch create-or-update a sync's datasets from a JSON array (--file <path> or stdin).
available <connectionId>	List every table or object a connection exposes for syncing, plus its exports usable as export-backed datasets.
fields <connectionId> <datasetName>	Read column-level detail for one table or object: data types, constraints, and delta cursor candidates.

---

celigo datasets list

List the datasets configured on a sync. Results are sorted by name, and the CLI follows the response's Link cursor to collect every page.

Signature

celigo datasets list --sync <syncId>

Arguments

None.

Flags

Flag	Type	Default	Description
--sync <syncId>	string	—	Required. Sync whose datasets to list.

Default table columns: _id, name, externalId, enable, ingestionMode, userActionRequired.

Example

celigo datasets list --sync 5f83a9b2c7d3e8f1a2b3c4d5 --format table

Corresponds to: GET /v1/syncs/{_syncId}/datasets (operationId: listDatasetsForSync)

---

celigo datasets get <datasetId>

Fetch one dataset by ID, including its per-column selections in dataElements.

Signature

celigo datasets get <datasetId> --sync <syncId>

Arguments

Argument	Type	Required	Description
<datasetId>	string	Yes	Dataset ID (the _id from datasets list).

Flags

Flag	Type	Default	Description
--sync <syncId>	string	—	Required. Sync the dataset belongs to. Datasets have no unscoped by-ID endpoint.

Example

celigo datasets get 6a559b547885d8f93921c2ba --sync 5f83a9b2c7d3e8f1a2b3c4d5 --jq '.dataElements[] | select(.enable)'

Corresponds to: GET /v1/syncs/{_syncId}/datasets/{_id} (operationId: getDataset)

---

celigo datasets upsert

Create and update a sync's datasets in one batch PUT. Read the body from a file with -f, --file, or pipe it on stdin.

The batch is all-or-nothing: one invalid entry rejects the whole request. Datasets you leave out of the array keep their saved configuration. upsert merges — it never deletes.

Signature

celigo datasets upsert --sync <syncId> --file <path>
celigo datasets upsert --sync <syncId> < datasets.json     # or pipe on stdin

Arguments

None.

Flags

Flag	Type	Default	Description
--sync <syncId>	string	—	Required. Sync to upsert datasets on.
-f, --file <path>	string	—	Read the JSON body from a file instead of stdin (--file - also means stdin).

Request body

A JSON array (not an object) of dataset entries, matching the PUT /v1/syncs/{_syncId}/datasets request schema. Each entry carries one identifier — externalId to create, _id to update:

Field	Purpose
_id	Updates the saved dataset with that ID. Omit when creating.
externalId	Creates a dataset for that source table or object — the name from datasets available — or the export's ID for an export-backed dataset. Omit when updating and send _id instead.
name	Display name of the dataset.
enable	Whether the sync replicates the dataset on each run.
ingestionMode	append, replace, or merge. Required whenever enable is true.
exportProperties	Extraction behavior — type: delta or all, plus delta.dateField for the cursor. Source-table datasets only.
isExport / tableName	Mark an export-backed dataset and name its destination table. tableName must be unique across the sync's datasets.
dataElements	Per-column selections. Omit to replicate every column.
driftPolicy	Per-dataset override of the sync's schema drift policy.

One entry creating a dataset, one entry disabling a saved one:

[
  {
    "name": "Account",
    "externalId": "Account",
    "enable": true,
    "ingestionMode": "merge",
    "exportProperties": { "type": "delta", "delta": { "dateField": "LastModifiedDate" } },
    "dataElements": [
      { "name": "Id", "enable": true, "isPrimaryKey": true },
      { "name": "Name", "enable": true }
    ]
  },
  { "_id": "6a559b547885d8f93921c2ba", "enable": false }
]

Example

celigo datasets upsert --sync 5f83a9b2c7d3e8f1a2b3c4d5 --file ./datasets.json

Corresponds to: PUT /v1/syncs/{_syncId}/datasets (operationId: upsertDatasetsForSync)

---

celigo datasets available <connectionId>

List every table or object the connection's application exposes for syncing. The listing also includes the connection's exports, which can back export-backed datasets. This is the whole candidate pool, not a list of what remains unselected.

The API returns two catalogs, and the CLI merges them into one list. A _catalog column tags each row: dataset for a source table or object, export for an export resource.

Signature

celigo datasets available <connectionId> [--sync <syncId>] [--type <datasets|exports|all>] [--refresh]

Arguments

Argument	Type	Required	Description
<connectionId>	string	Yes	Source connection to read the catalog from.

Flags

Flag	Type	Default	Description
--sync <syncId>	string	—	Merge the catalog with this sync's saved dataset selections, so saved and unsaved entries appear side by side. Saved entries carry an _id.
--type <type>	string	all	Restrict the catalog to datasets (source tables/objects), exports (export resources), or all. Sent as type; an unknown value is rejected locally before any request.
--refresh	boolean	false	Bypass the platform's cached catalog and re-read it from the source application. Sent as refreshCache=true.

Example

# Everything the connection could replicate, annotated with what this sync already selected
celigo datasets available 5f8d43a1b9e5a80011a35f2e --sync 5f83a9b2c7d3e8f1a2b3c4d5 --format table

# Only the source tables, re-read live from the application after a schema change
celigo datasets available 5f8d43a1b9e5a80011a35f2e --type datasets --refresh

Corresponds to: GET /v1/di/metadata/connections/{_connectionId}/datasets (operationId: listConnectionDatasets), or with --sync, GET /v1/di/metadata/sync/{_syncId}/connections/{_connectionId}/datasets (operationId: listSyncConnectionDatasets)

---

celigo datasets fields <connectionId> <datasetName>

Read column-level detail for one table or object on a connection. Each column reports its data type, length, precision, and constraints. The response also carries deltaFields: the columns usable as the delta export cursor (exportProperties.delta.dateField). Use fields to build a dataElements selection before calling upsert.

The API wraps its answer in a { "dataset": … } envelope. The CLI prints the inner object.

Signature

celigo datasets fields <connectionId> <datasetName> [--sync <syncId>] [--is-export] [--refresh] [--record-type <type>] [--display-name <name>]

Arguments

Argument	Type	Required	Description
<connectionId>	string	Yes	Source connection to read the columns from.
<datasetName>	string	Yes	Table or object name from datasets available. With --is-export, supply an export-backed identifier instead — see the gotchas below. The CLI URL-encodes the value, so names containing spaces are safe to pass.

Flags

Flag	Type	Default	Description
--sync <syncId>	string	—	Merge the source columns with this sync's saved dataset state and per-column selections.
--is-export	boolean	false	Treat <datasetName> as an export-backed entry and read the export's record structure instead of a source table. Sent as isExport=true.
--refresh	boolean	false	Bypass the cached column list and re-read it from the source application. Sent as refreshCache=true.
--record-type <type>	string	—	NetSuite record type backing a saved-search dataset. Sent as recordType; omit for other sources.
--display-name <name>	string	—	NetSuite saved-search display name. Sent as displayName, and only meaningful alongside --record-type.

Example

# Column catalog for one Salesforce object, with the delta-cursor candidates
celigo datasets fields 5f8d43a1b9e5a80011a35f2e Account --jq '{deltaFields, columns: [.dataElements[].name]}'

# The same object merged with what this sync has already selected
celigo datasets fields 5f8d43a1b9e5a80011a35f2e Account --sync 5f83a9b2c7d3e8f1a2b3c4d5

# A NetSuite saved-search dataset
celigo datasets fields 5f8d43a1b9e5a80011a35f2e "Sales Orders" \
  --record-type salesorder --display-name "Sales Orders"

Corresponds to: GET /v1/di/metadata/connections/{_connectionId}/datasets/{datasetName}/details (operationId: getConnectionDatasetDetails), or with --sync, GET /v1/di/metadata/sync/{_syncId}/connections/{_connectionId}/datasets/{datasetName}/details (operationId: getSyncConnectionDatasetDetails)

---

Gotchas

• --sync is required on list, get, and upsert. Datasets exist only as children of a sync. There is no unscoped /v1/datasets collection and no by-ID endpoint that works without the sync. Omitting --sync fails locally, before any request is sent.
• upsert merges; it never deletes. Datasets missing from the array keep their saved configuration. Removing an entry does not unselect the table. To stop replicating one, send it with its _id and enable: false.
• Every entry needs an identifier. Send externalId (the source table or object name) to create a dataset, and _id to update one that already exists. An entry with neither rejects the whole batch with 400 Bad Request and the error code dataset_externalId_id_required. To update, read the saved _id from datasets list --sync <syncId> and send that — it identifies the dataset unambiguously.
• dataElements replaces the whole column selection. The array you send becomes the dataset's complete selection. Omit it to replicate every column. Read the current selection first with celigo datasets get <datasetId> --sync <syncId>.
• enable: true requires ingestionMode, and merge requires a primary key. Enabling a dataset without an ingestion mode fails validation. So does merge with no enabled isPrimaryKey column.
• available is the candidate pool, not a comparison. It lists everything the connection exposes, whether or not the sync already replicates it. With --sync, only saved datasets carry an _id — that column separates saved from unsaved. The merged view omits dataElements to keep the payload small, so use fields to inspect columns.
• --refresh costs a live round trip to the source application. It re-reads the catalog or column list from the connected system instead of the platform cache. Expect a slower response, and a failure if the source is offline. Use it after a schema change in the source, not by default.
• fields identifies export-backed datasets differently. Pass --is-export, then supply the export's ID as <datasetName> for the connection-scoped lookup. When you also pass --sync, supply the saved dataset's _id instead.
• mismatchSyncConnection: true means the export drifted. The flag appears in list output when an export-backed dataset's export no longer uses the sync's source connection. Upserting that dataset fails with the error code connection_mismatch. Repoint the export at the sync's connection, or remove the dataset.
• upsert returns no body. The PUT succeeds with an empty payload, so the CLI prints a confirmation message instead of a resource. Read the saved state back with datasets list --sync <syncId> to confirm what landed.
• Datasets live only inside a syncs-type integration. An integration holds either flows or syncs, never both. Create the parent integration with syncs enabled before attaching datasets to a sync inside it.
• upsert requires full mode. It creates and reconfigures datasets, so it is gated like create and update. list, get, available, and fields work in read mode. See Profiles & regions.

Related

• syncs — the sync that owns these datasets, and where the source connection and destination are configured.
• sync-jobs — the runs a sync produces, showing whether the selected datasets replicated.
• connections — resolve the <connectionId> that available and fields read their catalogs from.
• exports — the exports that back export-backed datasets.