Data stores

Files, keyed values, and event streams that steps read from and write to between runs: staging files and the SQL queries that read them, cross-reference caches, durable state such as cursors and sequence numbers, and topics that hand records from one flow to another. The file server lets partners exchange files with Celigo Storage over SFTP.

Reading and writing records to external systems is done by Exports, imports & tools.

Resource	What it covers
File Server	A Celigo-hosted SFTP server per environment, whose partner accounts each read and write one Celigo Storage folder.
File Storage	Files held in Celigo-hosted storage instead of an external file provider.
Lookup Caches	In-memory key-value stores for deduplication and cross-reference lookups.
State	Durable JSON values keyed by name, account-wide or scoped to one resource.
Storage Query	Read-only SQL over CSV, JSON, NDJSON, and Parquet files in Celigo Storage, run as background jobs.
Topics	Durable event streams that let one flow publish messages for other flows to consume at their own pace.