Pipeline-Triggered Ingestion

A kept record checks its source on a schedule. When your pipeline owns the cadence instead, pause the schedule and trigger each check yourself. The pattern is four moves: pause the record, overwrite your file, trigger a check, pull the delta. The record stays paused the whole time: no scheduled checks run, and every ingest that happens is one you asked for.

The four moves

1. Pause the record

On the record page, choose Pause from the record menu, or call the API:

PATCH /v1/sources/:id
Content-Type: application/json

{ "status": "paused" }

The scheduled check is removed immediately. A paused record shows a paused status and a Manual badge in the dashboard: automated checks off, checks on demand.

2. Overwrite your file

For a file record, your pipeline uploads the new version over the API (the dashboard's Replace file button does the same):

curl -X POST "https://api.catalogian.com/v1/sources/:id/upload" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "[email protected]"

For a URL record, your pipeline updates the file at the origin URL; the check in the next step re-fetches whatever the URL serves. A paused record's re-upload does not resume the schedule: the ingest runs because you triggered it.

3. Trigger the check

One call starts the ingest and the diff:

curl -X POST "https://api.catalogian.com/v1/sources/:id/check" \
  -H "Authorization: Bearer YOUR_API_KEY"

The check needs a write-scoped key (see Authentication & Keys). The response is immediate: the ingest is queued, not run inline.

{
  "jobId": "source-check-src_abc123-manual-1790486400000",
  "sourceId": "src_abc123",
  "status": "queued",
  "manualChecks": {
    "allowance": 3,
    "used": 1,
    "remaining": 2,
    "resetAt": "2026-09-27T00:00:00.000Z"
  }
}

4. Pull the delta

Once the check finishes, the diff between the previous edition and the new one is a delta event. Read it with GET /v1/sources/:id/delta (cursor pagination) or grab the newest with GET /v1/sources/:id/delta/latest. If you would rather be pushed, a webhook delivers the event the moment it is detected. Full detail in Delta Events and Row Diffs.

Limits on the trigger

The trigger endpoint carries two honest limits. The burst limit is 5 checks per minute per key, so a pipeline loop cannot flood the queue. On top of that, manual checks are metered per record per day: 3 a day on free, 10 a day on watch, and unmetered on max. Accepted checks return the post-write numbers (allowance, used, remaining, resetAt) so your pipeline can pace itself. The per-tier numbers are part of the record ladder in Plans & Limits.

The two 429s

When the trigger is refused, the body says which limit bit you:

The rate-limit body reads "Rate limit: manual checks are capped at 5 per minute. Retry in N seconds." The allowance body carries the honest numbers (allowance, used, resetAt, tier) and an upgradeUrl pointing at /pricing. No silent stalls: a refused check is never queued.

Resume the schedule

When the pipeline hands the cadence back, resume the record and the scheduled checks return at the record's cadence floor:

PATCH /v1/sources/:id
Content-Type: application/json

{ "status": "active" }