Jobs & schedules
CommissionSight runs work in the background in two ways: ingest jobs (one per uploaded statement, processed off a queue) and scheduled maintenance (a cron that runs every 15 minutes). Everything here is idempotent — re-running never double-counts.
1. Ingest jobs (per statement)
When you POST /v1/files, the raw file is stored, a job is created, and a message is queued.
A worker then runs: parse → map → score → compute deltas → roll up.
- Lifecycle:
queued → processing → completed(orfailed). PollGET /v1/jobs/{jobId}or subscribe to a webhook. See Polling jobs. - Async + queued: uploads return
202immediately; processing happens on the edge. - Retries: a failed run is retried automatically up to 5 times; persistent failures land in
a dead-letter queue. You can also retry manually via
POST /v1/jobs/{jobId}/retry. - Idempotent: records upsert on a deterministic key and statuses are recomputed deterministically, so re-running a period yields identical results.
Upload limits
A statement is parsed and scored in a single in-memory pass, so there are guard rails (see Scale & performance):
| Limit | Value | What happens if exceeded |
|---|---|---|
| File size | 15 MB | 413 at upload — split the statement by period and upload each part |
| Rows per statement | 40,000 | the job fails with a clear message — split by period (or sub-carrier/plan) |
These cover essentially every real carrier statement; your total book size is unbounded (millions of members across carriers and periods).
2. Scheduled maintenance (every 15 minutes)
A cron trigger runs three idempotent tasks. You don’t call these — they keep the platform healthy automatically.
| Job | What it does | Bound |
|---|---|---|
| Stuck-job recovery | Any ingest job stuck in processing for > 15 min is re-queued (covers a worker crash/timeout). | up to 100 jobs/run |
| Webhook redelivery | Pending/failed webhook deliveries whose next attempt is due are retried with exponential backoff (0, 30s, 2m, 10m, 30m, 2h, max 6 attempts). | up to 100 deliveries/run |
| Retention purge | Raw statement bytes are deleted from object storage 24h after a file processes — or 3 days after upload if it never processes (its job is then canceled). Scored results are kept. See Data retention. | up to 500 files/run |
All three are safe to run repeatedly: stuck-job recovery only touches jobs past the timeout, webhook redelivery only retries due deliveries, and the retention purge skips files already purged.
Observability
- Ingest jobs:
GET /v1/jobslists your account’s jobs with status and stats, and supports one-click retry in the app. - Webhook deliveries: each attempt is recorded with status, attempt count, and the next scheduled retry.