Data retention & deletion

CommissionSight stores your data in three places, each with a different purpose and lifecycle.

What’s stored, and where

Store	Holds	Isolation
Raw statement files	The original files you upload (the CSV/XLSX bytes).	Per account, organized by account → carrier → period.
Member data	The parsed member records and scored statuses, deltas, timeline.	A dedicated database per account — never co-mingled.
Metadata	File/job metadata, period rollups, account/config. No member-level statement data.	Scoped per account by token.

Everything is encrypted in transit and at rest; access is entirely controlled by your API token, and each account’s member data lives in its own dedicated database.

Field-level data minimization

You don’t have to store a column you don’t want kept. Each carrier config can carry a privacy policy that is applied at ingest, before anything is persisted — so minimized fields never land in your database at all:

drop — listed columns are never stored (written as null). Use it for fields you don’t need for reconciliation (e.g. date of birth, email, phone, street address).
hash — listed columns are stored as a salted one-way hash instead of the clear value, so you can still group or join on them without retaining the original.
keepRawRow: false — discards the verbatim source row entirely, keeping only the mapped, canonical fields.

Crucially, member matching is unaffected: identity is computed from the real values first, then the redaction is applied — so cross-month scoring, drop/reappear detection, and owed math all keep working even when the identifier itself is hashed. Only optional identifier/PII columns are eligible (required scoring fields like the commission amount can’t be dropped). Tell the CommissionSight team which fields to minimize for a carrier and it’s enabled in that carrier’s config. Defaults store everything, so this is opt-in per carrier.

Two independent deletion controls

They’re complementary — use either or both:

Retract a period (clears scored data)

DELETE /v1/files/{fileId} retracts the file’s carrier+period: it deletes the period’s member records (members, statuses, deltas) and re-scores the following month. The raw file is left in place so the period could be reloaded.

In the app: the Remove action on the Files page. See Uploading statements.

Purge raw files (clears the original bytes)

POST /v1/files/{fileId}/purge deletes the raw statement bytes from object storage. The file row, its metadata, and any scored results remain — but the file can no longer be re-ingested. Idempotent.

In the app: a Purge raw action per file on the Files page (the row shows raw purged after).
On request, the CommissionSight team can purge raw files for your whole account, which purges every raw file the account has uploaded in one action.

# purge one file's raw bytes
curl -X POST "https://api.commissionsight.com/v1/files/$FILE_ID/purge" -H "Authorization: Bearer $TOKEN"

Full deletion

To remove both the raw file and the scored data for a period, retract it and purge it. Retract clears the member data; purge clears the raw file.

Automatic purge (retention policy)

Raw statement bytes are not kept indefinitely. A scheduled job purges them from object storage across all accounts:

Processed files — purged 24 hours after the ingest job completes. The scored results stay; only the raw source bytes are removed.
Unprocessed files — if a file never finishes processing, it’s purged 3 days after upload, and its job is canceled (recorded as failed with “raw file purged by retention policy”).

Purged files show r2_purged_at and can’t be re-ingested. The manual Purge raw / per-account purge above is for purging sooner than the automatic window. (You can always re-upload a statement to re-process a period.)

Exception files (rejected rows)

When an ingest rejects rows that fail validation, CommissionSight writes an exception file — a CSV of every rejected row, prefixed with its source row number (_row) and the validation errors (_errors), followed by the original cells verbatim so you can see exactly what to fix.

Download it from the Jobs page (a ”⚠ N rejected” button on any job with rejected rows), or via the API / SDK:

curl -L "https://api.commissionsight.com/v1/jobs/$JOB_ID/exceptions" \
  -H "Authorization: Bearer $TOKEN" -o exceptions.csv

const csv = await client.downloadExceptions(jobId); // CSV text

Retention: exception files are kept for 30 days, then purged from object storage by the same scheduled job. After that the endpoint returns 404 (the scored data and job metadata remain).

What purge does not do

Purging raw files does not remove the scored results derived from them — those live in the data plane and are cleared by a retract (or by deleting the account). This is deliberate: you can shed the raw source documents for retention while keeping the analytics you’ve already computed.