Employment & Income

The employment domain covers the user’s employment record in RDS, manual income verification in DynamoDB, transaction-based income detection, and the cleanup that runs when a user connects a new bank institution. Endpoint shapes are in API Specification; storage in PostgreSQL Schema and DynamoDB Tables.

Employment Records

POST and PUT /{user_id}/insights/employment behave identically: both insert a new RDS row, and the highest id becomes the current record.

Validate the body; failure returns 400.
When the Pave allow list is non-empty, read the previous record’s pay frequency (pgx.ErrNoRows is a known empty value; any other read error marks it unknown).
Insert the row; failure returns 500.
Delete the cached payday prediction (Same-Day Cache).
Run the Pave Re-mine on Eligibility check when the previous value is known.
Delete the user’s income verification row and call Underwriting Recalculation; both failures are logged only.
Return 201.

DELETE /{user_id}/insights/employment deletes every row for the user and the cached payday; returns 200.

GET /{user_id}/insights/employment returns the current record with its income verification status and a fresh payday prediction (404 with no record, 500 if prediction fails). GET /{user_id}/insights/v2/employment returns the record and verification as lists, empty when absent, without predicting.

Source: pkg/api/employment.go

Pave Re-mine on Eligibility

When an employment write moves a user from an ineligible to an eligible pay frequency under the pay-frequency gate, their transactions have never reached Pave. The API then calls the Transactions Service re-mine endpoint (POST /{user_id}/transactions/items/{item_id}/remine) for each active item, which republishes the user’s transactions onto the Kinesis stream the feeder consumes.

Condition Re-mine

Condition	Re-mine
Gate flag off for the user	No
New pay frequency not in the allow list	No
Previous pay frequency already in the allow list	No
Previous pay frequency unknown (RDS read error)	No
Otherwise	Yes; metric `insight.pave.remine.triggered` per item

Gate flag off for the user

New pay frequency not in the allow list

Previous pay frequency already in the allow list

Previous pay frequency unknown (RDS read error)

Otherwise

Yes; metric insight.pave.remine.triggered per item

Source: pkg/api/employment.go (remineOnPaveEligible)

Income Verification

A manual verification is a single income_verification row per user, reviewed out of band by Ollie or MX.

Endpoint Behaviour

Endpoint	Behaviour
`POST /{user_id}/insights/employment/verification`	`409 VERIFICATION_IN_PROGRESS` if a `pending` row exists. Otherwise writes the row (`status: pending`, `is_valid: false`, note `PROCESS STARTED`), deletes the user’s RDS employment rows, invalidates the cached payday; `202`
`GET /{user_id}/insights/employment/verification`	Returns the row; `404` when absent
`PUT /{user_id}/insights/employment/verification`	Called by Ollie and the admin API. Sets `status` and `source`, appends the note, sets `is_valid` to whether `status` is `approved`; `404` when absent; `202`

POST /{user_id}/insights/employment/verification

409 VERIFICATION_IN_PROGRESS if a pending row exists. Otherwise writes the row (status: pending, is_valid: false, note PROCESS STARTED), deletes the user’s RDS employment rows, invalidates the cached payday; 202

GET /{user_id}/insights/employment/verification

Returns the row; 404 when absent

PUT /{user_id}/insights/employment/verification

Called by Ollie and the admin API. Sets status and source, appends the note, sets is_valid to whether status is approved; 404 when absent; 202

Source: pkg/api/income.go

Income Detection

POST /{user_id}/insights/employment/detect finds payroll-like transaction groups for onboarding. It always returns 200; failures are encoded in the body (outcome, error_code, low_txns_detected).

Fetch 3 months of main-account transactions from the Transactions Service, retrying on PRODUCT_NOT_READY (backoff 0, 1, 3, 5, 10 s). Failure: outcome: not_found, error_code: NO_TRANSACTIONS, low_txns_detected: true.
low_txns_detected is true when there are fewer than 5 distinct transaction dates.
Run all five methods and write the full result set to site-incomedetection (schema).
Merge: C and D, plus the ML income predictions, plus A if non-empty else B.
Order payroll: the merged results first, then the credits the ML model scored but did not predict as income, each group by descending amount (ties: newest date, then name). Drop credits smaller than MIN_DETECTED_INCOME_AMOUNT, and drop ML-scored credits whose date and amount match a merged result. Empty: not_found with no error_code; any other detector error: error_code: DETECTION_FAILED.

Method Tag Rule

Method	Tag	Rule
A	`name_and_cat`	Credits at or below `MIN_INCOME_AMOUNT` whose Plaid category and name look like payroll
B	`stats`	Credits at or below `MIN_INCOME_AMOUNT`, not a known non-employer; buckets with at least 2 transactions, a date coefficient of variation under 0.15, and a mean gap near a valid pay frequency
C	`name_and_amt`	Credits at or below `MIN_INCOME_METHOD_C`, batched by similar name (Jaro-Winkler at least `MIN_JW_SCORE_METHOD_C`, up to `TXN_BATCH_SIZE_METHOD_C` + 1 per batch); keep batches with at least 2 amounts within `INCOME_MIN_BOUNDS` to `INCOME_MAX_BOUNDS` times the batch mean
D	`gig`	Credits whose name matches a gig-platform list and no exclusion term; no amount floor
ML	`ml`	SageMaker endpoint over all credits; always invoked when there are transactions. The model returns every credit it scored with a per-transaction prediction: credits predicted as income (`1`) are bucketed and merged like any other method; the rest are appended after the merged results (step 5). An endpoint error contributes nothing

name_and_cat

Credits at or below MIN_INCOME_AMOUNT whose Plaid category and name look like payroll

stats

Credits at or below MIN_INCOME_AMOUNT, not a known non-employer; buckets with at least 2 transactions, a date coefficient of variation under 0.15, and a mean gap near a valid pay frequency

name_and_amt

Credits at or below MIN_INCOME_METHOD_C, batched by similar name (Jaro-Winkler at least MIN_JW_SCORE_METHOD_C, up to TXN_BATCH_SIZE_METHOD_C + 1 per batch); keep batches with at least 2 amounts within INCOME_MIN_BOUNDS to INCOME_MAX_BOUNDS times the batch mean

gig

Credits whose name matches a gig-platform list and no exclusion term; no amount floor

ml

SageMaker endpoint over all credits; always invoked when there are transactions. The model returns every credit it scored with a per-transaction prediction: credits predicted as income (1) are bucketed and merged like any other method; the rest are appended after the merged results (step 5). An endpoint error contributes nothing

Source: pkg/employment/detector.go

Transactions are bucketed by cleaned name with Jaro-Winkler similarity above 0.9. Metrics: insight.employment.detection.success tagged success_reason:<tag>, and insight.employment.detection.failure.

API env var	prod	test
`MIN_INCOME_AMOUNT`	-200	-200
`MIN_DETECTED_INCOME_AMOUNT`	-5	-5
`MIN_INCOME_METHOD_C`	-150	-150
`MIN_JW_SCORE_METHOD_C`	0.8	0.8
`TXN_BATCH_SIZE_METHOD_C`	6	12
`INCOME_MIN_BOUNDS`	0.5	0.75
`INCOME_MAX_BOUNDS`	2	1.25

Source: deploy/lambda.tf, deploy/main.tf, deploy/variables.tf

Remediation Triggers (v2)

POST /{user_id}/insights/v2/employment/detect is the onboarding v4 variant. It fetches the same transactions and runs the same detection, but payroll is only the merged results (step 4, ordered and filtered as in step 5); the credits the ML model did not predict as income are not returned. It also returns remediation_triggers, the signals that the connected bank account is unlikely to support a float, in the order below. Window: the 30 days before today; income triggers include today, transaction triggers do not.

Trigger Fires when

Trigger	Fires when
`NO_INCOME`	`payroll` is empty
`LOW_INCOME`	`payroll` is not empty, and no single method’s (A, B, C, D, or the ML income predictions) credits in the window total more than `insight.income_detection.low_income_amount.config`
`NO_TRANSACTIONS_FOUND`	No main-account transactions in the window. Excludes the two triggers below
`NOT_ENOUGH_TRANSACTIONS`	At most `insight.income_detection.low_txn_count.config` transactions in the window
`NOT_ENOUGH_ACTIVITY`	The window’s transactions fall on at most `insight.income_detection.low_activity_days.config` distinct dates

NO_INCOME

payroll is empty

LOW_INCOME

payroll is not empty, and no single method’s (A, B, C, D, or the ML income predictions) credits in the window total more than insight.income_detection.low_income_amount.config

NO_TRANSACTIONS_FOUND

No main-account transactions in the window. Excludes the two triggers below

NOT_ENOUGH_TRANSACTIONS

At most insight.income_detection.low_txn_count.config transactions in the window

NOT_ENOUGH_ACTIVITY

The window’s transactions fall on at most insight.income_detection.low_activity_days.config distinct dates

Failures: error_code: NO_TRANSACTIONS evaluates no triggers; error_code: DETECTION_FAILED evaluates only the transaction triggers. Each request logs income remediation triggers with the outcome and remediation_triggers.

Source: pkg/api/employment.go (DetectEmploymentV2)

Results Log

Each run logs income detection results once, after the response is built. Every transaction appears once, in returned or dropped; the per-method lists and the model’s raw arrays are stored in the site-incomedetection row, not logged.

Field Content

Field	Content
`returned`	The response list in order. Each credit carries `id`, `date`, `amount`, `name`, `methods` (the tags above for every method that found it; empty for credits only the model scored), and the model’s verdict as `ml_prediction` and `ml_probability` (the income-class probability), both absent when the model did not score it
`dropped`	Credits a method found, or the model scored, that are not returned, in the same shape plus `reason`: `below_min_amount`, or `stats_skipped` (found only by B, whose results are merged only when A finds nothing)
`income_count`, `ml_rejected_count`, `dropped_count`	Sizes of the merged group, the appended model-rejected group, and `dropped`
`method_counts`	Transactions each method found, keyed by tag
`ml_prediction_id`	The model invocation’s ID, shared with the `site-incomedetection` row

returned

The response list in order. Each credit carries id, date, amount, name, methods (the tags above for every method that found it; empty for credits only the model scored), and the model’s verdict as ml_prediction and ml_probability (the income-class probability), both absent when the model did not score it

dropped

Credits a method found, or the model scored, that are not returned, in the same shape plus reason: below_min_amount, or stats_skipped (found only by B, whose results are merged only when A finds nothing)

income_count, ml_rejected_count, dropped_count

Sizes of the merged group, the appended model-rejected group, and dropped

method_counts

Transactions each method found, keyed by tag

ml_prediction_id

The model invocation’s ID, shared with the site-incomedetection row

Source: pkg/employment/detector.go (logDetectionResults)

Institution Change

site-insight-institution-change-handler consumes new_bank_account_added ({user_id, account_id}) tapped from EventBridge — see Events. Per message, in order:

Delete the income_verification row.
Delete the user’s RDS employment rows.
Delete the cached payday prediction (failure logged only).

A missing user_id or account_id, or a failure in steps 1 or 2, fails the batch item; the queue’s maxReceiveCount of 1 sends it straight to the DLQ (SQS Queues). Source: pkg/institution-change-handler/handler.go