Income Detection

The Insight Service provides two complementary income detection signals:

Real-time stream signal — the prod-insight-income-signaller Lambda watches the FloatMe transaction Kinesis stream and emits an income_txn event to EventBridge when it sees a qualifying transaction. This is fast (sub-second) but coarse: it only confirms that a deposit hit a user’s account today.
On-demand ML classification — the API Lambda’s POST /{user_id}/insights/employment/detect endpoint runs a user’s recent Plaid transactions through five detection methods (including a SageMaker ML model) and returns structured income source candidates. This is slower but richer, used during the onboarding employment-detection flow.

Income Signaller

Lambda: prod-insight-income-signaller
Trigger: Kinesis stream prod-txn-floatme-transactions (LATEST position)

The income-signaller consumes every transaction event on the FloatMe Kinesis stream. It applies two filters before emitting anything:

Filter	Rule
Amount	Transaction amount must be negative (negative values represent credits to the account in the FloatMe data model).
Date	Transaction date must equal today’s date in the America/Chicago timezone. Transactions dated in the past are silently dropped — using UTC here would cut off 6 hours of potential income detections near midnight CT.

Filter

Rule

Amount

Transaction amount must be negative (negative values represent credits to the account in the FloatMe data model).

Date

Transaction date must equal today’s date in the America/Chicago timezone. Transactions dated in the past are silently dropped — using UTC here would cut off 6 hours of potential income detections near midnight CT.

Transactions that pass both filters are emitted to the EventBridge default bus:

EventBridge Field Value

EventBridge Field	Value
Bus	`default`
Source	`insight-service.income`
Detail-Type	`income_txn`
Payload	The full `Transaction` object from the FloatMe transaction model, including `user_id`, `amount`, `date`, and account details.

Bus

default

Source

insight-service.income

Detail-Type

income_txn

Payload

The full Transaction object from the FloatMe transaction model, including user_id, amount, date, and account details.

Downstream Routing

Two EventBridge rules consume income_txn events from the default bus:

income_txn event
  ├─▶ income_detected_rule
  │     filter: amount < -${cc_notification_limit}
  │     target: prod-income-event-tap SQS
  │     consumer: prod-insight-funds-notifier Lambda
  │
  └─▶ (other consumers outside this service, e.g. Float Service income webhooks)

The income_detected_rule additionally filters on amount — only transactions where amount < -cc_notification_limit (a configurable Terraform variable) reach the funds-notifier. This prevents small incidental credits from triggering card funding notifications.

Funds Notifier

Lambda: prod-insight-funds-notifier
Trigger: SQS (prod-income-event-tap, via income_detected_rule EventBridge rule)

The funds-notifier handles credit card funding notifications for eligible cardholders. Each SQS message is a wrapped EventBridge income_txn event.

Processing Flow

income_event_tap SQS
  └─▶ prod-insight-funds-notifier
        ├─▶ Credit Card Service GET /user/{user_id}
        │     ├─ 404: user not a cardholder → skip
        │     └─ 200: continue
        │           ├─ initialDepositStatus == "completed" or "pending" → skip
        │           └─ continue
        │                 ├─▶ DynamoDB cc_income_notification.CheckAndStore(user_id)
        │                 │     └─ already notified within buffer → skip
        │                 └─▶ Segment SendIt("mvp_deposit_fund_alert", user_id, txn)

Credit Card Eligibility Check

Before sending any notification, the funds-notifier calls the Credit Card Service to verify the user is a cardholder. The check has three possible outcomes:

Response Behaviour

Response	Behaviour
`404 Not Found`	User is not a cardholder. No notification sent; record treated as success.
`200 OK` with `initialDepositStatus == "completed"` or `"pending"`	User’s card funding is already in progress or complete. No notification sent.
`200 OK` with any other `initialDepositStatus`	User is eligible. Proceed to deduplication check.
Any other status	Returns an error; SQS message added to the batch failure list for retry.

404 Not Found

User is not a cardholder. No notification sent; record treated as success.

200 OK with initialDepositStatus == "completed" or "pending"

User’s card funding is already in progress or complete. No notification sent.

200 OK with any other initialDepositStatus

User is eligible. Proceed to deduplication check.

Any other status

Returns an error; SQS message added to the batch failure list for retry.

Deduplication

The cc_income_notification DynamoDB entity is used to prevent sending duplicate notifications to the same user within a buffer window. CheckAndStore writes the entity and returns false (skip) if a record already exists within the window, or true (notify) and stores the record if not.

Error Handling

The Lambda processes SQS records in batch mode with per-message SQSBatchItemFailure responses. Failures during Credit Card Service calls (non-404 error status codes) are retried. Notification send failures (Segment.SendIt) are logged as errors but do not fail the batch item — the notification is considered best-effort once eligibility has been confirmed.

ML-Based Income Detection (API Lambda)

Endpoint: POST /{user_id}/insights/employment/detect
Lambda: prod-insight-api

This endpoint runs a user’s recent Plaid transactions through five detection methods to identify likely income sources. It is used during the employment onboarding flow to pre-populate employer and income data.

Detection Methods

Method Key Description

Method	Key	Description
A — Name + Category	`name_and_cat`	Matches Plaid transaction categories and names against known payroll patterns.
B — Statistical	`stats`	Analyses transaction date and amount statistics to identify regular payroll cadence.
C — Name + Amount	`name_and_amt`	Matches transaction names against income patterns and applies mean-amount thresholds.
D — Gig Income	`gig`	Matches transaction names against a curated list of known gig-economy income sources.
ML — SageMaker Model	`ml`	Invokes the `income-detection-endpoint` SageMaker endpoint with a structured request; returns per-transaction income probability scores.

A — Name + Category

name_and_cat

Matches Plaid transaction categories and names against known payroll patterns.

B — Statistical

stats

Analyses transaction date and amount statistics to identify regular payroll cadence.

C — Name + Amount

name_and_amt

Matches transaction names against income patterns and applies mean-amount thresholds.

D — Gig Income

gig

Matches transaction names against a curated list of known gig-economy income sources.

ML — SageMaker Model

ml

Invokes the income-detection-endpoint SageMaker endpoint with a structured request; returns per-transaction income probability scores.

The response returned to callers is the merged output of Methods C and D (always included). Methods A and B are logged for analysis but are not returned on their own — they are merged into the result only when C and D produce matches (A first, then B if A is empty). The ML method is gated by insight.income_detection.ml.enabled; when enabled, insight.income_detection.ml.percent controls the rollout percentage of users for whom ML results are also merged in. The merge logic is implemented in pkg/employment/detector.go (Detector.Detect).

SageMaker Integration

The income-detection-endpoint SageMaker endpoint receives a structured IncomeDetectionMLRequest and returns an IncomeDetectionMLReponse containing:

PredictionID — unique ID for the inference run
Predictions — per-transaction income/non-income label
Probabilities — per-transaction income probability score

All five method results plus the ML metadata (prediction ID, probabilities, whether ML was enabled and used) are written to the prod-insight-income-detection DynamoDB table for offline analysis regardless of which method’s results are returned to the caller.

GrowthBook Flags

Flag Key Default Effect

Flag Key	Default	Effect
`insight.income_detection.ml.enabled`	`false`	Enables invocation of the SageMaker ML endpoint during detection
`insight.income_detection.ml.percent`	`0`	Percentage of users (0–100) for whom ML results are returned instead of legacy method results when ML is enabled

insight.income_detection.ml.enabled

false

Enables invocation of the SageMaker ML endpoint during detection

insight.income_detection.ml.percent

0

Percentage of users (0–100) for whom ML results are returned instead of legacy method results when ML is enabled

Architecture — System context and Lambda inventory
Event Flows — income_txn EventBridge event routing and downstream consumers
DynamoDB Tables — cc_income_notification and income_detection entity schemas
Feature Flags — Full GrowthBook flag reference