DynamoDB Tables

The Insight Service reads and writes four DynamoDB tables. The primary table (prod-pave) uses a single-table design and holds all insight, score, alert, and verification entities. The other tables are either owned by the Insight Service or shared cross-service stores.

prod-pave — Primary Insight Table

All entities in this table share the same PK and SK string attributes. The DBType attribute stores the entity discriminator. Key construction uses utils.HashJoinPath(), which joins path segments with #.

Key Construction Reference

Entity PK SK TTL Field

insights (mine metadata)

MINE#<user_id>

TIME#<RFC3339 timestamp>

None

recurring (recurring expenses/income)

USERS#<user_id>

RECURRING#<last_mine_timestamp>#<uuid>

due_date + TTLDaysToAdd days

ritual (ritual expenses)

USERS#<user_id>

RITUAL#<last_mine_timestamp>#<uuid>

date + TTLDaysToAdd days

income (income sources)

USERS#<user_id>

INCOME#<user_id>

None

scores (cash advance scores)

USERS#<user_id>

CASHADVANCESCORE#<RFC3339 timestamp>

ScoreTTL days (default: 30)

balance (monthly savings balances)

USERS#<user_id>

SAVINGS_MONTH#<YYYY-MM>

1 year (365 days)

alert (balance alert state)

USERS#<user_id>

ALERT#<user_id>

None

pave_label (feeder mining trigger)

LABEL#<user_id>

PAVE#REFRESH_INSIGHTS

None

last_page (mining pagination state)

USERS#<user_id>

LASTPAGE#REQUESTID#<pave_request_id>

5 days

cc_income_notification (credit card funds dedup)

CCNOTIFY#<user_id>

CREATED#<created_ts>#EXPIRES#<expires_ts>

BufferTimeInDays days (default: 2)

income_verification (manual income verification)

USERS#<user_id>

VERIFICATION

None

payday (payday prediction result)

PAYDAY#<user_id>

EXPIRES#<next_payday_date>

None

Entity Descriptions

insights (mine metadata)

Written by the feeder for every Pave mining event. Tracks the mining request’s timestamp and request ID. Read by the miner to correlate incoming Pave webhook data.

recurring (recurring expenses/income)

Written in batch by the miner after a successful Pave mining run. Each row is one recurring transaction pattern (expense or income source) returned by Pave. The TTL is computed from the due date of the last recurrence plus a configured buffer (pave_expense_ttl, default 1 day). Old rows expire automatically; new rows are written fresh on each mining run.

ritual (ritual expenses)

Same lifecycle as recurring — written by the miner from Pave’s ritual expense results. A "ritual" in Pave’s model is a discretionary but regular spending pattern (e.g., a weekly coffee shop visit). TTL mirrors the recurring pattern.

income (income sources)

Written by the miner. Stores the raw Pave income source list for the user. Unlike recurring and ritual, there is only one row per user (the SK ends in #<user_id>) so newer writes overwrite the previous row.

scores (cash advance scores)

Written by the miner after calling Pave’s cash-advance score API. Each score row is timestamped; the TTL defaults to 30 days (pave_score_ttl). The API Lambda reads the most recent score for GET /{user_id}/scores/cash_advance without calling Pave directly.

alert (balance alert state)

Written by the balance-alert Lambda after dispatching a notification. Tracks UpdatedDate (timestamp of last alert) and Sent status. Read before sending any alert to enforce the 7-day cooldown.

pave_label (feeder mining trigger)

Written by the feeder to mark that a user’s Pave insight mining has been scheduled. The presence of this row gates mining — the feeder does not re-queue a user who already has an active label. Read by the feeder on each Kinesis event.

last_page (mining pagination state)

Written by the feeder to record that a given Pave request ID represents the last page of an upload batch. TTL is 5 days. When the miner processes a job, it checks whether this entity exists for the {user_id, request_id} pair; if it does, the miner emits the user_new_insights_available EventBridge event.

cc_income_notification (credit card funds dedup)

Written by the funds-notifier after sending a credit card funds push notification. Prevents duplicate notifications within the buffer window. Queried by PK prefix to check if a notification was already sent recently. TTL defaults to 2 days (cc_notification_buffer_days).

income_verification (manual income verification)

Written by the income verification flow (manual entry + optional MX/Ollie provider augmentation). Deleted by the institution-change-handler when a user connects a new bank. Stores employer name, salary, pay frequency, and start date.

payday (payday prediction result)

Written by the payday predictor during float creation (when loan_id is non-empty). Stores the winning prediction and all collection date variants (instant, standard, extended) from both the Pave and FM Legacy algorithms. The SK encodes the predicted next payday date for ordering — records with past dates are filtered out at query time (the query uses SK >= EXPIRES#<today>) but no TTL is set; records are not auto-deleted and are superseded by the latest write for a given user.

prod-incomedetection — Income Detection Results

Field Value

Table name

prod-incomedetection ({environment}-incomedetection)

PK

INCOMEDETECTION#<user_id>

SK

<run_date> (date string)

TTL

7 days

Written by the API Lambda after each income detection run. Stores the transaction set and detection method results for offline analysis. One record per user per day; older records expire after 7 days.

fmdatacapture — Cross-Service Data Capture

Field Value

Table name

prod-fmdatacapture ({environment}-fmdatacapture)

Owner

Shared cross-service table (Insight Service is a producer, not the owner)

The Insight Service writes several data-capture record types to this table:

Event Name SK Pattern Description

CASHFLOW_ANALYTICS

CASHFLOW_ANALYTICS#{date}

FloatMe-generated forecasts written alongside Pave results for comparison. TTL: 2 days.

PAYDAY_PREDICTION

PAYDAY_PREDICTION#{date}

Full payday prediction payload: all three algorithm results, all errors, loan_id, prediction date. Written during float creation. TTL: 2 days.

FM_PAYDAY_FILTERED_TXNS

FM_PAYDAY_FILTERED_TXNS#{date}

List of payroll-candidate transactions used by the V2 payday algorithm. One record per user per day. TTL: 2 days.

FM_PAYDAY_PREDICTED_PAYDAY

FM_PAYDAY_PREDICTED_PAYDAY#{date}

V2 predicted payday date and cadence. One record per user per day. TTL: 2 days.

Legacy Tables (us-east-1)

These tables are owned by other services and accessed read-only (or with minimal writes) by the Insight Service.

Table Name Region Used By

user-balance-settings

us-east-1

Balance-alert Lambda reads each user’s configured low-balance threshold (LowBalanceAlert attribute).

locks

us-east-1

Balance-alert Lambda acquires a per-user distributed lock before processing each account event, to prevent duplicate alerts in concurrent SQS batches.

Related Pages