DynamoDB Tables

The Insight Service reads and writes four DynamoDB tables. The primary table (prod-pave) uses a single-table design and holds all insight, score, alert, and verification entities. The other tables are either owned by the Insight Service or shared cross-service stores.

prod-pave — Primary Insight Table

All entities in this table share the same PK and SK string attributes. The DBType attribute stores the entity discriminator. Key construction uses utils.HashJoinPath(), which joins path segments with #.

Key Construction Reference

Entity PK SK TTL Field

insights (mine metadata)

MINE#<user_id>

TIME#<RFC3339 timestamp>

None

recurring (recurring expenses/income)

USERS#<user_id>

RECURRING#<last_mine_timestamp>#<uuid>

due_date + TTLDaysToAdd days

ritual (ritual expenses)

USERS#<user_id>

RITUAL#<last_mine_timestamp>#<uuid>

date + TTLDaysToAdd days

income (income sources)

USERS#<user_id>

INCOME#<user_id>

None

scores (cash advance scores)

USERS#<user_id>

CASHADVANCESCORE#<RFC3339 timestamp>

ScoreTTL days (default: 30)

balance (monthly savings balances)

USERS#<user_id>

SAVINGS_MONTH#<YYYY-MM>

1 year (365 days)

alert (balance alert state)

USERS#<user_id>

ALERT#<user_id>

None

pave_label (feeder mining trigger)

LABEL#<user_id>

PAVE#REFRESH_INSIGHTS

None

last_page (mining pagination state)

USERS#<user_id>

LASTPAGE#REQUESTID#<pave_request_id>

5 days

income_verification (manual income verification)

USERS#<user_id>

VERIFICATION

None

payday (payday prediction result)

PAYDAY#<user_id>

PREDICTION

2 days

Entity Descriptions

insights (mine metadata)

Written by the feeder for every Pave mining event. Tracks the mining request’s timestamp and request ID. Read by the miner to correlate incoming Pave webhook data.

recurring (recurring expenses/income)

Written in batch by the miner after a successful Pave mining run. Each row is one recurring transaction pattern (expense or income source) returned by Pave. The TTL is computed from the due date of the last recurrence plus a configured buffer (pave_expense_ttl, default 1 day). Old rows expire automatically; new rows are written fresh on each mining run.

ritual (ritual expenses)

Same lifecycle as recurring — written by the miner from Pave’s ritual expense results. A "ritual" in Pave’s model is a discretionary but regular spending pattern (e.g., a weekly coffee shop visit). TTL mirrors the recurring pattern.

income (income sources)

Written by the miner. Stores the raw Pave income source list for the user. Unlike recurring and ritual, there is only one row per user (the SK ends in #<user_id>) so newer writes overwrite the previous row.

scores (cash advance scores)

Written by the miner after calling Pave’s cash-advance score API. Each score row is timestamped; the TTL defaults to 30 days (pave_score_ttl). The API Lambda reads the most recent score for GET /{user_id}/scores/cash_advance without calling Pave directly.

alert (balance alert state)

Written by the balance-alert Lambda after dispatching a notification. Tracks UpdatedDate (timestamp of last alert) and Sent status. Read before sending any alert to enforce the 7-day cooldown.

pave_label (feeder mining trigger)

Written by the feeder to mark that a user’s Pave insight mining has been scheduled. The presence of this row gates mining — the feeder does not re-queue a user who already has an active label. Read by the feeder on each Kinesis event.

last_page (mining pagination state)

Written by the feeder to record that a given Pave request ID represents the last page of an upload batch. TTL is 5 days. When the miner processes a job, it checks whether this entity exists for the {user_id, request_id} pair; if it does, the miner emits the user_new_insights_available EventBridge event.

income_verification (manual income verification)

Written by the income verification flow (manual entry + optional MX/Ollie provider augmentation). Deleted by the institution-change-handler when a user connects a new bank. Stores employer name, salary, pay frequency, and start date.

payday (payday prediction result)

The same-day payday prediction cache, keyed PAYDAY#<user_id> / PREDICTION: one row per user, overwritten in place. Only the day’s first request writes it — every later request that day is a single GetItem and writes nothing. It stores the prediction that was served plus whatever each algorithm produced (pave, fm, v2), so a cache hit can report the full picture. pave and v2 are both nil-able, because the chain short-circuits.

prediction_date is the validity field — a row may only be served while it matches the current UTC day. The 2-day TTL is garbage collection only and is never the validity mechanism, since DynamoDB deletes expired items up to 48 hours late. Rows are deleted outright whenever the user’s employment record changes. See Forecasts & Payday.

prod-incomedetection — Income Detection Results

Field Value

Table name

prod-incomedetection ({environment}-incomedetection)

PK

INCOMEDETECTION#<user_id>

SK

<run_date> (date string)

TTL

7 days

Written by the API Lambda after each income detection run. Stores the transaction set and detection method results for offline analysis. One record per user per day; older records expire after 7 days.

fmdatacapture — Cross-Service Data Capture

Field Value

Table name

prod-fmdatacapture ({environment}-fmdatacapture)

Owner

Shared cross-service table (Insight Service is a producer, not the owner)

The Insight Service writes several data-capture record types to this table:

Event Name SK Pattern Description

CASHFLOW_ANALYTICS

CASHFLOW_ANALYTICS#{date}

FloatMe-generated forecasts written alongside Pave results for comparison. TTL: 2 days.

PAYDAY_PREDICTION

PAYDAY_PREDICTION#{date}

Full payday prediction payload: every algorithm result, all errors, loan_id, prediction date, which algorithm won (detection_used), and whether it was served from cache (from_cache). Written on every evaluation. TTL: 2 days.

FM_PAYDAY_FILTERED_TXNS

FM_PAYDAY_FILTERED_TXNS#{date}

List of payroll-candidate transactions used by the V2 payday algorithm. One record per user per day. TTL: 2 days.

FM_PAYDAY_PREDICTED_PAYDAY

FM_PAYDAY_PREDICTED_PAYDAY#{date}

V2 predicted payday date and cadence. One record per user per day. TTL: 2 days.

Legacy Tables (us-east-1)

These tables are owned by other services and accessed read-only (or with minimal writes) by the Insight Service.

Table Name Region Used By

user-balance-settings

us-east-1

Balance-alert Lambda reads each user’s configured low-balance threshold (LowBalanceAlert attribute).

locks

us-east-1

Balance-alert Lambda acquires a per-user distributed lock before processing each account event, to prevent duplicate alerts in concurrent SQS batches.

Related Pages