Insight Service

The Insight Service is the authority for financial insights and analytics within the FloatMe platform. It owns the full pipeline from transaction ingestion into the Pave third-party analytics platform through to income detection, payday prediction, cash-advance scoring, and employment tracking — surfacing these as both a REST API and as EventBridge events for downstream service consumption.

What This Service Owns

Domain Responsibility

Insights

Recurring expenses, ritual expenses, income sources, and bank balance snapshots sourced from Pave; cached in DynamoDB and served via the REST API

Pave Mining

Ingestion of Plaid transactions into Pave (feeder), retrieval of processed insights (miner), and webhook handling to coordinate the two (webhook)

Income Detection

ML-based classification of individual transactions via SageMaker (API Lambda)

Forecasts & Payday

Payday prediction and cash-flow forecasting (income, recurring expense, ritual expense) served via the API Lambda; combines Pave data, employment records, and GrowthBook feature flags

Employment

Manual employment records stored in RDS; updated by the API Lambda and cleared by the institution-change-handler when a user’s bank institution changes

Scores

Cash-advance repayment probability scores sourced from Pave; cached in DynamoDB and served via the API Lambda

Notifications

Balance alerts triggered when a new bank account is linked (balance-alert)

Events

Insight lifecycle events published to EventBridge for downstream service consumption (Float Service)

Lambda Functions

Function Trigger Purpose

prod-insight-api

API Gateway (IAM + JWT)

Primary REST API — forecasts, employment CRUD, income detection, payday prediction, cash-advance scores, expenses

prod-insight-feeder

Kinesis (prod-txn-plaid-transactions, LATEST)

Uploads Plaid transactions and balances to Pave; tracks per-user pagination state in DynamoDB

prod-insight-webhook

API Gateway (Pave webhook, HMAC-verified)

Receives Pave webhooks; enqueues USER_DATA_INSIGHTS_READY events to the miner SQS queue

prod-insight-miner

SQS (prod-insight-miner)

Fetches processed insights from Pave, writes all entity types to DynamoDB, emits user_new_insights_available to EventBridge

prod-insight-replay-feeder

SQS (prod-insight-replay-feeder)

Manual replay path — refetches Pave data with custom parameters and updates pagination state

prod-insight-balance-alert

SQS (prod-insight-balance-alert-event-tap, via EventBridge rule)

Sends balance alerts when a new bank account is linked; validates eligibility via the Underwriting Service

prod-insight-institution-change-handler

SQS (prod-insight-institution-change-handler-sqs-tap, via EventBridge rule)

Clears stale employment records from RDS when a user’s bank institution changes

External Dependencies

Service Role

Pave

Third-party financial analytics provider; receives transaction uploads from the feeder and returns processed insights (expenses, income, scores) to the miner and API

Transactions Service

Source of the prod-txn-plaid-transactions Kinesis stream; also queried directly by the miner and API for account and balance data

User Service

User profiles and identity data; queried by the API Lambda

Underwriting Service

Eligibility validation; queried by the API Lambda and the balance-alert Lambda

SageMaker

ML endpoint (income-detection-endpoint) for classifying individual transactions as income; invoked by the API Lambda on employment detect requests

GrowthBook

Feature flags controlling income detection method, payday prediction behaviour, and transaction batch sizes; fetched at Lambda startup

Segment

User analytics and push notification delivery; called by balance-alert

EventBridge (default bus)

Receives user_new_insights_available events from this service; routes events from txn-service to balance-alert and institution-change-handler

Documentation

  • Architecture — System context diagrams, Lambda functions, data storage, and integration overview

  • Pave Mining — Feeder, webhook, miner, and replay-feeder flows with sequence diagram

  • Income Detection — SageMaker ML income detection pipeline

  • Balance Alerts — Balance-alert Lambda trigger, eligibility check, and notification flow

  • Institution Changes — Institution-change-handler trigger and employment record invalidation

  • Forecasts & Payday — Payday prediction, cash-flow forecasting, and data-capture writes

  • Scores — Cash-advance repayment scores from Pave

  • Event Flows — EventBridge events produced and consumed, rules and routing

  • DynamoDB Tables — Single-table design, entity types, access patterns

  • PostgreSQL Schema — Employment table columns and access patterns

  • Infrastructure — Lambda configs, SQS queues, DynamoDB tables, Terraform layout

  • Feature Flags — GrowthBook feature flag reference

  • API Specification — Full OpenAPI spec (Swagger UI)