Event Flows

The Float Service participates in the FloatMe event-driven architecture in three ways: it publishes lifecycle events to EventBridge for downstream consumers, it consumes scheduled EventBridge rules to trigger collection and prenote runs, and it reacts to external signals (balance events) via SQS.

Overview

                        Float Service
                              │
      ┌───────────────────────┼───────────────────────┐
      │                       │                       │
      ▼                       ▼                       ▼
 Publishes               Consumes               Consumes
 EventBridge         scheduled rules           SQS queues
 events (outbound)   (inbound triggers)    (inbound signals)
      │                       │                       │
      ▼                       ▼                       ▼
Downstream         Collections Scheduler   Balance Worker
services           invoked on schedule     reacts to balance
consume            to queue collection     events in real time
user_float_created runs to SQS

Events Published

user_float_created

Published to EventBridge by the API Lambda immediately after a float record is successfully written to RDS.

Field Value

Event source

float-service.api

Event type

user_float_created

Payload

Full float object: id, user_id, type, amount, fee, debit_status, debit_date, credit_id, evaluation_id, created_date, is_custom_payback_date, default_payback_date

Trigger

POST /{user_id}/floats — after RDS write succeeds, before response is returned

Downstream services (e.g. underwriting, analytics) subscribe to this event via their own EventBridge rules. The Float Service does not manage those subscriptions.

Scheduled Collections

Two CloudWatch EventBridge rules trigger the prod-floats-collections-scheduler Lambda on weekdays. The scheduler queries RDS for qualifying floats at each stage and enqueues them to the prod-floats-collections SQS queue for the Collections Worker to process.

EventBridge Rules

Rule Schedule (UTC) Input Effect

float-collections

9:30 AM Mon–Fri

{"time": 6}

Triggers the Due Date (TODAY6AM) run. Queries for SCHEDULING floats whose due date is today.

float-collections-retry

8:30 AM Mon–Fri

{"time": 5}

Triggers the Daily Retry run. Queries for floats in RETRY, FAILED, ACHFAILED, and UNCOLLECTABLE statuses whose due date has passed.

Scheduler → Worker Flow

EventBridge rule fires
        │
        ▼
Collections Scheduler Lambda invoked
        │
        ▼
Query RDS for qualifying floats
  TODAY6AM: status=SCHEDULING, due date <= today
  RETRY:    status IN (RETRY, FAILED, ACHFAILED, UNCOLLECTABLE), due date passed
        │
        ▼
For each qualifying float:
  Enqueue Collection{float, processType} to SQS
        │
        ▼
Collections Worker Lambda dequeues and processes
(see xref:collections.adoc[Collections Engine] for decision logic)

Day-Before-ACH Fargate Job

The site-floats-day-before-ach Fargate task runs as a long-lived batch job rather than a Lambda. It is triggered by EventBridge Scheduler (not a CloudWatch rule) at 19:00 ET on weekdays — approximately two hours before the Usio ACH cutoff.

EventBridge Scheduler

Schedule Cron (ET) Effect

site-floats-day-before-ach

cron(0 19 ? * MON-FRI *) in America/New_York

Calls ecs:RunTask on the site-floats cluster with the site-floats-day-before-ach task definition. flexible_time_window.mode = "OFF". retry_policy.maximum_retry_attempts = 0 — ACH-adjacent workflows must not retry silently; missed fires require operator follow-up.

Flow

EventBridge Scheduler fires (weekdays 19:00 ET)
        │
        ▼
day-before-ach Fargate task starts
        │
        ▼
Page SCHEDULING floats due on next business day (Friday → Monday)
        │
        ▼
For each float (worker pool, rate-limited):
  Acquire per-user billing lock (DynamoDB)
        │
        ▼
  Run rule set:
    1. ruleFloatStillCollectable — re-fetch from RDS; skip if ACHSENT/COMPLETED
    2. ruleMaxACHAttempts — skip if ACH attempt limit reached
    3. ruleAchIfInvalidDebitCard — route to ACH if no valid primary debit card
        │
        ├── No rule routed → ActionNone (leave in SCHEDULING for Due Date pinless run)
        │
        └── ActionSubmitACHNextDay
              │
              ▼
        Submit next-day ACH to Payments Service
        Update float status (ACHSENT or RETRY)
        Write collection log to DynamoDB (Process: TOMORROW)

Webhook Worker

The prod-floats-webhook-worker Lambda is the webhook-triggered collection path, consuming several EventBridge-tapped triggers from the Transactions Service (one SQS tap per trigger, dispatched by detail-type). On a new_account balance update event (source txn-service.feeder) it applies a more conservative balance check before attempting collection. On a plaid_item_connection_lost / plaid_item_connection_revoked event (source txn-service.miner) it makes a pinless-only attempt on the user’s overdue float while the debit card is still usable (gated by floats.collections.plaid_disconnected.rollout).

Event Path

Transactions Service publishes balance update event
  (filter: new_account events from txn-service.feeder)
        │
        ▼
EventBridge rule: balance-detected-rule
        │
        ▼
SQS: prod-floats-balance-event-tap
        │
        ▼
Webhook Worker Lambda invoked (detail-type = new_account)
        │
        ▼
Check user has float in RETRY status
Enforce daily attempt cap, ACH attempt cap, and balance threshold
  (balance > float fee + float amount + $20 buffer, configurable)
Attempt collection (see xref:collections.adoc#balance-update[Balance Update])

The same Lambda also consumes the Plaid connection lost/revoked events:

Transactions Service (miner) publishes plaid_item_connection_lost /
  plaid_item_connection_revoked (source: txn-service.miner)
        │
        ▼
EventBridge rule: {env}-floats-plaid-disconnect
        │
        ▼
SQS: prod-floats-plaid-disconnect-event-tap
        │
        ▼
Webhook Worker Lambda invoked (detail-type = plaid_item_connection_*)
        │
        ▼
Gate on floats.collections.plaid_disconnected.rollout for the user
Pinless-only attempt on the user's overdue float, no ACH fallback
  (see xref:collections.adoc#plaid-connection-lost-revoked[Plaid Connection Lost / Revoked])

GrowthBook Flags

Flag Effect

floats.webhook.balance.buffer

Balance buffer applied on top of fee + amount before attempting collection. Default: $20.

floats.pinless.institutions

Array of institution IDs for which pinless debit routing is supported in the balance worker.

floats.collections.plaid_disconnected.rollout

Per-user gate for the Plaid connection lost/revoked path. When enabled, a plaid_item_connection_lost / plaid_item_connection_revoked event triggers a pinless-only collection attempt on the user’s overdue float. Off = the worker parses the event and no-ops.

Insights Worker (Next-Payday Refresh)

The prod-floats-insights-worker Lambda reacts to user_new_txns_batch_completed events from the Transactions Service (published by the refiner once a Plaid transaction batch has been refined for a user — only when the batch contained new transactions and it is the last page). For a user with an overdue float it refreshes the cached next_payday_date, keeping the payday-ACH job’s candidate selection accurate between collection attempts.

It does not attempt any collection or move money — it only updates the cached payday. It is additive to the payday-ACH job’s own per-attempt refresh (refreshNextPayday), not a replacement.

Event Path

Transactions Service publishes user_new_txns_batch_completed (source: txn-service.refiner)
        │
        ▼
EventBridge rule: {env}-floats-insights-available
        │
        ▼
SQS: {env}-floats-insights-event-tap  (30s delivery delay)
        │
        ▼
Insights Worker Lambda invoked
  Event payload: bare user id string (fmsdk Event in EventBridge detail)
        │
        ▼
Load user's overdue floats (RETRY / FAILED / ACHFAILED / UNCOLLECTABLE)
  none ──► no-op (no insight call)
        │
        ▼
GrowthBook flag floats.collections.payday_ach.rollout for user?
  No  ──► skip (not in the payday-ACH program; no insight call)
  Yes ──► call insight-service GetNextPayday
        │
        ▼
Per float:
  cached next_payday_date within 2 days of today? ──► leave unchanged (freeze window)
        │ (otherwise)
        ▼
  FirstPaydayAfter(float's due date)?   (strictly after ach_debit_date)
  nil (no payday after due date) ──► leave that float's next_payday_date unchanged
  date                           ──► UpdateNextPaydayDate on that float
Delivery delay. The tap queue sets delay_seconds = 30, so each event stays invisible for 30 seconds before the Lambda can receive it. The batch-completed event is published as soon as the refiner has stored the new transactions, which is ahead of the payday prediction seeing them; the delay gives that a head start. No Lambda invocation happens during the wait. Because the tap’s maxReceiveCount is 1 there is no second look — if the prediction is still working off pre-batch data, the worker caches a slightly older prediction and the next completed batch re-triggers the refresh.
The next payday is computed per float, strictly after that float’s due date (ach_debit_date), not after "today". This is a safety measure: if the worker runs on a float’s due date after a failed initial attempt, insight can return that same day as the next payday — caching it would make the payday-ACH job target the due date itself. FirstPaydayAfter excludes the reference date, so the cached value is always a later collection opportunity.
Freeze window. When a float already has a cached next_payday_date that falls within 2 calendar days of "today" ([today, today+2]), the worker leaves it unchanged rather than refreshing it. Close to a predicted payday, insight-service tends to roll the prediction forward to the following payday; overwriting then would risk pushing next_payday_date further out — past the attempt the payday-ACH job is already lined up for. A cached date already in the past is treated as stale (not imminent) and is still refreshed.
Reads (finding overdue floats) hit the RDS replica; the next_payday_date write targets the RDS main pool. next_payday_date writes are idempotent, so a record that fails mid-batch is returned as a batch-item failure and safely retried. See Collections Engine for how the payday-ACH job consumes next_payday_date.

Payday-ACH Rollout Observability

Metrics for monitoring the payday-ACH rollout. Note the two prefixes: Lambda-emitted custom metrics (via ddlambda) are float.collections. (singular); the payday-ACH Fargate job emits DogStatsD counters as floats.collections. (plural).

Metric Source Key tags / meaning

float.collections.next_payday_refreshed

insights-worker (Lambda)

Count of floats whose next_payday_date was refreshed from a user_new_txns_batch_completed event. One per float actually updated.

floats.collections.attempt

payday-ach job (Fargate)

The dispatch funnel, sliced by outcome: — flag_disabled (enabled flag off, no-op), dry_run (enabled on, rollout off — no money), achsent/failed/… (rollout on, real ACH). Also tagged rollout:true|false, attempt_reason:, job:retry-payday-ach.

floats.collections.check

payday-ach job (Fargate)

Per-float rule-chain result: passed:true|false, skip_reason: (e.g. max_ach_attempts_reached).

floats.collections.job.started / .completed / .duration_seconds / .floats_total

payday-ach job (Fargate)

Run-level: started/completed (tag status:success|failure), wall-clock duration, and candidate count. Filter by job:retry-payday-ach.

float.collections.payment with outcome:ach_suppressed_payday

legacy collectors (Lambda)

An overdue ACH was skipped because payday-ACH owns the user’s ACH budget. Sliced by process: — daily (retry), webhook_balance (balance).

Useful log queries (service:<application>): "refreshed next_payday_date for overdue floats" (insights-worker success), "payday-ach dry run" (rollout-off attempts), "skipping overdue ACH; payday-ACH rollout owns" and "skipping balance-webhook ACH; payday-ACH rollout owns" (suppression).

Prenote Scheduling

The prod-floats-prenote-scheduler Lambda runs on its own EventBridge schedule, independent of the collections schedule. It submits zero-dollar ACH prenotes ahead of the due date to validate account numbers with the processor before real collection debits are submitted.

EventBridge Rule

Rule Schedule (UTC) Effect

float-prenote-scheduler

10:00 PM daily

Fires two hours before midnight UTC. Triggers prod-floats-prenote-scheduler to enqueue prenote submissions for floats due in 5 calendar days.

Scheduler → Worker Flow

EventBridge rule fires (daily, 22:00 UTC)
        │
        ▼
Prenote Scheduler Lambda invoked
        │
        ▼
Determine target due date
  run_time + 5 calendar days
        │
        ▼
Query RDS replica: SCHEDULING floats due on target date
        │
        ▼
For each float, evaluate GrowthBook flag floats.prenotes for user_id
  Keep if flag is enabled for the user (default false)
        │
        ▼
SendMessageBatch ──► SQS: prod-floats-prenotes
        │
        ▼
Prenote Worker Lambda dequeues (concurrency capped at 3)
  Fetch user from User Service
  Call Payments.SubmitUsioPrenote (UsioAccount: float-debit)
  Batch-item failure on error (SQS retries up to maxReceiveCount)
The prenote path is decoupled from collection state. It does not acquire the per-user collection lock, write to collection-history, or change float status. Returns (e.g. closed accounts) are tracked by the Payments Service.

Reporter

The prod-floats-reporter Lambda generates a daily float origination summary and posts it to Slack.

Flow

EventBridge rule fires (daily, 12:00 UTC)
        │
        ▼
Reporter Lambda invoked
        │
        ▼
Query RDS read replica
  Group floats by date and loan type (ACH, PINLESS, RTP)
  Aggregate counts and amounts
  Window: last 10 days
        │
        ▼
Format plain-text table:
  Date (Mon YYYY-MM-DD) | ACH total | Pinless total | RTP total
        │
        ▼
Post to Slack channel: #feed-fm-collections
The reporter is enabled in production only. It is disabled in non-production environments via its EventBridge rule schedule.

Related Pages

  • Collections Engine — Full decision logic for each collection stage

  • ACH Processing — Kinesis-based ACH settlement callbacks

  • Float Lifecycle — When user_float_created is published relative to float creation

  • Architecture — System context diagram showing all inbound and outbound event paths