Collections Engine
The collections engine attempts to recover outstanding float balances through two parallel paths: a scheduled path that runs on a fixed cadence, and a webhook-driven path that responds to real-time income and balance events. A separate prenote path runs ahead of the due date to validate account numbers with the ACH processor before collection begins.
Overview
Scheduled Path (Lambda)
EventBridge (CloudWatch rules)
│
▼
Collections Scheduler ──► SQS: float-collections ──► Collections Worker ──┐
│
Scheduled Path (Fargate) ▼
EventBridge Scheduler (weekdays 19:00 ET) Payments Service
│ │
▼ │
day-before-ach Fargate task ───────────────────────────────────────────► │
│
Webhook Path │
SQS: income-event-tap ──► Webhook Worker ─────────────────────────────► │
SQS: balance-event-tap ──► Balance Worker ─────────────────────────────► │
│
Prenote Path │
EventBridge (daily 22:00 UTC) │
│ │
▼ │
Prenote Scheduler ──────► SQS: prenotes ──► Prenote Worker ──────────────►│
│
ACH Callback Path ▼
Kinesis: prod-payments ──► ACH Handler Float status updated
DynamoDB attempt logged
| The Lambda scheduled path (Collections Scheduler → Collections Worker) handles Due Date and Daily Retry only. T-1 Day ACH is handled exclusively by the Fargate batch job. |
The Collections Scheduler runs on a CloudWatch schedule and queries RDS for floats due at each stage. It enqueues them to SQS for the Collections Worker to process. The Webhook and Balance Workers react to external events without polling. The Prenote Scheduler runs on its own daily schedule and enqueues zero-dollar prenote submissions for the subset of users whose floats are due in 5 calendar days and whose floats.prenotes GrowthBook flag is enabled.
T-1 Day Collection
The day before a float’s due date, the day-before-ach Fargate batch job (pkg/collections/jobs) runs at 19:00 ET — approximately two hours before the Usio ACH cutoff — and re-fetches each float after acquiring the billing lock so stale producer state cannot cause a double-collection.
The Fargate job runs four rules per float in order:
-
ruleFloatStillCollectable— re-fetches the float from RDS; skips if alreadyACHSENTorCOMPLETED(guards the window between producer query and worker dispatch). -
ruleMaxACHAttempts— skips if the float has already reached the configured ACH attempt limit (excludes manual collection logs). -
ruleAchIfInvalidDebitCard— routes toActionSubmitACHNextDayif the user has no valid primary debit card; otherwise leaves the float inSCHEDULINGfor the Due Date pinless run. -
ruleACHIfFirstFloat— routes toActionSubmitACHNextDayfor users with float rank 0 (no prior completed floats) when the GrowthBook flagfloats.collections.day_before.first_float_achis enabled for them. The flag is a percentage rollout controlled in GrowthBook without a deploy.
If the user has no valid primary debit card, next-day ACH is submitted immediately. If they do have a valid card, the system checks float rank: rank-0 users (no prior completed floats) are routed to ACH when the floats.collections.day_before.first_float_ach GrowthBook flag is enabled for them; everyone else waits for the higher-priority pinless debit on the due date itself.
Float due date is tomorrow
│
▼
Valid primary debit card? ──No──────────────────────────────────────► Submit ACH
│ Yes │
▼ │
Float rank 0 (no prior completed floats)? │
│ │
├── No ──► No action — wait for Due Date Collection │
│ │
└── Yes │
│ │
▼ │
floats.collections.day_before.first_float_ach enabled? │
│ │
├── No ──► No action — wait for Due Date Collection │
│ │
└── Yes ────────────────────────────────────────────────────┤
▼
ACH accepted? ──Yes──► Mark float ACHSENT
│ No (awaits settlement)
▼
Leave float SCHEDULING
(log attempt; due-date
pinless run still fires)
Due Date Collection
On the float’s due date, the scheduler queries for floats in SCHEDULING status whose due date is today or earlier. Pinless debit is the preferred method; ACH is the fallback when the pinless attempt fails with an NSF error code.
Float is due today
│
▼
Valid primary debit card? ──No──────────────────────────────► Submit ACH
│ Yes │
▼ │
Submit pinless debit │
│ │
Payment successful? ──Yes──► Mark float COMPLETED │
│ No │
▼ ▼
NSF error? (code 62 or 05) ──Yes──► Submit ACH ACH accepted? ──Yes──► Mark float ACHSENT
│ No │ No (awaits settlement)
▼ ▼
Mark float RETRY ◄─────────────────────────────────────── ACH rejected
Daily Retry
Every morning the scheduler queries for floats in RETRY, FAILED, UNCOLLECTABLE, and ACHFAILED statuses whose due date has passed. Exit conditions are checked first; floats that pass are attempted using the same pinless-then-ACH routing as the Due Date stage.
Float is in RETRY (or similar) status
│
▼
ACH attempts ≥ configured limit? ──Yes──────────────────────► Mark float DEFAULTED
│ No
▼
Days since due date > 90? ──Yes─────────────────────────────► Mark float DEFAULTED
│ No
▼
Valid Plaid account (can check balance)?
│
├── No ──► Valid primary debit card?
│ │
│ ├── No ──► Mark float UNCOLLECTABLE
│ │ (re-evaluated next Daily Retry)
│ └── Yes ──► Do nothing, check again tomorrow
│
└── Yes
│
▼
Balance > float amount + $10?
│
├── No ──► Do nothing, check again tomorrow
│
└── Yes
│
▼
Valid primary debit card?
│
├── Yes ──► Submit pinless debit
│ │
│ Payment successful? ──Yes──► Mark float COMPLETED
│ │ No
│ ▼
│ NSF error? ──Yes──► Submit ACH ──► (see below)
│ │ No
│ └──────────► Mark float RETRY
│
└── No ──► Submit ACH
│
ACH accepted? ──Yes──► Mark float ACHSENT
│ No
└──────────► Mark float RETRY
UNCOLLECTABLE is not a terminal status. Floats in this state are re-queued by the Daily Retry scheduler on subsequent runs. If a valid debit card or Plaid account is later associated with the user, collection will be reattempted.
|
Webhook-Triggered Collection
Two event-driven workers respond to real-time signals without waiting for the next scheduled run.
Income Detection
The webhook-worker is triggered when an income event is received from the Insight Service. It targets floats already in RETRY status, subject to daily attempt limits.
Income detected for user
│
▼
User has a float in RETRY status? ──No──► Ignore
│ Yes
▼
ACH attempts ≥ limit? ──Yes──► Mark float DEFAULTED
│ No
▼
Already attempted ≥ 3 times today? ──Yes──► Ignore
│ No
▼
Cached balance ≥ $50? ──No──► No action (float stays RETRY)
│ Yes
▼
Valid primary debit card?
│
├── Yes ──► Submit pinless debit
│ │
│ Payment successful? ──Yes──► Mark float COMPLETED
│ │ No
│ └──────────► Mark float RETRY
│
└── No ──► Submit ACH
│
ACH accepted? ──Yes──► Mark float ACHSENT
│ No
└──────────► Mark float RETRY
Balance Update
The webhook-worker-balance is triggered when a balance update event arrives from the Transactions Service. It applies a more conservative balance check and enforces a per-day attempt cap.
Balance update event received for user
│
▼
Balance-based collection enabled for user? ──No──► Ignore
(GrowthBook feature flag)
│ Yes
▼
User has a float in RETRY status? ──No──► Ignore
│ Yes
▼
Attempts today ≥ 3? ──Yes──► Ignore
│ No
▼
ACH attempts ≥ limit? ──Yes──► Ignore
│ No
▼
Balance > float fee + float amount + $20? ──No──► No action
│ Yes
▼
Valid primary debit card?
│
├── Yes ──► Submit pinless debit
│ │
│ Payment successful? ──Yes──► Mark float COMPLETED
│ │ No
│ └──────────► Mark float RETRY
│
└── No ──► Submit ACH (if institution allows ACH)
│
ACH accepted? ──Yes──► Mark float ACHSENT
│ No
└──────────► Mark float RETRY
Both the balance threshold ($20 buffer) and the max ACH/daily attempt limits for the balance worker are configurable via GrowthBook feature flags (floats.webhook.balance.buffer, floats.collections.max_ach_attempts, floats.collections.max_attempts_on_day).
|
ACH Prenotes
Prenotes are zero-dollar ACH verification transactions submitted to the user’s bank 5 calendar days before a float’s due date. They give the ACH processor a chance to validate the account number and surface returns (e.g. closed accounts) before the actual collection debit, reducing NSF and return rates when the Due Date and Daily Retry stages later submit real debits against the same account.
The prenote path is fully decoupled from the collection path — it is driven by its own EventBridge schedule, its own SQS queue, and a dedicated scheduler/worker pair. It does not change float status, acquire the per-user collection lock, or write to the collection-history table.
Prenote Waiting Period
Per the payment processor, the wait before a live (real-money) debit may follow a prenote is not a rolling 72 hours measured from the submission timestamp. The clock starts at 00:00 UTC on the calendar date the prenote was submitted, three full days must then elapse, and the live transaction may not be submitted until the day after that.
Worked example from the processor: a prenote submitted on 5/25 cannot be followed by a live transaction until 5/29 — i.e. the earliest eligible live-debit date is the submission date + 4 calendar days.
This is why the scheduler prenotes 5 calendar days ahead of the due date (see below). A float due on D is prenoted on D − 5; that submission becomes eligible for a live debit on D − 1, leaving a one-day buffer before the Due Date collection runs against the same account.
|
The processor measures the waiting period in whole UTC calendar days, so the submission time of day does not matter — only the date. The earliest live-debit date is unaffected by whether a prenote is submitted at 00:01 UTC or 23:59 UTC on a given day. Per the processor, this waiting period is also not affected by bank holidays or weekends — every UTC calendar day counts toward the three-day wait, including Saturdays, Sundays, and holidays. |
|
This waiting period is described by the processor as a NACHA-compliance requirement, but the mechanics above are the processor’s own implementation, not a verbatim NACHA rule. NACHA’s underlying rule lets an Originator initiate live entries as soon as the third banking day following the prenote’s settlement date (with no Return or NOC received). That differs from the processor’s description on three points:
Other processors may apply the banking-day rule directly, so the exact eligible date can differ between processors for the same prenote. Our 5-calendar-day lead time clears both interpretations comfortably. The rule is stated verbatim in the 2017 Federal Register amendment that adopted it for federal ACH participation, Federal Register, FR Doc. 2017-19135 (Sep 11 2017): "This change permits an Originator that has originated a Prenotification Entry to a Receiver’s account to initiate subsequent Entries to the Receiver’s account as soon as the third Banking Day following the Settlement Date of the Prenotification Entry, provided that the ODFI has not received a return or NOC related to the Prenotification." |
Scheduler
The prenote-scheduler Lambda (cmd/prenote-scheduler) runs at 22:00 UTC every day — two hours before midnight UTC. It queries RDS for floats in SCHEDULING status whose due date is a fixed 5 calendar days out (run_time + 5 days). The schedule runs every day, including weekends, so no weekday adjustment is applied.
The run is deliberately scheduled close to the 00:00 UTC cutoff: because the waiting period is keyed to the UTC calendar date a prenote is submitted on (see above), running late in the UTC day lets us prenote as many of that day’s qualifying floats as possible while they still count against the current date’s waiting period, rather than slipping into the next day’s.
Each candidate float is then gated by the floats.prenotes GrowthBook feature flag, evaluated per user_id. Users for whom the flag is true are kept; everyone else is dropped. Rollout is controlled entirely in GrowthBook — the service ships with the flag defaulted off, and a float-service deploy is not required to dark-launch or adjust the sample.
Surviving floats are batched into a single SendMessageBatch call to the prod-floats-prenotes SQS queue. One message is enqueued per user (deduplicated by user_id) with payload {"user_id": "…"}.
EventBridge rule fires (daily, 22:00 UTC)
│
▼
Prenote Scheduler Lambda invoked
│
▼
Determine target due date
run_time + 5 calendar days
│
▼
Query RDS replica: floats WHERE status = SCHEDULING AND due_date = target
│
├── No matching floats ──► log and exit
│
▼
For each float, evaluate GrowthBook flag floats.prenotes for user_id
Keep if flag is enabled for the user; default is false
│
├── 0 kept ──► log and exit
│
▼
Enqueue one PrenoteMessage per kept user to SQS: prod-floats-prenotes
Worker
The prenote-worker Lambda (cmd/prenote-worker) consumes from prod-floats-prenotes with a reserved concurrency cap of 3 to limit load on the Payments and User services. It uses SQS partial-batch failure reporting (ReportBatchItemFailures) so a failing user in a batch does not stall the rest.
For each message, the worker fetches the user’s profile from the User Service and submits a prenote to the Payments Service’s USIO integration, keyed to the float-debit USIO account. The Payments Service is the authority for prenote logging and tracking; the Float Service records no local state for prenote attempts.
SQS message received
Payload: {"user_id": "..."}
│
▼
Unmarshal payload
│
├── Malformed ──► report as batch item failure (SQS retries up to maxReceiveCount)
│
▼
GET user from User Service
│
├── Error ──► report as batch item failure
│
▼
Payments Service: SubmitUsioPrenote
FirstName, LastName, Email, UserId, UsioAccount = "float-debit"
│
├── Error ──► report as batch item failure
│
▼
Log success and ack the message
Configuration
| Setting | Effect |
|---|---|
|
Boolean flag evaluated per |
|
URL of the |
|
Set to |
| The scheduler does not verify that the user has a valid ACH-eligible account before enqueuing. Eligibility is enforced downstream by the Payments Service, which rejects prenote submissions for users without an ACH-routable bank account. |
Collection Outcomes
| Status | Set by | Description |
|---|---|---|
|
Float creation; T-1 Day (ACH failed) |
Initial status. Float has been created and disbursement is being processed. Also retained when a T-1 Day ACH attempt fails — the due-date pinless run will still fire normally. |
|
T-1 Day, Due Date, Daily Retry, Webhook workers |
ACH debit submitted to the payment processor. Awaiting a settlement callback via the |
|
ACH Handler (success callback); Due Date and Daily Retry (pinless success) |
Float fully collected. The |
|
Due Date (all attempts failed), Daily Retry (attempt failed or balance too low), Webhook workers (NSF or failure) |
Collection attempted but unsuccessful. The float will be retried at the next Daily Retry run or on the next qualifying income/balance event. |
|
Daily Retry (ACH attempts ≥ limit or > 90 days past due); Webhook worker (ACH attempts ≥ limit) |
Float has exceeded the maximum automated collection attempts or age threshold. |
|
Daily Retry (no valid Plaid account and no valid primary debit card) |
No valid payment method exists to attempt collection. Re-evaluated on subsequent Daily Retry runs. |
| Prenote submissions do not change float status. A prenote return — if the account is closed or invalid — surfaces through the Payments Service’s own monitoring, not through the float record. |
Distributed Locking
Before processing any float, each collection Lambda acquires a per-user distributed lock stored in the locks DynamoDB table. The lock key is loan-processing:user_id:{userID} with a 60-second lease and a 1-second heartbeat. This prevents two collection paths (e.g., a scheduled run and an incoming income event) from attempting concurrent collection on the same user’s float.
If a lock cannot be acquired, the attempt is skipped without updating the float status. See DynamoDB Tables for the locks table schema.
The prenote worker does not acquire this lock. Because prenotes are zero-dollar and do not update float state, concurrent prenote and collection activity for the same user is safe.
Related Pages
-
Float Lifecycle — End-to-end float status state machine
-
DynamoDB Tables — Collection history and locks table schemas
-
PostgreSQL Schema — Float statuses and column reference
-
Architecture — Collections system context diagram