Infrastructure

Three workflows drive deploys, each calling .github/workflows/deploy-env.yaml once per target with that target’s account, region, state bucket, artifact bucket and legacy region as inputs. on_pr.yaml builds the Lambda tarball once and runs terraform plan against every continuous target with the read-only site-github-services-plan role. on_main.yaml builds the tarball and the collections-jobs image once, applies every continuous target with site-github-services-deploy, and attaches the tarball to the draft release when release-please creates one. on_release.yaml applies prod from the published release’s tarball and image tag. The continuous targets are the old test environment (account 267052520423, us-west-2), the new test account (299185625033, us-east-2) and staging (208295047603, us-east-2); prod is 267052520423 in us-east-2. Authentication is GitHub OIDC; there are no static credentials. Both roles, and the site-github-services-boundary permissions boundary every Lambda and Fargate role carries, are owned by the infrastructure repo. The Terraform providers use the workflow’s credentials directly, with no further role assumption. State lives in the floatme-site-media bucket under terraform/floats/ for old test and prod, in floatme-test-services-terraform-state for the new test account and in floatme-staging-services-terraform-state for staging, locked by the terraform-locking table in each account. Lambda packages upload to the bucket passed as artifact_bucket (floatme-test-services-artifacts in the new test account, floatme-site-media elsewhere). The provider versions are pinned by the committed deploy/.terraform.lock.hcl.

Peer service APIs (admin, user-service, underwriting, txn, insight, subscription-service, loc-service) are resolved by name at plan time and addressed by convention (https://{id}.execute-api.{region}.amazonaws.com). A peer that has not yet deployed to an environment yields an empty URL and an account-wide execute-api:Invoke grant until the next deploy picks up its id, so a fresh environment comes up in two passes rather than failing the plan. The payments API is a REST gateway with no list-by-name lookup, so payments-service must be deployed to an environment before this service.

Scheduled batch jobs run as ECS Fargate tasks on the site-floats cluster (awsvpc network mode, FARGATE launch type, platform LATEST, no public IPs). All tasks run the collections-jobs image and dispatch on a subcommand.

Each task is provisioned at 512 CPU / 1024 MB and runs three containers: the application binary, a FireLens log-router sidecar (logs ship to Datadog intake, no CloudWatch Logs group), and a Datadog Agent sidecar (APM and DogStatsD). Schedules are in EventBridge.

Each DLQ is named <queue>-dlq.

The ach-handler consumes the Payments Service’s prod-payments stream, filtered to six event types: FLOAT_DEBIT_COMPLETED, FLOAT_DEBIT_COMPLETED_PLAID, FLOAT_DEBIT_RETURNED, FLOAT_CREDIT_COMPLETED, FLOAT_CREDIT_RETURNED, FLOAT_DEBIT_CHARGED_BACK. Consumer config: batch size 100, parallelization factor 2, starting position LATEST, BisectBatchOnFunctionError enabled.

Schemas and access patterns are in DynamoDB Tables.

The three us-east-1 tables carry no environment prefix and are shared with other services. Client configuration: the three shared tables resolve their region from DYNAMO_LEGACY_REGION (the lock client separately from DYNAMO_LOCK_REGION); site-float-service from DYNAMO_REGION + DYNAMO_TABLE.

The API Lambda publishes user_float_created to the floatme-events bus — see Events. All scheduled triggers:

Event-tap rules routing external events to SQS:

All secrets are namespaced by environment (site/…​).

The site-floats API Gateway (AWS IAM / SigV4 auth) forwards all routes (ANY /{proxy+}) to the site-floats-api Lambda. Callers are FloatMe backend services and the global FloatMe API Gateway for authenticated mobile requests.

Datadog monitors and SLOs exist in prod only (local.create_datadog_monitors in deploy/datadog.tf). Alerts outside prod go unread, and the old and new test accounts both deploy as test, so Datadog would reject the second account’s identical monitors as duplicates.

Datadog SLOs cover the api and ach-handler Lambdas: error and throughput SLOs on both — each 99.9% target / 99.99% warning over 7-day and 30-day windows.