Data Architecture
=================

Data Domains
------------

Platform data is partitioned by service, with each microservice owning its own
PostgreSQL database (database-per-service). The principal domains are:

.. list-table::
   :header-rows: 1
   :widths: 25 40 35

   * - Domain
     - Owner service
     - Principal entities
   * - Identity & tenancy
     - Identity (auth)
     - Users, businesses/tenants, teammates, API keys, passcodes, IP
       whitelists, webhook configurations.
   * - Customer records
     - Customers
     - Per-tenant end-customer profiles (name, DOB, address, country,
       blacklist flag).
   * - Accounts
     - Internal Accounts
     - Internal accounts, virtual accounts (provider bank details),
       applications and application history, sweep rules and executions,
       deposits and deposit stages, token wallets.
   * - Money movement
     - Transfer
     - Transfer workflows, approval requests/decisions, fees and fee configs,
       exchange rates, remittance configurations.
   * - Transaction record
     - Transactions
     - Customer-visible transactions (integer minor-units amount + precision,
       direction, type, sending/receiving accounts, tenant ID).
   * - Beneficiaries
     - Payees
     - Payees, payee institutions, provider routing, domestic transfer
       configurations, settlement instructions.
   * - FX & OTC
     - FX
     - Currency pairs, rates, FX transactions, tenant markups and bonuses,
       OTC orders with approval fields.
   * - Billing
     - Billings
     - Fee configs, fees, invoices and line items, subscription plans and
       subscriptions, audit-log tables per entity.
   * - Onramp
     - Onramp
     - Transactions with hop state, payment intents, saved cards, provider
       accounts, webhook events, billing records, reference data (providers,
       countries, currencies, networks, assets).
   * - Crypto custody
     - Web3
     - Wallets, transfers, token registry/operations, Circle wire accounts and
       deposits, payment links, OpenTrade savings, workflow state machine.
   * - Compliance
     - Risk
     - Customers, KYC information and document data, address information,
       feature activations.
   * - Delivery & audit
     - Webhooks / Audit
     - Webhook events, call history, ingestion audit; audit logs, response
       logs, session documents.

Sensitive Data Classification
-----------------------------

.. list-table::
   :header-rows: 1
   :widths: 25 45 30

   * - Class
     - Examples
     - Handling
   * - Secrets & credentials
     - Provider API keys, JWT secrets, Kafka TLS material, Dfns signing keys
     - Stored exclusively in GCP Secret Manager for sandbox/live; never in
       code or images.
   * - Sensitive personal data
     - KYC documents, identity attributes
     - Encrypted at rest with a dedicated document encryption key in the Risk
       service; KYC processing delegated to Sumsub.
   * - Sensitive financial data
     - Bank account details, card tokens
     - Field-level AES encryption with a separate sensitive-data key
       (distinct from the default encryption key); card PANs are never stored
       — only provider-issued tokens (Coinflow card-on-file) and metadata.
   * - Cryptographic key material
     - Dfns ECDSA P-256 private key
     - Base64-stored secret; every mutating custody call is user-action signed
       (challenge → sign → DER → base64url).
   * - Operational data
     - Transactions, transfers, balances
     - Tenant-scoped via mandatory ``x-tenant-id`` metadata; multi-tenant
       isolation enforced at the service layer.

Data Flow Principles
--------------------

- **Tenant context propagation.** Every internal call carries
  ``x-tenant-id``, ``x-user-id``, ``x-environment``, ``x-session-id`` (and
  where relevant ``x-customer-id``, ``x-idempotency-key``) as gRPC metadata.
- **Environment separation.** Live and sandbox data never mix: the gateway
  maintains separate connection pools, sandbox traffic routes to dedicated
  staging deployments, and secrets are namespaced per environment.
- **Event sourcing of money movement.** Status transitions flow through Kafka
  topics; the Transactions service is updated only through
  ``transactions.update.status`` events, keeping a single system of record.
- **Idempotency.** POST operations carry idempotency keys stored in Redis at
  the gateway and forwarded downstream; billing charges and onramp
  tokenization are idempotent by design; the webhooks service deduplicates on
  idempotency keys.
- **Auditability.** Services emit trace events to ``audit.events.log``
  (fire-and-forget, gracefully degraded); the Audit service persists them with
  full request context.

Storage Components
------------------

.. list-table::
   :header-rows: 1
   :widths: 25 75

   * - Component
     - Role
   * - PostgreSQL
     - Primary datastore, one database per service. Migrations are versioned
       SQL (golang-migrate / goose / sqlx-migrate) applied at startup or
       deploy time.
   * - Redis
     - Gateway idempotency store and rate limiting; per-service caching
       (FX rates, currencies); Asynq task queues (onramp); Web3 wallet
       provisioning job queue; webhook deduplication.
   * - Kafka
     - Event backbone: provider webhook fan-in, transfer/transaction status
       propagation, billing events, audit logging, transaction monitoring.
       TLS-secured in sandbox/live.
   * - MongoDB
     - Session-based request logging (Audit service) and optional structured
       webhook delivery logs.
   * - Google Cloud Storage
     - Object storage (identity service document/bucket use).
   * - GCP Secret Manager
     - Uniform secrets backend for all services, keyed per environment.