Operational Consoles & Workflows
================================

Operational Consoles
--------------------

.. list-table::
   :header-rows: 1
   :widths: 25 75

   * - Console
     - Purpose
   * - Gravv dashboard
     - Tenant self-service: accounts, customers, transactions, recipients,
       wallets, cards, billing, invoices, subscriptions, OTC desk, stablecoin
       payment links, webhook configuration and history, team management, API
       keys, and profile.
   * - Admin console
     - Internal operations: oversight of businesses, customers, transactions,
       cards, and accounts across all tenants.
   * - Admin API surface
     - Super-admin-token-protected gateway routes exposing polymorphic admin
       operations per service (businesses, customers, cards, internal
       accounts, audit response logs, statistics, payee verification resets,
       idempotency-key resets).
   * - Slack alerting
     - Operational notification channels: payout escalations (Sokin),
       manual-processing queues, onramp low-balance warnings, and gateway
       alert middleware.

Key Operational Workflows
-------------------------

Transfer Lifecycle Management
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Transfer workflows progress through a state machine with status-indexed
pickup. Failed payouts emit compensation events
(``transfer.inflow.compensated``, ``transaction.compensated``); transfers
requiring intervention are flagged for manual processing and escalated to
Slack. Dual-authorization approvals gate configured transfer types.

Reconciliation
~~~~~~~~~~~~~~

- **Onramp reconciliation** runs at multiple cadences (routine, full, batch,
  deep) with stuck-transaction and critical-transaction thresholds.
- **Deposit reconciliation** verifies the inbound deposit pipeline stages.
- **Sweep reconciliation** verifies automated sweep executions against rules.
- Daily operational review is supported by transaction volume, statistics,
  and export RPCs across services.

Webhook Delivery Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Outbound deliveries are polled from the database in batches, signed, and
retried with exponential backoff up to a bounded retry count. Operators can
inspect webhook history and per-call detail, acknowledge ingestion-audit
entries, manually retry failed deliveries, and send raw webhooks for testing.

Fee & Billing Operations
~~~~~~~~~~~~~~~~~~~~~~~~

Failed fee charges retry up to three times before being marked permanently
failed; operators can retry failed fees, refund charged fees, link fees to
references, export fee data, and generate or export invoices (PDF, batch).

Compliance Operations
~~~~~~~~~~~~~~~~~~~~~

KYC/KYB reviews arrive as Sumsub webhook events and update verification state
automatically; document data is enriched and encrypted. Feature activation
(accounts, cards) is orchestrated based on verification outcomes. Transaction
monitoring verdicts gate settlement.

Scheduled Jobs
~~~~~~~~~~~~~~

- FX rate refresh and rate management (3-minute and 10-minute crons).
- Exchange-rate auto-update in the Transfer service.
- Payee verification scheduling.
- Web3 wallet-provisioning job queue with retry and scheduler locking.
- Tenant provisioning queue in the Identity service
  (``FOR UPDATE SKIP LOCKED`` polling).

Incident Response & Change Management
-------------------------------------

- All changes flow through pull requests on the ``dev`` → ``main`` branch
  model with CI (tests, lint, contract breaking-change detection) and GitOps
  deployment via a shared reusable workflow.
- Post-mortems and system-design changes are documented in the monorepo
  (``docs/`` — bug post-mortems, architecture change records with
  before/after states in ``changelogs/``).
- Health endpoints (``/healthz``, ``/livez``, ``/readyz``) support platform
  probes; graceful shutdown is implemented across services.