Actions & Deferred Tasks — The Engine Room
Almost every mutating operation in axis-service — activating a card, loading funds, suspending, transferring, ordering — speaks one shape: prepare → execute → (findOne | cancel), with a single ActionRequestEntity row as the durable state machine. What differs is how execute does the work: a single-card action runs inline and comes back Complete in the same request; a group action defers — it persists a task, returns Processing, and the real work runs later on a background worker. This is the engine the flow docs sit on top of.
The Shape of an Action
POST .../cards/actions/{prepare} · .../actions/{id}/{execute|cancel} · .../actions/findOne — and the /multi/ variants for groups
A caller never mutates a card directly. They open an action: prepare validates the request and stages an ActionRequestEntity (returning its id and a New preview), then execute commits it. findOne reads the current state by id or clientRef; cancel abandons it while still New. Single-card, multi-card, and group requests all use this same shape — only the URL (cards/actions/… vs cards/multi/actions/…) and the task behind it change.
prepare lands on New · execute moves to Processing · a still-New action can go Canceled (400) by an explicit cancel or the expiry timer (§03)
One row in cpm.action_request is the single source of truth for an action's life. findOne just reads it back.
| column | holds |
|---|---|
action_status | the state code (0 → 500 above), stamped with action_status_ts |
action_type | activate, load, suspend, p2pFundTransfer, … |
selector_json | which card(s)/account the request targets |
params_json | amount, currency, channel, reason (input-only) |
access_scope_json | the resolved cardIds + count |
clientRef | via entity_ext_ref — the idempotency key (§03) |
cpm prepare
Validate, resolve the selector, create the action_request at New, return the id + preview. No card is touched yet.
execute
Commit the staged action. Status moves to Processing — then runs inline or defers (§01).
cpm findOne
Read status by id or clientRef. For inline actions it's already terminal; for deferred ones, poll until it settles (§04).
Two Ways to Execute
The single most important distinction — it decides whether you get a result on the response or have to wait for it
CardAccountTask.execute() does the work in the request thread: flip to Processing, call the handler synchronously, set the terminal status — all before responding.
cpm Processing
updateActionStatus(PROCESSING).
tribe call the handler now
CardActionHandler.execute() runs in-thread — the Tribe activate/load/status call happens here. (P2P uses CardAccountTransferTask the same way.)
cpm terminal before responding
On success Complete, on error Failed — set before the HTTP response returns. The execute response already carries the outcome.
GroupCardAccountActionTask.execute() doesn't do the work — it schedules it and returns. Right for a batch: N cards × Tribe calls is far too long for one request.
queue schedule a task
TaskExecutor.scheduleForExecution(GroupCardActionHandler, ctx, now+100ms, actionId) persists a row to the deferred queue (§02).
cpm Processing, then return
updateActionStatus(PROCESSING) and respond immediately — the cards are not activated yet.
async work runs later
A background worker picks the task up, activates each card on Tribe, and settles the action to Complete / Partial Complete / Failed — then fires a callback (§04).
The Deferred-Task Engine
com.cambrist.axis.model.task — DeferredTaskEntry · TaskScanner · TaskExecutorImpl · TaskFunction
| column | holds |
|---|---|
task_class | FQN of the TaskFunction to run (loaded by reflection) |
task_ctx | JSON payload (e.g. partnerId, the actionId via task_ref) |
dueon_ts | earliest run time — the scanner only claims due rows |
status | 0 NEW → 1 PICKED → 2 COMPLETE |
exec_count / last_outcome | attempt count + the last run's result/error text |
batch_ref | stamped by the scanner when a batch is claimed |
scanner sweep on an interval
The DeferredTaskScanner daemon polls every scanIntervalMs (~40s prod / 30s stg), batch size 10.
cpm atomic claim
A single UPDATE … WHERE status=0 AND dueon_ts ≤ now … FOR UPDATE SKIP LOCKED … RETURNING * flips rows to PICKED. SKIP LOCKED is what makes it safe for multiple threads and multiple app instances — no task runs twice.
pool run in a transaction
Each claimed task runs on a 4-thread executor (queue 400), wrapped in TxSupport.executeWithinTransaction, with the partner RequestContext restored from task_ctx.
pool retry only if asked
Throwing DeferredTaskException(nextRetryTime, …) reschedules the row (status → NEW, new dueOn). Any other throw records the error and stops.
Not everything goes through the persisted queue. The lighter BackgroundTask.schedule(ms) just hands a Runnable to an in-JVM ScheduledFuture — no DB row, no scanner. It's used for short fire-and-forget work: the expiry timer (§03), the KYC webhook tasks, callback delivery (§04), and internal fund transfers. The trade-off is the headline difference: persisted-queue tasks survive a restart and are claimed across instances; in-memory tasks do not — a restart drops whatever was pending.
Staying Consistent
Idempotency at the door, and two timers that bound an action's life
At prepare, the validator looks up clientRef in entity_ext_ref (type ClientRef, scoped to the partner). If it already exists, prepare is rejected with nonUniqueReference — "req.ClientRef must be unique" (HTTP 417 · errorCode 2230). A successful prepare registers the ref against the new action_request. So a retried prepare with the same ref can't silently create a second action — and findOne by clientRef lets the caller recover the original.
An action prepared but never executed shouldn't live forever. Two independent mechanisms bound it — one rejects late calls, the other cleans up.
| timer | where | fires | does |
|---|---|---|---|
60s stale guardACTION_EXPIRY_MS | validator, on execute/cancel | request arrives > 60s after prepare | rejects it — StaleResourceException "this action has expired". Also rejects already-canceled/complete actions. |
120s auto-cancelsubscribeToExpiration | in-memory BackgroundTask armed at prepare | 120s after prepare | if the action is still New, marks it Canceled and deletes its staged group_card_action rows. A no-op once execute has moved it on. |
Getting the Result Back
Two ways to learn how a deferred action ended — pull, or be pushed
The simplest path: call findOne by id or clientRef and read action_status. Poll until it leaves Processing for a terminal state. Always available, no setup. Inline actions skip this entirely — their execute response is already terminal.
When a deferred action settles, the handler publishes a CallbackNotification. A partner that has registered a CallbackEndpoint gets an HTTP POST — no polling. Event types include ActionRequest_Complete / _PartialComplete / _Failed and KycInquiry_Pass / _Fail.
handler publish on settle
publishCallbackNotification(action) → CallbackNotificationService.publishOne() writes an api_callback_event row and raises an internal AppEvent.
cpm fan out to endpoints
CallbackEventHandler (on /internal-api/callback-events, role REST_API_APPEVENTNOTIFY) finds the partner's active CallbackEndpoints for that event type and schedules a delivery per endpoint.
background deliver with retries
CallbackDeliveryTask POSTs the notification JSON to the partner URL — optional bearer auth + an X-Setldpay-Signature HMAC — retrying with exponential backoff (up to ~5 attempts), each attempt recorded.
Who Rides the Mechanism
axis-service runs mutating work on three substrates — inline, the deferred queue, and in-memory background tasks. Some flows reach them through the prepare/execute action contract; others are kicked off by webhooks or internal events. The trigger and the substrate together decide how the result comes back.
| flow | task | substrate | triggered by | result via |
|---|---|---|---|---|
| single-card action activate, load, suspend, markLost… | CardAccountTask → CardActionHandler | inline | prepare / execute | the execute response |
| P2P fund transfer | CardAccountTransferTask | inline | prepare / execute | the execute response |
| group action bulk activate / load / status | GroupCardAccountActionTask → GroupCardActionHandler | deferred queue | prepare / execute | findOne poll · or callback |
| KYC inquiry initiate / screen | InitiateKycFlowTask · PersonScreeningTask | inline | KYC verify endpoint | KYC callback |
| KYC submission update | NewSubmissionStartTask · SubmissionUpdateTask | background | ID-PAL webhook | KYC callback |
| callback delivery | CallbackDeliveryTask | background | internal AppEvent | HTTP POST to the partner |