Actions & Deferred Tasks — The Engine Room
The Shape of an Action
POST .../cards/actions/{prepare} · .../actions/{id}/{execute|cancel} · .../actions/findOne — and the /multi/ variants for groups
A caller never mutates a card directly. They open an action: prepare validates the request and stages an ActionRequestEntity (returning its id and a New preview), then execute commits it. findOne reads the current state by id or clientRef; cancel abandons it while still New. Single-card, multi-card, and group requests all use this same shape — only the URL (cards/actions/… vs cards/multi/actions/…) and the task behind it change.
prepare lands on New · execute moves to Processing · a still-New action can go Canceled (400) by an explicit cancel or the expiry timer (§03)
One row in cpm.action_request is the single source of truth for an action's life. findOne just reads it back.
| Column | Holds |
|---|---|
| action_status | the state code (0 through 500 above), stamped with action_status_ts |
| action_type | activate, load, suspend, p2pFundTransfer, ... |
| selector_json | which card(s)/account the request targets |
| params_json | amount, currency, channel, reason (input-only) |
| access_scope_json | the resolved cardIds + count |
| clientRef | via entity_ext_ref -- the idempotency key (see section 03) |
prepare cpm
Validate, resolve the selector, create the action_request at New, return the id + preview. No card is touched yet.
execute
Commit the staged action. Status moves to Processing — then runs inline or defers (§01).
findOne cpm
Read status by id or clientRef. For inline actions it's already terminal; for deferred ones, poll until it settles (§04).
Two Ways to Execute
The single most important distinction — it decides whether you get a result on the response or have to wait for it
CardAccountTask.execute() does the work in the request thread: flip to Processing, call the handler synchronously, set the terminal status — all before responding.
Processing cpm
updateActionStatus(PROCESSING).
call the handler now tribe
CardActionHandler.execute() runs in-thread — the Tribe activate/load/status call happens here. (P2P uses CardAccountTransferTask the same way.)
terminal before responding cpm
On success Complete, on error Failed — set before the HTTP response returns. The execute response already carries the outcome.
GroupCardAccountActionTask.execute() doesn't do the work — it schedules it and returns. Right for a batch: N cards × Tribe calls is far too long for one request.
schedule a task queue
TaskExecutor.scheduleForExecution(GroupCardActionHandler, ctx, now+100ms, actionId) persists a row to the deferred queue (§02).
Processing, then return cpm
updateActionStatus(PROCESSING) and respond immediately — the cards are not activated yet.
work runs later async
A background worker picks the task up, activates each card on Tribe, and settles the action to Complete / Partial Complete / Failed — then fires a callback (§04).
The Deferred-Task Engine
com.cambrist.axis.model.task — DeferredTaskEntry · TaskScanner · TaskExecutorImpl · TaskFunction
| Column | Holds |
|---|---|
| task_class | FQN of the TaskFunction to run (loaded by reflection) |
| task_ctx | JSON payload (e.g. partnerId, the actionId via task_ref) |
| dueon_ts | earliest run time -- the scanner only claims due rows |
| status | 0 NEW -> 1 PICKED -> 2 COMPLETE |
| exec_count / last_outcome | attempt count + the last run's result/error text |
| batch_ref | stamped by the scanner when a batch is claimed |
sweep on an interval scanner
The DeferredTaskScanner daemon polls every scanIntervalMs (~40s prod / 30s stg), batch size 10.
atomic claim cpm
A single UPDATE … WHERE status=0 AND dueon_ts ≤ now … FOR UPDATE SKIP LOCKED … RETURNING * flips rows to PICKED. SKIP LOCKED is what makes it safe for multiple threads and multiple app instances — no task runs twice.
run in a transaction pool
Each claimed task runs on a 4-thread executor (queue 400), wrapped in TxSupport.executeWithinTransaction, with the partner RequestContext restored from task_ctx.
retry only if asked pool
Throwing DeferredTaskException(nextRetryTime, …) reschedules the row (status → NEW, new dueOn). Any other throw records the error and stops.
Not everything goes through the persisted queue. The lighter BackgroundTask.schedule(ms) just hands a Runnable to an in-JVM ScheduledFuture — no DB row, no scanner. It's used for short fire-and-forget work: the expiry timer (§03), the KYC webhook tasks, callback delivery (§04), and internal fund transfers. The trade-off is the headline difference: persisted-queue tasks survive a restart and are claimed across instances; in-memory tasks do not — a restart drops whatever was pending.
Staying Consistent
Idempotency at the door, and two timers that bound an action's life
At prepare, the validator looks up clientRef in entity_ext_ref (type ClientRef, scoped to the partner). If it already exists, prepare is rejected with nonUniqueReference — "req.ClientRef must be unique" (HTTP 417 · errorCode 2230). A successful prepare registers the ref against the new action_request. So a retried prepare with the same ref can't silently create a second action — and findOne by clientRef lets the caller recover the original.
An action prepared but never executed shouldn't live forever. Two independent mechanisms bound it — one rejects late calls, the other cleans up.
| Timer | Where | Fires | Does |
|---|---|---|---|
| 60s stale guard (ACTION_EXPIRY_MS) | validator, on execute/cancel | request arrives > 60s after prepare | rejects it -- StaleResourceException "this action has expired". Also rejects already-canceled/complete actions. |
| 120s auto-cancel (subscribeToExpiration) | in-memory BackgroundTask armed at prepare | 120s after prepare | if the action is still New, marks it Canceled and deletes its staged group_card_action rows. A no-op once execute has moved it on. |
Getting the Result Back
Two ways to learn how a deferred action ended — pull, or be pushed
The simplest path: call findOne by id or clientRef and read action_status. Poll until it leaves Processing for a terminal state. Always available, no setup. Inline actions skip this entirely — their execute response is already terminal.
When a deferred action settles, the handler publishes a CallbackNotification. A partner that has registered a CallbackEndpoint gets an HTTP POST — no polling. Event types include ActionRequest_Complete / _PartialComplete / _Failed and KycInquiry_Pass / _Fail.
publish on settle handler
publishCallbackNotification(action) → CallbackNotificationService.publishOne() writes an api_callback_event row and raises an internal AppEvent.
fan out to endpoints cpm
CallbackEventHandler (on /internal-api/callback-events, role REST_API_APPEVENTNOTIFY) finds the partner's active CallbackEndpoints for that event type and schedules a delivery per endpoint.
deliver with retries background
CallbackDeliveryTask POSTs the notification JSON to the partner URL — optional bearer auth + an X-Setldpay-Signature HMAC — retrying with exponential backoff (up to ~5 attempts), each attempt recorded.
Who Rides the Mechanism
axis-service runs mutating work on three substrates — inline, the deferred queue, and in-memory background tasks. Some flows reach them through the prepare/execute action contract; others are kicked off by webhooks or internal events. The trigger and the substrate together decide how the result comes back.
| Flow | Task | Substrate | Triggered by | Result via |
|---|---|---|---|---|
| single-card action (activate, load, suspend, markLost...) | CardAccountTask -> CardActionHandler | INLINE | prepare / execute | the execute response |
| P2P fund transfer | CardAccountTransferTask | INLINE | prepare / execute | the execute response |
| group action (bulk activate / load / status) | GroupCardAccountActionTask -> GroupCardActionHandler | DEFERRED QUEUE | prepare / execute | findOne poll or callback |
| KYC inquiry (initiate / screen) | InitiateKycFlowTask / PersonScreeningTask | INLINE | KYC verify endpoint | KYC callback |
| KYC submission update | NewSubmissionStartTask / SubmissionUpdateTask | BACKGROUND | ID-PAL webhook | KYC callback |
| callback delivery | CallbackDeliveryTask | BACKGROUND | internal AppEvent | HTTP POST to the partner |