Internal Docs/Actions & deferred tasks — the engine room
the engine room · prepare → execute → defer · one contract, many flows

actions & deferred tasks

Almost every mutating operation in axis-service — activating a card, loading funds, suspending, transferring, ordering — speaks one shape: prepare → execute → (findOne | cancel), with a single ActionRequestEntity row as the durable state machine. What differs is how execute does the work: a single-card action runs inline and comes back Complete in the same request; a group action defers — it persists a task, returns Processing, and the real work runs later on a background worker. This is the engine the flow docs sit on top of.

00

the shape of an action

POST .../cards/actions/{prepare} · .../actions/{id}/{execute|cancel} · .../actions/findOne — and the /multi/ variants for groups

one contract, four verbs

A caller never mutates a card directly. They open an action: prepare validates the request and stages an ActionRequestEntity (returning its id and a New preview), then execute commits it. findOne reads the current state by id or clientRef; cancel abandons it while still New. Single-card, multi-card, and group requests all use this same shape — only the URL (cards/actions/… vs cards/multi/actions/…) and the task behind it change.

New 0 Processing 100 Complete 200 Partial Complete 300 Failed 500
prepare lands on New · execute moves to Processing · a still-New action can go Canceled (400) by an explicit cancel or the expiry timer (§03)
ActionRequestEntity — the durable record

One row in cpm.action_request is the single source of truth for an action's life. findOne just reads it back.

columnholds
action_statusthe state code (0 → 500 above), stamped with action_status_ts
action_typeactivate, load, suspend, p2pFundTransfer, …
selector_jsonwhich card(s)/account the request targets
params_jsonamount, currency, channel, reason (input-only)
access_scope_jsonthe resolved cardIds + count
clientRefvia entity_ext_ref — the idempotency key (§03)
the round trip

prepare cpm

Validate, resolve the selector, create the action_request at New, return the id + preview. No card is touched yet.

execute

Commit the staged action. Status moves to Processing — then runs inline or defers (§01).

findOne cpm

Read status by id or clientRef. For inline actions it's already terminal; for deferred ones, poll until it settles (§04).

01

two ways to execute

The single most important distinction — it decides whether you get a result on the response or have to wait for it

inline — single-card & transfers

CardAccountTask.execute() does the work in the request thread: flip to Processing, call the handler synchronously, set the terminal status — all before responding.

Processing cpm

updateActionStatus(PROCESSING).

call the handler now tribe

CardActionHandler.execute() runs in-thread — the Tribe activate/load/status call happens here. (P2P uses CardAccountTransferTask the same way.)

terminal before responding cpm

On success Complete, on error Failed — set before the HTTP response returns. The execute response already carries the outcome.

deferred — group actions

GroupCardAccountActionTask.execute() doesn't do the work — it schedules it and returns. Right for a batch: N cards × Tribe calls is far too long for one request.

schedule a task queue

TaskExecutor.scheduleForExecution(GroupCardActionHandler, ctx, now+100ms, actionId) persists a row to the deferred queue (§02).

Processing, then return cpm

updateActionStatus(PROCESSING) and respond immediately — the cards are not activated yet.

work runs later async

A background worker picks the task up, activates each card on Tribe, and settles the action to Complete / Partial Complete / Failed — then fires a callback (§04).

Why it matters to a caller. After execute, an inline action's response is the final answer. A deferred action's response only says Processing — you learn the outcome by polling findOne or receiving a callback. Same contract, very different timing.
02

the deferred-task engine

com.cambrist.axis.model.task — DeferredTaskEntry · TaskScanner · TaskExecutorImpl · TaskFunction

execute()
schedule
persist a task row, due now + 100ms
cpm
DEFERRED_TASK_QUEUE
status NEW (0) · task class + ctx JSON + dueOn
sweep
~30–40s
daemon thread
TaskScanner
claims a batch · status → PICKED (1)
4-thread
pool
TaskRunner (txn)
TaskFunction.execute
GroupCardActionHandler does the work
scheduled by execute() persisted queue row scanner + worker pool
DEFERRED_TASK_QUEUE — the persisted row
columnholds
task_classFQN of the TaskFunction to run (loaded by reflection)
task_ctxJSON payload (e.g. partnerId, the actionId via task_ref)
dueon_tsearliest run time — the scanner only claims due rows
status0 NEW1 PICKED2 COMPLETE
exec_count / last_outcomeattempt count + the last run's result/error text
batch_refstamped by the scanner when a batch is claimed
claim, run, settle

sweep on an interval scanner

The DeferredTaskScanner daemon polls every scanIntervalMs (~40s prod / 30s stg), batch size 10.

atomic claim cpm

A single UPDATE … WHERE status=0 AND dueon_ts ≤ now … FOR UPDATE SKIP LOCKED … RETURNING * flips rows to PICKED. SKIP LOCKED is what makes it safe for multiple threads and multiple app instances — no task runs twice.

run in a transaction pool

Each claimed task runs on a 4-thread executor (queue 400), wrapped in TxSupport.executeWithinTransaction, with the partner RequestContext restored from task_ctx.

retry only if asked pool

Throwing DeferredTaskException(nextRetryTime, …) reschedules the row (status → NEW, new dueOn). Any other throw records the error and stops.

the other substrate — in-memory BackgroundTask

Not everything goes through the persisted queue. The lighter BackgroundTask.schedule(ms) just hands a Runnable to an in-JVM ScheduledFuture — no DB row, no scanner. It's used for short fire-and-forget work: the expiry timer (§03), the KYC webhook tasks, callback delivery (§04), and internal fund transfers. The trade-off is the headline difference: persisted-queue tasks survive a restart and are claimed across instances; in-memory tasks do not — a restart drops whatever was pending.

Sharp edges worth knowing.
  • Not instant. A deferred task is due at now+100ms but only starts on the scanner's next sweep — so group work begins seconds later, not immediately.
  • Rows stay PICKED. On success the task is not marked COMPLETE (the markComplete() call is commented out) — completed rows linger in DEFERRED_TASK_QUEUE at status 1.
  • No dead-letter. A task that fails without requesting a retry simply stops with its error in last_outcome; there's no max-attempt escalation or dead-letter queue.
03

staying consistent

Idempotency at the door, and two timers that bound an action's life

clientRef — the idempotency key

At prepare, the validator looks up clientRef in entity_ext_ref (type ClientRef, scoped to the partner). If it already exists, prepare is rejected with nonUniqueReference — "req.ClientRef must be unique" (HTTP 417 · errorCode 2230). A successful prepare registers the ref against the new action_request. So a retried prepare with the same ref can't silently create a second action — and findOne by clientRef lets the caller recover the original.

two timers — don't confuse them

An action prepared but never executed shouldn't live forever. Two independent mechanisms bound it — one rejects late calls, the other cleans up.

timerwherefiresdoes
60s stale guard
ACTION_EXPIRY_MS
validator, on execute/cancelrequest arrives > 60s after preparerejects it — StaleResourceException "this action has expired". Also rejects already-canceled/complete actions.
120s auto-cancel
subscribeToExpiration
in-memory BackgroundTask armed at prepare120s after prepareif the action is still New, marks it Canceled and deletes its staged group_card_action rows. A no-op once execute has moved it on.
The 60s window is the contract; the 120s sweep is the cleanup. A well-behaved caller executes within 60s. The 120s timer is the safety net for a prepare that's abandoned (client crash, network drop) — but because it's an in-memory task, a JVM restart in that window drops it, and the orphaned New action is left until something else reaps it. The 60s validator guard still protects execute regardless.
04

getting the result back

Two ways to learn how a deferred action ended — pull, or be pushed

poll — findOne

The simplest path: call findOne by id or clientRef and read action_status. Poll until it leaves Processing for a terminal state. Always available, no setup. Inline actions skip this entirely — their execute response is already terminal.

push — callback notifications

When a deferred action settles, the handler publishes a CallbackNotification. A partner that has registered a CallbackEndpoint gets an HTTP POST — no polling. Event types include ActionRequest_Complete / _PartialComplete / _Failed and KycInquiry_Pass / _Fail.

how a callback reaches the partner

publish on settle handler

publishCallbackNotification(action)CallbackNotificationService.publishOne() writes an api_callback_event row and raises an internal AppEvent.

fan out to endpoints cpm

CallbackEventHandler (on /internal-api/callback-events, role REST_API_APPEVENTNOTIFY) finds the partner's active CallbackEndpoints for that event type and schedules a delivery per endpoint.

deliver with retries background

CallbackDeliveryTask POSTs the notification JSON to the partner URL — optional bearer auth + an X-Setldpay-Signature HMAC — retrying with exponential backoff (up to ~5 attempts), each attempt recorded.

05

who rides the mechanism

flows by execution substrate

axis-service runs mutating work on three substrates — inline, the deferred queue, and in-memory background tasks. Some flows reach them through the prepare/execute action contract; others are kicked off by webhooks or internal events. The trigger and the substrate together decide how the result comes back.

flowtasksubstratetriggered byresult via
single-card action
activate, load, suspend, markLost…
CardAccountTaskCardActionHandlerinlineprepare / executethe execute response
P2P fund transferCardAccountTransferTaskinlineprepare / executethe execute response
group action
bulk activate / load / status
GroupCardAccountActionTaskGroupCardActionHandlerdeferred queueprepare / executefindOne poll · or callback
KYC inquiry
initiate / screen
InitiateKycFlowTask · PersonScreeningTaskinlineKYC verify endpointKYC callback
KYC submission updateNewSubmissionStartTask · SubmissionUpdateTaskbackgroundID-PAL webhookKYC callback
callback deliveryCallbackDeliveryTaskbackgroundinternal AppEventHTTP POST to the partner
Not on the action engine. Bulk card generation and import are CLI commands (BulkCardGenCommand, BulkCardImport in com.cambrist.axis.cpm.commands); the single-card order path also mirrors a card in by calling CardImportTask inline. None of these go through prepare/execute or the deferred queue — the bulk provisioning flow has its own writeup (card provisioning).
The one persisted-queue rider. Of everything above, only the group action path uses the durable DEFERRED_TASK_QUEUE (GroupCardActionHandler is the sole TaskFunction). Everything else is either inline or rides the in-memory BackgroundTask substrate — which is why a restart is survivable for batch group work but can drop a pending KYC-webhook, callback delivery, or the 120s expiry timer.