Overview
Live state read from PostgreSQL and the running worker process.
Controls
Every action creates real Dhara tasks. Nothing is simulated in the browser.
Burst jobs
PENDING → RUNNINGEnqueue echo jobs (2s each) and watch the worker pool claim them concurrently.
Flaky jobs
RETRYRetrying jobs fail temporarily, then succeed. “Always fail” exhausts attempts and dead-letters.
Slow tasks
TIMEOUTRun slow_task. The handler exceeds its deadline and is retried, then dead-lettered.
Worker crash
REAPERAn isolated worker claims a task, then the playground kills it. Heartbeats stop, the reaper requeues the task, and a restarted worker completes it.
Worker pool
Dhara exposes process gauges and database leases, not per-worker heartbeats.
—| Worker lease | Current task | Attempt | Last heartbeat |
|---|---|---|---|
| No running task leases. | |||
Idle worker identities are not synthesized. Rows appear only when PostgreSQL shows a real locked_by lease.
Recent tasks
Click any task to inspect its real state and, for your own tasks, payload and logs.
| Task | Type | State | Attempts | Worker | Next run | Duration | Created |
|---|---|---|---|---|---|---|---|
| Loading tasks… | |||||||
Prometheus metrics
Raw output from Dhara’s existing client and worker handlers.
expand
Client / queue gauges /metrics
Not loaded.
Worker execution /worker/metrics
Not loaded.
What you are watching
Claiming
Workers claim rows with SELECT … FOR UPDATE SKIP LOCKED. Each PENDING task is claimed by exactly one worker.
Retries
Failed attempts are recorded in PostgreSQL and rescheduled with exponential backoff plus full jitter. The attempt counter is incremented on claim.
Heartbeat vs timeout
A handler timeout is an ordinary handler error. The reaper only acts when a RUNNING task stops refreshing locked_at — for example after the worker process dies.
Dead letters
Once attempts are exhausted, Dhara writes a dead_letters row. Retrying a DEAD task resets attempts and returns it to the ready queue.