Skip to main content
API docs by Redocly

Querator API (0.1.0)

Download OpenAPI specification:Download

Querator Team: querator@kapetan.io URL: https://github.com/kapetan-io/querator License: Apache 2.0

Querator is a Distributed Durable Execution System built on top of an Almost Exactly Once Delivery (AEOD) Queue.

Core Features

Almost Exactly Once Delivery

Querator provides high-throughput, lease-based queue operations with Almost Exactly Once Delivery semantics. Unlike streaming systems, items are leased to consumers who gain exclusive processing rights until completion or retry.

Lease-Based Processing

  • Exclusive Access: Consumers lease items with guaranteed exclusive processing rights
  • Timeout Protection: Automatic re-queuing if lease deadline expires
  • Flexible Completion: Items can be completed, retried, or moved to dead letter queues

Scheduled Delivery

Support for scheduled item delivery via enqueue_at timestamps, enabling delayed processing and retry scheduling.

Horizontal Scaling

  • Partitioned Queues: Each queue can have multiple partitions for parallel processing
  • Automatic Load Balancing: Consumers are automatically distributed across partitions
  • Pluggable Storage: Support for multiple storage backends (BadgerDB, Memory, with PostgreSQL/MySQL planned)

API Design

The API follows a lease-based workflow:

  1. Produce items to queues with optional scheduling
  2. Lease items for exclusive processing with configurable timeouts
  3. Complete processed items or Retry failed items
  4. Manage queues and inspect storage state

All operations support batch processing for high throughput scenarios.

Storage Backends

Querator supports pluggable storage backends with minimal requirements (ordered primary keys only):

  • Memory: In-memory storage for testing and ephemeral workloads
  • BadgerDB: Embedded storage for single-instance deployments
  • Planned: PostgreSQL, MySQL, SurrealDB, FoundationDB

For more information, visit querator.io or the GitHub repository.

Querator V1

Core queue operations including produce, lease, complete, retry, and queue management.

Queue Operations:

  • queue.produce - Add items to queues with optional scheduling
  • queue.lease - Lease items for exclusive processing
  • queue.complete - Mark leased items as successfully processed
  • queue.retry - Retry failed items or move to dead letter queue
  • queue.stats - Get queue performance statistics
  • queue.clear - Clear queue data (destructive operation)

Queue Management:

  • queues.create - Create new queues with specified configuration
  • queues.list - List all available queues with pagination
  • queues.info - Get detailed information about a specific queue
  • queues.update - Update queue configuration
  • queues.delete - Delete queues and associated data

Storage Operations:

  • storage/items.* - Direct storage inspection and manipulation
  • Low-level operations for debugging and data migration

Produce an item on a queue

Produce an item on a queue. Calls to the endpoint may provide multiple items to produce. The call will not return until all items provided have been written to storage.

Timeout Semantics

The call will return when the item(s) provided are written to storage or when request_timeout has been reached. If the call returns with a non 200 response code, the client should consider the items provided as not written to the queue. The client should NOT cancel the request before request_timeout has been reached. If the call fails to return after request_timeout is reached the client can safely cancel the request.

Request Body schema: application/json
queue_name
required
string

The name of the queue

request_timeout
required
string
Default: "5m"

The duration this request should block before the client will cancel the request

required
Array of objects (QueueProduceItem) [ 1 .. 1000 ] items

List of items to be queued

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "request_timeout": "30s",
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "code": 200
}

Lease items from the queue

Lease a items from the requested queue. When a items is lease by a consumer, that consumer gains exclusive rights to the items. No other consumer will be provided the same items unless the lease_deadline has expired and the consumer with the lease has not marked the items via /queue.complete or /queue.retry

If a items is not acknowledge via /queue.complete within the lease_deadline period it will be offered to another consumer.

Timeout Semantics

The call will return when a lease is successful or when request_timeout has been reached. If the call returns with a 454 Retry Request response code, this means there were no items in the queue and the request_timeout was reached. In this case the client should retry the lease request.

The client should NOT cancel the request before request_timeout has been reached. If the call fails to return after request_timeout is reached the client can safely cancel the request.

Request Body schema: application/json
queue_name
required
string

The name of the queue

client_id
required
string

A unique id which identifies this client. Duplicate client id's are not allowed

request_timeout
required
string
Default: "1m"

The duration this request should block before the client will cancel the request.

batch_size
integer [ 1 .. 1000 ]

The maximum number of items to be leased for this request

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "batch_size": 1000,
  • "client_id": "client-01",
  • "request_timeout": "30s"
}

Response samples

Content type
application/json
{
  • "items": [
    ],
  • "queue_name": "webhooks",
  • "partition": 0
}

Retry items

Retry an item to be processed again at a later date/time. Once the retry_at time is reached, querator will enqueue this item into the specified queue and the attempted counter for this item will increment by 1.

If the item should not be retried, but failed immediately. Set dead: true and the item will be placed in the dead letter queue regardless of the number of attempts left on the items.

If you wish to fail a items such that it is deleted from the queue and never offered to another consumer, nor placed into the dead letter queue, then call /queue.complete instead.

Request Body schema: application/json
queue_name
required
string

The queue name provided when the item was leased. If the queue_name does not match the original queue_name provided when the item was leased, then the retry request will return a non 200 status code.

partition
required
integer

The partition provided when the item was leased. If the partition does not match the original partition provided when the item was leased, then the retry request will return a non 200 status code.

required
Array of objects (QueueRetryItem)

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "partition": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
{
  • "code": 200
}

Complete leased items

Mark leased items as completed. The item can then be removed from the queue by Querator and will not be offered to any other consumers.

If any of the ids provided cannot be marked as completed then the entire request will be rejected with a code 453 Request Failed. It is the responsibility of the client to remove the offending id and try again.

Timeout Semantics

The call will return when the id(s) provided are completed to storage or when request_timeout has been reached. If the call returns with a non 200 response code, the client should consider the items provided as not completed to the queue. The client should NOT cancel the request before request_timeout has been reached. If the call fails to return after request_timeout is reached the client can safely cancel the request.

Request Body schema: application/json
queue_name
required
string

The name of the queue these ids are from

partition
required
integer

The Partition these ids are from

request_timeout
required
string

The duration this request should block before the client will cancel the request

ids
required
Array of strings

A list of the item ids to mark as complete

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "partition": 0,
  • "request_timeout": "30s",
  • "ids": [
    ]
}

Response samples

Content type
application/json
{
  • "code": 200
}

List all available queues

List all the available queues

Request Body schema: application/json
pivot
required
string

The name of the queue to pivot upon when paging through lists of queues

limit
required
integer <= 1000

The maximum number of items to return in a single list response

Responses

Request samples

Content type
application/json
{
  • "pivot": "queue-302",
  • "limit": 500
}

Response samples

Content type
application/json
{
  • "items": [
    ]
}

Create a queue

Create a new queue with the provided characteristics

Request Body schema: application/json
queue_name
required
string

The name of the queue

created_at
required
string <date-time>

The timestamp the queue was created

updated_at
required
string <date-time>

The timestamp the queue was last updated

dead_queue
required
string

The name of the dead letter queue for this queue. If this is a dead letter queue then this field will be empty when retrieved via /queue.list

reference
required
string

This is a user supplied field which could contain metadata or specify who owns this queue

lease_timeout
required
string

The default lease timeout for this queue

expire_timeout
required
string

How long an item can wait in the queue regardless of attempts before it is moved to the dead letter queue

max_attempts
required
integer

The maximum number of times an item can be retried by a consumer before it is placed in the dead letter queue or removed. Infinite attempts if max_attempts is set to '0'. lease_timeout violations are included as attempts

requested_partitions
required
integer

The number of partitions this queue has requested. This might be different than the number of currently active partitions as the system grows or contracts the number of partitions.

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "dead_queue": "queue-name-dead",
  • "reference": "jake@statefarm.com",
  • "lease_timeout": "60m",
  • "expire_timeout": "24h",
  • "max_attempts": 10,
  • "requested_partitions": 25
}

Response samples

Content type
application/json
"{\n \"code\": 200,\n\n}"

Update a queue

Update the details of a queue.

Request Body schema: application/json
queue_name
required
string

The name of the queue

created_at
required
string <date-time>

The timestamp the queue was created

updated_at
required
string <date-time>

The timestamp the queue was last updated

dead_queue
required
string

The name of the dead letter queue for this queue. If this is a dead letter queue then this field will be empty when retrieved via /queue.list

reference
required
string

This is a user supplied field which could contain metadata or specify who owns this queue

lease_timeout
required
string

The default lease timeout for this queue

expire_timeout
required
string

How long an item can wait in the queue regardless of attempts before it is moved to the dead letter queue

max_attempts
required
integer

The maximum number of times an item can be retried by a consumer before it is placed in the dead letter queue or removed. Infinite attempts if max_attempts is set to '0'. lease_timeout violations are included as attempts

requested_partitions
required
integer

The number of partitions this queue has requested. This might be different than the number of currently active partitions as the system grows or contracts the number of partitions.

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "dead_queue": "queue-name-dead",
  • "reference": "account-12345",
  • "lease_timeout": "60m",
  • "expire_timeout": "24h",
  • "max_attempts": 35
}

Response samples

Content type
application/json
{
  • "code": 200
}

Get Info about a queue

Request Body schema: application/json
name
required
string

The name of the queue to retrieve info about

Responses

Request samples

Content type
application/json
{
  • "name": "queue-name"
}

Response samples

Content type
application/json
{
  • "name": "queue-name",
  • "created_at": "2024-01-01 12:00:00.0000",
  • "updated_at": "2024-01-01 12:00:00.0000",
  • "dead_queue": "queue-name-dead",
  • "reference": "jake@statefarm.com",
  • "lease_timeout": "60m",
  • "expire_timeout": "24h",
  • "max_attempts": 50,
  • "requested_partitions": 456
}

Delete a queue

Delete a queue and all associated data. Use the force parameter to delete queues that have active items or leases.

Request Body schema: application/json
queue_name
required
string

The name of the queue to delete

force
required
boolean

force indicates the deletion should ignore any current open leases or items in the queue and delete all data related to the queue. In addition, this forcibly cancels all in progress client lease requests.

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "force": true
}

Response samples

Content type
application/json
{
  • "code": 200
}

Fetch stats for a queue

Retrieve statistics about a queue. Queue stats provide information on the health of a partition in a queue

Request Body schema: application/json
queue_name
required
string

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name"
}

Response samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "logical_queues": [
    ]
}

Clear a queue

Clears all data from a queue, optionally clears all retry and scheduled items

Request Body schema: application/json
queue_name
required
string

The name of the queue to clear

retry
required
boolean

Indicates the 'retry' queue will be cleared. If true, any items scheduled to be retried at a future date will be removed.

scheduled
required
boolean

Indicates any 'scheduled' items in the queue will be cleared. If true, any items scheduled to be enqueued at a future date will be removed.

queue
required
boolean

Indicates any items currently waiting in the FIFO queue will clear. If true, any items in the queue which have NOT been leased will be removed.

destructive
required
boolean

Indicates the Retry, Scheduled, Queue operations should be destructive in that all data regardless of status will be removed. For example, if used with ClearRequest.Queue = true, then ALL items in the queue regardless of lease status will be removed. This means that clients who currently have ownership of those items will not be able to "complete" those items, as querator will have no knowledge of those items.

Responses

Request samples

Content type
application/json
{
  • "queue_name": "queue-name",
  • "retry": false,
  • "scheduled": false,
  • "queue": true,
  • "destructive": true
}

Response samples

Content type
application/json
{
  • "code": 200
}

List Items in a Partition

List items in a queue partition

Request Body schema: application/json
queue_name
required
string
partition
required
integer
pivot
required
string
limit
required
integer

Responses

Request samples

Content type
application/json
{
  • "queue_name": "string",
  • "partition": 0,
  • "pivot": "string",
  • "limit": 0
}

Response samples

Content type
application/json
[
  • {
    }
]

Import Items to a Partition

Import items into a partition

Request Body schema: application/json
queue_name
required
string
partition
required
integer
required
Array of objects (StorageItem)

Responses

Request samples

Content type
application/json
{
  • "queue_name": "string",
  • "partition": 0,
  • "items": [
    ]
}

Response samples

Content type
application/json
[
  • {
    }
]

Delete items from a partition

Request Body schema: application/json
queue_name
required
string
partition
required
integer

The partition number these ids belong to

ids
required
Array of strings

Responses

Request samples

Content type
application/json
{
  • "queue_name": "string",
  • "partition": 0,
  • "ids": [
    ]
}

Response samples

Content type
application/json
{
  • "code": 200
}