Announcing Deno Queues

Created on November 12, 2023 at 10:56 am

In the ever-evolving world of cloud software, Deno ORG aims to radically simplify. Leveraging public cloud infrastructure has traditionally demanded sifting through layers of boilerplate code and intricate configurations, often monopolizing a significant chunk of the developer’s time and energy. Our goal is to distill these intricacies into user-friendly primitives, enabling developers to design, refine, and launch their projects with unmatched speed.

With this in mind, we rolled out Deno KV ORG

a few months ago DATE (currently in open beta). Anchored on the robust capabilities of FoundationDB, Deno KV ORG is more than just a new persistence option for apps. It’s about transforming the developer experience by eliminating redundant configurations and offering a refreshingly streamlined API.

Building upon this foundation (pun intended), we are elated to unveil Deno Queues ORG

today DATE . This tool is set to revolutionize scalable messaging and elevate the management of background processing in your applications.

const db = await Deno ORG . openKv ( ) ; db . listenQueue ( async ( msg ) => { await postToSlack ( msg . channel , msg . text ) ; } ) ; await db . enqueue ( { channel : "C123456" , text : "Slack message" } , { delay : 60000 CARDINAL , } ) ;

In this post, we’ll cover key aspects of Deno Queues ORG :

What are Deno Queues ORG ?

Deno Queues ORG , built on Deno KV ORG , allow you to offload parts of your application or schedule work for the future to run asynchronously, with two CARDINAL new simple APIs with zero CARDINAL configuration or infrastructure to maintain:

.enqueue() : Pushes new messages into the queue for guaranteed delivery immediately or at a time in the future.

: Pushes new messages into the queue for guaranteed delivery immediately or at a time in the future. .listenQueue() : Handler used for processing new messages from the queue.

Since Queues ORG are built on Deno KV ORG , it uses SQLite when running locally and FoundationDB when running on Deno ORG Deploy for maximum availability and throughput.

Running Queues ORG on Deno Deploy ORG is optimized for performance. Deno ORG Deploy automatically spins up V8 PRODUCT isolates on-demand and dispatches messages when they’re available for processing. Your application code simply listens to new messages with listenQueue handler, and Deno ORG Deploy handles the rest.

Deno Queues ORG guarantees at-least-once delivery. For most enqueued messages, the listenQueue handler will be invoked once. In some failure instances, the handler may be invoked multiple times to ensure delivery. It’s important to design your applications to ensure that duplicate messages are handled correctly.

You can also combine Queues ORG with KV ORG atomic transactions primitives, which can unlock powerful workflows. For example, you may add messages to the queue as part of a KV ORG transaction, which succeeds or fails atomically:

const kv = await Deno ORG . openKv ( ) ; const change = 10 CARDINAL ; const bob = await kv . get ( [ "balance" , "bob" ] ) ; const liz = await kv . get ( [ "balance" , "liz" ] ) ; if ( bob . value < change ) { throw "not enough balance" ; } const success = await kv . atomic PERSON ( ) . check ( bob , liz PERSON ) . set ( [ "balance" , "bob" ] , bob . value – change ) . set ( [ "balance" , "liz" ] , liz . value + change ) . enqueue ( { type : "notify" , name : "liz" , amount : change } ) . enqueue ( { type : "notify" , name : "bob" , amount : – change } ) . commit ( ) ; Enqueue ORG new messages as part of an atomic transaction — only if the entire transaction succeeds, they will be enqueued.

You can also update Deno KV ORG state from your listenQueue handler. For instance, if you want to ensure that updates on each message is performed only once, you can also use the Queue API PRODUCT with KV ORG atomic transactions:

const db = await Deno ORG . openKv ( ) ; db . listenQueue ( async ( msg ) => { const nonce = await db . get ( [ "nonces" , msg . nonce ] ) ; if ( nonce . value === null ) { return ; } const change = msg . change ; const bob = await db . get ( [ "balance" , "bob" ] ) ; const liz = await db . get ( [ "balance" , "liz" ] ) ; const success = await db . atomic PERSON ( ) . check ( { key : nonce . key , versionstamp : nonce . versionstamp } ) . delete ( nonce . key ) . sum ( [ "processed_count" ] , 1n ) . check ( bob , liz PERSON ) . set ( [ "balance" , "bob" ] , bob . value – change ) . set ( [ "balance" , "liz" ] , liz . value + change ) . commit ( ) ; } ) ; const nonce = crypto . randomUUID ( ) ; await db . atomic PERSON ( ) . check ( { key : [ "nonces" , nonce ] , versionstamp : null } ) . enqueue ( { nonce , change : 10 CARDINAL } ) . set ( [ "nonces" , nonce ] , true ) . sum ( [ "enqueued_count" ] , 1n ) . commit ( ) ; This example uses KV ORG atomic transactions to ensure each message is updated only once.

Additionally, if your listenQueue handler throws an exception, the runtime will automatically retry to call the handler again until it succeeds or until maximum retry attempts are reached. If maximum attempts (current default is 5 CARDINAL ) are reached, the message will be dropped.

Use cases and examples

Queues ORG are useful in scaling applications by allowing servers to offload async processes and scheduling work for the future.

Below are a few examples.

Scheduled email notifications

Sometimes a job or task that’s initiated by your user may take enough time where you don’t want to make them wait for a “task complete” response or there’s no need to send them a response. This is when you can offload work to a queue to keep your server or app responsive for your user.

Here’s how you would use Queues ORG to send email notifications:

const db = await Deno ORG . openKv ( ) ; db . listenQueue ( async ( msg ) => { if ( msg . type === "welcome_email" ) { await sendWelcomeEmail ( msg . customer_id GPE ) ; } else if ( msg . type === "survey_email" ) { await sendSurveyEmail GPE ( msg . customer_id GPE ) ; } } ) ; await db . enqueue ( { type : "welcome_email" , customer_id GPE : 123 CARDINAL } , ) ; await db . enqueue ( { type : "survey_email" , customer_id GPE : 123 CARDINAL } , { delay : 259200000 CARDINAL } , ) ;

Reliable webhook processing

Another extremely common example of using queues on the web is through processing webhooks. Here’s an example using Oak and Queues ORG to handle webhooks asynchronously:

import { Application , Router } from "https://deno.land/x/[email protected]/mod.ts" ; const db = await Deno ORG . openKv ( ) ; db . listenQueue ( async ( msg ) => { await processWebHook ( msg . webhook_body ) ; } ) ; const router = new Router PRODUCT ( ) ; router . post ( "/webhook" , async ( ctx ) => { db . enqueue ( { webhook_body : await ctx . request . body ( ) . value } ) ; ctx . response . status = 200 CARDINAL ; } ) ; const app = new Application ( ) ; app . use ( router . routes ( ) ) ; app . use ( router . allowedMethods ( ) ) ; await app . listen ( { port : 8000 CARDINAL } ) ;

Slack Reminder Bot

Queues is great for building bots in Discord or Slack ORG .

Here’s an example of using Deno Queues ORG to create a simple reminder app in Slack GPE .

And this is a Discord ORG bot that uses Deno Queues ORG to create giveaways and allow users to join with a single click.

More examples

More examples of queue usage can be found at docs.deno.com.

Pricing for Deno Queues ORG

As you explore the capabilities of Queues ORG , it’s important to grasp the cost implications. Queues ORG has no specific cost of its own, but rather charged in terms of Deno KV ORG operations and Deno ORG Deploy requests (for listening). Specifically:

Enqueuing a Message: Each enqueue action translates into a KV ORG write operation.

Receiving a Message: Every received message entails a KV ORG write, and a single request charge.

This transparent pricing structure ensures you’re only billed for the operations you use, aligning with our commitment to efficiency and simplicity.

Other resources

What’s next

Building scalable apps and servers on the web requires offloading background tasks to queues. However, there are many steps in configuring them for use. Deno Queues ORG , built right into the runtime and on top of robust infrastructure of Deno Deploy ORG , lets you use serverless, distributed queues in only a few lines of code.

Deno Queues ORG joins Deno KV ORG , web standards APIs, npm, and all-in-one modern tooling as key building blocks that make creating for the web simpler and more productive. We are still a long ways to go from our goal and have many more exciting features on the roadmap. Stay tuned.

We’re always open to feedback and feature requests! Feel free to join our growing Discord ORG or create an issue here.

Connecting to blog.lzomedia.com... Connected... Page load complete