How Prisma saved 98% on distribution costs with Cloudflare R2

Created on November 12, 2023 at 10:33 am

7 min TIME read

The following is a guest post written by Pierre-Antoine Mills ORG , Miguel Fernández PERSON , and Petra Donka PERSON of Prisma. Prisma provides a server-side library that helps developers read and write data to the database in an intuitive, efficient and safe way.

Prisma PERSON ’s mission is to redefine how developers build data-driven applications. At its core, Prisma PERSON provides an open-source, next-generation TypeScript Object-Relational Mapping (ORM) library that unlocks a new level of developer experience thanks to its intuitive data model, migrations, type-safety, and auto-completion.

Prisma ORM has experienced remarkable growth, engaging a vibrant community of developers. And while it was a great problem to have, this growth was causing an explosion in our AWS ORG infrastructure costs. After investigating a wide range of alternatives, we went with Cloudflare ORG ’s R2 PRODUCT storage — and as a result are thrilled that our engine distribution costs have decreased by 98% PERCENT , while delivering top-notch performance.

It was a natural fit: Prisma PERSON is already a proud technology partner of Cloudflare ORG ’s, offering deep database integration with Cloudflare Workers ORG . And Cloudflare ORG products provide much of the underlying infrastructure for Prisma Accelerate ORG and Prisma Pulse PERSON , empowering user-focused product development. In this post, we’ll dig into how we decided to extend our ongoing collaboration with Cloudflare ORG to the Prisma ORM, and how we migrated from AWS S3 ORG + CloudFront ORG to Cloudflare R2, with zero CARDINAL downtime.

Distributing the Prisma ORM and its engines

Prisma ORM simplifies data access thanks to its type-safe Prisma Client ORG , and enables efficient database management via the Prisma CLI PRODUCT , so that developers can focus on product development.

Both the Prisma Client ORG and the Prisma CLI rely on the Prisma Engines ORG , which are implemented in Rust ORG and distributed as platform-specific compiled binaries. The Prisma Engines ORG perform a variety of tasks ranging from providing information about the schema for type generation, or migrating the database, to transforming Prisma ORG queries into SQL ORG , and executing those queries against the database. Think of the engines as the layer in the Prisma ORM that talks to the database.

As a developer, one CARDINAL of the first ORDINAL steps to get started with Prisma PERSON is to install Prisma Client ORG and the Prisma CLI from npm. Once installed, these packages need the Prisma Engines ORG to be able to function. These engines have complex target-platform rules and were originally envisioned to be distributed separately from the npm package, so they can be used outside of the Node.js ecosystem. As a result, they are downloaded on demand by the Prisma CLI, only downloading what is strictly required for a given project.

As of mid-2023 DATE , the engines account for 100 million CARDINAL downloads a month and 250 terabytes QUANTITY of egress data transfer, with a continuous month-over-month DATE increase as our user base grows. This highlights the importance of a highly available, global, and scalable infrastructure that provides low latency engine downloads to Prisma ORG users all around the world.

Our original solution: AWS S3 & CloudFront ORG

During the early development of the Prisma ORM, our engineering team looked for tools to build the CDN ORG for engine distribution. With extensive AWS ORG experience, we went with the obvious: S3 CARDINAL blob storage for the engine files and CloudFront ORG to cache contents closer to the user.

A simplified representation of how the Prisma Engines ORG flow from our CI where they are built and uploaded, to the Prisma CLI downloading the correct engine for a given environment when installing Prisma PERSON , all the way to the user being able to use it.

We were happy with AWS ORG for the most part, and it was able to scale with our demands. However, as our user base continued to grow, so did the costs. At our scale of traffic, data transfer became a considerable cost item that we knew would only continue to grow.

The continuously increasing cost of these services prompted us to explore alternative options that could better accommodate our needs while at least maintaining the same level of performance and reliability. Prisma is committed to providing the best products and solutions to our users, and an essential part of that commitment is being intentional about the allocation of our resources, including sensible spending to enable us to serve our growing user base in the best way possible.

Exploring distribution options

We extensively explored different technologies and services that provided both reliable and fast engine distribution, while being cost-effective.

Free solutions: GitHub & npm MONEY

Because Prisma ORM is an open-source solution, we have explored various ways to distribute the engines through our existing distribution channels, at no cost. In this area, we had both GitHub Releases and npm as candidates to host and distribute our engine files. We dismissed GitHub Releases early on as the quality of service was not guaranteed, which was a requirement for us towards our users, so we can be sure to provide a good developer experience under all circumstances.

We also looked at npm, and confirmed that hosting the engine files would be in agreement with their Terms of Service ORG . This made npm a viable option, but also meant we would have to change our engine download and upload logic to accommodate a different system. Additionally, this implied that we would have to update many past Prisma CLI versions, requiring our users to upgrade to take advantage of the new solution.

Paid solutions: CDNs & Cloudflare ORG

We then considered only replacing CloudFront ORG , which accounted for 97% PERCENT of our distribution costs, while retaining S3 ORG as the origin. When we evaluated different CDNs ORG , we found that alternatives could lead to an estimated PERCENT 70% cost reduction.

We also explored Cloudflare ORG ’s offerings and were impressed by Cloudflare R2 PRODUCT , an alternative to AWS ORG S3 + CloudFront ORG . It offers robust blob storage compatible with S3 ORG and leverages Cloudflare ORG ’s network for global low-latency distribution. Additionally, it has no egress costs, and is solely priced based on the total volume of data stored and operations on that data. Given our reliance on Cloudflare ORG ’s product portfolio for our Data Platform ORG , and extensive experience with their Workers ORG platform, we already had high trust in the quality of Cloudflare ORG ’s products.

To finalize our decision, we implemented a test to confirm our intuitions about Cloudflare ORG ’s quality of service. We deployed a script to 50 CARDINAL cities across the globe, representative of our incoming traffic, to measure download latencies for our engine files ( ~15MB QUANTITY ). The test was run multiple times, with latencies for the different cache statuses recorded and compared against our previous AWS ORG -based solution. The results confirmed that Cloudflare R2’s ORG reliability and performance were at least on par with AWS S3 ORG + CloudFront ORG . And because R2 is compatible with S3 ORG , we wouldn’t need to make substantial changes to our software in order to move over to Cloudflare ORG . These were great results, and we couldn’t wait to switch!

Our solution: moving to Cloudflare ORG ’s R2 PRODUCT

In order to move our engine file distribution to Cloudflare ORG , we needed to ensure we could make the switch without any disruption or impact to our users.

While R2 CARDINAL URLs match S3 ORG ‘s format, Prisma CLI PRODUCT uses a fixed domain to point to the engine file distribution. This fixed domain enabled us to transition without making any changes to the code of older Prisma PERSON versions, and simply point the existing URLs to R2 CARDINAL . However, to make the transition, we needed to change our DNS ORG configuration to point to Cloudflare ORG . While this seems trivial, potential issues like unexpected DNS propagation challenges, or certificate validation problems when connecting via TLS ORG , required us to plan ahead in order to proceed confidently and safely.

We modified the Prisma PRODUCT ORM release pipeline to upload assets to both S3 ORG and R2 CARDINAL , and used the R2 Super Slurper ORG for migrating past engine versions to R2 PRODUCT . This ensured all Prisma PERSON releases, past and future, existed in both places. We also established Grafana PERSON monitoring checks to pull engine files from R2 PRODUCT , using a DNS ORG and TLS ORG configuration similar to our desired production setup, but via an experimental domain. Those monitoring checks were later reused during the final traffic cutover to ensure that there was no service disruption.

As ensuring no impact or disruption to our users was of utmost importance, we proceeded with a gradual rollout of the DNS ORG changes using DNS ORG load balancing, a method where a group of alias PERSON records assigned to a domain are weighted differently. This meant that the DNS ORG resolver directed more traffic to heavier-weighted records. We began with a load balancing configuration simulating our old setup, with one CARDINAL record (the control) pointing to AWS CloudFront ORG , and the other (the candidate) pointing to R2 CARDINAL . Initially, all weight was on the control, effectively preserving the old routing to CloudFront ORG . We also set the lowest TTL ORG possible, so changes in the record weights took effect as soon as possible, creating more control over DNS ORG propagation. Additionally, we implemented a health check that would redirect all traffic to the control if download latencies were significantly higher, or if errors were detected, ensuring a stable fallback.

At this point, everything was in place and we could start the rollout.

Our DNS load balancing setup during the rollout. We assigned increasing weights to route traffic to Cloudflare R2 PRODUCT . The health check that would fail over to AWS CloudFront ORG never fired.

The rollout began with a gradual increase in R2 PRODUCT ‘s DNS ORG weight, and our monitoring dashboards showed that Cloudflare ORG downloads were proportional to the weight assigned to R2 CARDINAL . With as little as 5% PERCENT traffic routed to Cloudflare ORG , cache hit ratios neared 100% PERCENT , as expected. Latencies PERSON matched the control, so the health checks were all good, and our fallback never activated. Over the duration of an hour, we gradually increased R2 PRODUCT ‘s DNS ORG weight to manage 25% PERCENT , 50% PERCENT , and finally 100% PERCENT of traffic, without any issues. The cutover could not have gone any smoother.

After monitoring for an additional two days DATE , we simplified the DNS ORG topology and routed to Cloudflare ORG exclusively. We were extremely satisfied with the change, and started seeing our infrastructure costs drop considerably, as expected, not to mention the zero CARDINAL downtime and zero CARDINAL reported issues from users.

A success

Transitioning to Cloudflare R2 was easy thanks to their great product and tooling, intuitive platform and supportive team. We’ve had an excellent experience with their service, with consistently great uptime, performance and latency. Cloudflare proved once again to be a valuable partner to help us scale.

We are thrilled that our engine distribution costs have decreased by 98% PERCENT . Cloudflare ORG ‘s cost-effective solution has not only delivered top-notch performance but has also brought significant savings to our operations. An all around success!

To learn more about how Prisma PERSON is building Data DX ORG solutions with Cloudflare ORG , take a look at Developer Experience Redefined: Prisma & Cloudflare Lead the Way to Data DX.

And if you want to see Prisma PERSON in action, get started with the Quickstart guide.

Connecting to Connected... Page load complete