From 76% to 100% cache hits: Rebuilding docs caching on Cloudflare

Web Performance & Frameworks

About the session

Mintlify serves over 200 million monthly page views across thousands of developer docs sites, and our Next.js ISR caching couldn’t keep up with deploy velocity. Nearly one in four visitors hit a cold start, and our cache hit rate was stuck at 76%. This session shares the engineering story of how we rebuilt our caching layer on Cloudflare: a Worker proxy in front of all traffic, version-aware cache keys for HTML and RSC, webhook-driven revalidation, Durable Objects for distributed locking, and Cloudflare Queues for batched cache warming. With these, we effectively reached a 100% cache hit rate. You’ll leave this session with concrete, reusable patterns for caching a high-velocity multi-tenant app at the edge.
View transcript

0:03Hi, everyone. Thanks for coming to our talk. Nice to meet you. I'm Hanbi, and I'm a co -founder and CTO at Mitlify. Hi, everyone. I'm Veland. I'm a product engineer at Mitlify. And today, we are here to talk to you about how we eliminated cold starts for pages utilizing Cloudflare. So, if you aren't familiar with what Mitlify is... We specialize in public-facing developer documentation for companies. So think help guides, tutorials, API references, things of that nature. You use us to host the content, maintain it, customize the look and feel, and edit the content using us. We work with companies like Anthropic, Coinbase, Scale AI, companies of that nature.

0:56Now, you might be wondering, what's complicated about hosting static docs? Before we jump into the technical problem, I want to walk you through some of the product requirements. So I'm sure everyone here is a developer, and as developers, we really like things that are fast. And so for us as a company, it was really important to prioritize a user experience where the pages stay snappy. regardless of when you're visiting them and that when customers update the content that it updates quickly as well. And so to start off with we started with doing things that don't scale and we had what you're probably imagining a system where you have a single tenant and you're hosting it and like every time you make an update you're rebuilding the docs.

1:44Eventually we got to a point where we had enough customers. where there were so many builds going on simultaneously that it was not scalable. And so we switched to a model where essentially all the tenants were in one project and we were building all the pages during build time. However, we then reached to another scaling problem where you can't build like tens and thousands, like hundreds of thousands of pages during build time. Like the build time would just be far too long. And so. we switched to a concept called the isr cache which is something that first cell offers which isr stands for incremental static revalidation and essentially it allows you to build the page on first visit so whoever visited the page for the first time after the we released a new version like they would encounter a like a slightly slower experience like fetching the content and then every subsequent visit the will be cached and then we kind of run into more problems there and i'll let lilan speak to that

2:55yeah so this approach that hamby just talked about took us from zero to one uh but once we were at this point we quickly hit you know scaling limits and we were noticing that our customer pages were just very slow uh and the main case where this was happening was when we were having hits. So when a user requests would come in, it would be a miss in the ISR cache and then end up going all the way to our database and our backend and compiling the MDX, which was a very slow process. And this was something that was happening many, many times a day because we were essentially clearing the ISR cache every time we made a code deploy. So this was happening throughout the 10 plus times, depending on our shipping velocity.

3:49So the solution to this, or sorry, yeah, this was our cash hit rate, 76%, which essentially meant that one in four requests were a cold hit, which really wasn't something acceptable for our use case. So the solution here is to have a amalgamation of Cloudflare primitives.

4:18This diagram may look very over-engineered and over-complex, but it's my hope that during this presentation, I'll be able to convince you all that every primitive and every box in this diagram is really necessary for this system and the requirements that we need to fulfill.

4:38So starting off with. the proxy layer we have a docs proxy worker which is the core worker in this system its main job is to handle a user request fetch it from Cloudflare CDN and return it back to the user the way that we actually accomplished our 100 % cash hit rate is by utilizing stale wall revalidate for our client version changes. The reason why this is OK is because generally, our codebase changes don't need to be immediately reflected to the user. Usually, they're small UI changes or a small bug fix.

5:24Obviously, in the case where we're applying a hot fix, we do need to have a way to immediately bump everyone to the latest version. But in the general case, stale while revalidate works here.

5:36You might be thinking, though, like, if we're just doing stale while revalidate, why do we need all of this here? Why can't we just, you know, return a stale while revalidate tag from the Vercel origin and let Cloudflare handle it natively from there? The reason that we're not able to do this is because of an issue called version skew. And essentially version skew is when you have pages on different... deployment versions of the code base. So a quick example is if you have page 1, 2, and 3 all on Vercel deployment A. And we ship a new feature, a new change, and now we have deployment B as the latest version.

6:24When we do stale while we validate on a page by page level, we, for example, would go to page 1, and this would... you know, do still or without it, and we get eventually bumped to version B. When you navigate between version B and, you know, other pages on different versions, you get this nasty, like, flash-in full-page refresh behavior, as well as, like, inconsistency if we change the navigation bar layout, for example. So what this means is that we need to perform... Stale while revalidate at the sitemap level So yeah, that's that's the general problem statement here and why the system we need we created Kind of needs to be this complex Yeah, we're essentially manually implementing stale while revalidate for a whole sitemap

7:23So in order for us to do this the first thing we need to figure out is is how do we actually version and serve stale content?

7:35The first thing to do this is having a way to actually fetch different versions of a client. Thankfully, this is something that we're able to natively handle with Vercel, going back to deployment A and deployment B. So we're able to pass to the origin as a header, a specific deployment. deployment version in order to fetch that version of the build. So that's how we're able to fetch specific versions. We also need a way to version the cache to have that also align with the version that we're fetching. If we didn't version the cache key, we would have, you know, all these different versions because the deployments kind of overriding each other in the cache and we it would lead to the same version SKU issue.

8:29So that's why in our cache key, we have the cache prefix, which corresponds to the customer. And then we also have the Vercel deployment ID, which corresponds to the deployment that we want to fetch. And what this allows us to do is we can have, like we can have a deployment, or sorry, not a deployment, a... customer doc site pointing to a specific deployment. So it can continue pointing to deployment A, the old version. And what we're able to do in the background is warm deployment B into the cache. And then the last component of this is being able to atomically cut over the current deployment that we're serving.

9:14So if we take just a generic docs.com, for example, we need a way to say. for docs .com right now we're serving deployment a and then once the revalidation completes and like everything for deployment b is loaded into the cache we need a way to say for docs .com serve b now the way the way we do this is by having a domain to deployment mapping that's held in the kv and this is read on the hot path and it determines what deployment we serve and then also what goes into the cache key at that moment

10:00yeah so the next thing we need to consider is how do we know when something's actually stale and how do we know when we want to kick off this whole revalidation flow obviously because it's a revalidation at the site level it's quite expensive for doc sites with thousands of pages it's thousands of requests that end up going to our back end so we want to know exactly when to to kick off this workflow so kind of just logically speaking you need two things you need a way to know what the latest version is and then you also need to know what the version is being served for specific response so yeah what we have to keep track of the latest version is honestly quite simple we just have a webhook on when whenever a new deployment goes into Vercel and that's written to DKV and mapped by like the project version and then the way we're able to detect a mismatch between what's being currently served and what the latest version is by simply using the response headers from whatever's returned from the page.

11:19So we're able to have a got version, which is the version that we got from a page response. And then we're also able to compare that to the want version, which is the canonical latest version. And then when you have this mismatch, this is when we're actually kicking off this work. flow so that's when should revalidate evaluates to true so now getting into you know the meat of the problem how do we actually do an atomic site map revalidation this is the you know final approach but we went through several iterations to actually get to this point Our original approach scratching all of this stuff was just in this layer within the docs proxy worker what we had was a context dot wait which allows us to you know like run a function after after the worker has been invoked without blocking the request so then when we detected this version mismatch that's when we kicked off that revalidation flow and ran it within one worker invocation the first problem that we ran to here was that every time we publish a new version there is a you know a huge revalidation

12:52storm and a big thundering herd issue this is because you know these the stale wall revalidate process isn't happening instantly so while it's still revalidating we're continuing to like re like every other requests is also triggering its own revalidations so the way we solved this was you know we needed we we knew that we needed to have some sort of state right and the way we handled this was with a revalidation durable object uh what this did was that it effectively just acted like a lock and only allowed for one revalidation job to be running at the same time so instead essentially moving whatever we had in the context.wait into the execution of a durable object.

13:41Now, this worked when pages in a sitemap were less than 500 or so. But once we scaled past that, we quickly ran into the execution time limit of a single invocation of a durable object. So then that's what led us to our current architecture where we have the revalidation durable object essentially queuing all the requests for a given sitemap to a revalidation queue and having it consumed by the revalidation And it was also a lot faster because within the durable object, we were only able to have six requests in flight at a given time.

14:27So with the queue in the worker system, we were able to shod that. out much more effectively yeah so but one complexity that this approach introduced was the fact that now all of our warming requests are sent out in batches right so what the durable object is doing is is taking you know for example 1,000 page splitting that up into a bunch of batches and firing that off to the queue so we also needed a way to know when all those jobs or all those batches were finished the way we were able to do that was using the you know sequel light backed durable objects and we would have sequel light tables for you know revalidation jobs that were currently running as well as like essentially rows for every single batch that we sent out and then we'd only be able to mark a revalidation as complete once every single one of those batches like returned successfully.

15:32And we know that all the pages were successfully warmed.

15:38And then once we have that, we're able to release the lock on the durable object so that future revalidations can happen as well as updating the KV, which as you remember before, holds the mapping to which deployment we're serving so once B deployment B or deployment ID B is warmed into the cache we're able to serve for Docs .com B instead of A

16:13and yeah so that solves the case where we're making updates to our own code base but it doesn't solve the issue where customers are updating their own docs. The reason is is because we're not able to perform stale while revalidate for our customer docs. This is because it's part of Mitlefy's value prop to be a platform that provides an easy and most importantly a very fast way to update their docs. So it just wasn't an acceptable user experience to have. still will revalidate flow so what we actually do on customer doc updates is we do a cache clear on you know either pages that were just updated or the entire site map depending on the type of update made and after that we do a similar flow where we still warm everything into the cache but instead of instead of flipping you know the flag or flipping what we're serving after fact we're doing a premature cache clear so that um in this way we get the benefit of you know having fast cached pages as well as quick updates yeah uh so the result of these changes is that we're able to bring our cash hit rate from around 76 which was

17:45very bad to effectively 100% for our end users. And because of the granularity that Cloudflare allows us to have with our cache, we're able to have a 15-day Edge TTL. So, you know, we're able to keep those pages in the cache for a long time. Yeah, and just to reiterate and wrap things up, you know, things that were important to us was that the pages are always fast, regardless of how quickly frequently our team is shipping and that the updates happen quickly when customers expect them to change. And yeah, like doing this at a massive scale. Not only do we have like tens of thousands of customers on our platform, but we also have hundreds of million developers reading the docs every single month.

18:37And so we need to ensure that that experience is fast because otherwise you know how developers are they're super finicky in terms of ensuring that things are fast. And in case you're also looking to be a founder one day or build your own multi -tenant SaaS application these are the primitives that we encourage you to use in order to build a system that's similar. And I know this was a very technically dense presentation and we 20 minutes with you guys today. If you're interested in learning more, feel free to read the full write -up on our blog. And also, if these problems are interesting to you, please feel free to reach out to either me or Leland on LinkedIn or X, and we'd be more than happy to chat about engineering opportunities at Mitlify as well.

Browse all
Connect 2027

Connect returns in 2027

Be the first to know when registration opens and the agenda goes live.

By submitting this form, you agree to our Privacy Policy.

cloudflare-connect-2026

---
title: "From 76% to 100% cache hits: Rebuilding docs caching on Cloudflare"
url: "https://www.cloudflare.com/connect/2026/session/from-76-to-100-cache-hits-rebuilding-docs-caching-on-cloudflare"
year: 2026
---

# From 76% to 100% cache hits: Rebuilding docs caching on Cloudflare

[Watch session](https://www.cloudflare.com/connect/2026/session/from-76-to-100-cache-hits-rebuilding-docs-caching-on-cloudflare)

## Session details

- **Track:** Web Performance & Frameworks
- **Topics:** App modernization, Cloudflare product innovations, Programmable Platforms
- **Solution areas:** Developer Platform, Core Platform

## About this session

Mintlify serves over 200 million monthly page views across thousands of developer docs sites, and our Next.js ISR caching couldn’t keep up with deploy velocity. Nearly one in four visitors hit a cold start, and our cache hit rate was stuck at 76%. This session shares the engineering story of how we rebuilt our caching layer on Cloudflare: a Worker proxy in front of all traffic, version-aware cache keys for HTML and RSC, webhook-driven revalidation, Durable Objects for distributed locking, and Cloudflare Queues for batched cache warming. With these, we effectively reached a 100% cache hit rate. You’ll leave this session with concrete, reusable patterns for caching a high-velocity multi-tenant app at the edge.

## Speakers

- [Hahnbee Lee](https://www.cloudflare.com/connect/speakers/2512893) — Founder and CTO, Mintlify

## Transcript

[0:03] Hi, everyone. Thanks for coming to our talk. Nice to meet you. I'm Hanbi, and I'm a co -founder and CTO at Mitlify. Hi, everyone. I'm Veland. I'm a product engineer at Mitlify. And today, we are here to talk to you about how we eliminated cold starts for pages utilizing Cloudflare. So, if you aren't familiar with what Mitlify is... We specialize in public-facing developer documentation for companies. So think help guides, tutorials, API references, things of that nature. You use us to host the content, maintain it, customize the look and feel, and edit the content using us. We work with companies like Anthropic, Coinbase, Scale AI, companies of that nature.

[0:56] Now, you might be wondering, what's complicated about hosting static docs? Before we jump into the technical problem, I want to walk you through some of the product requirements. So I'm sure everyone here is a developer, and as developers, we really like things that are fast. And so for us as a company, it was really important to prioritize a user experience where the pages stay snappy. regardless of when you're visiting them and that when customers update the content that it updates quickly as well. And so to start off with we started with doing things that don't scale and we had what you're probably imagining a system where you have a single tenant and you're hosting it and like every time you make an update you're rebuilding the docs.

[1:44] Eventually we got to a point where we had enough customers. where there were so many builds going on simultaneously that it was not scalable. And so we switched to a model where essentially all the tenants were in one project and we were building all the pages during build time. However, we then reached to another scaling problem where you can't build like tens and thousands, like hundreds of thousands of pages during build time. Like the build time would just be far too long. And so. we switched to a concept called the isr cache which is something that first cell offers which isr stands for incremental static revalidation and essentially it allows you to build the page on first visit so whoever visited the page for the first time after the we released a new version like they would encounter a like a slightly slower experience like fetching the content and then every subsequent visit the will be cached and then we kind of run into more problems there and i'll let lilan speak to that

[2:55] yeah so this approach that hamby just talked about took us from zero to one uh but once we were at this point we quickly hit you know scaling limits and we were noticing that our customer pages were just very slow uh and the main case where this was happening was when we were having hits. So when a user requests would come in, it would be a miss in the ISR cache and then end up going all the way to our database and our backend and compiling the MDX, which was a very slow process. And this was something that was happening many, many times a day because we were essentially clearing the ISR cache every time we made a code deploy. So this was happening throughout the 10 plus times, depending on our shipping velocity.

[3:49] So the solution to this, or sorry, yeah, this was our cash hit rate, 76%, which essentially meant that one in four requests were a cold hit, which really wasn't something acceptable for our use case. So the solution here is to have a amalgamation of Cloudflare primitives.

[4:18] This diagram may look very over-engineered and over-complex, but it's my hope that during this presentation, I'll be able to convince you all that every primitive and every box in this diagram is really necessary for this system and the requirements that we need to fulfill.

[4:38] So starting off with. the proxy layer we have a docs proxy worker which is the core worker in this system its main job is to handle a user request fetch it from Cloudflare CDN and return it back to the user the way that we actually accomplished our 100 % cash hit rate is by utilizing stale wall revalidate for our client version changes. The reason why this is OK is because generally, our codebase changes don't need to be immediately reflected to the user. Usually, they're small UI changes or a small bug fix.

[5:24] Obviously, in the case where we're applying a hot fix, we do need to have a way to immediately bump everyone to the latest version. But in the general case, stale while revalidate works here.

[5:36] You might be thinking, though, like, if we're just doing stale while revalidate, why do we need all of this here? Why can't we just, you know, return a stale while revalidate tag from the Vercel origin and let Cloudflare handle it natively from there? The reason that we're not able to do this is because of an issue called version skew. And essentially version skew is when you have pages on different... deployment versions of the code base. So a quick example is if you have page 1, 2, and 3 all on Vercel deployment A. And we ship a new feature, a new change, and now we have deployment B as the latest version.

[6:24] When we do stale while we validate on a page by page level, we, for example, would go to page 1, and this would... you know, do still or without it, and we get eventually bumped to version B. When you navigate between version B and, you know, other pages on different versions, you get this nasty, like, flash-in full-page refresh behavior, as well as, like, inconsistency if we change the navigation bar layout, for example. So what this means is that we need to perform... Stale while revalidate at the sitemap level So yeah, that's that's the general problem statement here and why the system we need we created Kind of needs to be this complex Yeah, we're essentially manually implementing stale while revalidate for a whole sitemap

[7:23] So in order for us to do this the first thing we need to figure out is is how do we actually version and serve stale content?

[7:35] The first thing to do this is having a way to actually fetch different versions of a client. Thankfully, this is something that we're able to natively handle with Vercel, going back to deployment A and deployment B. So we're able to pass to the origin as a header, a specific deployment. deployment version in order to fetch that version of the build. So that's how we're able to fetch specific versions. We also need a way to version the cache to have that also align with the version that we're fetching. If we didn't version the cache key, we would have, you know, all these different versions because the deployments kind of overriding each other in the cache and we it would lead to the same version SKU issue.

[8:29] So that's why in our cache key, we have the cache prefix, which corresponds to the customer. And then we also have the Vercel deployment ID, which corresponds to the deployment that we want to fetch. And what this allows us to do is we can have, like we can have a deployment, or sorry, not a deployment, a... customer doc site pointing to a specific deployment. So it can continue pointing to deployment A, the old version. And what we're able to do in the background is warm deployment B into the cache. And then the last component of this is being able to atomically cut over the current deployment that we're serving.

[9:14] So if we take just a generic docs.com, for example, we need a way to say. for docs .com right now we're serving deployment a and then once the revalidation completes and like everything for deployment b is loaded into the cache we need a way to say for docs .com serve b now the way the way we do this is by having a domain to deployment mapping that's held in the kv and this is read on the hot path and it determines what deployment we serve and then also what goes into the cache key at that moment

[10:00] yeah so the next thing we need to consider is how do we know when something's actually stale and how do we know when we want to kick off this whole revalidation flow obviously because it's a revalidation at the site level it's quite expensive for doc sites with thousands of pages it's thousands of requests that end up going to our back end so we want to know exactly when to to kick off this workflow so kind of just logically speaking you need two things you need a way to know what the latest version is and then you also need to know what the version is being served for specific response so yeah what we have to keep track of the latest version is honestly quite simple we just have a webhook on when whenever a new deployment goes into Vercel and that's written to DKV and mapped by like the project version and then the way we're able to detect a mismatch between what's being currently served and what the latest version is by simply using the response headers from whatever's returned from the page.

[11:19] So we're able to have a got version, which is the version that we got from a page response. And then we're also able to compare that to the want version, which is the canonical latest version. And then when you have this mismatch, this is when we're actually kicking off this work. flow so that's when should revalidate evaluates to true so now getting into you know the meat of the problem how do we actually do an atomic site map revalidation this is the you know final approach but we went through several iterations to actually get to this point Our original approach scratching all of this stuff was just in this layer within the docs proxy worker what we had was a context dot wait which allows us to you know like run a function after after the worker has been invoked without blocking the request so then when we detected this version mismatch that's when we kicked off that revalidation flow and ran it within one worker invocation the first problem that we ran to here was that every time we publish a new version there is a you know a huge revalidation

[12:52] storm and a big thundering herd issue this is because you know these the stale wall revalidate process isn't happening instantly so while it's still revalidating we're continuing to like re like every other requests is also triggering its own revalidations so the way we solved this was you know we needed we we knew that we needed to have some sort of state right and the way we handled this was with a revalidation durable object uh what this did was that it effectively just acted like a lock and only allowed for one revalidation job to be running at the same time so instead essentially moving whatever we had in the context.wait into the execution of a durable object.

[13:41] Now, this worked when pages in a sitemap were less than 500 or so. But once we scaled past that, we quickly ran into the execution time limit of a single invocation of a durable object. So then that's what led us to our current architecture where we have the revalidation durable object essentially queuing all the requests for a given sitemap to a revalidation queue and having it consumed by the revalidation And it was also a lot faster because within the durable object, we were only able to have six requests in flight at a given time.

[14:27] So with the queue in the worker system, we were able to shod that. out much more effectively yeah so but one complexity that this approach introduced was the fact that now all of our warming requests are sent out in batches right so what the durable object is doing is is taking you know for example 1,000 page splitting that up into a bunch of batches and firing that off to the queue so we also needed a way to know when all those jobs or all those batches were finished the way we were able to do that was using the you know sequel light backed durable objects and we would have sequel light tables for you know revalidation jobs that were currently running as well as like essentially rows for every single batch that we sent out and then we'd only be able to mark a revalidation as complete once every single one of those batches like returned successfully.

[15:32] And we know that all the pages were successfully warmed.

[15:38] And then once we have that, we're able to release the lock on the durable object so that future revalidations can happen as well as updating the KV, which as you remember before, holds the mapping to which deployment we're serving so once B deployment B or deployment ID B is warmed into the cache we're able to serve for Docs .com B instead of A

[16:13] and yeah so that solves the case where we're making updates to our own code base but it doesn't solve the issue where customers are updating their own docs. The reason is is because we're not able to perform stale while revalidate for our customer docs. This is because it's part of Mitlefy's value prop to be a platform that provides an easy and most importantly a very fast way to update their docs. So it just wasn't an acceptable user experience to have. still will revalidate flow so what we actually do on customer doc updates is we do a cache clear on you know either pages that were just updated or the entire site map depending on the type of update made and after that we do a similar flow where we still warm everything into the cache but instead of instead of flipping you know the flag or flipping what we're serving after fact we're doing a premature cache clear so that um in this way we get the benefit of you know having fast cached pages as well as quick updates yeah uh so the result of these changes is that we're able to bring our cash hit rate from around 76 which was

[17:45] very bad to effectively 100% for our end users. And because of the granularity that Cloudflare allows us to have with our cache, we're able to have a 15-day Edge TTL. So, you know, we're able to keep those pages in the cache for a long time. Yeah, and just to reiterate and wrap things up, you know, things that were important to us was that the pages are always fast, regardless of how quickly frequently our team is shipping and that the updates happen quickly when customers expect them to change. And yeah, like doing this at a massive scale. Not only do we have like tens of thousands of customers on our platform, but we also have hundreds of million developers reading the docs every single month.

[18:37] And so we need to ensure that that experience is fast because otherwise you know how developers are they're super finicky in terms of ensuring that things are fast. And in case you're also looking to be a founder one day or build your own multi -tenant SaaS application these are the primitives that we encourage you to use in order to build a system that's similar. And I know this was a very technically dense presentation and we 20 minutes with you guys today. If you're interested in learning more, feel free to read the full write -up on our blog. And also, if these problems are interesting to you, please feel free to reach out to either me or Leland on LinkedIn or X, and we'd be more than happy to chat about engineering opportunities at Mitlify as well.

## Related sessions

- [A runtime for developers and their agents](https://www.cloudflare.com/connect/2026/session/a-runtime-for-developers-and-their-agents)
- [100 billion profiles, one early bet on Cloudflare Durable Objects](https://www.cloudflare.com/connect/2026/session/100-billion-profiles-one-early-bet-on-cloudflare-durable-objects)
- [TanStack Start, and the rest of the TanStack ecosystem](https://www.cloudflare.com/connect/2026/session/tanstack-start-and-the-rest-of-the-tanstack-ecosystem)

## More from Connect

- [All sessions](https://www.cloudflare.com/connect/)
- [Get notified for Connect 2027](https://www.cloudflare.com/connect#2027)