13/03/2024
Clustering - a behavior describing the handling of requests/responses within a Fastly POP by it's Nodes.
The clustering nodes
In a Point of Presence (PoP), there are several nodes that work independently. Each node has its own hash table to keep track of cached objects. When a user sends a request, if the node handling the request doesn’t have the requested object in its hash table, the request must go to the origin.
However, this process is inefficient because the chances of the node responsible for responding having the requested object in its cache are not high.
How can we make sure the origin server doesn’t get overloaded?
What we can do instead is create a single node responsible for making requests to the origin
Designate a single node responsible for the object
In this example, there are three users asking for a file. Each user’s request is handled by a separate node. If the requested file isn’t in the cache of the node handling the request, that node will go to another designated node (the chosen one) to fetch the data from the origin.
Reminder on the hash keys and hash table
The parameters used to generate the hash key are:
- The host like :
fastly.antoinebrossault.com - The URL :
/hello - And a generation ID :
1– e.g : 1
Hash for fastly.antoinebrossault.com/hello = 21357f4e1d9a-13a7bc88b63d-1
Recap of the caching behavior regarding of the hashtable :
- A request with a difference in either the Host or the URL will generate a different Hash
Key. - A cacheable response object from a request will be cached with that Hash Key on the
Hash Table. - When another request comes in and generates the same Hash Key, we can check for the matching Hash Key on the Hash Table and can serve the cached response object.
- No key match on the table? It’s not cached, it’s a MISS, fetch it from the backend.
Anatomy of a Point Of Presence :
In a point of presence, there are many nodes. A load balancer in front of these nodes decides which one will handle each incoming request.
Delivering and Fetching node logic in a POP
Here’s a simplified overview of the logic in a POP:
- 1) Request hits the load balancer.
- 2) Load balancer selects a delivering node.
- 3) In the delivering node:
- Check if the request’s hash is in the hashtable.
- If yes, return the object from the node cache.
- If not, proceed to the fetching node.
- 4) In the fetching node:
- Check if the request’s hash is in the hashtable.
- If not, go to the origin.
- 5) After fetching from the origin:
- Populate the cache of the fetching node.
- Send the object back to the delivering node.
- The delivering node caches the object and updates its hashtable.
- 6) Finally, the delivering node sends back the data to the user.
Schema :
Here’s a schema of the logic in a POP, to be honest I wanted to make it simpler, and I ended up with a not 100% clean schema, so sorry about this….
In this schema the black arrows are going forwards when the blue dashed lines represent the request going back.
When Delivering and Fetching node behaviors are disabled
PASS & Restart disable clustering
PASS :
When a request has a « PASS » instruction, caching is unnecessary, so we turn off the clustering behavior.
Restart :
Restart is a way to handle errors from the origin. In this case, we don’t cache the origin response; instead, we return it directly to the user.
A secondary fetching node as a fallback
The usual behavior is as I described earlier, which is shown in this diagram.
But what if the fetching node gets too busy or stops responding? In that case, the delivery node can choose a backup secondary node automatically. This logic is built into the algorithm.
Takeaways
• Clustering is a set of Node behaviors within a POP, that improves caching performance and reduces request load to the origin.
• Nodes can fulfil both the Deliver Behavior and the Fetch Behavior.
• The request hash key is used via algorithm to determine the Primary Node for that request. The Primary then acts as the Fetch Node.
• The Primary, when acting as the Delivery Node, will use the Secondary Node as it’s
Fetch Node.
• When the Primary Node is down, the Secondary Node acts as the Fetch Node.
• Different Hash Key = Different Fetch
