In a previous article, I discussed blogging software that runs on Google’s App Engine. I use similar software to run this blog; in this article, I discuss one scheme for adding page caching to the software.
The Problem: Using Too Much CPU
Converting the markup when we save the article is more efficient, but it means we have to store the generated HTML and reconvert all previously saved articles whenever we change the templates or the style sheet. It’s simpler to convert on the fly. If this strategy ends up causing a performance problem, we can always go back later and add page caching.
Well, this strategy does end up causing a performance problem. On every page view, GAE was dumping this message to my application’s log:
This request used a high amount of CPU, and was roughly 6.6 times over the average request CPU limit. High CPU requests have a small quota, and if you exceed this quota, your app will be temporarily disabled.
The number varies and is sometimes twice as high as 6.6.
The Solution: A Page Cache
I profiled my blog application and confirmed my assumption that the CPU usage was primarily caused by the conversion of RST to HTML. I decided to address this problem with a cache, on the theory that the RST-to-HTML markup was causing the CPU spike, rather than the database lookup.
GAE’s documentation further states:
The Memcache service provides your application with a high performance in-memory key-value cache that is accessible by multiple instances of your application. Memcache is useful for data that does not need the persistence and transactional features of the datastore, such as temporary data or data copied from the datastore to the cache for high speed access.
However, there’s a limit to how much stuff you can cram into your application’s memcache area. GAE can and will evict items from memcache due to “memory pressure” (i.e., if there’s too much stuff in there). Since my primary goal is to reduce CPU usage, rather than round-trip time to the data store, I decided to implement a two-level cache. The primary cache is the memcache service. The secondary cache is the data store. The theory is that it’s still “cheaper” to go to the data store cache than to re-render the page from RST.
I also decided to remove the preview IFRAME from the edit page, since I can always go to another browser tab or window and preview a draft article by entering the direct URL for it. That way, it only re-renders when I ask for it.
In the remainder of this article, I’ll discuss how I implemented the page cache, using the picoblog software I described in my previous article, Writing Blogging Software for Google App Engine. The modified picoblog software (as well as the original non-caching software) is available at http://software.clapper.org/python/picoblog/.
Data Model Changes
The first change is to add the appropriate data items for the
second-level cache. In
models.py, I added the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
The first class,
PageCacheEntry, represents a single cached page in the
data store. The second class,
PageCache, isn’t persisted; it’s the public
face of the second-level cache, and it serves as a means to manipulate the
page cache. That is,
PageCache is the class the rest of the code will
use; no code outside of
models.py will ever touch
With the data model changes finished, I created a
with the following code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133
TwoLevelCache is the actual cache used by the rest of the application.
Let’s take a brief tour of each method (except
get() retrieves an item from the cache.
- It first checks the Memcache service. If the page is there, it’s returned.
- If the page is not in Memcache, then
get()checks the data store-resident cache. If the page is there, then it’s re-added to Memcache (which could fail) and returned.
add() puts a rendered page in both Memcache and the disk cache.
delete() removes a page from both Memcache and the disk cache.
flush_all() clears both caches.
flush_for_article() is the most complicated method. It’s called when an
article has been edited and saved, and it attempts to be intelligent about
which parts of the cache to clear. The caller passes both the changed
article and a copy of the article before it was changed.
The uncaching rules are relatively simple:
- If the article is not a draft, and the title or the tags have changed, then every rendered page could be affected. The tags appear on every page, and the most recent n article titles appear on every page. (And it’s likely that the article being edited is a more recent one.) So, in this case, we clear the entire cache.
- Otherwise, uncache: the page itself, the pages of articles with the same tags, and the page that displays all articles in the same month as the changed article.
Those rules can be further refined, but they’re a good start at uncaching only what needs to be uncached.
Changes to the Admin Code
First, as I noted above, I removed the preview frame from the edit window. With the preview frame in place, I’m constantly paying the penalty of re-rendering the article. It’s not necessary, since I can always look at the draft article in another window.
Removing the preview frame is a matter of removing these lines from
1 2 3 4 5 6
Then, I changed the
SaveArticleHandler class in
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
These changes ensure that saving an article adjusts the cache
copy.deepycopy() method comes from the
Finally, I had to change the main request handlers in
use the cache.
All handlers in
blog.py already extend an
The necessary changes break down into just a few things:
AbstractPageHandler. This method will attempt to fetch the page from the cache, based on its unique path. If the page isn’t in the cache,
get()will call out to the subclass’s
do_get()method to render the page; then, it’ll cache the newly-rendered page.
Rename all the subclasses’
do_get(), and change those methods to return the rendered page instead of writing it directly to the HTTP response object.
status()method that subclasses can override to set the HTTP status to something other than 200. This is mostly for the
Here’s the new
get() method and
status() method in
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Both are pretty straightforward.
Let’s look at just one of the handlers to show the changes that
need to be made; the same change needs to be made to all handlers.
RSS2FeedHandler class is nice and small, so I’ll use that
RSS2FeedHandler looked before caching:
1 2 3 4 5 6 7 8 9 10 11
Here’s how it looks now:
1 2 3 4 5 6 7 8
The only handler that’s even slightly different is the
1 2 3 4 5 6 7 8 9 10
Note the addition of the
status() method, returning a 404 (“not found”)
That’s it. The blogging software now has a two-level page cache.
Did it help?
Based on profiling data, adding the page cache helped a lot. But, just as important, the blog now seems much more responsive, and there are far fewer “high CPU” warning messages in the log.
There are certainly other caching implementations and approaches one could use. For instance, my friend and colleague, Mark Chadwick, added caching to his GAE application using a python decorator; his code caches anything that a decorated function happens to return. The approach I outline in this article is just one implementation–one that happens to work well for me.
Related Brizzled Articles
- Writing Blogging Software for Google App Engine
- Making XML-RPC calls from a Google App Engine application
- Experimenting with Google App Engine, by Bret Taylor.
- Building Scalable Web Applications with Google App Engine (presentation), by Google’s Brett Slatkin.
- Google App Engine documentation