Writing a Caching Engine for Flash Player

Not to be confused with Object Pooling, I’m referring to saving data you need to persist in your application across sessions, and more specifically about saving images in your own local cache as opposed to the browser cache.

As usual, I ask a question on Flex coders and get no response. This usually arises from the fact that its an open ended question without a simple answer, and thus, would take a long time to answer… say, the amount of time it takes to write a blog entry like this. My question was in regards to building caching engines. I’d never done one, and figured all the Java/Ruby/Python/ColdFusion server-heads who do Flex would know / have experiences to share. Well, it’s a month later, so here’s how it went for me.

Why Cache?

Flash Player already has caching facilities. Most uses of Flash Player are in the browser and it utilizes the networking API’s of the browser it is within.  Thus it uses the caching abilities of said browser. If you load an image utilizing Loader, it’ll cache that image like any other image loaded from the web. If you load a text file utilizing URLLoader, same thing. Sometimes caching doesn’t work. There are a variety of reasons why this happens, but bottom line you can depend on the browser cache for loading normal assets.

Loading from cache, however, is not instantaneous. If you utilize an item renderer in a Flex/Flash DataGrid, and scroll the DataGrid, you’ll see what I mean. Ely created a SuperImage component that actually puts the bitmap in a stored list as BitmapData for later retrieval. Since retrieval from RAM is immediate, this allows you to show the image without a flicker. Combined with a lot of images, it is a very compelling visual. Scrolling through a bunch of images now looks visually a lot better.

For the software product I’m currently building, I have the need to load an insane amount of images. Amongst these images are an image, or multiple images of interest amongst a specific time period. Imagine a timeline you can scroll horizontally. The left is this morning, and the right is this evening. All the images represent video screen captures of people walking by your desk. You are trying to find the one that shows someone stealing your coffee. That’s basically what I’ve coded.

Another metaphor to use is like the CoverFlow components that abound, duplicating the user interface functionality iTunes popularized. You are looking for a specific album cover, so you scroll from left to right, right to left, looking for it. Same type of thing.

To do so implies you have the images already converted to BitmapData, and thus in RAM ready to use.

Bitmaps and ByteArray’s

Putting images in RAM is all well and good when creating user interfaces that need to show a lot of images quickly, but there are 2 problems. First, bitmaps take up a LOT of RAM. While Flash was a traditionally more of a CPU hog than a RAM hog, times have changed now that we can yuke mad pixels into memory. So, you need an effective way to grab a bunch of images, and throw them into RAM.

The second problem is that when you refresh the browser or come back to the application at a later time, you need to re-download the images again. They could be in the browser cache, yes, but you still need to load them into BitmapData objects.

If you’re interface needs to load data immediately, in my case images, the easiest way is to save them as ByteArray’s in a local SharedObject, aka, a Flash Cookie.

The quickest you can read and display those images is in 2 frames (aka 2 screen redraws). The first frame is loading all of the cached Bitmap ByteArray’s from the Flash Cookie, creating a bunch of Loader’s, and calling loadBytes on them. Even though Loader.loadBytes is asynchronous, it returns almost always (in my testing anyway) one frame later. This is really dang quick. That means you can load a bunch of saved Bitmaps from a Flash Cookie and display them extremely quickly; 2/30th’s of a second if your movie/app is running at 30 frames per second. The problem? RAM. The ByteArray’s you loaded are now doubled in size because the Loader.loadBytes create’s a separate memory space for the Bitmap data to draw. So, if you saved 20 megs of Bitmap’s, and then load them into your application for display, you’ve now got 40 megs.

Notes on Garbage Collection

To solve that problem, you merely null out the ByteArray’s. If you need to clean up the bitmaps, you first call bitmapData.dispose, and then null out their references. Doing this in the same stack (aka same frame) will usually initiate Garbage Collection to eat it. This is because, in theory, the same rules for Garbage Collection still apply in the new one implemented in Flash Player 8 (and still used in 9 and 10). The old 2 rules are:

1. GC will run every 60 seconds.
2. GC will run every time memory allocation jumps 20% or above in the current frame.

I have no clue if they still are true; the Macromedia peeps would randomly, uncomfortably acknowledge those rules, but never admit they were true for business reasons. Grant Skinner has more details if you’re interested. Now, you could utilize the LocalConnection hack to force GC to run, but I’ve found that when I do the technique above, GC runs every time. I’d use a timer to update the current System.totalMemory, and you could see it spike when I loaded the bitmaps in, and nose dive when I killed their references and called bitmapData.dispose. This leads me to believe #2 is still an accurate rule.

How to Cache Images

One weird thing about Flash Cookie’s is that it supports primitives, and ByteArray (since he’s in the flash.utils package), but not Bitmaps or BitmapData’s. You can put those in a local SO and flush it, but when you go to retrieve those objects later, they are cast to Object and not usable as DisplayObject’s anymore. My guess is they never got around to properly implementing IExternalizable interface (thanks for the tip, Steven!) for the Bitmap and BitmapData classes, so they don’t serialize/de-serialize properly.

You have one option beyond using Base64 encoded strings. You can utilize the PNG and JPEG encoders that come with Flex, or other image open source image encoders for ActionScript. They will take a BitmapData and give you a ByteArray back. A ByteArray DOES work in a local SharedObject. You then throw these guys in an array, save using some unique hashing system, and voila. To use them later, you read out the ByteArray’s, throw in a Loader using loadBytes, and boom, you’ve got your Bitmap back.

Keep in mind, if you aren’t doing what I’m doing, and saving gigabytes worth of images, you don’t have to call flush immediately.  This can be an expensive operation performance wise since it’s writing a local file.  If you can afford to keep the data around in memory, the SO will get written to disk when either GC eats the SO, or your browser closes.

The way we are getting images is I ask the server for a table of contents for a given date range and special ID.  If I get back an empty dictionary (Python’s “Object”), I just assume there are no images for that date and ID.  If I get back a dictionary full of data, I then make a request for an image blob.  This is basically a gigantic ByteArray (500k to 4 megs) of all of the images for the given date range.  Once downloaded, I use the table of contents to get my images from the blob.  Imagine a JPEG the size of a table, and the table of contents tells you where to “cut on the dotted lines” to get each image out.  Same theory.  Helps me get a TON of images in only 2 server calls instead of 200.  Additionally it helps server performance as well.

The downside is it takes awhile to download 3 megs and display it.  So, whilst downloading we’ll use a regular image request in a Flex image control (aka a Loader) to get an image to show while the big ones download in the background.  That way, a user can click on the timeline and see something while the ones that allow them to scrub for a range are downloaded in the background.  The point here is that caching does not have to replace utilizing the standard way of getting images from a server; both can be used in tandem.

Hashing

When I say hashing, I mean, creating a unique name for whatever Flash cookie I’m saving. I’ve found that saving many smaller Flash cookies is better than creating one mammoth one in terms of performance. Therefore, you have to have some way of finding all of those images.

Databases utilize primary key’s; usually a unique number starting from 1 on up to identify a row of data in a table. In my case, each set of images is associated with a particular ValueObject property and date accurate to days. So, if I want to ensure each block of images is unique, I can create a hash (or filename depending on how you look at it) using those variables like so:

private static const SO_NAME:String = "thumbnailsCache";
public function getHashName(date:Date, specialField:String):String  
{
        return SO_NAME + "-" + specialField + "-" + date.fullYear + "-" + date.month + "-" + date.date
}

 

Utilizing this unique name, I can then save and read SharedObject’s with assurance that they will be unique like the sample below shows:

var so:SharedObject = SharedObject.getLocal(getHashName(vo.date, vo.specialField));

 

Debugging

You’ll find that in creating your cache, it may get screwed up during development.  Maybe you didn’t save enough data, or saved the wrong data.  Or, your hash name format needs to change.  While there are tools out there for examining and manipulating Flash Cookies, I find it’s easier to just create a table of contents of all the SO’s you’ve saved, and then create methods to delete them all.  In my current classes, they have have 2 methods for saving data, save and read.  In the save method, you just add another line to save a pointer to it.

private static const SO_INDEX_NAME:String = "thumbnailsCacheTOC";
protected function saveIndex(date:Date, specialField:String):void
{
        var so:SharedObject = SharedObject.getLocal(SO_INDEX_NAME);
        so.data[getHashName(date, specialField)] = {date: date, specialField: specialField};
        so.flush();
}

 

Then, later if I need to delete all of the SharedObject’s I’ve ever saved, I can call my deleteAll method.  This method will loop through all of the SO’s I’ve saved, delete them, then delete the table of contents SO.

public function deleteAll():void
{
        var so:SharedObject = SharedObject.getLocal(SO_INDEX_NAME);
        for(var hashName:String in so.data)
        {
                var localSO:SharedObject = SharedObject.getLocal(hashName);
                localSO.clear();
        }
        so.clear();
}

 

Keep in mind this isn’t fool proof.  Since Flash Player’s local SharedObject’s are globally managed by the Flash Player and the settings manager online, you could have SharedObject’s deleted without your knowledge.  This could result in your Table of Contents being incorrect.  Code accordingly.

Handling Space Requirements

Flash Player defaults to 1 meg of space allowed by default per domain.  We only have 1 SWF in our domain doing this kind of caching, so it’s technically 1 SWF.  That said, 1 meg isn’t enough; I’m saving 2 megs on average per SO, and saving at least 30 or so of them.  The settings manager in Flash Player (right click on a SWF > settings) allows you to up the size to unlimited.  If, however, you know you’ll only utilize a certain amount, you can simply inform your user via some text that you’d like a certain amount, and then pass that certain amount in your flush method.  This’ll bring up the settings dialogue allowing them to allot more space… and return a pending for your flush operation.

What we’re probably planning on doing instead is creating a settings screen during application registration to ask for unlimited, and bring up the dialogue via System.showSettings ( SecurityPanel.LOCAL_STORAGE ).  No point in confusing the user with random dialogues.  I just assume if the return value from a flush is pending, it didn’t work.  Besides, there isn’t an easy way to correlate a given save operation with a SharedObject.onNetStatus of SharedObject.Flush.Success unless you start utilizing unique tokens for save operations.  That’s what?  A pain in the arse…  I really wish the Flash Communication Server / Flash Media Server classes like SharedObject were upgraded correctly to ActionScript 3 and supported event dispatching methods instead like everyone else.

Conclusions

Creating a caching engine has allowed my application to instantaneously show megs of images instantly where utilizing the browser cache would not.  The only trade off is I need to be extremely thorough in cleaning up not only my Bitmaps for garbage collection, but the rest of the app as well.  This ensures there is enough RAM around for 60 megs to magically appear.  I haven’t run into fragmentation problems yet, but if I do, I’ll report back.

For more generic ValueObjects, such as the kind to draw graphs, this is in the 24k range and is a lot easier.  I just keep in RAM during the session.  When the user logs back in, since I know the data in the past can’t change, it immensely speeds up the application because I don’t need to make multiple server calls beyond login.

I firmly believe that SharedObject’s weren’t created to do what I’m using them for.  That said, they work well.  If I were utilizing AIR, I could use the local SQLLite / database connectivity or local file reading/writing features instead.  Since I’m in a browser, and there are a no ways for Flash to integrate with a browser’s cache, local SharedObject’s are a the next best thing.