Post Microsoft MIX 2008 Thoughts

If you are in a hurry, here are some links with excerpts about the section.


Continue reading “Post Microsoft MIX 2008 Thoughts”

I have a hard time writing slow ActionScript 3

I have a problem going backwards. When I learned attachMovie in Flash 5, that function was the death knell for my Director career. “Create something from nothing!?!?” :: queue Quake sound :: “GODLIKE!!! (nsfw)“.

In Director, everything you wanted to be on a depth had to have had something there originally, put there on the “score” (Director’s timeline). If you didn’t do this at author-time, you couldn’t swap things out on that depth. The attachMovie function in Flash was great because this didn’t matter; I had thousands of depths to play with; the sky was the limit!

I later learned the sky on _root had a limit; it was called the Settings Panel. Lakitu wouldn’t even go near that elevation. It didn’t matter though; creating new MovieClip’s allowed millions of depths that designers & coders never used; it was wonderful.

To be fair, you could abstract this stuff in Director. The crew over at Director-Online had at least 2 sprite engines that emulated what Flash did, with extra features. Furthermore, I couldn’t really appreciate this stuff back then anyway; I couldn’t even return values from functions. I just had functions set global variables. Class? Que? n00b.

I tried going back after I learned some things, but I just couldn’t do it. Flash offered too much. It was soo fast. So much more effective. ActionScript just seemed more organized than Lingo.

After getting over my frustration at AS1, I learned that I could create re-usable stuff. While I loved to re-invent the wheel ALL THE FRIKIN’ TIME, it did get tiring after awhile, especially when my 4th generation Scrollbar still lacked track clicking and scroll buttons. Creating re-usable tools was hot.

After getting over my furious frustration with AS2, and learning enough design patterns to pretend like I knew what I was doing, I slowly developed a loathing for AS1. I then had a hatred for anything not AS1; aka, code puked on frames.

I remember seeing a lot of creative things done with prototype in ActionScript 1, however, and even some still in AS2. So, I tried to go back, and experiment. I always ended up getting bit in the arse in the end.

When AS3 came out, the early version, it actually wasn’t that different (for me) from AS2. AddChild vs. createNewMovieClip, aka the new DisplayList was pretty easy. Protected vs. private, and other namespace features made sense pretty quickly too. The only hard thing was learning the immensely large new API. It’s one thing to remember that a CheckBox emits both a click AND a change event in the Flex SDK, and you need to care about the latter… it’s another to remember that navigateToURL is in the flash.utils.* package. Even the built-in stuff is immensely large.

What is interesting, though, is that I never developed a hatred for AS2. Because of the complete and utter lack of communication between the 2 AVM’s, combined with Flex’ slow rise to power meant a lot of parallel AS2 and AS3 work. If you asked me if I wanted to do an AS2 or AS3 project, I’d probably respond AS3, unless your deadline was a week or less.

…and that’s my point. Writing AS3 is slow. It’s finally reached a maturity as a programming language; a real language, not a “scripting language” because it now officially compiles to machine code on 3 differentchip sets. As a real language, it has all the features (most, hehe… private constructor what, abstract classes who?) a traditional programmer would expect from a programming language.

My boss, a professional Java Developer who now does a lot of PHP (and I’m sure has a larger illustrious career I don’t know about yet), said that a strongly-typed language should be making me faster, not slower.

In shock and anger, I pulled my phone from my pocket, called up “Bullshit”, and requested he leap upon my bosses’ head.

There is a reason I click the “Flash” icon on my task bar, do File > New, hit F9, and test a new String parsing algorithm there in AS3 instead of going the ActionScript project route in Flex Builder… or even “new ActionScript File” and making it launching it as an Application. Flash is faster. Sure, once you get rolling, you can have some good test bed code combined with test cases even run by ANT in Flex. I’d still argue the instant gratification in Flash beats Flex.

…but it doesn’t stop there. I mean, after hearing about things such as “int promotion” and “compiler optimizations based on clear, strongly-typed variables”, AS3 can get out of control. Parsing a pipe (|) delimited string, properties separated by commas (,), with name value pairs separated by equal signs (=) has gone from 15 lines in pre AS1 to 40 in AS3 if you ensure every parsed variable is strongly typed. For an algorithm that’s only run one time very rarely, AS3 doesn’t really benefit here.

Fine? Then why not NOT strongly type so much? Only strongly-type to ensure you don’t get any type casting exceptions, and move on with life?

…because… I … can’t. It’s hard to stop. I’ve been trained by the fear of the software engineer army to strongly-type everything. To ensure the compiler has a crystal clear understanding of everything, and that my code will run at moch-10. I can have faith that the old AVM will never even need gas, because the key in the ignition for it will never be turned. All the code I write now ensures everything is typed, and all namespaces are correct.

I’ve tried abandoning namespaces and using *, Object, and even just abandoning type-casting in general for quick tests or prototypes. It’s just as time goes on, I find the code I add to those things starts getting more AS3 like.

Bottom line, it’s really really hard to break the habit. Every time I go back to AS2 for some random project, I’m reminded how damn fast it is to create things in it. Half the exceptions I get in AS3 I really don’t care about, and don’t need to know about; they don’t impact the project from working if they are small in scope. A lot of my data is either primitives or simply arrays.

For smaller projects, AS2 is still the bomb. For smaller algorithms, less strong-typing just results in less, more readable code.

I can see why professional Ruby, JavaScript, and Python developers still exist. I don’t care what anyone says, loosely typed scripting languages still kick ass. While I wish their engines weren’t so damn slow for larger projects, lately, I’ve just been finding that for a lot of smaller scoped projects or areas, I’m glad I have the loosely typed option.

While I fully believe the majority of the industry will use C# for Silverlight 1.1 going forward, MAN was it refreshing to write it in JavaScript for 1.0. Taking a break from AS3’s strict ways was great. SmartFoxServer has an option similar to later builds of Red5. You can write in the standard Java for performance reasons OR in your scripting language of choice (JavaScript, Python, etc). For a lot of work, scripting languages are good enough, and since they are good enough, you can write less code, faster to get the same result. That last part is debatable, but not by me. After having a career in both, to me they both clearly have their place.

…what I can’t figure out is how to drop the “good habits” that AS3 teaches you to go back to fast and furious ways of scripting languages.

Invalidation Strategies for Flash Player

Skip Intro

Had the honor of meeting up with an old friend yesterday. Chris Skogen, one of the talented gents at Gizmolabs, was my manager & tech lead when I worked at BellSouth. He’s an extremely talented programmer, doing a lot of C, C++, assembly, and Java. Below is some of his team’s coding + hardware work in action. This is not actually the record music playing, but rather, the record’s time code being used to scrub an MP3. Plethora of other features but… isn’t this one alone enough to make you go, “OMFG!!!”

He’s been doing it for a long time, and since I respect anyone who has a lot of experience in this industry, I pretty much spent the day absorbing every word. We were supposed to meet for lunch since her majesty and I skipped out on going to a Christmas party of a friend of his that we RSVP’d too. Figured, I could buy him lunch and he’d forgive me. Anyway, instead of staying an hour at noon, stayed till 7pm mainly talking shop, finishing with Gears of War coop. He’s the guy that got me typing my brackets on a new line after he and another C coder at BellSouth started making fun of my if statements (Flex Builder is biased against this way of coding, btw…bleh !). He was also the guy who first introduced me to the Command pattern, and wondered if the Flash community had any framework like that. Four months later, I was all into ARP, which has the Command pattern as its central crux.

I spent some of the time asking about invalidation methods. Invalidation being defined as “painting the screen with pixels in an efficient manner”, also known as redraw. Since he develops for multiple platforms, he knows the libraries and strategies involved for screen redrawing on Linux, OSX, Windows, and various devices. I remember one of Bruce Epstein’s books, Director in a Nutshell describing “The Great White Flash”. It’s been so long I can’t really remember the cause, but the redraw of the window caused the application to start with this white flash, basically a redraw of the screen that sabotaged the initial experience with various ways around it. When Chris would describe Windows redraw events compared to other platforms, it sounded all too familiar. The Director mantra back then was, “If you build it on PC, you know for a FACT it’ll work just fine on a Mac. If you develop on a Mac, it will NOT work on a PC.” A lot of the intricacies of when something redraws an area, how you have to manage it, etc. seemed really gross, and painful. However, like a lot of C guys I’ve talked to, he seemed to take it stride, view it as a conquered fiefdom, and begin to relay the double-buffering strategy employed in some cross platform library. Must… get… that… attitude!!!!

In particular, I recently had the good fortune of getting Greg Burch’s, bloke employed at Adobe on the Mobile Team, feedback on my Flash Lite 2 component set. The guy clearly knows his stuff, has massive knowledge in ActionScript optimization, and is very focused in achieving that goal. He also understands mobile GUI development as well as the experience users expect on devices. After talking to him on the phone, I can clearly see why he is employed with Adobe on the Mobile Team. In our discussions, both offline and on, he basically set out in the opposite direction as me. While I was originally interested in creating a nice API as a tertiary benefit to the original 3 goals, he instead approached it from a “we must make it lightweight”. Even 10 seconds of his recounting of past projects with no supportive details clearly indicates that he has earned in blood a lot of great techniques and understandings. Basically, he is making the code look as much like ActionScript 1 as possible (even C for that matter), not as an intention, just a result that optimization and ActionScript 2 & OOP do not go hand in hand. While he is confident and happy with these potential changes, I was somewhat disheartened. His suggestions all met the first goal of “if it runs Flash Lite 2, the framework needs to work”, but I felt it was losing a lot of it’s utility to developers; they would now have to do a lot more work (typing basically, and not all of it comfortable). However, if you are going to run under the insane CPU & RAM restraints, no one is left unscathed. Welcome to the suck.

After talking to Chris, however, he pretty much corroborated everything Greg was suggesting and had little empathy for my forlorn yearning for OOP. This lead to a very interesting discussion about inheritance vs. interfaces. He’s a big believer in programming by contract; having classes implement interfaces instead of using the extends keyword. After hearing him recount the challenges of being “a slave to the base class”, I totally understand and agree… for large scale projects with uber-knowledgeable individuals. If you are on a deadline, and have a variety of skill sets on your team, abstracting details seems to me like a good thing. I hate those projects, though,hehe! Flash Lite, however, has no room for interfaces as they too are basically objects like classes, and take up RAM. I digress I know, but it’s fun stuff.

Anyway, let me give some context first. There are multiple ways to redraw things to the screen in the Flash Player, with different repercussions in performance and developer approach. You basically have the following options in Flash Player 9:

  1. DisplayList
  2. Blitting
  3. Drawing

Both blitting and drawing depend on the DisplayList, but they are not mutually exclusive. The DisplayList is the act of adding a graphic object to a list of “things to draw”. If the Flash Player see’s it in there, it’ll get drawn. You can remove the object from the list, and it’ll no longer be drawn. This has a lot of positive repercussions in that a graphical object can exist in RAM without taking up CPU resources being drawn. Flash Player 8 and below did not do this. If you created a graphical object, like a MovieClip for example, even if it was invisible, it would still take up system resources being drawn every frame.

Suddenly, the programmer now has control over what is drawn, and when. This makes coding easier, too, because you can choose when to draw things, therefore writing more scalable & efficient applications… just what Flex needed when you started writing these gigantor Enterprise apps that made Flash Player 7 & 8 scream in pain.

There is something, however, more efficient than the above. This is called blitting. It is the act of painting a bitmap to the screen. A Bitmap is a DisplayObject in Flash Player 9, meaning it can be painted, or not painted, to the screen at the developer’s discretion. While Flash Player 9 has removed the problem of “too many objects == too much system resources”, Bitmap never has this problem even if those objects are in the billions. This is because you only ever have 1 bitmap (usually) that you paint on top of. Therefore, you actually have multiple small objects in memory that are then redrawn at some interval to the screen, and ONLY those areas that have changed. This is actually what is probably under the hood of the Flash Player, written in C/C++, that is redrawing all of your DisplayObjects. However, if you know going in that you’ll have a lot of objects which aren’t interactive, blitting is a wonderful technique that scales well, uses less RAM than the DisplayObject , and CPU is minimal because Bitmaps don’t use a lot of CPU to redraw compared to vector graphics. This technique can be used in the Flash Player 8 as well, although, there is no DisplayList so even if the Bitmap isn’t being visibly shown, it is still taking up CPU cycles to redraw it. Both can be combined with double-buffering where you paint to an off-screen bitmap, and copy pixels onto a on screen, smaller one.

Finally, there is drawing. Drawing is the act of using the Flash Player’s vector drawing API to create vector graphics. These are wonderful in that the engine to create them is small, and they take next to no RAM. They look nice in terms of anti-aliasing, and the API is simple enough you can create some pretty sophisticated results by building atop that foundation. The downside is that drawing doesn’t scale very well. A lot of “anything” in Flash Player 8 and below used a lot of CPU, and vector content is definitely a prime candidate to use those resources. There are various techniques you can apply in Flash Player 8 and 9 using cacheAsBitmap, scrollRect, or even blitting the resulting drawing and thus removing the vector part.

The way each of the above is rendered is doing on an interval, called a “frame”. Flash Player was built originally as a vector to pixel renderer , and later incorporated a timeline to show changes in these images over time. In effect, a lightweight animation tool was the result, hence Flash Player’s rising popularity for design in the Internet during the modem, low bandwidth days. These frame ticks run on some developer / designer set time code, like “15 frames a second”, or “120 frames a second”.

What that means is that the Flash Player will update the screen 15 times a second, but more no more. It is not guaranteed to update the screen that many times for various reasons such as screen refresh, CPU usage, etc., but it IS guaranteed not to update more than the set frame rate. This is useful for animators to convey motion. As Ted Patrick has written about in the past, the introduction of the ActionScript Virtual Machine share’s this time between rendering. The important thing to understand, though, is that ActionScript for the most part is single-threaded. This means that once it starts, it doesn’t stop till it’s done running all of it’s code. If you have a lot of “stuff” happening when a user clicks a button, that code is run, and nothing else can really happen in the meantime. There are various threads that the Flash Player spawns to handle network operations, keep tabs to see if theAVM stopped responding for whatever reason, but none of these are accessible to the developer. Bottom line, if you’re code is busy doing things, you’ve allocated less time to the Flash Player graphics renderer to redraw the screen, and thus you’ll lose redraw fidelity.

How do you then lessen this impact, and create an effective invalidation solution given the technology?

In Flash MX and Flash MX 2004, an invalidation routine was created that ensured that components were only ever changed visually once per frame. This was done because while the developer was more than capable of changing a MovieClip‘s x and y coordinates on the screen multiple times in a few milliseconds, it would only actually render once. Changing those properties directly actually triggers a need to redraw event in the Player, and thus you are being very inefficient in your use of the Flash Player. To remove this problem, the first technique was a convention called drawing and invalidation. You would have a few private properties in a base class that abstracted MovieClip , and allowed you to change it’s properties with the ability to then invalidate on your own time. Some would invalidate immediately. It was a good first attempt, but was more formalized and better implemented when ActionScript 2 came onto the scene in Flash MX 2004. While AS2 still outputted the same bytecode as AS, it still implemented the main function that made this possible, introduced flexibly in Flash Player 6: onEnterFrame.

The enter frame event is something emitted every frame. In Flash Player 9, DisplayObjects do this regardless of whether something has a timeline or not, 1 frame or 10 billion such as MovieClip & Sprite. While the name itself is somewhat legacy, it is useful to understand what it really means. First, it is guaranteed to be fired once per frame. Unlike setInterval which can fire randomly (at most 10 times per frame, or less for longer intervals), you can depend on the onEnterFrame function to run when you expect it to run. When it actually fires within a frame context isn’t important. What is important to know is that it is synonymous with “redrawing now”. Since you can only redraw once per frame, it fits in nicely with knowing when you are best suited to do so effectively. FlashMX & Flash MX 2004 would use the invalidate method for graphical components. This meant, “redraw next frame”. So, the developer could change a bunch of properties, styles, etc. and theMovieClip wouldn’t actually redraw itself visually (by setting _x, _width, applying a color, etc.) until the next screen redrawn, no matter how many were changed, or how many times the same one was changed.

This was great in theory, but a lot of the race conditions with measurement and styles made it hard to effectively do. Regardless, it was THE best way to ensure effective redraw. This was slightly improved upon in the Flex 1.5 framework with a more formal system:
– set a property
– if it causes a redraw, invalidate a draw function, otherwise have that property’s dirty flag set
– if a dirty flag was set, determine what type of redraw to perform, a draw, size, measurement, or style change
– invalidate one of those functions if applicable
– one of the invalidated functions would run at the next redraw

This made possible larger Flash Player applications. The problem was, though, that even if things weren’t redrawing and just sitting there, they were still taking up resources, regardless of the wonderful invalidation implementation. Hence Flash Player 9’s DisplayList being invented.

Blitting is kind of a low-level, boilerplate affair, and it’s currently not used a lot by the Flex framework. This is a shame. While the Charting components and Flex components both make extensive use of caching as bitmap and setting effective scrollrects , I think for some of the non-interactive parts of charting, they could scale effectively better. I have not, however, read reports of problems with the charting components not scaling. Props to the power of ActionScript 3!

A significant amount of the framework is being drawn with vector graphics via the Drawing API, and all uses extensive use of theDisplayList.

So, returning to my invalidation implementation, I chose to use the Flex framework invalidation methodology in my implementation of a Flash Lite 2 component framework. Granted, I didn’t have the added or render event’s, but enter frame is all you really need. Seemed logical enough; you want something to efficiently redraw on a device that REALLY needs to be efficient. As I later found, some on my own, some with Greg’s help, there are a few problems with this.

1. Most dirty flags in the Flex framework start out with default values of false. This default value takes up RAM, RAM that I don’t have.
2. I am making an assumption that a developer wants to have the component redrawn when they change a property. This is an incorrect assumption to make.

Regarding #1, one solution is to simply not define default values for the dirty flags. The if ( flag == true ) still results in a correct evaluation if the default value is false, undefined, or null. That way, the flags would only last for 1 frame. Unfortunately, that is still a lot because Objects, what all data-types derive and thus extend in pre-AS3, take up a lot of overhead so even a Boolean has a valid cost to think about.

Regarding #2, there are numerous cases where data isn’t ready for consumption by GUI controls, or some other process hasn’t completed yet. Some of that data or relevant process may be needed to do internal calculations for redrawing. I’m slowly to see ways for developers to implement their own call to invalidate. So, instead of doing this:

attachMovie ( ComboBox.SYMBOL_NAME, “cb”, 0 );
cb.prompt = “– select a value –“;
cb.dataProvider = someArray;
cb.setClickHandler ( this, onClick );

You would instead do (per Greg):

attachMovie ( ComboBox.SYMBOL_NAME, “cb”, 0 );
cb.prompt = “– select a value –“;
cb.dataProvider = someArray;
cb.setClickHandler ( this, onClick );

Notice the only difference is the addition of a invalidate method call. This triggers a redraw. The former, every single public property set, and some methods may trigger a redraw, usually via getter / setters (which have a lot of overhead…). Yes, it’s efficient in that it supports an infinite amount of property setting with only one redraw, but again, I’m assuming the developer WANTS to redraw. Maybe they have the data, want to set it, throw it away, but are awaiting more. Maybe they need the above, plus they are doing it to multiple controls over a sequence so they can effectively deliver a large amount of data through a small series of steps the device can handle. Later, they can then redraw when they deem it necessarey to do so.

In discussing this stratgedy with Chris, he’d recount a lot of techniques Apple’s Cocoa uses as well as some of his own implementations. He found the most successful (and I’m paraphrasing badly here) is when parents are responsible for drawing their children. This allows you can redraw just one section of the GUI, or if you need to redraw everything, you can call invalidate on the main parent, and it’ll cascade down the tree of children, who in turn call invalidate on their children.

What I find interesting is that this technique could not only work in Flash Lite 2 / 2.1 which is basically Flash Player 7, but also in blitting in Flash Player 8 as well as utilizing the DisplayObject API in Flash Player 9. In fact, it’d be really nice if Flex 3 gave us this option. I think the reason it’s not in Flex 2 is a lot of people need an accessible framework; meaning, low learning curve. Some people have never developed or programmed before, or not done that level of heavy GUI work, etc. Bottom line, I think the decision by the Flex team is “developers shouldn’t have to know how the Flash Player works to effectively write applications atop it”. I totally agree. I also believe, however, that there should be a way to optionally allow those of us who know what we’re doing to redraw when we see fit. It’d really help those larger applications, or when you are trying to squeeze all you can out of an implementation, say, Flash Lite 3 which could possibly use the new AVM on a device :: crosses fingers ::.

While it’s more work for the developer, I understand why Greg suggested what he did now, and have more context as to the ramifications what power this really gives developers. I guess I just don’t really have in my head best practices on implementing this. The abstracted invalidation model that Flex has removes any care, and provides a nice convention for me to build atop of. Now, I suddenly have to care when I redraw things which could be different for each GUI control. Furthermore, the less utility I add to a framework by making it more efficient instead, the more the expectations of the Flash Lite community shifts to wanting more features out of controls. Not sure the best approaches to either. Fun, regardless!

Yahoo Toolbar with Shockwave

Went to Yugop yesterday and had to install Shockwave on this machine because it’s new. However, after going through the don’t-remember-my-page-and-close-my-browser-you-coohh#$#$ installer, I noticed part of the installation process had the option of installing Yahoo toolbar. Maybe I’ve been out of the Director community and missed this detail, but why the integration? What’s the point of offering it? Can you write extensions via Shockwave? It was really confusing as to why that was added to the installation process, and although I am on a G12 (or whatever’s faster than a T3), I was still a little irritated that something I didn’t ask for was potentially downloaded with Shockwave. Just thought it was weird.