Software, Fitness, and Gaming – Jesse Warden

Blog

Server-Side FFmpeg Converter: Flash, ARP, & SWFStudio

Preface

As part of my Flash & Flex speech I’ve been giving throughout the year, I wanted to create a real-world app where you would want utilize Flash & Flex together. Back in late July, I started creating a YouTube clone. There was a good blog entry posted awhile ago on this. The goal was to create a really rich front-end in Flex that pretty much required the use of Flash. I purchased a design from Templatemonster.com that looked impossible to do in Flex, and set out to implement it.

Introduction

The one thing you need for a YouTube clone is a server-side tier that can upload videos of various formats, and convert them to FLV’s. After receiving the cliche “You could of searched on Google instead of asking that question” on FFMPEG’s developer list, I realized I wasn’t going to get any help from the community on finding a scriptable executable, at least in my aggressive time frame, especially for Windows. Stefan from the chattyfig Flashcom list hooked me up with a Windows compiled ffmpeg.exe. I wanted to build the whole back-end in PHP, though, since although I’d prefer to use Ruby, I don’t really have the time with this specific project so had to go with what I know. I spent 3 days trying to get the FFmpeg php module to compile as a PHP add in. I even went so far as to try to learn MAKE and friends.

On the 4th day, exasperated, I gave up and took drastic action. I decided to create the back-end converter in Flash. I’ve only once before created a server program in Flash. Flash is typically used on the client as a front end technology, in a web browser, or as rich desktop apps. There was one time in the past, however, where it was perfect as a server application. Macromedia (now Adobe) had a program called Central. It was their first foray onto the desktop client world. In Central 1.5, they incorporated an API that allowed you to access AOL instant messenger & ICQ messaging capabilities via code. It was pretty neat. I couldn’t for the life of me figure out a good use for it on the client, though. Instead, I created a bot server. The bot, called FEEDBot, allowed people to IM it. If you were a buddy of FEEDBot, you could then ask it to monitor your favorite blogs. It’d check every hour, and if a feed was updated, it’d IM you with a link to the blog and an excerpt. I’d basically run the Central program, leave my computer running so it’d act like a server.

I did a few tests and found that via SWFStudio, I could talk to FFMPEG via Flash 8 pretty well. I didn’t have a legal license of Zinc back then, and didn’t want to create it in Flex 2/ActionScript 3 because for a quick and dirty server app, strong-typing was too slow to develop in. Flash 8 was perfect. So, I proceeded to create a video converter for the back-end in Flash 8, SWFStudio, and Flashcom. Flashcom allowed the client to convert video and get real-time updates. It also allowed the EXE to act as a service so to speak, so multiple clients could all ask Flashcom to convert videos after it uploaded them, and the EXE would then tell FFMPEG to churn away.

How does it work?

Through a few technologies and steps.

A client will upload a video. This video is uploaded via PHP. Upon a successful upload, the client then tells Flashcom to convert the video. Flashcom then tells the VideoConverter application, which also exists on the same server as PHP, to start converting the video. The VideoConverter then tells FFmpeg to convert the video to FLV with certain parameters. When FFmpeg is done, it tells the VideoConverter. The VideoConverter then tells Flashcom that it is done with that particular video. Flashcom then tells the client that it is done, and the client can now play the FLV via Flashcom.

Since the above process utilizes Flashcom, this allows the client to get real-time updates as to the process, unlike the “wait for frikin’ ever for a respone” one gets on YouTube. Additionally, you can use the same server to play back the video. Obviously having the same computer converting video via FFmpeg, running a Flashcom messaging server, as well as a Flashcom video serving server won’t scale. That wasn’t the point; the point was to get a proof of concept application up and running. Naturally, I bit off more than I could chew. The video conversion part works, but the actual client portion of the application is taking longer than expected, mainly because of my full-time consulting work; I’m traveling a lot, thus don’t really have enough time to devote to finishing it.

However, since the server portion DOES work, I figured I’d release the code in case it might benefit someone.

Points of Interest

One particular architecture challenge I had with the server portion is the same problem I’ve had with Flex Data Services and Cairngorm 2 for Flex 2. Cairngorm, as well as ARP, do not natively support a methodology for doing server push. They are built around a request / response model. This means that the client initiates a request and the server responds. The entire framework is built around this concept. Therefore, there is no real model built around the server pushing a message to the client, and the client acting upon it.

So, I came up with a concept an Observer. My boss & architect on a previous project, Thomas Burleson, helped me hash out the cleanest way to implement it into Cairngorm. I used that same 10 minute conversation to do it in this implementation via ARP since both frameworks are very similar, and Cairngorm 2 won’t work in Flash 8.

This Observer class basically listens for events coming from the server in a push fashion, in this case, utilizing Flash Media Server which supports a real-time binary socket connection with the Flash Player. This helps in 2 ways.

First, Flash Player 9 doesn’t seem to support the old slash syntax that has been in Flash for years, at least with regards to how the old Flashcom server-side framework expects it to. A lot of my server-side code expects this to work, so I had to find a way that didn’t require a major rewrite of the existing Flashcom server-side component framework since I like how it works and didn’t have time to invent a new one. The client to server method calls work, but the server to client ones did not. Therefore, I merely pass in the namespace string and have the Observer parse it to something useful. This requires a very minute change in the ASC code on the Flashcom server; instead of passing a namespace as a method call, you merely pass the method name as the first call, and the namespace as the first parameter.

Second, I can segregate the server push, the part that doesn’t fit well into Cairngorm/ARP, into it’s own class. This base class is extended by business specific classes. For the VideoConverter, I extend this Observer class, and act on specific messages by dispatching events. The Observer doesn’t actually respond to the server, nor does it send messages. Instead, it dispatches events like normal. This allows Cairngorm/ARP Commands to determine if there is a need to send a message to the Flashcom server in the classic request/response way. Since I can encapsulate the fact I’m talking to a real-time server in the Business Delegates, Cairngorm/ARP work great!

One minor detail is that you can, via a Command, turn the Observer on and off. Additionally, I’ve hacked ARP to have a singleton event dispatcher much like Cairngorm 2 does.

SWFStudio API Parameters

Another thing is that SWFStudio uses vanilla (un-typed) Objects as parameters to it’s methods. I despise this. While I enjoy the ability to get away some loosely typed things in Flash, those particular cases make it extremely challenging to debug. So, for the relevant methods, I created strongly typed ValueObjects so I can ensure all my calls to SWFStudio are correct; if something messes up, I clearly know where to look for the problem. It makes unit testing the SWFStudio to FFmpeg calls easier & quicker.

I’ve only implemented one shell method and one file copy method for SWFStudio but because I used ARP, it’ll be easy to add more as time goes on. Hopefully the Northcode peeps could uh… you know… do it themselves, hehe! I know most Flash devs don’t really strongly type everything, so I highly doubt it’s a common enough use case to support development effort on. Since they DO support Flex 2, I know they care, but probably not enough for Flash 8 and below.

Anyway, here’s the code, hope it is helpful/beneficial to you in some way.

Video Converter – Source ZIP

September 2, 2006
Flash 9 Button in Flex 2

This is an example of how to use a Flash 9 Alpha (Blaze) button in Flex 2. The reason one would do this is that you want to utilize Flash content in your Flex app. Flash is superior in creating artistic details for interactive elements compared to Flex, thus having them work together allows the superior interactive design that Flash offers with the powerful development & programming environment Flex offers.

There are many different approaches to getting Flash content into Flex 2. I’ve been speaking about 3, and plan to show more examples at this years Adobe MAX 2006 in Las Vegas later this year. This example in particular shows where you want to use pure Flash elements, and want to keep your timeline animations intact. This requires a few steps. You’ll create your Flash button in Flash 9, create 2 ActionScript 3 classes for your button, and utilize in Flex 2 via MXML.

Creating the Button in Flash 9

First, you create your button in Flash 9 Alpha (Blaze). If you don’t have Flash 9, you can download it at the Adobe labs (http://labs.adobe.com/technologies/flash9as3preview/ ) if you own Flash 8. This is a typical button that has labels for up, over, and off states. I didn’t include a down state for brevity’s sake. One thing to pay attention to is the “nuked by Flex” layer. This is typically called “Actions” and, if you’re a purist, where your stop actions go. However, no ActionScript on frames remains when you embed a SWF into Flex 2. The mxmlc compiler strips it all out. I keep it here anyway so we can test in Flash at authortime and at runtime. Labels, however, are kept intact.

Another thing to take note of is that using Flash 8 & 9 is currently the only way to get the Saffron engine (FlashType) to render in Flex 2. While you can embed fonts in Flex 2, they won’t look as good unless you use Flash to do it. There is a RAM hit, but better RAM than CPU I say. So, if you dig in the Label MC, you’ll see the TextField embeds Verdana. Feel free to embed whatever font you want. You don’t have to embed a font, I figure since we’re using Flash, we might as well.

Purists would argue that you should at least set the font CSS. I agree, but have to catch a plane in a few hours, so no time. I’ll show you at MAX if you really want to know how.

Finally, one workflow thing to note. You only have to recompile (Test Movie) in Flash 9 if you are changing graphical assets. If you are just changing code, you don’t have to recompile. When Flex 2 embeds the SWF, it recompiles the ActionScript 3 class for you. So, you basically only have to compile in Flash once, and then from then on, you can compile in Flex Builder. MTASC users should be familiar with this concept.

While Flash 9 no longer supports symbols, you’ll notice by right clicking and selecting “Linkage…” that the class name I assigned to the symbol is “com.jxl.BlueTechButtonSymbol”. The word “Symbol” was used in the class name on purpose. I’ll describe why later. You can name the symbols in the library whatever you want.

I suggest you save and compile the FLA & SWF in the same directory as your Flex project.

The ActionScript 3 Classes

I have felt it’s nice to have 2 classes to utilize the button instead of 1, Ockham’s razor be damned. There are a few reasons for this. First, Flex 2 does not support anything to be used in it’s framework that doesn’t at least extend mx.core.UIComponent. This is because Flex has a lot of layout engine stuff that requires MovieClip’s to measure themselves and support a common set of methods. Thus, while one could get away with merely wrapping a MovieClip or Sprite into a UIComponent, that’s a hack that’ll get you into trouble in some layout situations. Since we’re dealing with Flash for a design purpose, we should also respect other design concepts like layout. Having a class strictly measure the Flash asset and abstract the Flash asset via composition is more flexible, clean, and ensures layout is maintained. Second, and supporting the first reason, is that all ActionScript is removed from your frames when Flex embeds a SWF. Even simple things like stop and gotoAndPlay are gone. You have to put them somewhere, so you put them into a class.

Thus, I have 2 classes. One class merely wraps the MovieClip button. It handles the stops and gotoAndPlay’s. That’s really about it. Some can manage sound as well, unless you wish to use the timeline for that; user preference. Another usage is interacting with nested elements. For example, it’s easier to animate a TextField by wrapping it in a MovieClip. This allows ActionScript 3’s strict nature to be satiated by encapsulating it correctly in an ActionScript 3 class, defining the MovieClip as public (there was an old bug where authortime elements HAD to be defined as public), amongst other things. So, instead of outside classes/components going my_btn.label_mc.label_txt.text just to set text, they can merely go my_btn.label = “some text”.

The second class extends UIComponent, handles the measuring, exposes public properties, and utilizes the Flash class via composition. It basically exposes the Flash button’s capabilities to the outside world, the outside world in this case being Flex.

How do you know the difference? I go back to the “Symbol” naming convention. Flash Developers will know what this means. Flex Devs might not see the correlation here, but for us old skoolerz, we know that Symbol refers to the actual MovieClip symbol in Flash’s library that we are using. That way, if you see a FlashButton class and a FlashButtonSymbol class, you immediately know that FlashButton is usable in Flex 2, and the FlashButtonSymbol is usable in Flash.

One important thing to reiterate is that you can code both your BlueTechButtonSymbol class and the BlueTechButton class both in Flex Builder 2. When you recompile your Flex 2 app, it’ll recompile the BlueTechButtonSymbol class inside the Flash 9 SWF for you. This makes it easier to program in just Flex, and allow Flash to do what it does best.

Additionally, you’ll notice at the top of BlueTechButtonSymbol that there is a metatag called “Embed”. The first parameter is a relative path to the SWF you’re embedding the class to, and the 2nd is the symbol you want to this class to be embedded to. Granted, you could omit the symbol parameter, but I like to be explicit. The Symbol class extends MovieClip. This may seem obvious, but it’s important to note. I’ve put some constants up top for both frame labels and frame numbers for use in gotoAndPlay & currentFrame actions, but you could hardcode these if you wanted to. Again, all this class’ does is wrap what would usually be put on frames.

The BlueTechButton class follows the standard UIComponent model of exposing public properties, using dirty flags for intelligent redraw, and calling internal invalidation methods. It also hardcodes the measuring size since we aren’t allowing our SWF button to resize (although it could with scale 9 or animation).

Notice in the commit properties function it’s calling the setLabel function on the SWF class. It doesn’t have to know about all the guts of label_mc and label_txt being inside of it. This also frees the designer up to change the MovieClip later, and only having to modify the Flash classes.

Utilizing Your Flash Button in MXML

If you look in ButtonExample.mxml, you’ll see that I simply use 1 tag to instantiate the button, and have it’s click event trace out some text in a TextArea. That’s it! All the rollover animations are all where they belong; in the SWF (which is now embedded in your main Flex 2 SWF, giving you 1 SWF).

Again, if all you are doing is changing code, you don’t need to recompile in Flash. You only need to do so if you are changing actual art and animations. Then, click your project folder in Flex 2, and hit F5 to refresh it; this’ll ensure it copies the most recent SWF to the bin directory.

Getting Funky

There is another example in the source called AnimationExample. This animates the button in Flex 2. You could do this with Flex 2 components obviously, or you could even animate this in Flash, but there are a ton of cool, key features here that shouldn’t be overlooked when animating in Flex instead.

First, you have no timing issues. If you want a button to be clickable, you simply put a click event on it’s tag. Flex will handle internally when to attach this event, and still supports strict-typing. In Flash, you’d have to either attach the event to a top level MovieClip, and pray the designer didn’t change his mind about the animation, or didn’t accidentally delete the code.

Second, you can diff Flex animations, you can’t diff a binary FLA. So, if an animation changes, and you use source control with your Flex apps, you can see how the animation changed, and who changed it, as well as reverting.

To me, this blends the best of both worlds. While the assets know how they should look, they don’t always know in what context they’ll be used. While one would hope the design comps or wire frames would specify this, it’s not always clear to the designer. This helps offload some of the animation integration to Flex to really decide how the Views’ function without having the Flex developer work around some of the Flash animations. For example, the little rollover effect in the Flash Button in no way affects the Flex Developer’s code. However, if the button faded in, while the code will still technically work, the user would have to “wait” to click on it. Again, this is ok from a code perspective, but you got “lucky”. Now, if the animation is an elaborate building sequence, suddenly the programmer has to know when this is “done”. Granted, he could still attach a click event on the animation, but it’d be pointless since that’s not really want you want. Using Flex states & transitions, you have events that are generated at each point of the animation, giving more programmatic control.

This is all debatable of course.

Transitions are also separate from states. So much so, the purist in me argues no design specifics, even including x, y, and size values should be put within state tags, only transition tags.

This is all debatable of course, hehe!

Anyway, as far as I see it, the Symbol classes could be written by a Flash Developer or a designer whose comfortable writing a little code that he can actually test in Flash. I don’t see this as a common use-case however since a lot of houses have a developer of some sort to handle such things. This is just my stab at it. Comments are in the source code.

Flash 9 Button in Flex 2 – Button Example | Animation Example | Source | ZIP

*** Note: If you have issues viewing the samples above, simply compile locally using Flex 2 and Flash 9. Apparently my copy of Flex Builder 2 has some leftover alpha & beta bits that are wreaking havoc.

August 27, 2006
Dr Watson at Detroit Airport

Gotcha, bitch!

August 26, 2006
Symbol Naming Strategy for Flash
My co-worker, Tony Martin, found a weakness in the mx.core.UIObject symbolName pattern that we have been using. It’s really frustrating because we’re already half-way into the project, and I wished I could of known about it at the beginning so I could formulate a less sweeping refactoring change. How many times have you heard that before?

To give you some background, one of the problems with early Flash Development was the fact that visual elements in Flash can be stored as symbols. These symbols can be reused, much like a class can be reused as instances. Early on, designers started making these symbols do common things, and even started associating code with them, the first components. These MovieClips quickly became problem prone because of the lack of initialization order control, and keeping both linkage name and class name in sync. Linkage names were utilized to name a symbol. That way, when you wanted to attach it visually, you had a name to identify it with. Some people would make both the name of the class and the symbol the same, which was a pretty good convention.

ActionScript 2 came along and introduced packages, much like Java. The early framework thus kept the same scenario where your package name was the same as your symbol name. The occasional issue arose where the 64 character limits on symbol names would cause an unexpected refactoring, but other than that, she worked good.

…till today. I’ve seen the questions in the past on the Flashcoders list of how you associate another Symbol with the same class. There are a couple ways. You can either simply put the class in the AS2 class name in the linkage panel, or associate it at runtime via Object.registerClass.

I never understood the point, though. To me, it didn’t make any sense; why would you create a brand new asset, but re-use the old class that had specific functionality? Why not just do a new class, problem solved? I saw one scenario where the mx framework actually did this, utilizing mx.skins.SkinElement in this way. It was an extremely lightweight class that had like 3 properties, and 3 methods. It acted like a UIObject, but didn’t have all the extra baggage.

Today, we’ve coded a generic List renderer, and our design comps show 2 kinds; one that is thick, and one that has no background and is just text and an arrow. Tony, being prudent, asked if we could just use the same class we already wrote. It was a specific class, inherited a lot of nice functionality from base classes, and the only thing we were changing was artwork and animations. Seemed logical to me.

The problem? The static symbolName property. Typically, when extending UIObject, you implement a symbolName property as a static class member. This typically is the same as your class, only a String. You then copy and paste it into both the linkage ID and AS2 class name fields for the MovieClips. That way, all 4 are in sync. The compiler doesn’t check for a missing symbol name, so if it doesn’t work, you misspelled it; basically ensuring you don’t have a magic string. Secondly, you can utilize the symbolName in the attachMovie and ensure you never misspell it, like:
```
attachMovie ( MyClass.symbolName, "mc", 0 );
```
The problem is, “static is the new _global”, and you can’t change static on the fly without affecting all other creations of the components. Even if you made it a class property, you’d have the same problem; anytime a new class would be created, it’d be referencing the new symbolName. Could you change it back? Sure… I guess. I mean, Flash Player is single-threaded, so you could technically time it I guess, but that’s not a risk I want to take.

It gets worse when some of our container classes actually depend on this property, and thus you realize your kind of stuck with it.

We built our lightweight lists around the same way the DataGrid works with cellRenderers; you basically pass in a class you want it to “make”. It works like:
```
my_list.childClass = Label;
```
It internally knows that it is dealing with our base component class, so knows to look for the symbolName.

So how’d we get around it? Uber-hack:
```
var newSymbolForClass:Function = function(){};
newSymbolForClass.symbolName = "AnotherListRenderer";
ourList.childClass = newSymbolForClass;
```
Voodoo, but it works. All the symbolName really is a String that tells attachMovie what MovieClip to attach; it doesn’t care if it’s an anonymous function or a true class.

This does, however, pose the challenge. Now that I finally understand the use case for allowing multiple MovieClip symbols to utilize the same class, how does one utilize the symbolName strategy to get rid of the magic strings, but be flexible enough to support different symbols for the same class? How does this work in container classes that “know” they are dealing with these types of classes?

For now, I’m thinking we’ll keep the above hack, but in the future, it may be better to make it an instance level property. This, however, makes it harder to pass generic classes to the components who will create them in a factory type fashion. They no longer need only the class, but now need to know which symbol to make. Step backwards, but certainly gives the flexibility that design projects need. Forcing people to create a new class just for new artwork is ridiculous, so maybe this is the best compromise.

This is pointless in ActionScript 3 since everything is a class, and there are no symbol names anymore. You’ll have to basically write a parent class that allows multiple child classes to be drawn into it via Composition.
August 26, 2006