Insightful ramblings ... well, ramblings

As promised here are my slides and links to sample code from my Flex for Android presentation at the Toronto Flex User Group meeting. Had a great time presenting for the first time in a long time, will certainly be offering to do it again.

You can find all the source code for the samples and build script on github.

Here are the links mentioned at the end of the presentation.

• http://opensource.adobe.com/wiki/display/flexsdk/Hero
• http://opensource.adobe.com/wiki/display/flexsdk/Mobile+Application
• http://labs.adobe.com/technologies/air2/android/
• http://www.openscreenproject.org/
• http://developer.android.com/sdk/index.html
• http://developer.android.com/guide/practices/ui_guidelines/index.html
• http://www.teehanlax.com/blog/2010/02/01/ipad-gui-psd/
• http://www.teehanlax.com/blog/2009/06/18/iphone-gui-psd-30/

For those of you who were present, 10 out of 11 demos worked perfect, number 11 died when I tried to run the debugging. I forgot to add the 'android.permission.INTERNET' to the application descriptor, my bad. You can find it in the source code.

I am getting ready to do a presentation today for the Toronto Flex User Group on using Adobe Flex and AIR to create Android applications. Developing Flex apps for Android is an exciting new frontier for Flex developers. Adobe is working to release a new version of the Flex framework, named Hero, that will be optimized for mobile devices, but this doesn't mean that the current framework can't be used to develop Flex applications for the Android platform.

My presentation will cover:

• size considerations
• user interactions
• device capabilities and how to access them
• optimization
• demos
• using the Android SDK tools

Check back soon for the presentation slide deck and demo sample code.

For those of you who have had a chance to create Flex applications for Android, you have quickly discovered how useless the standard Spark components are. Too small and no true touch support (aside from clicking). This is the first in what I hope will be many demos of reworked Spark components. My goal is to make them work much better on mobile and touch sensitive devices.

The first two components I have reworked are the List and DropDownList, both are virtually unusable in the standard format on touch devices.

The standard List component is not swipeable so the first goal was to add the ability to swipe up/down to scroll the list. Creating a custom scroller class made that possible. The next step was reworking the list to prevent selections on mouse down events. What you don't want is a list item to be selected every time you touch the list to swipe it. This was a little more involved and required some hacks to get at private methods and properties of the List and BaseList classes.

Creating a replacement for the DropDownList was a small leap after getting the swipe list working. It's essentially a text input and button that you can assign a dataprovider to. When you click the button, a modal window is opened with the swipe list. Select an item, click OK and you're done.

You can get the source code on GitHub, enjoy.

I recently had a request for some help on skinning a Flex 4 Spark List with a custom scroll bar and item renderers. Even though the skinning process has gotten much easier with Flex 4 and the Spark components there is still a learning curve and a few things that are either learned through a lot of trial and error or by having someone point it out to you. Hopefully with this post I will be able to save some other Flex developers the "trial and error" route.

This is the comp of the list that needed to get created. Nothing to complicated at first glance, the standard Spark scroll bar needs to get replaced and a custom itemrenderer will be needed for the list.

So how does one go about turning the image above into a functioning Spark List, follow along.

Step 1 – Prep the skin pieces

The scroll bar track and thumb from the comp need to be turned into assets that we can use in the Spark skin. The itemrenderer can be completely done with code. If you are familiar with Photoshop turn off the surrounding layers, select the thumb and track, respectively, and then do a 'copy merged' and paste each as a new image. Save them as transparent PNG's that can be embedded into the Flex swf.

Step 2 – Create the Spark Skins

Three skins will be required for this task, one for the thumb, the track and the vertical scroll bar (right click, view source on the swf at the bottom to view all the source code).

The track and thumb skins simply embed the pngs that we created in step 1.

The track skin:

<s:BitmapImage source="@Embed('assets/scroll-track.png', scaleGridLeft='2', scaleGridTop='5', scaleGridRight='12', scaleGridBottom='275')" 
				   left="0"  top="0" right="0" bottom="0" />
    

And the thumb skin:

<s:BitmapImage source="@Embed('assets/scroll-thumb.png', scaleGridLeft='1', scaleGridTop='5', scaleGridRight='13', scaleGridBottom='97')" 
				   left="0"  top="0" bottom="0" right="0" />

In both cases we use the scale nine attributes to make sure the graphic scales cleanly in the horizontal and vertical directions.

The scroll bar skin sets the skins class for the track and thumb buttons to the new skins we have created.

<s:Button id="track" top="0" bottom="0" right="0"  width="15"
		  focusEnabled="false"
		  skinClass="com.dgrigg.skins.VScrollBarTrackSkin" />

<s:Button id="thumb" width="14"
		  focusEnabled="false" visible.inactive="false"
		  skinClass="com.dgrigg.skins.VScrollBarThumbSkin"   />

That's it for skin creation.

Step 3 – Styling via CSS

Once the skins are created the next step is creating a css file to get the List component to look the way we want. Using css selectors we can tell the List to use the new skin class we have created for the vertical scroll bar on the list. The 'fixedThumbSize' style tells the scroll bar if it should adjust the size of the thumb based on the list length. In this case we want the thumb to always be the same size so it gets set to 'true'. We have also used CSS to turn off the horiztontal scroll bar, sometimes with custom item renderers the horizontal scroll bar will make an appearance when you don't want it to.

s|List 
{
	contentBackgroundAlpha:.5; 
	contentBackgroundColor: #000000;
	borderColor:#000000;
}

s|List #scroller 
{
	horizontalScrollPolicy: off;
}

s|List s|VScrollBar {
	skinClass: ClassReference("com.dgrigg.skins.VScrollBarSkin");
	fixedThumbSize:true;
}

Remember to load the style sheet into the main application class and the new pimped list is ready to go.

Step 4 – ItemRenderer creation

The final step in bring the comp to life is creating the ItemRenderer. Fortunately Flex 4 has made this very easy. Navigate to the package you want to create the renderer in and then right click and select 'New/MXML Item Renderer'. A basic item renderer will be created that you can begin working with. Here is the code for the item renderer we will be using.

<?xml version="1.0" encoding="utf-8"?>
<s:ItemRenderer xmlns:fx="http://ns.adobe.com/mxml/2009" 
			xmlns:s="library://ns.adobe.com/flex/spark" 
			xmlns:mx="library://ns.adobe.com/flex/mx" 
			autoDrawBackground="false" height="85" width="385">
<fx:Script>
	<![CDATA[
		
		override public function set data(value:Object):void 
		{
			super.data = value;
			
			if (value)
			{
				titleLbl.text = data.title;
				contentLbl.text = data.text;
				titleLbl.visible = true;
				contentLbl.visible = true;
				readLbl.visible = true;
			}
			else
			{
				titleLbl.text = "";
				contentLbl.text = "";
				titleLbl.visible = false;
				contentLbl.visible = false;
				readLbl.visible = false;
			}
			
		}
	]]>
</fx:Script>
<s:states>
	<s:State name="normal" />
	<s:State name="hovered" />
	<s:State name="selected" />
</s:states>

<s:Rect left="0" right="0" top="0" bottom="0">
	<s:fill>
		<s:SolidColor color.selected="0x383c40" color.normal="0x23252a" color.hovered="0x383c40"
					  alpha.selected="0.8" alpha.hovered="0.5" alpha.normal="0.8" />
	</s:fill>
</s:Rect>

<s:Label id="titleLbl" 
		 x="15" y="15" 
		 width="370" color="0xffffff"/>

<s:Label id="contentLbl" 
		 x="15" y="30" 
		 width="370" height="30" 
		 color="0xeeeeee" fontSize="11"  />

<s:Label id="readLbl" 
		 x="15" y="65" 
		 color="0x336699" color.selected="0xCCCCCC" 
		 text="READ MORE" fontSize="11"/>

<s:Line left="0" right="0" bottom="0" width="1">
	<s:stroke>
		<s:SolidColorStroke color="0x000000"/>
	</s:stroke>
</s:Line>

</s:ItemRenderer>

A few things to note in the itemrenderer code. First, the 'autoDrawBackground' attribute on the ItemRenderer base class is set to 'false'. This turns off the  selected/rollover/hover background that normally gets drawn and displayed. The s:Rect instance is used with the state selectors on the color attribute to create custom rollover color/alpha combinations. The normal list will allow you to change the color but you can't tweak the alpha so this is your best option. Second, the 'set data' method was overwritten in order to set the values of the various labels. This is a must step for pretty much any custom item renderer. The overwritten method does a small check to see if the incoming value exists, if not the labels get hidden. This is always a good practice to prevent empty rows from appearing in the list.

Wrap up

That's it. A custom scroll bar and item renderer in four steps. No rocket science, just a little Flex know how. You can view the source code here.

And this is the final product, not bad.

One of the nice features of the Flex mx:List component is that you can set the renderers to be editable without having to create a new itemrenderer. With the Spark List (s:List) that option is no longer available. Fortunately it is quite easy to create a custom editable itemrenderer for the List.

Select an item in the left column and change the value, you can see the change events being traced out. Right click to view the source code.

There are three steps making an editable itemrenderer for the Spark List component.

First

Create a custom itemrenderer with a TextInput and a Label.

 <?xml version="1.0" encoding="utf-8"?>
 <s:ItemRenderer xmlns:fx="http://ns.adobe.com/mxml/2009" 
             xmlns:s="library://ns.adobe.com/flex/spark" 
             xmlns:mx="library://ns.adobe.com/flex/mx" 
             autoDrawBackground="true">
 
    
    <s:Label id="labelDisplay" 
           text="{data.label}"   
           top="7" bottom="5" left="5" right="3"/>
    
    <s:TextInput id="inputTxt"  
              visible="false"  
              top="1"  bottom="1" left="1" right="0"/>
    
 </s:ItemRenderer>
  

Second

In order to toggle between the read and write states we need to add a click handler to the label and focusOut handler to the text input. You will notice that I simply set the respecitive components to either visible or not instead of uses the spark states. Using the spark states would require extending the list component to include an editHover state and for this example I wanted to keep things simple.

 <fx:Script>
   <![CDATA[
 	 import spark.components.supportClasses.ListBase;
 	 
 	 private function onChange(event:Event):void 
 	 {
 		isEdit(false);
 		
 	 }
 	 
 	 private function onEdit(event:Event):void 
 	 {
 		inputTxt.text = data.label;
 		isEdit(true);
 		//set cursor postion to end
 		inputTxt.selectRange(inputTxt.text.length,inputTxt.text.length+1);
 		inputTxt.setFocus();
 	 }
 	 
 	 private function isEdit(value:Boolean):void 
 	 {
 		labelDisplay.visible = !value;
 		inputTxt.visible =  value;
 	 }
 	 
   
   ]]>
 </fx:Script>
 
 
 <s:Label id="labelDisplay" 
 	  text="{data.label}"   
 	  click="onEdit(event)" 
 	  top="7" bottom="5" left="5" right="3"/>
 
 <s:TextInput id="inputTxt"  
 		 visible="false"  
 		 focusOut="onChange(event)"
 		 top="1"  bottom="1" left="1" right="0"/>
 
 

Third

This is the not so obvious step. You need to manually dispatch an event through the List’s dataProvider in order to notify the List and anything else that is using the data source that it’s contents have changed.

private function onChange(event:Event):void 
 {
 
 var oldValue:String = labelDisplay.text;
 
 if (oldValue != inputTxt.text)
 {
    data.label = inputTxt.text;
    labelDisplay.text = inputTxt.text;
    
    
    //dispatch the data update event
    var list:ListBase = this.owner as ListBase;
    list.dataProvider.itemUpdated(data, 'label', oldValue, inputTxt.text); 
 }
 isEdit(false);
 
 }

And that’s it. A basic editable list itemrenderer for the Spark List in 3 easy steps. Right click on the swf to see the source code.

Since Apple kiboshed Adobe’s hopes of iPhones running Flash content, Android (Google) has stepped up and offered Flash and Flex developers a mobile platform to development and deliver content on. This has come as great news to Flash platform developers who want the opportunity to dive into the mobile environment. Many Flash (and Flex) developers I have spoken to about mobile development are still unsure about what it will take to deliver Flash player based content and applications to mobile devices.

The good news is the development and delivery process is almost identical to the current process. Use Flash or Flash Builder to create a swf file and then either deploy that to a website, to be served up via an HTML page or wrap it with the AIR runtime and packge it for installation on an Android phone as an native Android installer.  Really the only difference in developing for the Android platform is the extra step required to create an ‘.apk’ file if you want to create a stand alone application.

The bad news is the environment that the content and applications are running in is quite different from anything the vast majority of Flash platform developers have worked in. The screen is small, the bulk of user interactions are touch based, the screen can easily be rotated and tilted, the user is often quickly looking at the screen while interacting with the environment around them (ie texting and walking at the same time) and the processor doesn’t have the juice most modern desktops and laptops have. This means that while the swf delivery and development process is quite familar the UI and UX development is anything but. Add in the processing constraints and most Flash platform developers will quickly realize there is a significant learning curve to developing Flash and Flex content for Android (and mobile in general) devices.

So what are the key areas developers need to be aware of?

1. Screen size
2. User interaction
3. Device capabilities
4. Performance

1. Size, it really does matter

Below are two screen shots of the same Flash application running on a Google Nexus One and a normal desktop computer. The application is 480 x 800. The first image shows the screen as seen on a normal desktop computer with the screen resolution set at 1280 x 1024 (fairly standard desktop resolution). The second image shows the same screen scaled to the physical size as seen on the Nexus screen. It might seem obvious that the mobile screen will be smaller than the desktop screen but watch how screen resolution affects something that is the "same" size.

Desktop screen shot

Android (Nexus One) screen shot

As you can see there is a huge difference in the physical size of the two screens even though the application is viewed at 480 x 800 pixels on both devices. Mobile devices have much higher screen resolutions than normal computers, 240dpi vs 72dpi. As you can see from the images above this makes a huge difference. 

For mobile applications this has two very big implications.

First a smaller screen with a much higher resolution means you have much less physical real estate to work with when laying out the user interface. Designers need to be acutely aware of this when creating screens and views. Add in the fact the users are often quickly glancing at the screen which means screens can not afford any clutter and designers have a difficult task in packing a lot of meaningful content into a very small area.

Second, since everything appears much smaller on mobile screens things like text and UI controls (button, checkboxes, etc) need to made larger for readability and clickability (if a beer company can use drinkability I can use clickability). On a mobile device the main pointer is the finger tip which is a lot larger and less precise than a mouse pointer. In a desktop application the typical button size is around 25 pixels in height, on a mobile device 60 pixels is the norm, anything smaller than that and it becomes very challenging to click the button you want.

I couldn’t help but be reminded of this Simpson’s clip after my first experiments building a Flash application for the Android.

This leads to my second point.

2. User interaction, it’s all about the finger

Apple and the iPhone ushered in a whole new era of computer interaction. Sure ‘touch’ has been around for a long time, but it was Apple who made it mainstream. Point and click are things of the past, now it’s point, flip, swipe, pinch, wipe. Sounds more like a heated game of rock-paper-scissors. The great news for Flash platform developers is that AS3 has full gesture support built in and ready to be used. Events are dispatched for a multitude of user gestures including tapping, swiping and rotating. AS3 also supports multi touch for instances where multiple touch points need to be tracked. 

This is a whole new territory for Flash and Flex developers, we generally just interact with the mouse and keyboard, a few of us (myself included) have developed for larger touch screens, but not small mobile screens. Fortunately there is some good information to be found on designing for human interaction. Google has provided some guidelines for user interface design and Apple has some really good (and thorough) documentation on Human Interface design.

This is an area that will make or break an application. It won’t matter how great the apps processes information or what information it displays or what it allows users to do, if the control is non-intutive and confusing users will drop it faster than a hot potato (and they may just throw their phone).

3. Device Capabilities, the swiss army knives of personal computing

Most mobile devices have as many sensors and media inputs as swiss army knives have blades. There are GPS sensors, accelerometers, touch screens, cameras, microphones, video cameras, scroll balls, the list goes on. The trick for developers is figuring out how to leverage the various sensor inputs (and outputs) to create unique and pleasant user experiences.

For the Flash and Flex developer most (if not all) of these sensors will be available to AIR based applications and most of them will be available to browser based swfs. This an area where Flash applications will really be able to shine. Flash has supported camera and video input for a long time, working with these types of media inputs should be second nature to many Flash platform developers. Some of the other sensor inputs are new (ie accelerometer, microphone and gps) but Flash has the advantage of being able to access and use them, sorry HTML5 and javascript.

Personally I think Flash is uniquely positioned to offer mobile users amazing experiences based on the capabilities of the devices. Flash has been used for years to create amazing, immersive, media rich user experiences. With increased ways to receive and provide feedback between the user and device who knows what applications will start to look like and how they will function.

4. Performance, where the rubber meets the road

Performance, performance, performance. This is "allegedly" why Apple doesn’t offer Flash on the iPhone/iPad. I don’t buy it. I have seen Flash running on Android devices first hand, both in the browser and as stand alone applications and trust me it runs perfectly. Adobe has worked very closely with mobile manufacturers to provide hardware acceleration, reduce battery usage and deliver maximum performance with the upcoming Flash 10.1 release.

The critical issue for developers is to remember they are developing for devices that don’t have the same processing resources of typical desktop and laptop computers. Special attention needs to be paid to optimizing performance as much as possible. Grant Skinner has a great presentation on Flash performance, where gains can be found and where trouble lurks. Garbage collection, green threading, blitting and generics are all areas that AS3 developers should understand in order to optimize and maximize the performance Flash can deliver.

The old computer mantra is "garbage in …garbage out". If your application runs slow, freezes or crashes, don’t blame the device, virtual machine, Adobe, Google, Android or your mother, first look at your code. This is where the majority of backlash against Flash performance should really be pointed. It’s not the Flash plugin and runtime, it’s the fact that anyone can "develop" a Flash application and many people that do, don’t understand the performance and memory implications of what they are doing. With the ability to release Flash applications and content to mobile devices, Flash will be under increased scrutiny, do the community a favor and write clean, efficient code.

Where to next?

Who knows? The mobile environment is like the internet was 10-15 years ago, the wild wild west. A lot of really cool things will get made and a lot of dogs will come and go. The only thing I am sure of is that the Flash platform will have a large place in how applications and content get developed, deployed and used. It’s an exciting time for Flash platform developers, especially after all the HTML5/Apple rhetoric. Get a mobile device, get some neat ideas or find someone who has them and build something amazing, I know I will be.