Test 1: New York TimesNew York Times full site took 18 seconds over Wi-Fi with Flash enabled. It takes a moment for Flash content to become visible after the page load finishes, so we're using that as the end of the loading time in this test. The page has several Flash ads, as well as an embedded video. Double tapping on the video automatically zooms in on it. When playing a video, the Android operating system will tie the volume rocker to the volume of the Flash content, which is a nice touch.
Playback was surprisingly smooth, and all the controls seemed to work. Even the sound worked well. It did take a few tries to hit some of the Flash elements, like the all important "Play" button. When we did manage to get into it, we were even able to skip around in the video by dragging across the seek bar. We were able to work the volume control as well, but it was more hassle than it was worth. Scrolling around the page did exhibit some more lag than we are used to seeing on the Android browser. You can see the Flash content jerking around as you move around the screen. The lag was minimized when zoomed in on a different area of the page.
For comparison, we cleared our cache and reloaded the page with the Flash plug-in disabled. Without Flash content loading we got a load time of 14 seconds--a 4 second difference. That's a respectable, but not massive, improvement. Scrolling around on the page is very snappy without the Flash content, however.
Test 2: TimeTime magazine website. It too uses Flash content for ads and videos on its main page in addition to regular non-Flash text and images. Under the same conditions as we used for The New York Times site, Time.com took 20 seconds to load all of its content. Without Flash, this site loaded in only 12 seconds. That's a much bigger difference than we saw on the NY Times site.
The player on this site did not seem to be as compatible with Flash on a mobile device. Due to the page formatting, a double click did not fit the video correctly. We zoomed in manually and were, however, able to play the clip easily. This is where we hit a snag. The Time player, like many others, hides the controls unless you mouse over the video. Since there is no mouse on a phone, we were unable to skip around in the video at all. Likewise we couldn't adjust the volume of the internal player.
The video quality was again quite respectable, with no detectable frame rate drops. Audio quality was an issue here though. We heard some distinct crackling, and after pausing and resuming, audio sync was lost. Page scrolling here was lagging to a similar degree to the New York Times site, that is to say not a lot, but noticeable. The Flash plug-in is workable on this site if you just need to watch a quick clip, but we became frustrated with the Time player.
Test 3: VimeoVimeo is mostly just used for Flash video, we ran the non-Flash page load speed for this one as well. We loaded the full page (as opposed to the mobile version with non-Flash mobile video) since we need to test Flash. To our surprise, Vimeo's full site loaded complete with Flash video in only 5 seconds. Without Flash it was just a hair under 5 seconds. This is probably thanks to the relative lack of other content on the page.
The video filled the screen almost completely with a double tap, and looked quite nice at first. Unfortunately, Vimeo defaults to an HD stream. After a few seconds, the stream was stuttering badly. It took some doing to hit the button to switch modes. Like the Time player, Vimeo uses hidden controls that pop up when you mouse over them. A long press was sufficient to bring them up without pausing the video. Due to the size of the button, it took a few tries to select the non-HD stream. Once we did though, it actually looked excellent.
We were able to play/pause, seek, and even change to volume within the player. Once we realized a long press would bring up the controls, it was a snap. Navigating the page was much smoother than the other sites we tested, but that's understandable with fewer page elements on the Vimeo site. We really felt like the Vimeo player was an excellent experience to use, even on a phone.
Overall, we're encouraged by the beta for Adobe Flash player. In most cases, playback is smooth and the load times aren't terrible. One thing that does concern us is the clear inconsistency in the experience. It takes a few seconds to figure out how a player works (if it works well at all), and we'd like to see some more standardization. Almost every video we tried worked, but some had bad sound or unusable controls. Also, the phone got disconcertingly hot. Our understanding is that this beta of Flash is not yet hardware accelerated, so the final version may fix the laggy page scrolling and the heat issue to some degree. Are you running Froyo on a Nexus One? What are your experiences with Flash?