Technomicon Media's Channel

Asus Transformer Prime TF 201

by Mark W. Hibben
5/20/12

An Ambitious but Flawed Effort

In a recent article on CNN.com, Stuart Miles called the Transformer Prime (TP) “. . . one of the best tablets on the market.  No it really is that good.”  It’s hard to believe that Miles gave the TF 201 more than a cursory examination, and the initial and superficial impression one gets of the TP is generally favorable.  It’s attractive and appears well made, and before the new iPad arrived, it had one of the best tablet screens around, capable of 720p HD video.  The many flaws of the TP only become apparent after extended use.  This was clearly a rushed effort, both in design and execution.  Although Android fans have been quick to defend the Transformer Prime against critics such as myself, its failings have led to a class action lawsuit against the manufacturer, Asus.  Instead of making excuses for the Prime, Android supporters should be demanding better, both from Asus and Google.

By the Numbers

At the time we received the TF 201 in late December, the Prime certainly seemed to have the creds for a viable competitor to the iPad 2, as the table below shows. The Prime had a better screen that the iPad 2, and a better processor in the quad core nVidia Tegra 3, as well as more memory and a better camera.  What was not to like? 

Specifications Summary of the Devices in Our Comparison

Device OS Processor Processor Cores Screen Resolution Rear Camera
iPad 2 iOS 5.1 Apple A5 @ 1GHz 2 1024 x 768 720p 30fps video, 0.7 Mpixel stills
new iPad iOS 5.1 Apple A5X @ 1GHz 2 2048 x 1536 1080p 30fps video, 5 Mpixel stills
Asus Transformer Prime TF 201 Android 4.0.3 nVidia Tegra 3 @ 1.3 GHz 4 1280 x 800 1080p 30fps video, 8 Mpixel stills

 


Device
RAM Flash Storage GPS WiFi Wireless Bluetooth
iPad 2 512 MB 64 GB assisted 802.11abgn AT&T 3G (GSM) 2.1 + EDR
new iPad 1 GB 64 GB assisted 802.11abgn Verizon 4G 4.0
Asus Transformer Prime TF 201 1 GB 64 GB yes, but no longer listed in specs 802.11bgn NA 2.1 + EDR

Much Functionality, Some Dis-Functionality

The Prime also came with a host of features that all seemed like good ideas.  Most of them work, but some don’t, or work imperfectly.  The tablet itself features a built in micro-HDMI port for easy connection to a high definition monitor, a built in micro-SD memory card slot and a docking port for USB connection to a charger or PC.  The docking port also features a mechanical latch mechanism that mates with the docking keyboard.  The keyboard is a must-have accessory that features an additional battery, a full size USB port and a standard SD card slot.  The keyboard provides good tactile feel and despite being a little small, is fairly convenient to type on.  Generally all the ports worked as expected, affording convenient memory expansion and file movement through SD cards or USB thumb drives.  You can even connect a USB mouse thereby obtaining a cursor (which also appears if you use the keyboard touchpad). 

The micro-HDMI port, which I had high hopes for initially, turned out to be a mixed bag.  The connector is extremely finicky, being sensitive to the slightest change in position of the Prime or the HDMI cable.  You can use the HDMI output for video mirroring at 720p HD video, and this comes on automatically, but usually some wiggling of the cable or connector is necessary in order to see anything on the HD monitor.  Because of its location on the beveled edge of the tablet, the micro-HDMI port appears inadequately supported and allows too much flex in the connector/port interface.  In our view, this made the HDMI a non-starter for game play using the tablet as a controller.  You can use a USB based game controller instead, as of Android 4.0, but it’s up to the developer to implement the API’s.  For Grand Theft Auto III (chosen because it’s a game in common between iOS and Android), we found that the Xbox 360 controller implementation worked, but joystick based steering was actually more difficult than using the tablet accelerometers.   In general, we like being able to use the tablet as the controller, especially since this keeps game play consistent regardless of the display.

Our biggest disappointment with the Transformer Prime came when we tried to watch a movie rented from the Android Market (now called Play) via the HDMI output.  There was absolutely no output.  This wasn’t a connector problem, since we could watch videos taken with the Prime’s camera with no problems.  Here, it appears that the snag is Digital Rights Management.  Normally this is solved using HDCP (High-bandwidth Digital Content Protection) which encrypts the HDMI output.  HDCP is built into the HDMI driver chips that manufacturers like Asus use, so the problem can’t be the hardware, but is probably a software driver issue.  I realize that most people do not yet regularly use their mobile devices to watch movies at home, but this will become increasingly common as personal digital content moves into the Cloud to be distributed where ever the user wants.  Apple is already there with iTunes and iCloud, so Android can be expected to follow suit. 

Another disappointment was the GPS.  Google offers some really nice location based services, but you need a GPS to use them.  The Prime’s GPS wasn’t quite as complete a no-show as movies over HDMI.  I seem to recall getting a GPS lock once or twice, but not in the past few months.  The GPS problems of the Prime are the subject of the aforementioned class action law suit, along with complaints about the WiFi.  Interestingly, we never had any problems with WiFi signal strength whatsoever, but the lack of GPS is more than an annoyance.  It’s a product defect that Asus should fix, and they are trying.  Registered owners will receive a clip-on GPS dongle.  The GPS dongle attaches where the keyboard would, so they can’t both be used at the same time, but then who would need location-based services while using the keyboard? 

And then there’s the camera.  The camera has many good features and is usable under most circumstances.  It can take 8 Mpixel stills and record 1080p video at 30 fps.  The lens has auto-focus and there’s also an LED flash.  For close in work, the camera’s image and video quality are excellent.  However, when recording stills or videos of landscape scenes (where the lens is set to infinity focus) we noticed that the left 20% of the image is out of focus.  Apparently this is due to mechanical misalignment when the lens is set to infinity focus (either manually or through auto-focus). 

We’ve never seen this sort of defect in a mobile device camera, but it significantly reduces the utility of the camera in our view, especially since infinity focus is set for objects more than 4 or 5 meters away from the camera. To better see the image blur, click on the image above to see the full size original. 

We’re going to try to get this fixed under warranty, so we’ll follow up with an account of our experience with Asus in this matter.

Performance Measurements

We might have been able to overlook the functional deficiencies of the TP if the device’s Tegra 3 had lived up to its promise, but here the device also fell short.  There seems to be a myth in the Android community that the Prime is “much faster” than the iPad (either 2 or new), but that myth simply isn’t supported by the facts.  For this series of tests, we compared the TP to the iPad 2 and new iPad.  We tested the TP in three different configurations:

1) Configured as a typical consumer would use the tablet.  Live wallpaper was enabled, and there were a few widgets running on the home screen.  Performance was set to Balanced Mode, which limits the maximum clock rate of the CPU to 1.2 GHz.  Using the process management widget, we halted all other apps except the bench mark app under test. 
2) Configured as in 1), but with Performance Mode enabled.  Performance Mode shortens battery life, since it allows the Tegra 3 to run at its maximum clock of 1.3 GHz.
3) A “Reconfigured” mode adopted at the urging of Android supporters, who felt that the test conditions of 1) unfairly disadvantaged the TP compared to the iPads.  In this mode, the standard Asus (non-animated) wall paper was used, and widgets were removed from the home screen.  Performance Mode was enabled.  Not having the app management widget, we used the standard Settings app to go through all the apps and widgets to stop them manually and clear their caches.  We also enabled 2D hardware graphics acceleration in the Developer portion of the Settings app.  We did however leave the settings app running concurrently after verifying that it made no difference in terms of performance measurements.  For this test we also downloaded and installed the latest build of ICS 4.0.3 from Asus, which was 9.4.2.21, whereas configurations 1) and 2) used the previous build, but the same Android version.

For the first set of benchmark tests, we used the built-in Browser of Android and the Safari browser of iOS to run the Sunspider 0.9.1 JavaScript Benchmark.  No other apps were running concurrently for either iOS or Android except for widgets and live wallpapers for the Android configurations 1) and 2), and the Settings app in configuration 3).  Shown in the table below are the results with best results highlighted in green (shorter times better):

Device Total Time in msec 3D Access Bitops Control flow Crypto Date Math Regexp String
iPad 2 1813.9 225.2 221.6 140.5 15.6 133.8 264.3 175.6 70.4 566.9
new iPad 1810.4 211.8 225 140.6 15.6 123.8 273.3 176.2 70.9 573.2
1) Trans. Prime - Balanced 2274.4 411.5 222.4 158.2 16 184.8 358.5 154.4 110.9 657.7
2) Trans. Prime - Perform. 1955.5 353.7 191.2 136.6 15.8 145.8 306 133.3 95.1 578
3) Trans. Prime – Reconfig. 2279.7 403.6 212.8 141.2 15.1 154.1 373.7 161.5 122.9 694.8

 

In no case did any of the changes advocated by the Android supporters change the ultimate outcome: the iPads were still markedly faster overall, although switching to Performance mode in configuration 2) did produce a more or less proportional time decrease.  Since so many changes were introduced in configuration 3), it’s not clear which produced the slight degradation of performance, but I think the results illustrate that running live wallpapers and widgets didn’t have the severely deleterious effect claimed by the Android supporters. 

Android is a well-designed operating system, being based on Linux.  Widgets and live wallpapers, once parked in the background, don’t consume significant processor resources and therefore don’t impair the performance of the foreground benchmarking applications.

The next series of benchmarks summarizes the results of Passmark Software’s Performance Test Mobile, for which Android and iOS versions are available.  Both versions perform the identical series of tests.  We used Android version 1.0.2 and iOS version 0.4.4.  The table below summarizes the CPU intensive Performance Tests with best results highlighted in green (larger scores better):

  CPU            
Device Summary Integer Math Mops/sec Floating Point Math Mops/sec Find Prime Numbers thous/sec Random String Sorting thous/sec Data Encryption MB/sec Data Compression MB/sec
iPad 2 11223 347 288 68.3 1237 49.2 1.3
new iPad 10939 340 287 67 1160 48.4 1.23
1) Trans. Prime - Balanced 7965 115 380 92.9 1096 4.11 1.13
2) Trans. Prime – Perform. 8817 149 393 107 1174 4.53 1.23
3) Trans. Prime – Reconfig. 9306 175 438 107 1134 4.43 1.33

In these tests, configuration 3) of the Prime did show some improvement and scored the best of the three configurations, but didn’t change the ultimate outcome of the iPad 2 having the best overall score. 
The next table summarizes results for the Performance Test Disk and Memory access times.  Here “Disk” is a euphemism for the non-volatile flash memory storage of these devices.

  Disk     Memory    
Device Summary Storage Write MB/sec Storage Read MB/sec Summary Memory Write MB/sec Memory Read MB/sec
iPad 2 4644 32 30 2178 373 381
new iPad 9662 53.1 83.5 2413 422 383
1) Transformer Prime - Balanced 1785 10.7 23.7 1024 162 449
2) Transformer Prime - Performance 3187 16.6 68.6 1072 175 444
3) Transformer Prime - Reconfigured 3743 8.16 112 1041 170 431

 

Once again, although the TP’s configuration 3) improved overall performance, it wasn’t enough to overcome the commanding lead of the new iPad. 

The next table summarizes 2D Graphics performance in PTM.  Here, the Transformer Prime was clearly superior to the iPads, and configuration 3) improved on this lead only slightly:

  2D Graphics          
Device Summary Solid Vectors vec/sec Transparent Vectors vec/sec Complex Vectors vec/sec Image Rendering im/sec Image Filters filt/sec
iPad 2 610 1002 937 12.6 479 31.6
new iPad 858 1723 1459 3.44 542 2.7
1) Transformer Prime - Balanced 2223 4306 3355 109 1991 58.6
2) Transformer Prime - Performance 2319 4399 3496 116 2008 66.3
3) Transformer Prime - Reconfigured 2363 4449 3572 112 1996 72.8

The last table summarizes 3D graphics performance:

  3D Graphics    
Device Summary Simple Test frames/sec Complex Test frames/sec
iPad 2 1432 54.6 53.4
new iPad 1453 56.8 53.3
1) Transformer Prime - Balanced 1188 60.1 42.7
2) Transformer Prime - Performance 1230 59.9 45.5
3) Transformer Prime - Reconfigured 1231 59.9 45.6

 

Once again, although configuration 3) did improve TP performance, it wasn’t enough to overcome the lead of the new iPad.  In none of the performance categories did configuration 3) of the Transformer Prime change the ultimate outcome. 

Our real world testing of application launching and web browsing tended to confirm the relative slowness of the Transformer Prime.  All applications including the app to be launched on the TP were first shut down using the multitasking widget for configuration 1).  For configuration 3) all applications were shut down using the Settings app (except for Settings) and their caches were cleared.  For iOS devices, any concurrent apps were halted in the multi-tasking bar, including the app to be launched. 

We recorded via HDMI output or using an overhead camera the application launch process for the devices, and used the recorded video to determine start, completion, and elapsed times.  Time resolution was limited by the frame rate of the videos at 1/30 sec. 

For the browser launch and navigation comparisons, the launch times are necessarily dependent on the content of the CNN home page at the time of the launch, so we provided a re-test of the new iPad so that the comparison with TP configuration 3) is really apples-to-apples.  Not all tests were repeated for the configuration 3).  Launching of Netflix and GTA III involves the extra time of setting up an overhead camera, so this wasn’t done for the configuration 3), and no tests were done using TP configuration 2).  The table below summarizes the results:

 

Response Time in seconds      
Action New iPad Transformer Prime Config. 1) New iPad Re-test Transformer Prime Config. 3)
         
Kindle Launch 1.37 2.27   2.00
Google Earth Launch 15.83 11.67   12.17
Google Maps Launch 1.40 2.07   3.20
YouTube Launch 2.80 2.17   6.93
Netflix Launch 10.20 12.33    
GTA New Game Start 5.67 8.43    
Browser Launch to CNN 9.13 10.73 7.63 15.63
Time to Leave CNN to Technomicon 3.63 6.77 4.37 7.10
Preview Button Response 0.23 0.63 0.23 0.77

The various benchmark results point to a relative slowness of the Transformer Prime very much at odds with my initial expectations as well as the assumptions of many Android users and TF 201 users.  At this point, I can only point to areas for further investigation rather than specific explanations:

1)  JavaScript Implementation.  The latest build of Android 4.0.3 seemed to take a step back in terms of implementation speed.  This was evident both in the Sunspider results and in the Technomicon Preview Button response test.
2) Flash Support.  Unlike iOS, Android does support Adobe Flash, and this did appear to impact the CNN page load test.
3) Thread Management.  Although the Tegra 3 offers 4 processor cores, Android apps may not be able to take full advantage of these.  There can be any number of reasons for this.  The app algorithms may not lend themselves to parallel processing, or the app developer may not have written code to support multi-threading, or the Android OS may restrict processor utilization, or all of the above.  Passmark’s Performance Test for Windows 7 can take advantage of as many cores as the operating system makes available at any given time, and I would expect them to take a similar approach for Performance Test Mobile, but there’s no assurance of this. 
4) Application Architecture.  Unlike iOS and most other computer operating systems, Android inserts a Dalvik virtual machine into the front end of every running app.  The purpose of the Dalvik virtual machine is to convert the Android app’s generic “byte code” into the machine code of the target processor.  This affords developers greater portability, especially for Java apps with which Dalvik is supposed to be compatible, but at the cost of some execution speed.  Google offers developers the option to develop code that runs natively on the target processor in order to improve processing performance, but this doesn’t eliminate the Dalvik virtual machine, nor does Google guarantee that processing performance will actually improve.

 Ecosystems

When a consumer buys a tablet or smart phone these days, the consumer is not merely buying a device but is buying into an ecosystem of services and products associated with the device.  The consumer can and should consider this ecosystem when making a purchase decision, and therefore so did we when making our comparisons between the Transformer Prime and the iPad.  These ecosystems strongly affect the quality of the user’s experience and have become very extensive for both Android and iOS and include Cloud based services, apps and digital content of all types, and hardware accessories. 

Google was a cloud services company before the term came into common use, and as such their cloud support for Android is very good.  System and app updates arrive wirelessly directly to the Android device, with no need for a PC connection to install updates.  Unlike Apple, Google has fully embraced internet hosted apps, allowing users to create and share Office type documents on-line.  Google Drive for Android now incorporates document creation and sharing, as well as 5 GB of Cloud storage for any user files.  App and media purchases on-line are inherently cloud-based apps, and the Play Store is as easy to use as anything.  The number of apps and games available on Play has reached 450,000 by the latest count, achieving rough parity with iTunes at 500,000.  Most of the popular apps and games are available for both platforms.

Some of the included apps in the Transformer Prime could use some work, most notably the Browser.  As I pointed out in the new iPad vs. Transformer Prime comparison videos, the Browser had some difficulty handling screen rotation.  Google has been working on a replacement in Chrome Beta, but so far it doesn’t support Flash and can be painfully slow at times.  We expect that Chrome eventually to supplant Browser as the default in Android OS, but it’s not there yet.  Another included app is Polaris Office, which creates Office documents in the 97-03 formats (.doc, .xls, .ppt) as well as imports PC generated versions.  As an Office substitute it’s not bad, but does offer somewhat reduced functionality compared to the Microsoft original. 

File importation works imperfectly, and we found that Excel charts sometimes got mangled.  Compared to iWork on iOS, Polaris is easier to use, however, since iWork for iOS can’t directly import Office files and requires iWork for Mac OS to convert them to a compatible format. 

Our big disappointment in the Play Store came when we rented a movie, only to find that it wasn’t HD.  Since the Prime can display 720p HD content, we certainly think Android users deserve to have HD content available on the Store, but apparently there’s no HD content either for movies or TV shows.  This is an area where Google still lags Apple. 

Conclusions

Although the performance comparisons and results seemed to evoke the most strident objections from Android supporters, the benchmark results weren’t really critical in our assessment of the device.  If benchmark performance had been the only difference between the iPads and the Prime, we would have declared the contest a draw.  We just don’t think benchmarks are that important.  It was the functional issues that really sank the Prime.  When a company such as Asus (or Apple, or Google) advertises a certain functionality for a device, consumers have a reasonable expectation that it should work.  The GPS didn’t work.  The HDMI output didn’t work for protected movie playback.  The camera didn’t work right for landscape photography and videography.  It was disfunctionality in these areas that caused us to rate the new iPad as superior. 

In the following video sequence, I demo testing of the Transformer Prime in configuration 3) and summarize the results.

 

  • 1.
    Ambitious but...
  • 2.
    By the Numbers
  • 3.
    Dis- Function
  • 4.
    No HDMI Movies
  • 5.
    Blurred Image
  • 6.
    TP Reconfig.
  • 7.
    Sunspider Results
  • 8.
    CPU Results
  • 9.
    Memory Results
  • 10.
    2D Graph. Results
  • 11.
    3D Graph. Results
  • 12.
    App Launch
  • 13.
    Impacting Factors
  • 14.
    Eco- systems
  • 15.
    No HD Play
  • 16.
    To Conclude
  • 17.
    TF 201 Retest
Technomicon Media
on Facebook