iPhone 4S

iPhone 4

iPhone 3GS

iPhone 3GS (alternate white balance point)

Earlier this year I gave a talk (slides) outlining the latest and greatest in mobile performance, including a bit of my own unscientific research into carrier latency and bandwidth thanks to boomerang.js.

I realized that interest in mobile performance has exploded recently, especially with Steve Souders announcing his focus on mobile, and I thought it was time for an update, this time in blog form. Also, my old slides have been somewhat embarrassing. For some strange reason, at the time I wanted to give S5 a try – that outdated, ancient, not-performant slideshow framework. The result is a slideshow on performance that loads slowly… doh! (incidentally, I recommend deck.js as an alternative).

In any case, it was time for a roundup of mobile performance best practices, in blog form. I’m not sure if it’s properly called a manifesto, but it is what it is! Onward!

For fun: Latency and bandwidth tests

Before we start.. just a little fun data! It’s always been a pleasure to fiddle around with boomerang.js and compare results. With my latest trip to Japan, I had the opportunity to run Boomerang on Japan’s e-mobile 3G network in the remotest of places, up in Hakuba/Nagano in the Japanese alps.

When I got back, I ran the same tests from downtown San Francisco, which was closer to my California-based test server, and should’ve been faster, right? Theoretically…

Boomerang Tests (running on a California-hosted server)

Yes, e-mobile didn’t have the greatest ping, but it handily beat ATT, even though it was going across the entire Pacific Ocean! Note that it came out the best in the bandwidth tests however…

(Note: it’s unknown if or how Steve Souders’s latest research affects these findings)

There’s a lot of factors that could’ve been involved, so look to something like OpenSignalMaps for more data.

Ok, onto the tips!

Page Organization

For feature phones that have little to no caching, aggressively combine requests (deliver HTML/CSS/JS all in one package). For smart phones, take advantage of caching by mirroring desktop frontend best practices: separate HTML, CSS, and JS so they can be cached (per-session and across sessions).

There’s old research about extremely small cache sizes on iOS in particular, but this research has been followed up on by more recent research by Ryan Grove (Yahoo!) and Steve Souders (Google), which shows that we shouldn’t be so paranoid, since caching is pretty decent across all the major mobile browsers.

Of particular interest is a browser’s capability to cache files in a current session, browsing from page-to-page, which is what most users will end up doing (caching of the page across sessions is another matter). What is the maximum file size a browser will cache during a session? The results (via Browserscope) end up being encouraging:

Avoid redirects (foo.com -> m.foo.com)

If possible, perform the redirection behind the scenes on the server, which should be transparent to the user. When a user performs a Google search and clicks on your page, they’re already getting redirected once by Google (check it yourself). Your own redirects are adding a second redirect where there need not be one.

Optimize Images

Deliver appropriately-sized images to devices. The philosophy of responsive design makes it easy to simply downscale images to fit the screen, but avoid this where possible, as this means wasted bandwidth.

You can optimize images through CSS media queries or in JavaScript (see below). Though you want to reduce your dependence on cookies, it may be a good idea to store these width/height values into a cookie (or localStorage if you are fetching images in nontraditional ways) so the values can be read by the server, which can deliver appropriately-sized images. This technique has been implemented by Filament Group, so you should probably read about their experiences before trying to roll your own.

Also note that where applicable, at the expense of performance you may want to serve higher resolution images for better screens, such as for Retina displays (which can be detected with JavaScript or CSS).

JavaScript examples

window.innerHeight;       // max height actually available
window.innerWidth;        // max width actually available
window.devicePixelRatio;  // pixel density (standard is 1, high resolution is generally > 1)

(note that screen.width and screen.height are also available to tell you the dimensions of the entire screen, but this isn’t all available due to the space taken by the OS and browser chrome)

Media query examples

@media only screen and (max-width: 480px) {
  /* small screen styles */
}

@media only screen and (min-width: 481px) {
  /* large screen styles */
}

@media (-webkit-min-device-pixel-ratio: 1.5),
       (-o-min-device-pixel-ratio: 3/2),
       (min--moz-device-pixel-ratio: 1.5),
       (min-device-pixel-ratio: 1.5) {
  /* high resolution styles */
}

navigator.connection (Android only)

Use navigator.connection if it’s available to serve different assets based on connection speed (3G vs WIFI, for instance).

Here’s the contents of the navigator.connection object (with a phone running on a 3G connection):

{
  type: 4,
  UNKNOWN: 0,
  ETHERNET: 1,
  WIFI: 2,
  CELL_2G: 3,
  CELL_3G: 4
}

And an example of how you could write code for each type:

(function(){  // sandbox our code
  if(!navigator.connection) navigator.connection = {type:0, UNKNOWN: 0};  // polyfill
  var connection = navigator.connection;

  if(connection.type === connection.WIFI || connection.type === connection.ETHERNET) {
    // high bandwidth
  } else {
    // normal bandwidth
  }
})();

Here’s how we might be able to use Modernizr to help us out:

(function(){  // sandbox our code
  if(!navigator.connection) navigator.connection = {type:0, UNKNOWN: 0};  // polyfill
  var connection = navigator.connection;

  // add a custom test to Modernizr
  Modernizr.addTest('highbandwidth', function(){
    return connection.type === connection.WIFI || connection.type === connection.ETHERNET;
  });

})();

We can now target special high-bandwidth assets with CSS:

.highbandwidth .logo {background-image:url('logo-high.jpg');}
.no-highbandwidth .logo {background-image:url('logo.jpg');}

base64 encode small UI images

You can base64 encode binary image data in HTML and CSS:

HTML

<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==" alt="Red dot">

CSS

.dot {
  background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==');
}

There seems to be the mistaken belief floating around that base64 images aren’t cacheable. However, because you can embed them in your CSS, base64 encoded images will be cached along with the rest of your CSS.

You’ll probably just want to limit this to small UI icons however. I wrote a quick post on the drawbacks, and also a comparison with sprites.

If you’re using Compass/SASS, base64 encoding images is pretty trivial and easy to maintain with the inline-image helper.

Unicode and Emoji

☆ (HTML entity: #9734)
(HTML entity: #x1f468)

Before even considering base64 images, take advantage of icons that are already made for you: unicode characters and emoji (where supported). Keep in mind that these will look different across browsers, so this might not be an option for everyone.

Because of inconsistent implementations of Emoji character codes between Asian carriers, representatives from Apple and Google made a proposal to add Emoji to the Unicode standard. Unfortunately it seems that only iPhone supports them for now. Here’s a handy chart of all the Unicode emoji characters available.

Take advantage of CSS3

CSS3 offers many replacements for things we needed images for previously. RGBA values replace the need for a semitransparent image for overlays and such. Likewise, border-radius, box shadow, linear-gradients, radial-gradients all reduce the need for images.

However, do be aware that though phones support these new features, it doesn’t mean they’re necessarily ready for primetime. Something I’ve encountered recently to remind me of this fact is severe color banding issues when using a CSS radial gradient. The solution was to fake the radial gradient using two linear gradients, or to (unfortunately) use an old-fashioned image.

Avoid using cookies

Cookies get thrown into every request on a per-domain basis, so limit your usage of them. Use localStorage/sessionStorage instead, where possible.

App cache

Take advantage of the HTML5 app cache, though it’s an unwieldy beast to tame. This will mean you will need to make your site work offline, which might be tricky.

Assets that are traditionally cached get rechecked on page refresh (the server sends back a 304 response if your cache is still up-to-date). Whereas with the app cache, only one file (your manifest) is rechecked on page load.

Deferred JavaScript execution

We know that deferring downloading of scripts is advantageous (by placing scripts at the bottom of a page or by using the async attribute), but deferred execution of JavaScript is even more important. Sure, we can use the defer attribute, but that’s only really relevant for when the page is loading (defer lets the browser know that the UI doesn’t depend on the JavaScript, so it can be safely deferred).

But what about JavaScript that runs after the page loads, such as XHR and JSONP requests? What does this mean for the user? It means that the UI freezes up unexpectedly when JavaScript is being downloaded and executed in the background. But this doesn’t mean that you want to completely avoid background downloading of JavaScript. The Gmail mobile team came up with a clever solution: by commenting all of their code and dynamically eval’ing it when needed, they split up the JavaScript downloading from its execution.

Perceived performance

Do whatever you have to do to let the user know that the UI is still responding. This sometimes means faking that something is happening. Communication is key! If they clicked on a button, give some indication that they clicked on it. If they clicked on something that requires a request to the network, show a spinner right away, even if it means you haven’t even sent out a request. The user doesn’t need to know the nitty-gritty details – they just want to know that their intent was communicated.

Onclick delay

On several major mobile operating systems there’s a several hundred millisecond delay on the onclick event. This is because of the double-tap-to-zoom functionality. When a user first taps on the screen, there’s a hard-coded delay that waits for the second tap. If there’s no second tap, the onclick event is then fired. Unfortunately the only way to get around the delay is to tap into touch events instead, which is a bit more complicated than it appears on first glance.

For an overview of some of the challenges of implementing the workaround, see this article by Google’s Ryan Fioravanti.

Take advantage of hardware acceleration

Use hardware-accelerated CSS transforms where possible (translate3d, translateZ, rotate3d, and scale3d). An element that is hardware accelerated is turned into a graphic, which is perfect for the GPU to manipulate, taking away the burden from the CPU.

However, the GPU isn’t all-powerful, so don’t try to apply hardware accelerations to everything. Also, these elements still need to be refreshed periodically, and it turns out that you can make some good optimizations here. First, you’ll want to debug the composited layers on your desktop browser:

Debugging hardware acceleration

Chrome

1. Type the following in the address bar: about:flags
2. “Composited render layer borders” -> Enable

Safari

1. Open a terminal
2. $ defaults write com.apple.Safari IncludeInternalDebugMenu 1
3. $ defaults write com.apple.Safari CA_COLOR_OPAQUE 1
4. Open (or restart) Safari
5. Debug -> Show Compositing Borders

(to turn these off, run the same commands with a boolean FALSE: i.e. defaults write com.apple.Safari CA_COLOR_OPAQUE FALSE)

Keep in mind that each composited layer has a limited width and height. For instance, if you’re creating an image carousel, chances are the dimensions of the element will be too big to fit the layer into memory as one piece. This means that when the element is animated, the GPU has to break up the layer manually into several manageable chunks. It’s much better to chunk it yourself. To do this, you trigger hardware acceleration on each chunk.

So you will change this:

.carousel {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

To this:

.carousel {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

.carousel-pane {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

Where carousel-pane represents each child element of the carousel.

More info: (slides) WebKit in Your Living Room (Matt Seeley, Netflix)

Also see HTML5 Techniques for Optimizing Mobile Performance.

HTTP Pipelining

Take advantage of HTTP Pipelining, which is often overlooked, but has broad support on mobile. This virtually eliminates round trip times, for all but the first request.

Opera and Android support pipelining, and the newly released iOS 5 has added support for it.

Great! So how do you make sure your server is taking advantage of pipelining?

The first request to every server is sent by itself (only one request on the connection), and the browser looks for two properties in the response:

1. Use of HTTP/1.1
2. An explicit “Connection: Keep-Alive” header (required by Android)

-Blaze.io

If these criteria are met, subsequent requests will be pipelined. Sweet!

DNS Prefetching

Take advantage of DNS Prefetching. This theoretically speeds up load times, but there have been some issues, so be sure to test it.

To turn off DNS Prefetching, serve this meta tag:

<meta http-equiv="x-dns-prefetch-control" content="off">

You can also explicitly force a DNS lookup:

<link rel="dns-prefetch" href="http://www.example.com/">

Avoid library code bloat

If you’re developing for smart phones, you may be able to dramatically reduce the data over the wire by simply using new JavaScript APIs instead of a full-blown library or framework such as jQuery Mobile (which is rather a UI framework and depends on the desktop version of jQuery).

Among the things offered by newer browsers, which should reduce your dependence on a library:

• querySelector/querySelectorAll: CSS selectors that replace the need for a selector engine
• getElementsByClassName
• classList – replaces the need for hasClass, addClass, removeClass helpers (available on iOS5, but not sure what else)
• XMLHttpRequest – it’s probably time to learn how to do this natively instead of using a wrapper that takes care of IE’s old implementation that requests an ActiveX object. Cross-Origin Resource Sharing also means we can easily share resources across domains.
• HTML5 history (pushState, etc) – though an older bug means you should maybe hold off for now (note: the fix is present in iOS 5, which is based on a newer WebKit)
• HTML5 Form Validation reduces our need for JavaScript-based validation
• HTML5 input types that reduce our need for custom UI controls (date, time, slider [range], etc)

You’ll want to consider using either a library that was optimized for the browsers you’re targeting (such as Zepto.js) or simply bring in a microlibrary for specific tasks you want to perform.

Clientside databases

Most mobile browsers currently support WebSQL, which is being phased out in support of indexedDB, which is not widely implemented. However, if you use a wrapper such as Lawnchair, the transition from one to the other is relatively painless.

TODO: testing

More

(to be expanded on later)

• use standard desktop best practices (for smart phones)
• requestAnimationFrame instead of setTimeout
• setImmediate (where available) instead of setTimeout(fn(){},0)
• Ajax parsing times: you might want to use Multipart XHR so you can yield the process as it’s going through big responses (to prevent the UI locking up)
• avoid rgba, box shadows, text shadow, etc, as this greatly degrades performance (especially on animations)
• high resolution screens may suffer from animation/transition slowness – it’s been shown that manipulating the viewport tag to scale the page will speed up canvas rendering, and high-dpi scaling concerns are also evident in CSS transforms
• WebKit animations are faster than animations done via JavaScript
• if you do use JavaScript-based animations, limit the UI updates to ~17ms, which is equivalent to 60fps, which is the fastest your display will refresh anyway, so it’s pointless to try to do anything faster than that. Keep track of the time since the last UI update and don’t do anything if 17ms haven’t passed.

On the horizon…

• SPDY: already implemented in Amazon’s Silk browser for Kindle. Will possibly be on Android phones when the browser is powered by Chrome
• indexedDB replacing WebSQL

Tools

Mobile Perf bookmarklet
pcapperf (Packet Capture Web Performance Analyzer)
Blaze Mobitest – tests the loading time of your site on actual phones

More Resources

Mobile Web Application Best Practices
Mobile Performance (Steve Souders)
Strangeloop Web Performance Hub / Mobile

(odometer image via henrybloomfield on Flickr)

It looks like there’s finally some major improvements in mobile Safari, some of which I’ve found below on my “first glance” after downloading the SDK. Chime in if you find anything yourself!

Bad news first

Much to my disappointment, browser-based file uploads are still not supported (input type=”file”), even though they have been supported on Android since version 2.2 (which they have continued to improve and refine). The best use case for this is uploading pictures and videos straight from the browser, but Apple seems more interested in making those functions possible directly from the operating system itself, supporting only big names like Twitter and YouTube. So much for the little guys.

There’s other stuff on my wishlist that hasn’t yet been implemented, such as WebGL and indexedDB, along with some HTML5 input types, but regardless it’s good to see some real progress in iOS 5. (update: it appears that some WebGL features are starting to be supported, although the demos I tried didn’t seem to work).

Now let’s move on to the awesome new stuff that IS implemented!

position:fixed

A point of contention since the original iPhone was released in 2007, position:fixed now works on iOS5! What this means to developers: it’s now very easy to create top and bottom toolbars for your web apps. (there’s a long history of hacks and workarounds which will still need to be implemented as a fallback for the time being, but I’ll save that for another post. I should also mention that position:fixed has worked on Android since version 2.2, and webOS seems to support a somewhat hacky implementation).

Here’s a video demo of position:fixed on iOS5:

(check out the demo for yourself)

overflow:scroll

This is a similar problem to position:fixed, but now it works on iOS5! For content that doesn’t fit in its container element, previously the content was completely cut off (similarly with iframes). On the desktop we have the option to show scrollbars instead, allowing the user to scroll to see the content, but iOS made this difficult by requiring two fingers to scroll. Now overflow:scroll works and the user can scroll through the container’s content with a single finger. Developers can use new CSS to enable native scrolling (with scrollbars and momentum): -webkit-overflow-scrolling: touch; (via Johan Brook).

It’s assumed that it’s up to the developer to implement some sort of visual indicator to let the user know they can interact with the element.

One criticism so far: there’s no inertia scroll on this element, so it doesn’t quite feel the same as scrolling the page.

Here’s a quick video demo of overflow:scroll on iOS 5.

(check out the demo for yourself)

New Input Types

HTML5 defined many new helpful input types, and mobile operating systems have been slowly implementing them and creating optimized keyboards for the inputs. iOS5 now supports these input types: date, datetime, month, time, range. Click on the images to see their respective demo pages:

And a fun one – my iPhone slider hack based on input type range, which now actually works on the iPhone itself (because input type=”range” is now supported)!

Web Workers (JavaScript)

JavaScript is single-threaded, and it happens to run on the same thread as the UI, which means computationally expensive operations can lead to an unresponsive UI, and no one likes that. Web Workers help by offloading tasks to another thread and only returning output when it needs to. In other words, it’s a good thing for us web developers.

According to my own tests, Web Workers seemed to be briefly enabled on early versions of iOS3, but were soon disabled thereafter. Now with iOS5, they’re back again!

New ECMAScript 5 stuff (JavaScript)

(based on kangax’s ECMAScript 5 compatibility table)

HTML5Test score

iOS 4.3.2 scores 206 (7 bonus points) on the HTML5 Test, whereas iOS 5 beta scores 267 (9 bonus points). Much of these gains seem to be made in the “Forms” and “Parsing rules” sections, as well as the “Web Workers” section.

Miscellaneous new stuff

• Inline SVG in text/html
• float32array, int8array, int16array, int32array in JavaScript
• window.matchmedia() (JavaScript)
• onsearch Event (JavaScript) – it looks like this was just broken in later iOS 4.3 builds, but it’s back now in iOS 5
• MathML elements recognized (based on html5test)
• Improved speed of Canvas rendering (source)
• UIWebView and home screen web apps now running on Nitro? (expected but not tested)
• Newly supported CSS gradient syntax (-webkit-linear-gradient versus the older -webkit-gradient syntax)
• classList (JavaScript)
• A major WebKit HTML5 History (popstate) bug is now fixed
• contentEditable now works properly

These are some notes I collect as I read through Cameron Moll’s Mobile Web Design.

Introduction

8. Further info:

Designing the Mobile User Experience (Barbara Ballard)
dotMobi Web Developer’s Guide (dev.mobi)
Global Authoring Practices for the Mobile Web (Luca Passani) (passani.it/gap)
Constant Touch: A Global History of the Mobile Web (Jon Agar)
Personal, Portable, Pedestrian: Mobile Phones in Japanese Life (Mizuko Ito)
Mobile Interaction Design (Matt Jones)

13. iPhone released just before publication

Mobile Web Fundamentals

15. Putting 2.7 Billion in Context: Mobile Phone Users

18. Phone is smaller. Different input methods. Searching for specific data (directions or a phone number) rather than a truckload of data available via the desktop. One handed interaction.

19. Limitations: small screen size, difficulty of data input, user agent inconsistency

Opportunities: location-specific data, on-the-go-messaging, voice communication.

20. Weather Channel – greater reach on mobile web compared to desktop (source)

21. Rollable display – Polymer Vision Readius (video)

Many devices and user agents, making testing expensive and impractical. Emulators and testing on multiple devices will lead to successful deployment.

22. Opera Mini – far better UI than most native browsers

Mostly must be installed manually, except for T-Mobile shipping hundreds of thousands with it pre-installed

23. “…the mobile web experience is often a small screen, intermittent, one-handed experience”

24. understanding of target audience, their usage and what they want, and the contextual relevance (phone numbers, etc.) of data

25. Initiate a phone call: wtai scheme for WML, tel scheme for HTML.

Some browsers (Opera Mini, Safari Mobile) automatically convert links to phone numbers

WML scheme to add a number to the address book: wtai://wp/ap;+5555555;Amy%20Miller

26. carrier myopia: devs thinking the only way to reach the audience is by landing on the carrier deck. Mobile User Experience Conference (MEX) 2007: “Tearing down the walled garden will enhance the mobile content experience and release value for the industry. The objective should be a free market for content and applications, based on open standards and accessible to all. We think the current fragmentation of formats and channels to market is holding back growth.”

29. “cell phone” is an outdated term. Use “mobile” or “device”.

30. Japanese market: “keitai” (=”something you carry with you”)

Four Methods, Revisited

31. Mobile Web Design ~ The Series (Cameron Moll, 2005)

32. Method 1: Do nothing. If the markup is meaningful and standards-based, the better mobile browsers can adapt.

33. Good browsers such as the iPhone’s allow the viewing of the full site (with zooming).

34. Advantages: mobile browsers shoulder the burden of reformatting content, no additional effort needed, users have same content and possibly the same experience as on a PC

Disadvantages: doesn’t address contextual relevance or exploit device capabilities, users with zoom-enabled devices are a small market share

35. Method 2: Reduce images and styling

-presentation styling and images reduced on the fly

mowser.com (Russell Beattie, formerly Yahoo! Mobile)
skweezer.net – an older service dating back to 2001

2-minute mobile mod: http://www.mikeindustries.com/blog/archive/2005/07/make-your-site-mobile-friendly

36. Advantages: relies on implied hierarchy in HTML markup, viewable by a variety of devices, faster download, many browsers override CSS anyway, so why bother?

Disadvantages: ignores contextual relevance of mobility, file size may still be excessive

Method 3: Use Handheld Style Sheets

37. media=”handheld” style sheet

38. Not widely supported.

Advantages: handheld style sheet is built into CSS, at most one extra style sheet, one web address
Disadvantages: ignores contextual relevance of mobility, media=”handheld” support is inconsistent and nearly unreliable

Method 4: Create Mobile-Optimized Content

39. Kayak.com/moby

41. Advantages: contextually relevant (addresses first how the content is consumed, and presentation secondly), pages are leaner

Disadvantages: denies access to content by making assumptions about the user and device, two sets of files, alternate web addresses

42. Author’s opinion: doing nothing and mobile-optimized content are the two viable approaches

XHTML/CSS Development

43. WML deck and card metaphor. Decline giving way to XHTML. WAP protocol, WML language.

44. With WAP 2.0, XHTML-MP is the preferred markup language

45. Markup Test Pages, Mobile Web Design: Tips & Techniques (2005)

47. No need to define new standards just for the web. Same regardless of device: “The underlying standards for the greater web, regardless of device, person, or place, are the same: semantic markup, separation of structure and presentation, accessibility, and so on.”

W3C Mobile Web Best Practices 1.0

Global Authoring Practices for the Mobile Web (Luca Passani)

48. XHTML-MP template

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.0//EN" "http://www.wapforum.org/DTD/xhtml-mobile10.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <title>Site Title</title>
  <link rel="stylesheet" type="text/css" href="mobile.css"/>
  <meta http-equiv="Cache-Control" content="max-age=600" />
</head>
<body>
  <h1>Page Heading</h1>
  <p>Content</p>
</body>
</html>

49. DeveloperHome.com’s XHTML-MP Tutorial

50. XHTML-MP Tags List

Good case examples:
Flickr Mobile
Fandango Mobile

This is designed to be a “living” post of available materials which will hopefully be updated frequently.

Right now there’s not much, just some links, but I hope to keep adding materials, so check back!

The List

Sencha Touch
PhoneGap – native wrapper which provides access to native hardware through their own JavaScript APIs
jQuery Mobile – a separate mobile UI which has jQuery as a dependency
iAd – an update to Apple’s PastryKit, but probably intended only for ads
Zepto.js
Jo HTML5 Mobile App Framework
Sproutcore Touch
Unify
YUI3 – the desktop library has taken its first steps in supporting mobile (there’s no separate version just for mobile)
Spring Mobile: an extension to Spring MVC.
pyxis mobile
Wink Toolkit
ChocolateChip Mobile
WebApp.Net
baseJS
XUI
iWebKit
jQTouch
MooTouch – acquired and integrated into Sencha
UiUIKit (Universal iPhone UI Kit)
CiUI – CNET iPhone UI
UiUIKit
iUI (on Google Code) – the first library for the iPhone as far as I know.
PastryKit – Apple’s proprietary framework, peeked into just for curiousity (not recommended for use)

More from the Mobile Web series:

Part 1: The viewport metatag
Part 2: The mobile developer’s toolkit
Part 3: Designing buttons that don’t suck
Part 4: On designing a mobile webpage
Part 5: Using mobile-specific HTML, CSS, and JavaScript
Part 6: Dealing with device orientation
Part 7: Mobile JavaScript libraries and frameworks

I just came back from Mobile Monday Silicon Valley: Best Practices for Mobile Design at the Computer History Museum in Mountain View. This was a panel discussion with representatives from Google, Microsoft, Nokia, RIM, and Punchcut. There were quite a few little nuggets of wisdom which I tried to write down in bullet form:

Sue Booker, UED (Nokia)

• 1 trillion revenue worldwide from disabled users. 60m disabled users in the US.
• assume the user is multitasking
• apps will be used in totally unexpected ways
• not just physical disabilities.. also people in different situations or environmental limitations (no voice, no audio, limited bandwidth, etc)
• introduce new complicated functionality gradually. It’s easy to tell if it’s a new user or a repeat user, so slowly give repeat users more info.
• final takeaway: think of all types of users

Chris Nesladek, Commerce UI Lead (Google)

• 1. keep it simple
• 2. content is king
• 3. make it fun
• 4. make it fast
• 5. do one thing as an experiment
• new UI challenge in the future: flexible screens?
• discoverability of new features… for an Android feature: link to video demo on vimeo
• final takeaway: designers, make friends with developers, because that’s how the best products get made

Anne Sullivan, UI Designer (Microsoft/Tellme Networks)

• her focus: designing for speech on mobile
• problem: discoverability of speech
• understand the capabilities and limitations of speech
• graceful degradation for speech: what happens when a voice command isn’t recognized?
• be transparent about what happens about processing in the background. For instance, geolocation. Allow the user to change or override this info.
• paper prototypes are very helpful
• discoverability of voice command activation is hard.. have to test on device itself (i.e. to find the right button placement, etc.)
• final takeaway: forget powerpoint! do presentations on the phone itself

Jared Benson, Design (Punchcut)

• iterate quickly
• no substitute for device itself
• final takeaway: involve designers from the beginning

Wesley Yun, UX (RIM)

• don’t make assumptions about touch interfaces (common gestures). The touchscreen is still pretty new, so not everyone is used to this yet
• don’t be too clever for your users. Think of users first, and do what makes sense.
• prototype and fail quickly. If you’re not failing, you’re not innovating
• make sure unusual new interaction isn’t a core functionality (for instance, four-fingered scroll)
• if you make something redundant, you’ve failed as a designer
• final takeaway: make it obvious to users

If you read part 4 of this series, you’ll know that because of the variety of screen sizes, the best way to develop for mobile is to develop fluid layouts that take 100% of the available space on the screen.

What you probably didn’t think of is that there’s different screen widths even on the same device! This is due to screen orientation. And when the user changes the screen orientation, stuff may break (hey, it’s not a perfect world). In my experience this has meant needing to tweak percentage widths on elements, but I imagine there’s even more needs. Imagine an image carousel that can only fit three images across in portrait mode but can possibly fit four images in landscape mode. It might be nice to re-initialize the carousel to accommodate a forth image when the phone is in landscape mode (then again, that might create a lot of unnecessary overhead…).

In any case, I hope you can agree that it would sometimes be useful to know the screen orientation.

window.orientation and the orientationchange event

Luckily on the latest smartphones you have some goodies available to you that you don’t have on the desktop (since desktop users aren’t in the habit of constantly turning their screens sideways!).

window.orientation: this property gives the current screen orientation: 0 in portrait mode, 90 when rotated left, -90 when rotated right (no special value when the screen is upside-down)

orientationchange event: this window event fires after every 90 degrees of rotation and, like other events, can be applied in various ways:

// DOM Level 0 (avoid)
window.onorientationchange = function(){};

// DOM Level 2
window.addEventListener('orientationchange', function(){}, false);

Some websites recommend using orientationchange to dynamically add an orient attribute on the body element and target the orientation with CSS selectors (body[orient=landscape]), but this is in error. As it turns out, orientationchange is only fired AFTER the screen has been rotated (which also triggers a CSS reflow), which means this attribute is updated later (after the reflow). And unfortunately editing this orient attribute doesn’t trigger another CSS reflow. The result? When you rotate the device, these new CSS styles don’t get applied!

The fix is to add the orientation as a CSS class, which does trigger a CSS reflow. So our code at this point will look something like this:

(function(){
var init = function() {
  var updateOrientation = function() {
    var orientation = window.orientation;

    switch(orientation) {
      case 90: case -90:
        orientation = 'landscape';
      break;
      default:
        orientation = 'portrait';
    }

    // set the class on the HTML element (i.e. )
    document.body.parentNode.setAttribute('class', orientation);
  };

  // event triggered every 90 degrees of rotation
  window.addEventListener('orientationchange', updateOrientation, false);

  // initialize the orientation
  updateOrientation();
}

window.addEventListener('DOMContentLoaded', init, false);

})();

Now we can target elements like this in CSS:

.portrait body div { width: 10%; }
.landscape body div { width: 15%; }

With a little help from media queries

You may have heard of media queries being used to target mobile devices (based on screen pixel width) or to target the iPhone 4′s Retina display, but you may not have known that you can also target screen orientation!

@media all and (orientation: portrait) {
  body div { width: 10%; }
}

@media all and (orientation: landscape) {
  body div { width: 15%; }
}

The orientation media query is available on iOS 3.2+, Android 2.0+, and some other browsers.

This is a lot cleaner than the above JavaScript example in the sense that it’s pure CSS, and it’s part of the CSS that gets reflowed when the screen is rotated.

(Minor note: iOS 4 on the iPhone Simulator running 4.0.0 looks like it’s stuck in landscape orientation, but the media queries work correctly on my 3GS with 4.0.1)

Fallback: when window.orientation and media queries aren’t available…

If window.orientation isn’t available on a device, chances are the orientationchange event and media queries (for orientation) will also not be available. Oh no, what do we do now?

Even though this isn’t an entirely foolproof method, we can dynamically measure the window width and height and guess orientation based on that:

(function(){
var HTMLNode = document.body.parentNode;
var updateOrientation = function() {
  // landscape when width is biggest, otherwise portrait
  var orientation = (window.innerWidth > window.innerHeight) ? 'landscape': 'portrait';

  // set the class on the HTML element (i.e. )
  HTMLNode.setAttribute('class', orientation);
}
var init = function() {
  // initialize the orientation
  updateOrientation();

  // update every 5 seconds
  setInterval(updateOrientation, 5000);
}
window.addEventListener('DOMContentLoaded', init, false);
})();

Ok, so it’s not pretty, but it seems to work. The overhead in this fallback example is the fact that we have to use a polling technique (in this case every 5 seconds [5000 milliseconds]) to check for changes in orientation.

Note: there’s also the strong possibility that these browsers will not support the DOMContentLoaded event, but we’ll ignore that for the purposes of this article. (if you have problems, change DOMContentLoaded to load)

Putting it all together

Ok, so if you want the fallback example to work in addition to newer methods, unless you want to duplicate your CSS, then avoid using media queries to target orientation. Instead we’ll rely on adding a class to the html tag (or the body tag if you prefer).

Once we put everything together, we get something that looks like this:

(function(){
var supportsOrientation = (typeof window.orientation == 'number' && typeof window.onorientationchange == 'object');
var HTMLNode = document.body.parentNode;
var updateOrientation = function() {
  // rewrite the function depending on what's supported
  if(supportsOrientation) {
    updateOrientation = function() {
      var orientation = window.orientation;

      switch(orientation) {
        case 90: case -90:
          orientation = 'landscape';
        break;
        default:
          orientation = 'portrait';
      }

      // set the class on the HTML element (i.e. )
      HTMLNode.setAttribute('class', orientation);
    }
  } else {
    updateOrientation = function() {
      // landscape when width is biggest, otherwise portrait
      var orientation = (window.innerWidth > window.innerHeight) ? 'landscape': 'portrait';

      // set the class on the HTML element (i.e. )
      HTMLNode.setAttribute('class', orientation);
    }
  }

  updateOrientation();
}
var init = function() {
  // initialize the orientation
  updateOrientation();

  if(supportsOrientation) {
    window.addEventListener('orientationchange', updateOrientation, false);
  } else {
    // fallback: update every 5 seconds
    setInterval(updateOrientation, 5000);
  }

}
window.addEventListener('DOMContentLoaded', init, false);
})();

Minified (540 bytes):

(function(){var e=typeof window.orientation=="number"&&typeof window.onorientationchange=="object",f=document.body.parentNode;function c(){c=e?function(){var d=window.orientation;switch(d){case 90:case -90:d="landscape";break;default:d="portrait"}f.setAttribute("class",d)}:function(){f.setAttribute("class",window.innerWidth>window.innerHeight?"landscape":"portrait")};c()}window.addEventListener("DOMContentLoaded",function(){c();e?window.addEventListener("orientationchange",c,false):setInterval(c,5E3)},false)})();

Conclusion

And that’s it! Now we can reliably target different screen orientations with some straightforward CSS:

.portrait body div { width: 10%; }
.landscape body div { width: 15%; }

Again, in my experience I’ve used this to fix bugs. But I’m sure you can find more creative uses for it!

Related

iPad web development tips (Nicholas C. Zakas)
iPhone window.onorientationchange Code (Ajaxian)
The orientation media query (Quirksmode)

More from the Mobile Web series:

Part 1: The viewport metatag
Part 2: The mobile developer’s toolkit
Part 3: Designing buttons that don’t suck
Part 4: On designing a mobile webpage
Part 5: Using mobile-specific HTML, CSS, and JavaScript
Part 6: Dealing with device orientation
Part 7: Mobile JavaScript libraries and frameworks

Mobile-specific HTML

Viewport tag

Use the viewport tag to properly fit the content to the screen:

<meta name="viewport" content="width=device-width, initial-scale=1.0"/>

Tel scheme (to initiate phone calls)

<a href="tel:18005555555">Call us at 1-800-555-5555</a>

Sms scheme (to initiate text messages)

<a href="sms:18005555555">
<a href="sms:18005555555,18005555556">              <!-- multiple recipients -->
<a href="sms:18005555555?body=Text%20goes%20here">  <!-- predefined message body -->

Disable automatic telephone number linking

<meta name="format-detection" content="telephone=no">

iOS-specific HTML (some work on Android as well)

You also have access to several Apple-specific tags to use in your iOS applications (iPhone, iPad, and don’t forget the iPod Touch!).

<link rel="apple-touch-icon" href="icon.png"/>


<link rel="apple-touch-icon-precomposed" href="icon.png"/>


<link rel="apple-touch-icon" sizes="72x72" href="touch-icon-ipad.png" />
<link rel="apple-touch-icon" sizes="114x114" href="touch-icon-iphone4.png" />


<link rel="apple-touch-startup-image" href="startup.png">


<meta name="apple-mobile-web-app-capable" content="yes" />


<meta name="apple-mobile-web-app-status-bar-style" content="black" />

Turn off autocorrect, autocomplete, and autocapitalize

And also some handy attributes to turn off annoying autocorrect features:

<input autocorrect="off" autocomplete="off" autocapitalize="off">

Mobile-specific CSS

position:fixed and overflow:scroll

Mobile browsers are now starting to support these basic CSS properties better. Position:fixed will work on Android 2.2+ and iOS 5+. Overflow:scroll works with one finger on iOS 5+.

Also, iOS 5 has additional CSS to give the native scrollbar and momentum/intertia to elements with overflow:scroll: -webkit-overflow-scrolling: touch;

Media queries

Media queries enable you to target specific features (screen width, orientation, resolution) within CSS itself. Media queries themselves are actually quite old and are not mobile specific (they used to be popular for making a print-friendly version of webpages).

You can use them two ways: 1) inline in a CSS stylesheet or 2) as the “media” attribute in the link tag, which loads an external stylesheet. The following is an example of inline CSS that’s applied only when the device is in portrait mode:

@media all and (orientation: portrait) {
	body { }
	div { }
}

Here’s the same media query using the other method, which points to an external stylesheet (not recommended):

<link rel="stylesheet" media="all and (orientation: portrait)" href="portrait.css" />

This is not recommended because it creates an additional HTTP request (bad for performance). Also, in the case of screen orientation, the separate CSS stylesheet is NOT downloaded when the screen is rotated.

Here’s a few examples of using inline CSS:

// target small screens (mobile devices or small desktop windows)
@media only screen and (max-width: 480px) {
  /* CSS goes here */
}

/* high resolution screens */
@media (-webkit-min-device-pixel-ratio: 2),
             (min--moz-device-pixel-ratio: 2),
             (min-resolution: 300dpi) {
  header { background-image: url(header-highres.png); }
}

/* low resolution screens */
@media (-webkit-max-device-pixel-ratio: 1.5),
             (max--moz-device-pixel-ratio: 1.5),
             (max-resolution: 299dpi) {
  header { background-image: url(header-lowres.png); }
}

Read more: Media queries (Mozilla Developer Center)

Miscellaneous CSS

-webkit-tap-highlight-color (iOS): override the semitransparent color overlay when a user clicks a link or clickable element. To completely disable it, set the value to ‘transparent’ or ‘rgba(0,0,0,0)’

-webkit-user-select: none; – prevent the user from selecting text (also works on desktop WebKit)

-webkit-touch-callout: none; – prevent the callout toolbar from appearing when a user touches and holds an element such as an anchor tag.

Mobile-specific JavaScript

window.scrollTo(0,0);

This hides the address bar and takes advantage of the entire device screen. You’ll have to set this in a timeout and make sure to get the timing right. See Remy Sharp’s post for more details.

window.matchMedia()

(iOS 5+) Again, just as CSS media queries aren’t specific to mobile, they do come in especially useful for mobile, so it’s worth mentioning their JavaScript counterpart. window.matchMedia() is a JavaScript-based version of media queries. You can use respond.js as a polyfill for devices that don’t support this functionality natively.

navigator.connection

(Android 2.2+) Determine if the phone is running on WiFi, 3G, etc. Example:

if (navigator.connection.type==navigator.connection.WIFI) {
  // code for WiFi connections (high-bandwidth)
}

window.devicePixelRatio

Determine screen resolution (analogue to the CSS media query). (iPhone 4 has the value 2, while Nexus One has the value 1.5).

window.navigator.onLine

Not strictly mobile, but helpful for apps to determine if they’re being run offline.

window.navigator.standalone

(iOS 2.1+): determine if it’s running in full-screen mode

Touch and gesture events

touch events (iOS, Android 2.2+): touchstart, touchmove, touchend, touchcancel

gesture events (Apple only, iOS 2+): gesturestart, gesturechange, gesturend give access to predefined gestures (rotation, scale, position)

Screen orientation (every 90 degrees)

orientationchange event: triggered every 90 degrees of rotation (portrait and landscape modes). The current orientation is available through window.orientation

Device orientation (more fine-grained)

The deviceorientation event will fire very frequently, and gives more fine-grained information about the device’s orientation in three dimensions.
MozOrientation (or onmozorientation?) (Fennec/Firefox Mobile, Firefox 3.5+): also not strictly mobile. Gives access to the device’s accelerometer (x-y-z orientation data), updated periodically. Works on Android phones running Mobile Firefox. On the desktop this works with laptops such as Thinkpads and MacBooks.

devicemotion (shake gestures, etc.)

devicemotion fires when the user shakes or moves their device. Devicemotion taps into the accelerometer, which is fires off when the device accelerates. Contrast this with the deviceorientation event, which taps into the device’s gyroscope (if it has one), which only measures the 3D angle orientation, even when the device is at rest.

Media capture API

While iOS is still lacking basic file inputs, Android is forging ahead, giving developers fine-grained control over content users can upload.

<!-- regular file upload (Android 2.2+, NO iOS) -->
<input type="file"></input>

<!-- opens directly to the camera (Android 3.0+) -->
<input type="file" accept="image/*;capture=camera"></input>

<!-- opens directly to the camera in video mode (Android 3.0+) -->
<input type="file" accept="video/*;capture=camcorder"></input>

<!-- opens directly to the audio recorder (Android 3.0+) -->
<input type="file" accept="audio/*;capture=microphone"></input>

BlackBerry specific

If you’re developing for a BlackBerry Widget, you have access to proprietary information through the blackberry object (which gives access to useful information such as blackberry.network [returns values such as CDMA and Wi-Fi] and blackberry.system).

You also have the option to use PhoneGap, which augments JavaScript and gives you access to more phone features that native apps would have access to.

Use a mobile-optimized JavaScript library

I’ve created a separate entry for the available mobile libraries and frameworks.

Because smartphone browsers are standards-based, the aim of a JavaScript library on mobile is less towards API normalization and more towards providing an actual UI framework, usually to emulate the feel of native apps (and to provide easier workarounds to lack of access to position:fixed). We’ve seen a few libraries released that emulate the iPhone UI, and in the future we might see libraries emulating the Android UI, as well as entirely new UIs.

There’s also a bit to be said about simply loading full desktop JavaScript libraries into mobile clients. In my opinion this doesn’t particularly make sense, especially in a world where latency and bandwidth are so much more of a concern. It doesn’t make sense to force the user wait longer and download code that’s ultimately useless to them (hacks for desktop browsers such as IE 6, etc).

Take advantage of new stuff!

While not specific to mobile, there’s a lot of new stuff in general that you can use. If you limit yourself to the top smartphones (iPhone, Android, and maybe webOS), compared to the desktop you immediately have access to an even wider array of new stuff, especially many Webkit proprietary features, since most of these top smartphones have browsers based on Webkit.

-HTML: new tags (HTML5 (I’m sure you’ve heard of it by now…))
-CSS: 2d transforms, 3d transforms, animation, border radius, custom fonts with @font-face, etc.
-JavaScript: strict mode, constants, block scope, Date.now(), etc.

Slides

More from the Mobile Web series:

Part 1: The viewport metatag
Part 2: The mobile developer’s toolkit
Part 3: Designing buttons that don’t suck
Part 4: On designing a mobile webpage
Part 5: Using mobile-specific HTML, CSS, and JavaScript
Part 6: Dealing with device orientation
Part 7: Mobile JavaScript libraries and frameworks

Summary

Thanks to my coworker for lending me his phone for a few minutes (he waited in line for 9 hours yesterday when it first went on sale!).

This is quite an improvement over the iPhone 3GS running iOS4, which ran the SunSpider test in 13787ms. The iPhone 4 ran the same test in 10669ms, which makes it ~22% faster head-to-head.

Processors

The iPhone 3GS has a Samsung S5PC100 ARM Cortex-A8[5] 833 MHz (underclocked to 600 MHz), while the iPhone 4 has an Apple A4 1GHz (likely underclocked to an unknown speed).

Why would you want to install obsolete AVDs? In my case it’s a matter of research: I simply want to see the progress of features being added to its browser, and track those changes over time.

Step 1: Manually download and inspect repository.xml

When updating from “Available Packages” there’s a little one-line error saying “Some packages were found but are not compatible updates.”

Ok, so let’s check out the XML for ourselves to see if we can find anything. So point your browser to https://dl-ssl.google.com/android/repository/repository.xml (to see the XML, right click to view the page source if you’re using a Webkit-based browser).

Search for the AVD version you want. In this case we want 2.0 and 2.0.1, so a simple search find the relevant blocks of code. And we also find the XML tag that’s the cause of our troubles, which prevents us from easily getting the AVDs:

<sdk:obsolete />

Step 2: Get the AVDs!

At this point you could do two things, either save repository.xml to your computer and remove these “obsolete” tags (then add the XML to your AVD Manager by clicking “Add Add-on Site…”), or simply find the path to the AVD and download it manually.

The second option is to simply manually download the paths, which are easy to find in the XML and are listed here for your convenience:
Android 2.0 AVD (Linux)
Android 2.0 AVD (Mac OSX)
Android 2.0 AVD (Windows)

Android 2.0.1 AVD (Linux)
Android 2.0.1 AVD (Mac OSX)
Android 2.0.1 AVD (Windows)

Step 3: Install!

Create directories under your Android SDK installation’s “platforms” folder. In this case we have 2.0 (API level 5) and 2.0.1 (API level 6), so I created these folders: android-5 and android-6. Now just unzip the contents into these folders.

Start up the AVD Manager and click on “Installed Packages”. If you don’t see your new (obsolete) packages you just installed, hit the Refresh button and you should see them.

Now you can create new AVDs with these obsolete packages!