Archive for the ‘performance’ Category

iOS5 SunSpider: iPhone 4S vs iPhone 4 vs iPhone 3GS

Friday, October 14th, 2011

Result table (SunSpider 0.9.1, all devices running iOS 5)

Test iPhone 4S iPhone 4 iPhone 3GS
Total 2270ms 3483ms 4903ms
3D 281ms 486ms 672ms
Access 279ms 410ms 614ms
Bitops 177ms 200ms 270ms
Controlflow 20ms 26ms 34ms
Crypto 164ms 211ms 315ms
Date 332ms 528ms 770ms
Math 219ms 433ms 553ms
Regexp 88ms 117ms 151ms
String 711ms 1072ms 1525ms

Raw results

iPhone 4S (iOS5)

iPhone 4 (iOS5)

iPhone 3GS (iOS5)

Bonus: FishIETank (10 fish) (Canvas test)

Test iPhone 4S iPhone 4 iPhone 3GS
Total 50fps 35fps 25fps

One thing to note here: the 3GS has a bit of an advantage because of a non-retina screen. However, it’s still outperformed easily by the iPhone 4 and 4S.

Related

JavaScript SunSpider benchmark: iOS 4.3 vs iOS 4.0
BlackBerry Torch SunSpider results (JavaScript benchmark)
iPhone 4 SunSpider test results (22% faster than iPhone 3GS)
JavaScript SunSpider test: iOS 3.1.3 versus iOS 4 GM
JavaScript SunSpider: HTC Evo versus HTC Incredible versus Nexus One

Mobile Performance Manifesto

Tuesday, October 11th, 2011

Earlier this year I gave a talk (slides) outlining the latest and greatest in mobile performance, including a bit of my own unscientific research into carrier latency and bandwidth thanks to boomerang.js.

I realized that interest in mobile performance has exploded recently, especially with Steve Souders announcing his focus on mobile, and I thought it was time for an update, this time in blog form. Also, my old slides have been somewhat embarrassing. For some strange reason, at the time I wanted to give S5 a try – that outdated, ancient, not-performant slideshow framework. The result is a slideshow on performance that loads slowly… doh! (incidentally, I recommend deck.js as an alternative).

In any case, it was time for a roundup of mobile performance best practices, in blog form. I’m not sure if it’s properly called a manifesto, but it is what it is! Onward!

For fun: Latency and bandwidth tests

Before we start.. just a little fun data! It’s always been a pleasure to fiddle around with boomerang.js and compare results. With my latest trip to Japan, I had the opportunity to run Boomerang on Japan’s e-mobile 3G network in the remotest of places, up in Hakuba/Nagano in the Japanese alps.

When I got back, I ran the same tests from downtown San Francisco, which was closer to my California-based test server, and should’ve been faster, right? Theoretically…

Boomerang Tests (running on a California-hosted server)

Verizon 3G (SF) ATT 3G (SF) e-mobile 3G (Hakuba)
Latency 251ms 901ms 401ms
Bandwidth 62kbps 3kbps 95kbps

Yes, e-mobile didn’t have the greatest ping, but it handily beat ATT, even though it was going across the entire Pacific Ocean! Note that it came out the best in the bandwidth tests however…

(Note: it’s unknown if or how Steve Souders’s latest research affects these findings)

There’s a lot of factors that could’ve been involved, so look to something like OpenSignalMaps for more data.

Ok, onto the tips!

Page Organization

For feature phones that have little to no caching, aggressively combine requests (deliver HTML/CSS/JS all in one package). For smart phones, take advantage of caching by mirroring desktop frontend best practices: separate HTML, CSS, and JS so they can be cached (per-session and across sessions).

There’s old research about extremely small cache sizes on iOS in particular, but this research has been followed up on by more recent research by Ryan Grove (Yahoo!) and Steve Souders (Google), which shows that we shouldn’t be so paranoid, since caching is pretty decent across all the major mobile browsers.

Of particular interest is a browser’s capability to cache files in a current session, browsing from page-to-page, which is what most users will end up doing (caching of the page across sessions is another matter). What is the maximum file size a browser will cache during a session? The results (via Browserscope) end up being encouraging:

Maximum Cache Size (MB)
Android 2.1 4+
Android 2.2 2
Android 2.3 2
Android 3.0 4+
Blackberry 6.0.0 4+
iPad 4.3.5 4+
iPad 5.0 4+
iPhone 4.3.5 4+
iPhone 5.0 4+
Opera Mini 4 4+
Opera Mini 5 4+
Opera Mini 6 4+
webOS 2.0 1

Avoid redirects (foo.com -> m.foo.com)

If possible, perform the redirection behind the scenes on the server, which should be transparent to the user. When a user performs a Google search and clicks on your page, they’re already getting redirected once by Google (check it yourself). Your own redirects are adding a second redirect where there need not be one.

Optimize Images


Deliver appropriately-sized images to devices. The philosophy of responsive design makes it easy to simply downscale images to fit the screen, but avoid this where possible, as this means wasted bandwidth.

You can optimize images through CSS media queries or in JavaScript (see below). Though you want to reduce your dependence on cookies, it may be a good idea to store these width/height values into a cookie (or localStorage if you are fetching images in nontraditional ways) so the values can be read by the server, which can deliver appropriately-sized images. This technique has been implemented by Filament Group, so you should probably read about their experiences before trying to roll your own.

Also note that where applicable, at the expense of performance you may want to serve higher resolution images for better screens, such as for Retina displays (which can be detected with JavaScript or CSS).

JavaScript examples

window.innerHeight;       // max height actually available
window.innerWidth;        // max width actually available
window.devicePixelRatio;  // pixel density (standard is 1, high resolution is generally > 1)

(note that screen.width and screen.height are also available to tell you the dimensions of the entire screen, but this isn’t all available due to the space taken by the OS and browser chrome)

Media query examples

@media only screen and (max-width: 480px) {
  /* small screen styles */
}

@media only screen and (min-width: 481px) {
  /* large screen styles */
}

@media (-webkit-min-device-pixel-ratio: 1.5),
       (-o-min-device-pixel-ratio: 3/2),
       (min--moz-device-pixel-ratio: 1.5),
       (min-device-pixel-ratio: 1.5) {
  /* high resolution styles */
}

navigator.connection (Android only)

Use navigator.connection if it’s available to serve different assets based on connection speed (3G vs WIFI, for instance).

Here’s the contents of the navigator.connection object (with a phone running on a 3G connection):

{
  type: 4,
  UNKNOWN: 0,
  ETHERNET: 1,
  WIFI: 2,
  CELL_2G: 3,
  CELL_3G: 4
}

And an example of how you could write code for each type:

(function(){  // sandbox our code
  if(!navigator.connection) navigator.connection = {type:0, UNKNOWN: 0};  // polyfill
  var connection = navigator.connection;

  if(connection.type === connection.WIFI || connection.type === connection.ETHERNET) {
    // high bandwidth
  } else {
    // normal bandwidth
  }
})();

Here’s how we might be able to use Modernizr to help us out:

(function(){  // sandbox our code
  if(!navigator.connection) navigator.connection = {type:0, UNKNOWN: 0};  // polyfill
  var connection = navigator.connection;

  // add a custom test to Modernizr
  Modernizr.addTest('highbandwidth', function(){
    return connection.type === connection.WIFI || connection.type === connection.ETHERNET;
  });

})();

We can now target special high-bandwidth assets with CSS:

.highbandwidth .logo {background-image:url('logo-high.jpg');}
.no-highbandwidth .logo {background-image:url('logo.jpg');}

base64 encode small UI images

You can base64 encode binary image data in HTML and CSS:

HTML

<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==" alt="Red dot">

CSS

.dot {
  background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==');
}

There seems to be the mistaken belief floating around that base64 images aren’t cacheable. However, because you can embed them in your CSS, base64 encoded images will be cached along with the rest of your CSS.

You’ll probably just want to limit this to small UI icons however. I wrote a quick post on the drawbacks, and also a comparison with sprites.

If you’re using Compass/SASS, base64 encoding images is pretty trivial and easy to maintain with the inline-image helper.

Unicode and Emoji

☆ (HTML entity: #9734)
(HTML entity: #x1f468)

Before even considering base64 images, take advantage of icons that are already made for you: unicode characters and emoji (where supported). Keep in mind that these will look different across browsers, so this might not be an option for everyone.

Because of inconsistent implementations of Emoji character codes between Asian carriers, representatives from Apple and Google made a proposal to add Emoji to the Unicode standard. Unfortunately it seems that only iPhone supports them for now. Here’s a handy chart of all the Unicode emoji characters available.

Take advantage of CSS3

CSS3 offers many replacements for things we needed images for previously. RGBA values replace the need for a semitransparent image for overlays and such. Likewise, border-radius, box shadow, linear-gradients, radial-gradients all reduce the need for images.

However, do be aware that though phones support these new features, it doesn’t mean they’re necessarily ready for primetime. Something I’ve encountered recently to remind me of this fact is severe color banding issues when using a CSS radial gradient. The solution was to fake the radial gradient using two linear gradients, or to (unfortunately) use an old-fashioned image.

Avoid using cookies

Cookies get thrown into every request on a per-domain basis, so limit your usage of them. Use localStorage/sessionStorage instead, where possible.

App cache

Take advantage of the HTML5 app cache, though it’s an unwieldy beast to tame. This will mean you will need to make your site work offline, which might be tricky.

Assets that are traditionally cached get rechecked on page refresh (the server sends back a 304 response if your cache is still up-to-date). Whereas with the app cache, only one file (your manifest) is rechecked on page load.

Deferred JavaScript execution

We know that deferring downloading of scripts is advantageous (by placing scripts at the bottom of a page or by using the async attribute), but deferred execution of JavaScript is even more important. Sure, we can use the defer attribute, but that’s only really relevant for when the page is loading (defer lets the browser know that the UI doesn’t depend on the JavaScript, so it can be safely deferred).

But what about JavaScript that runs after the page loads, such as XHR and JSONP requests? What does this mean for the user? It means that the UI freezes up unexpectedly when JavaScript is being downloaded and executed in the background. But this doesn’t mean that you want to completely avoid background downloading of JavaScript. The Gmail mobile team came up with a clever solution: by commenting all of their code and dynamically eval’ing it when needed, they split up the JavaScript downloading from its execution.

Perceived performance

Do whatever you have to do to let the user know that the UI is still responding. This sometimes means faking that something is happening. Communication is key! If they clicked on a button, give some indication that they clicked on it. If they clicked on something that requires a request to the network, show a spinner right away, even if it means you haven’t even sent out a request. The user doesn’t need to know the nitty-gritty details – they just want to know that their intent was communicated.

Onclick delay

On several major mobile operating systems there’s a several hundred millisecond delay on the onclick event. This is because of the double-tap-to-zoom functionality. When a user first taps on the screen, there’s a hard-coded delay that waits for the second tap. If there’s no second tap, the onclick event is then fired. Unfortunately the only way to get around the delay is to tap into touch events instead, which is a bit more complicated than it appears on first glance.

For an overview of some of the challenges of implementing the workaround, see this article by Google’s Ryan Fioravanti.

Take advantage of hardware acceleration

Use hardware-accelerated CSS transforms where possible (translate3d, translateZ, rotate3d, and scale3d). An element that is hardware accelerated is turned into a graphic, which is perfect for the GPU to manipulate, taking away the burden from the CPU.

However, the GPU isn’t all-powerful, so don’t try to apply hardware accelerations to everything. Also, these elements still need to be refreshed periodically, and it turns out that you can make some good optimizations here. First, you’ll want to debug the composited layers on your desktop browser:

Debugging hardware acceleration


Chrome

  1. Type the following in the address bar: about:flags
  2. “Composited render layer borders” -> Enable

Safari

  1. Open a terminal
  2. $ defaults write com.apple.Safari IncludeInternalDebugMenu 1
  3. $ defaults write com.apple.Safari CA_COLOR_OPAQUE 1
  4. Open (or restart) Safari
  5. Debug -> Show Compositing Borders

(to turn these off, run the same commands with a boolean FALSE: i.e. defaults write com.apple.Safari CA_COLOR_OPAQUE FALSE)

Keep in mind that each composited layer has a limited width and height. For instance, if you’re creating an image carousel, chances are the dimensions of the element will be too big to fit the layer into memory as one piece. This means that when the element is animated, the GPU has to break up the layer manually into several manageable chunks. It’s much better to chunk it yourself. To do this, you trigger hardware acceleration on each chunk.

So you will change this:

.carousel {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

To this:

.carousel {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

.carousel-pane {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

Where carousel-pane represents each child element of the carousel.

More info: (slides) WebKit in Your Living Room (Matt Seeley, Netflix)

Also see HTML5 Techniques for Optimizing Mobile Performance.

HTTP Pipelining

HTTP Pipelining diagram

Take advantage of HTTP Pipelining, which is often overlooked, but has broad support on mobile. This virtually eliminates round trip times, for all but the first request.

Opera and Android support pipelining, and the newly released iOS 5 has added support for it.

Great! So how do you make sure your server is taking advantage of pipelining?

The first request to every server is sent by itself (only one request on the connection), and the browser looks for two properties in the response:

  1. Use of HTTP/1.1
  2. An explicit “Connection: Keep-Alive” header (required by Android)

-Blaze.io

If these criteria are met, subsequent requests will be pipelined. Sweet!

DNS Prefetching

Take advantage of DNS Prefetching. This theoretically speeds up load times, but there have been some issues, so be sure to test it.

To turn off DNS Prefetching, serve this meta tag:

<meta http-equiv="x-dns-prefetch-control" content="off">

You can also explicitly force a DNS lookup:

<link rel="dns-prefetch" href="http://www.example.com/">

Avoid library code bloat

If you’re developing for smart phones, you may be able to dramatically reduce the data over the wire by simply using new JavaScript APIs instead of a full-blown library or framework such as jQuery Mobile (which is rather a UI framework and depends on the desktop version of jQuery).

Among the things offered by newer browsers, which should reduce your dependence on a library:

  • querySelector/querySelectorAll: CSS selectors that replace the need for a selector engine
  • getElementsByClassName
  • classList – replaces the need for hasClass, addClass, removeClass helpers (available on iOS5, but not sure what else)
  • XMLHttpRequest – it’s probably time to learn how to do this natively instead of using a wrapper that takes care of IE’s old implementation that requests an ActiveX object. Cross-Origin Resource Sharing also means we can easily share resources across domains.
  • HTML5 history (pushState, etc) – though an older bug means you should maybe hold off for now (note: the fix is present in iOS 5, which is based on a newer WebKit)
  • HTML5 Form Validation reduces our need for JavaScript-based validation
  • HTML5 input types that reduce our need for custom UI controls (date, time, slider [range], etc)

You’ll want to consider using either a library that was optimized for the browsers you’re targeting (such as Zepto.js) or simply bring in a microlibrary for specific tasks you want to perform.

Clientside databases

Most mobile browsers currently support WebSQL, which is being phased out in support of indexedDB, which is not widely implemented. However, if you use a wrapper such as Lawnchair, the transition from one to the other is relatively painless.

TODO: testing

More

(to be expanded on later)

  • use standard desktop best practices (for smart phones)
  • requestAnimationFrame instead of setTimeout
  • setImmediate (where available) instead of setTimeout(fn(){},0)
  • Ajax parsing times: you might want to use Multipart XHR so you can yield the process as it’s going through big responses (to prevent the UI locking up)
  • avoid rgba, box shadows, text shadow, etc, as this greatly degrades performance (especially on animations)
  • high resolution screens may suffer from animation/transition slowness – it’s been shown that manipulating the viewport tag to scale the page will speed up canvas rendering, and high-dpi scaling concerns are also evident in CSS transforms
  • WebKit animations are faster than animations done via JavaScript
  • if you do use JavaScript-based animations, limit the UI updates to ~17ms, which is equivalent to 60fps, which is the fastest your display will refresh anyway, so it’s pointless to try to do anything faster than that. Keep track of the time since the last UI update and don’t do anything if 17ms haven’t passed.

On the horizon…

Tools

Mobile Perf bookmarklet
pcapperf (Packet Capture Web Performance Analyzer)
Blaze Mobitest – tests the loading time of your site on actual phones

More Resources

Mobile Web Application Best Practices
Mobile Performance (Steve Souders)
Strangeloop Web Performance Hub / Mobile

(odometer image via henrybloomfield on Flickr)

When to base64 encode images (and when not to)

Sunday, August 28th, 2011

Introduction

Ever since Steve Souders started evangelizing web performance, it’s been pounded into our heads that extra HTTP requests add a lot of additional overhead, and that we should combine them if possible to dramatically decrease the load time of our web pages.

The practical implication of this has been to combine our JavaScript and CSS files, which is relatively easy and straightforward, but the harder question has been what to do with images.

Sprites

Image sprites are a concept taken from video games: the idea is to cram a ton of image assets into one file, and rearrange a “viewport” of sorts to view only specific pieces of that file at a time. For instance, a simple sprite that holds two images might have one viewport that only looks at the top half of the sprite (image #1), and another viewport that only looks at the bottom half (image #2).

On the web side of things, this means that those multiple requests have now been combined into one request. This is nice because it saves both the overhead of additional HTTP requests as well as the overhead of setting up an image’s file header each time.

But there’s a few drawbacks with using image sprites:

  • hard to maintain and update: without some tool to help, manually editing and putting together image sprites is quite a chore
  • increased memory consumption (possibly very dramatic): this is often overlooked. The time to deliver the images is decreased at the expense of a bigger memory and CPU footprint, especially for large sprites and sprites with a lot of whitespace.
  • bleedthrough: for sprites that don’t have much whitespace to separate images, there’s an increased chance of nearby images visibly bleeding through other elements

Data URIs and Base64 encoding

Data URIs (see this, this, and this) and Base64 encoding goes hand-in-hand. This method allows you to embed images right in your HTML, CSS, or JavaScript.

Just like sprites, you save HTTP requests, but there’s also some drawbacks:

  • base64 encoding makes file sizes roughly 33% larger than their original binary representations, which means more data down the wire (this might be exceptionally painful on mobile networks)
  • data URIs aren’t supported on IE6 or IE7
  • base64 encoded data may possibly take longer to process than binary data (anyone want to do a study on this?) (again, this might be exceptionally painful for mobile devices, which have more limited CPU and memory) (side note: CSS background-images seem to actually be faster than img tags)

The “33% larger” claim is generally accepted truth now, despite the fact that the figure varies wildly depending on the type of content. This is exactly what I wanted to test, albeit in a pretty limited and nonscientific way.

Before I tested, I wanted to keep in mind a few unverified intuitions (which aren’t entirely my own, but seem to be ideas that are floating around out there). Here’s a few questions I had before going to test:

  • Is base64 encoding with gzipping roughly equal to the original filesize of the binary file?
  • Is base64 encoding best for small images?
  • Is base64 encoding best for small and simple icons and not good for pictures and photos?
  • Is base64 encoding best when multiple files are merged together?

There’s something else I wanted to test: whether Gzipping binary image data made much difference. I know text compresses well, but is it even worth compressing JPEG files with Gzip, for instance?

I ran three tests: one with a set of small UI icons, one with a set of small photographs, and one with a set of the same photographs in a larger size. Though my tests were by no means extensive, they do show that care should be taken in making assumptions about base64.

Just a note about the tables: they are comparing the binary form (original png or jpeg) with the base64 form as it would appear in a CSS stylesheet, and comparing each of those with their gzipped form, which is most likely how they would be sent down the wire. The CSS representation has a few practical declarations and looks something like this:

.address-book--arrow {
	background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAp1JREFUeNqEU21IU1EYfu7unW5Ty6aBszYs6MeUjGVYokHYyH5E1B9rZWFEFPQnAwmy6Hc/oqhfJsRKSSZGH1JIIX3MNCsqLTD9o1Oj6ebnnDfvvefezrnbdCHhCw/n433P8z7nPe/hBEEAtX0U7hc164uwuvVSXKwZLoOmaRDim+7m9vZa0WiEKSUFFpNpCWlmMyypqTDRuYn6t3k8vmQ2gRDCxs0t9fW45F52aBTROJLtZl7nEZad2m+KtoQCQ0FBARyOCGRZ/q92I1WgqqXlfdd95VsrK8/pChIEqqpCkiQsiCII0aBQZZoWl8lzFDwsFjMl0DBLY8Lj41hBwK4jSQrWOIphL6xYyhwJDWGo6wFSaH1Y3PTCAsITE1oyAa8flhWkbSiCLX8vun11eiGIpiJ/z2nYdx5HqLdVV7elrOzsuqysL3rmBIGiKPizKCHHWY4PLVeQbnXAdegqdhy+hu8dDTBnbqQJZJ1A7u+vz7RaiymWCZgCRSF6Edk8b9cx+B/W6WuVxPaZnyiqXoPpyUmVYvkKTIFClHigEieKjYuSvETUllaF4GAUM1NT6ooaJDKx+aDfC9fByxj90REb+9ppmIoAscH/6leg8MS9DJXPAM9xHCM443K57C6biMjcHDaVVCHw9RmCA2/RGC5C00AqXk/m4p20HZK4CM/J3Zk9n0ecMBhDQnJHcrTisyMfdQXOilrdMfxcwoHq/fg5R59TiQV3hYGKo6X2J/c7LyQIjOx9GXhOw/zoJ8wEevRGyp53o/lGMNYsBgPtEwLecwov7/jGDKa1twT6o3KpL4MdZgGsWZLtfPr7f1q58k1JNHy7YYaM+J+K3Y2PmAIbRavX66229hrGVvvL5uzsHDEUvUu+NT1my78CDAAMK1a8/QaZCgAAAABJRU5ErkJggg==);
	width: 16px;
	height: 16px;
	background-repeat: no-repeat;
}

Ok, onto the tests!

Test #1: Five 16×16 icons from the Fugue Icon set (PNG)

File Binary Binary Gzipped CSS + Base64 CSS + Base64 Gzipped
abacus 1443 1179 2043 1395
acorn 1770 1522 2478 1728
address-book–arrow 763 810 1153 948
address-book–exclamation 795 848 1199 988
address-book–minus 734 781 1113 919
Total 5,505 5,140 7,986 5,978
Combined file (5,505) (4,128) 7,986 4,423

* All numbers are byte sizes
** Numbers in parenthesis represent actual but impractical data. Unfortunately, images cannot be combined and delivered together in their binary form.

Takeaways:

  • The binaries are always smaller.
  • Sometimes Gzipping makes the files larger.
  • Gzipping the base64 version brings the filesize close to the size of the original binary, but this ignores the fact that the binaries get Gzipped as well. The Gzipped binaries (how they would be delivered to the client) are always smaller than the Gzipped base64 images
  • Combining files together dramatically reduces filesizes.

Practically, the developer has two options: deliver 5,140 bytes to the user in 5 separate HTTP requests, or 4,423 bytes in one HTTP request (CSS with base64 encoded image data). Base64 is the clear winner here, and seems to confirm that small icons compress extremely well.

Test #2: Five Flickr 75×75 Pictures (JPEG)

File Binary Binary Gzipped CSS + Base64 CSS + Base64 Gzipped
1 6734 5557 9095 6010
2 5379 4417 7287 4781
3 25626 18387 34283 20103
4 7031 6399 9491 6702
5 5847 4655 7911 5077
Total 50,617 39,415 68,067 42,673
Combined file (50,617) (36,838) 68,067 40,312

Takeaways:

  • (some of the same takeaways as Test #1)
  • Separately, photos aren’t too much bigger when base64 encoded and Gzipped. It’s very much within reason.

Practically, the developer can deliver 39,415 bytes in 5 separate requests, or 40,312 in 1 request. Not much filesize difference here, but 1 request seems preferable when we’re talking about 40kb.

Test #3: Five Flickr 240×160 Pictures (JPEG)

File Binary Binary Gzipped CSS + Base64 CSS + Base64 Gzipped
1 24502 23403 32789 23982
2 20410 19466 27333 19954
3 43833 36729 58561 38539
4 31776 31180 42485 31686
5 21348 20208 28581 20761
Total 141,869 130,986 189,749 134,922
Combined file (141,869) (129,307) 189,749 133,615

Takeaways:

  • (some of the same takeaways as Test #1)
  • Larger photos seem to bring the Gzipped binary and Gzipped base64 filesizes MUCH closer together, making the difference very minimal

The developer must choose between delivering 130,986 bytes in 5 HTTP requests, or 133,615 bytes in one HTTP request. Any good Souders follower would opt for the one request, BUT I would be careful here…

Caution: things aren’t always as they seem

There’s a huge caveat here: it may actually be more beneficial for perceived performance to deliver the images in 5 separate requests.

Why? Because 133,615 bytes is a lot to deliver all in one package to an end user who will be staring at blank placeholders for the duration. If the 5 base64 images all come in one request, that request will have to complete before ANYTHING is shown on the screen. All 5 images go from blank placeholders to almost immediately decoded from base64 and displayed in place.

Compare this with 5 requests that are most likely made in parallel and actually give a visual indicator to the user that actual image content is being downloaded, by showing parts of the images as they’re downloaded (you can also try a throwback to progressive JPEGs – really anything will be better than just a blank screen). That’s why it might actually be beneficial for perceived performance to just load images in the good old fashioned way. They will most likely load in parallel anyway, so the extra HTTP requests may actually not really make a difference. Not to mention it will be easier to let the browser manage the cache for each file instead of having to make your JavaScript manage your cache and prevent you from downloading an image that’s already stored away in localStorage or sessionStorage.

This being said, it’s generally advisable to put your common UI icons in base64 in your CSS, then let that whole chunk get cached by the browser. Those are usually clean vector icons as well, which seem to get compressed quite well (see Test #1).

But for image content, where there is nothing to be saved but HTTP requests, you should definitely think twice about base64 encoding to save requests. Yes, you will save a few HTTP requests, you won’t really be saving bytes, and the user might actually think the experience is slower because they can’t see the image content as it’s being downloaded. Even if you shave off a few milliseconds of wait time, the perceived performance is what matters most.

(EDIT: changed the wording of the “unverified intuitions” section from “not verified” to actual questions, to make it clearer)

JavaScript SunSpider benchmark: iOS 4.3 vs iOS 4.0

Saturday, January 22nd, 2011

Result table (SunSpider 0.9.1)

Test iOS 4 (3GS) iOS 4.3 beta 2 (3GS) % change
Total 13787ms 5357ms -61.1%
3D 1917ms 737ms -61.6%
Access 1893ms 617ms -67.4%
Bitops 1239ms 289ms -76.7%
Controlflow 221ms 35ms -84.2%
Crypto 850ms 308ms -63.8%
Date 1065ms 706ms -33.7%
Math 1511ms 606ms -59.9%
Regexp 1916ms 153ms -92.0%
String 3175ms 1907ms -39.9%

Raw results (iOS 4.3 beta 2 running on my iPhone 3GS)

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                  5357.3ms +/- 2.3%
--------------------------------------------

  3d:                    736.9ms +/- 7.5%
    cube:                260.1ms +/- 3.5%
    morph:               206.0ms +/- 23.2%
    raytrace:            270.8ms +/- 4.4%

  access:                617.4ms +/- 4.1%
    binary-trees:        118.8ms +/- 4.7%
    fannkuch:            180.4ms +/- 0.7%
    nbody:               222.0ms +/- 11.7%
    nsieve:               96.2ms +/- 0.8%

  bitops:                288.7ms +/- 9.6%
    3bit-bits-in-byte:    38.0ms +/- 4.3%
    bits-in-byte:         70.5ms +/- 39.0%
    bitwise-and:          85.1ms +/- 1.9%
    nsieve-bits:          95.1ms +/- 2.1%

  controlflow:            34.7ms +/- 3.2%
    recursive:            34.7ms +/- 3.2%

  crypto:                308.3ms +/- 1.1%
    aes:                 175.2ms +/- 0.5%
    md5:                  79.3ms +/- 3.9%
    sha1:                 53.8ms +/- 0.8%

  date:                  706.2ms +/- 5.5%
    format-tofte:        341.3ms +/- 11.0%
    format-xparb:        364.9ms +/- 1.1%

  math:                  605.9ms +/- 1.1%
    cordic:              224.5ms +/- 1.0%
    partial-sums:        263.4ms +/- 2.3%
    spectral-norm:       118.0ms +/- 1.2%

  regexp:                152.5ms +/- 5.5%
    dna:                 152.5ms +/- 5.5%

  string:               1906.7ms +/- 0.7%
    base64:              247.9ms +/- 2.4%
    fasta:               268.5ms +/- 0.6%
    tagcloud:            370.4ms +/- 3.0%
    unpack-code:         676.2ms +/- 0.3%
    validate-input:      343.7ms +/- 1.4%

Raw results for iOS 4.0

Related

BlackBerry Torch SunSpider results (JavaScript benchmark)
iPhone 4 SunSpider test results (22% faster than iPhone 3GS)
JavaScript SunSpider test: iOS 3.1.3 versus iOS 4 GM
JavaScript SunSpider: HTC Evo versus HTC Incredible versus Nexus One

BlackBerry Torch SunSpider results (JavaScript benchmark)

Wednesday, August 18th, 2010

Results

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                 322.2ms +/- 4.9%
--------------------------------------------

  3d:                   55.0ms +/- 15.3%
    cube:               19.8ms +/- 12.1%
    morph:              16.6ms +/- 35.8%
    raytrace:           18.6ms +/- 22.5%

  access:               32.8ms +/- 12.4%
    binary-trees:        1.8ms +/- 30.9%
    fannkuch:           14.2ms +/- 7.3%
    nbody:              12.6ms +/- 26.7%
    nsieve:              4.2ms +/- 24.8%

  bitops:               29.4ms +/- 10.2%
    3bit-bits-in-byte:   2.4ms +/- 28.4%
    bits-in-byte:        8.0ms +/- 15.5%
    bitwise-and:         8.6ms +/- 21.9%
    nsieve-bits:        10.4ms +/- 21.7%

  controlflow:           2.4ms +/- 28.4%
    recursive:           2.4ms +/- 28.4%

  crypto:               22.0ms +/- 10.6%
    aes:                 9.8ms +/- 27.5%
    md5:                 6.2ms +/- 22.0%
    sha1:                6.0ms +/- 20.7%

  date:                 33.2ms +/- 11.7%
    format-tofte:       16.6ms +/- 20.2%
    format-xparb:       16.6ms +/- 13.6%

  math:                 32.6ms +/- 15.2%
    cordic:             12.0ms +/- 29.3%
    partial-sums:       15.2ms +/- 20.4%
    spectral-norm:       5.4ms +/- 20.6%

  regexp:               15.6ms +/- 7.1%
    dna:                15.6ms +/- 7.1%

  string:               99.2ms +/- 14.0%
    base64:              9.2ms +/- 14.8%
    fasta:              14.4ms +/- 27.7%
    tagcloud:           27.2ms +/- 23.2%
    unpack-code:        31.8ms +/- 14.5%
    validate-input:     16.6ms +/- 18.0%

Related articles

iPhone 4 SunSpider test results (22% faster than iPhone 3GS)
JavaScript SunSpider test: iOS 3.1.3 versus iOS 4 GM
JavaScript SunSpider: HTC Evo versus HTC Incredible versus Nexus One
Sencha: BlackBerry Torch: The HTML5 Developer Scorecard