Archive for October, 2011

Web development / Silicon Valley rant

Saturday, October 22nd, 2011

(if you didn’t read the title – warning: rant…)

As far as web development goes, I guess I could rant on and on about the generalities. Of all things ephemeral. The lust after money, gadgets, and other worthless material crap that we use to stuff inside our oversized houses.

The startling creativity and in equal amounts the lack of creativity. The amount of amazing invention and the equal amount of copying and ripping off.

I could complain of amazing shortsightedness, of fads, and of constant new waves of developers that have to relearn and be convinced of old tried-and-true best practices (while still being able to identify the outdated best practices).

I could possibly ponder some of the stupid things we concern ourselves with, which are criticisms more aimed at the Silicon Valley mentality in general. The sheer amount of idiotic startups. The desire to have a startup only for the payout, for the acquisition. The one worthwhile and meaningful startup surrounded in a sea of hangers-ons, social network clones, photo sharing service clones, Facebook game clones, and on and on ad nauseum.

I could complain about the few passionate ones, surrounded by a pool of passionless jumpers-on-the-bandwagon, people looking for a free ride, and people looking to coast through their secure jobs, content with nothing more than receiving their paycheck.

I could complain about the lack of real heroes. People who made a difference to our small isolated communities, spoke at conferences, authored books, wrote popular blog articles, but were perhaps never themselves. People who we could not look up to as model human beings, but as model toolmakers and nothing more. I suppose it’s folly to look for too much else there anyhow.

I could complain about the praise of life imbalance. Of prideful people showing off, working longer hours than their peers, feeling like they have to in order to keep up, and bragging about losing sleep and becoming unbalanced and unhinged individuals. Competition is good and pushes everyone to be more detailed and more polished, but there seems to be no admission of a limit, of reaching a life imbalance. The more time spent coding, the better. If you don’t code in your free time, you must not be a true coder. If you don’t enjoy learning new languages, you must be a crappy coder. The amount of pride and self-delusion here is scary.

I could talk about corporations seeking young programmers without commitments in order to suck dry every last ounce of their passion, essence, and thirst, distracting their focus away from all things important. Of those same corporations sticking those young impressionables onto outdated projects, maintaining and fixing bugs. All for what?

I could speak of self-delusion that programming language X will save the world, and on self-delusion in general, which I believe is rampant here and elsewhere, and is possibly our greatest weakness as human beings.

I suppose now you’ll tell me I’ve lost my head and gone crazy, but I would disagree. These things have never been so clear and the arguments have never been so cogent as they are now. There is no utopia, no perfect occupation, but I believe there are some that are better than others. Everything has its faults, and web development isn’t for everyone. Perhaps it’s simply not for me?

I got into web development as a means to an end – as a way of publishing what I enjoyed doing. As a tool I picked up and put down when I was finished, and then got on with my life. Perhaps it’s time to move it back to that corner of my life, and even better, to use it as a tool to help me build new, meaningful, less ephemeral things. Something I could be proud of in more than a few years. Something more than being an “HTML5 expert” or “Mobile expert” or “JavaScript expert”.

No. I want to use these things as tools that are put back in their shed after I’m done. Tools to build something greater than the sum of its parts. Tools to make a meaningful contribution that may not amount to a hill a beans in this crazy world, but could at least be something I’m proud of.

Programming as a profession is only moderately interesting. It can be a good job, but you could make about the same money and be happier running a fast food joint. You’re much better off using code as your secret weapon in another profession.

People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines.

(Advice From an Old Programmer)

Somewhat related: Where Do All of the Old Programmers Go?, The ACL is Dead, Silicon Valley’s Dark Secret: It’s All About Age [and the disturbing reddit thread to accompany that])

Addendum

It’s notable to add that I’ve been comparing “Kyoto culture” to “Silicon Valley culture”. They are so dissimilar that it’s doubtful we’re even using “culture” in the same sense in both cases. Kyoto culture, a long proud, rich, colorful history of a people, and Silicon Valley culture: an atmosphere of tech, founded by money and driven by money. In one, it’s almost a crime to stay inside and not to explore and enjoy the city. In the other, our wealth lets us build up a castle, buy devices and home entertainment to keep us busy in our little fiefdom. Silicon Valley is more like a boomtown which exists only for one sick purpose, and consequentially lacks a soul even more severely. Would you rather be underpaid in a city with a soul or overpaid in a soulless city?

iOS5 SunSpider: iPhone 4S vs iPhone 4 vs iPhone 3GS

Friday, October 14th, 2011

Result table (SunSpider 0.9.1, all devices running iOS 5)

Test iPhone 4S iPhone 4 iPhone 3GS
Total 2270ms 3483ms 4903ms
3D 281ms 486ms 672ms
Access 279ms 410ms 614ms
Bitops 177ms 200ms 270ms
Controlflow 20ms 26ms 34ms
Crypto 164ms 211ms 315ms
Date 332ms 528ms 770ms
Math 219ms 433ms 553ms
Regexp 88ms 117ms 151ms
String 711ms 1072ms 1525ms

Raw results

iPhone 4S (iOS5)

iPhone 4 (iOS5)

iPhone 3GS (iOS5)

Bonus: FishIETank (10 fish) (Canvas test)

Test iPhone 4S iPhone 4 iPhone 3GS
Total 50fps 35fps 25fps

One thing to note here: the 3GS has a bit of an advantage because of a non-retina screen. However, it’s still outperformed easily by the iPhone 4 and 4S.

Related

JavaScript SunSpider benchmark: iOS 4.3 vs iOS 4.0
BlackBerry Torch SunSpider results (JavaScript benchmark)
iPhone 4 SunSpider test results (22% faster than iPhone 3GS)
JavaScript SunSpider test: iOS 3.1.3 versus iOS 4 GM
JavaScript SunSpider: HTC Evo versus HTC Incredible versus Nexus One

iPhone 4S, iPhone 4, and iPhone 3GS photo comparison

Friday, October 14th, 2011

More photos likely to come.. this is just after I unboxed my new 4S. Time to play with it a bit!

iPhone 4S

Canon 7D, shot with an iPhone 4S

iPhone 4

Canon 7D, shot with an iPhone 4

iPhone 3GS

Canon 7D, shot with an iPhone 3GS

iPhone 3GS (alternate white balance point)

Canon 7D, shot with an iPhone 3GS

Mobile Performance Manifesto

Tuesday, October 11th, 2011

Earlier this year I gave a talk (slides) outlining the latest and greatest in mobile performance, including a bit of my own unscientific research into carrier latency and bandwidth thanks to boomerang.js.

I realized that interest in mobile performance has exploded recently, especially with Steve Souders announcing his focus on mobile, and I thought it was time for an update, this time in blog form. Also, my old slides have been somewhat embarrassing. For some strange reason, at the time I wanted to give S5 a try – that outdated, ancient, not-performant slideshow framework. The result is a slideshow on performance that loads slowly… doh! (incidentally, I recommend deck.js as an alternative).

In any case, it was time for a roundup of mobile performance best practices, in blog form. I’m not sure if it’s properly called a manifesto, but it is what it is! Onward!

For fun: Latency and bandwidth tests

Before we start.. just a little fun data! It’s always been a pleasure to fiddle around with boomerang.js and compare results. With my latest trip to Japan, I had the opportunity to run Boomerang on Japan’s e-mobile 3G network in the remotest of places, up in Hakuba/Nagano in the Japanese alps.

When I got back, I ran the same tests from downtown San Francisco, which was closer to my California-based test server, and should’ve been faster, right? Theoretically…

Boomerang Tests (running on a California-hosted server)

Verizon 3G (SF) ATT 3G (SF) e-mobile 3G (Hakuba)
Latency 251ms 901ms 401ms
Bandwidth 62kbps 3kbps 95kbps

Yes, e-mobile didn’t have the greatest ping, but it handily beat ATT, even though it was going across the entire Pacific Ocean! Note that it came out the best in the bandwidth tests however…

(Note: it’s unknown if or how Steve Souders’s latest research affects these findings)

There’s a lot of factors that could’ve been involved, so look to something like OpenSignalMaps for more data.

Ok, onto the tips!

Page Organization

For feature phones that have little to no caching, aggressively combine requests (deliver HTML/CSS/JS all in one package). For smart phones, take advantage of caching by mirroring desktop frontend best practices: separate HTML, CSS, and JS so they can be cached (per-session and across sessions).

There’s old research about extremely small cache sizes on iOS in particular, but this research has been followed up on by more recent research by Ryan Grove (Yahoo!) and Steve Souders (Google), which shows that we shouldn’t be so paranoid, since caching is pretty decent across all the major mobile browsers.

Of particular interest is a browser’s capability to cache files in a current session, browsing from page-to-page, which is what most users will end up doing (caching of the page across sessions is another matter). What is the maximum file size a browser will cache during a session? The results (via Browserscope) end up being encouraging:

Maximum Cache Size (MB)
Android 2.1 4+
Android 2.2 2
Android 2.3 2
Android 3.0 4+
Blackberry 6.0.0 4+
iPad 4.3.5 4+
iPad 5.0 4+
iPhone 4.3.5 4+
iPhone 5.0 4+
Opera Mini 4 4+
Opera Mini 5 4+
Opera Mini 6 4+
webOS 2.0 1

Avoid redirects (foo.com -> m.foo.com)

If possible, perform the redirection behind the scenes on the server, which should be transparent to the user. When a user performs a Google search and clicks on your page, they’re already getting redirected once by Google (check it yourself). Your own redirects are adding a second redirect where there need not be one.

Optimize Images


Deliver appropriately-sized images to devices. The philosophy of responsive design makes it easy to simply downscale images to fit the screen, but avoid this where possible, as this means wasted bandwidth.

You can optimize images through CSS media queries or in JavaScript (see below). Though you want to reduce your dependence on cookies, it may be a good idea to store these width/height values into a cookie (or localStorage if you are fetching images in nontraditional ways) so the values can be read by the server, which can deliver appropriately-sized images. This technique has been implemented by Filament Group, so you should probably read about their experiences before trying to roll your own.

Also note that where applicable, at the expense of performance you may want to serve higher resolution images for better screens, such as for Retina displays (which can be detected with JavaScript or CSS).

JavaScript examples

window.innerHeight;       // max height actually available
window.innerWidth;        // max width actually available
window.devicePixelRatio;  // pixel density (standard is 1, high resolution is generally > 1)

(note that screen.width and screen.height are also available to tell you the dimensions of the entire screen, but this isn’t all available due to the space taken by the OS and browser chrome)

Media query examples

@media only screen and (max-width: 480px) {
  /* small screen styles */
}

@media only screen and (min-width: 481px) {
  /* large screen styles */
}

@media (-webkit-min-device-pixel-ratio: 1.5),
       (-o-min-device-pixel-ratio: 3/2),
       (min--moz-device-pixel-ratio: 1.5),
       (min-device-pixel-ratio: 1.5) {
  /* high resolution styles */
}

navigator.connection (Android only)

Use navigator.connection if it’s available to serve different assets based on connection speed (3G vs WIFI, for instance).

Here’s the contents of the navigator.connection object (with a phone running on a 3G connection):

{
  type: 4,
  UNKNOWN: 0,
  ETHERNET: 1,
  WIFI: 2,
  CELL_2G: 3,
  CELL_3G: 4
}

And an example of how you could write code for each type:

(function(){  // sandbox our code
  if(!navigator.connection) navigator.connection = {type:0, UNKNOWN: 0};  // polyfill
  var connection = navigator.connection;

  if(connection.type === connection.WIFI || connection.type === connection.ETHERNET) {
    // high bandwidth
  } else {
    // normal bandwidth
  }
})();

Here’s how we might be able to use Modernizr to help us out:

(function(){  // sandbox our code
  if(!navigator.connection) navigator.connection = {type:0, UNKNOWN: 0};  // polyfill
  var connection = navigator.connection;

  // add a custom test to Modernizr
  Modernizr.addTest('highbandwidth', function(){
    return connection.type === connection.WIFI || connection.type === connection.ETHERNET;
  });

})();

We can now target special high-bandwidth assets with CSS:

.highbandwidth .logo {background-image:url('logo-high.jpg');}
.no-highbandwidth .logo {background-image:url('logo.jpg');}

base64 encode small UI images

You can base64 encode binary image data in HTML and CSS:

HTML

<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==" alt="Red dot">

CSS

.dot {
  background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==');
}

There seems to be the mistaken belief floating around that base64 images aren’t cacheable. However, because you can embed them in your CSS, base64 encoded images will be cached along with the rest of your CSS.

You’ll probably just want to limit this to small UI icons however. I wrote a quick post on the drawbacks, and also a comparison with sprites.

If you’re using Compass/SASS, base64 encoding images is pretty trivial and easy to maintain with the inline-image helper.

Unicode and Emoji

☆ (HTML entity: #9734)
(HTML entity: #x1f468)

Before even considering base64 images, take advantage of icons that are already made for you: unicode characters and emoji (where supported). Keep in mind that these will look different across browsers, so this might not be an option for everyone.

Because of inconsistent implementations of Emoji character codes between Asian carriers, representatives from Apple and Google made a proposal to add Emoji to the Unicode standard. Unfortunately it seems that only iPhone supports them for now. Here’s a handy chart of all the Unicode emoji characters available.

Take advantage of CSS3

CSS3 offers many replacements for things we needed images for previously. RGBA values replace the need for a semitransparent image for overlays and such. Likewise, border-radius, box shadow, linear-gradients, radial-gradients all reduce the need for images.

However, do be aware that though phones support these new features, it doesn’t mean they’re necessarily ready for primetime. Something I’ve encountered recently to remind me of this fact is severe color banding issues when using a CSS radial gradient. The solution was to fake the radial gradient using two linear gradients, or to (unfortunately) use an old-fashioned image.

Avoid using cookies

Cookies get thrown into every request on a per-domain basis, so limit your usage of them. Use localStorage/sessionStorage instead, where possible.

App cache

Take advantage of the HTML5 app cache, though it’s an unwieldy beast to tame. This will mean you will need to make your site work offline, which might be tricky.

Assets that are traditionally cached get rechecked on page refresh (the server sends back a 304 response if your cache is still up-to-date). Whereas with the app cache, only one file (your manifest) is rechecked on page load.

Deferred JavaScript execution

We know that deferring downloading of scripts is advantageous (by placing scripts at the bottom of a page or by using the async attribute), but deferred execution of JavaScript is even more important. Sure, we can use the defer attribute, but that’s only really relevant for when the page is loading (defer lets the browser know that the UI doesn’t depend on the JavaScript, so it can be safely deferred).

But what about JavaScript that runs after the page loads, such as XHR and JSONP requests? What does this mean for the user? It means that the UI freezes up unexpectedly when JavaScript is being downloaded and executed in the background. But this doesn’t mean that you want to completely avoid background downloading of JavaScript. The Gmail mobile team came up with a clever solution: by commenting all of their code and dynamically eval’ing it when needed, they split up the JavaScript downloading from its execution.

Perceived performance

Do whatever you have to do to let the user know that the UI is still responding. This sometimes means faking that something is happening. Communication is key! If they clicked on a button, give some indication that they clicked on it. If they clicked on something that requires a request to the network, show a spinner right away, even if it means you haven’t even sent out a request. The user doesn’t need to know the nitty-gritty details – they just want to know that their intent was communicated.

Onclick delay

On several major mobile operating systems there’s a several hundred millisecond delay on the onclick event. This is because of the double-tap-to-zoom functionality. When a user first taps on the screen, there’s a hard-coded delay that waits for the second tap. If there’s no second tap, the onclick event is then fired. Unfortunately the only way to get around the delay is to tap into touch events instead, which is a bit more complicated than it appears on first glance.

For an overview of some of the challenges of implementing the workaround, see this article by Google’s Ryan Fioravanti.

Take advantage of hardware acceleration

Use hardware-accelerated CSS transforms where possible (translate3d, translateZ, rotate3d, and scale3d). An element that is hardware accelerated is turned into a graphic, which is perfect for the GPU to manipulate, taking away the burden from the CPU.

However, the GPU isn’t all-powerful, so don’t try to apply hardware accelerations to everything. Also, these elements still need to be refreshed periodically, and it turns out that you can make some good optimizations here. First, you’ll want to debug the composited layers on your desktop browser:

Debugging hardware acceleration


Chrome

  1. Type the following in the address bar: about:flags
  2. “Composited render layer borders” -> Enable

Safari

  1. Open a terminal
  2. $ defaults write com.apple.Safari IncludeInternalDebugMenu 1
  3. $ defaults write com.apple.Safari CA_COLOR_OPAQUE 1
  4. Open (or restart) Safari
  5. Debug -> Show Compositing Borders

(to turn these off, run the same commands with a boolean FALSE: i.e. defaults write com.apple.Safari CA_COLOR_OPAQUE FALSE)

Keep in mind that each composited layer has a limited width and height. For instance, if you’re creating an image carousel, chances are the dimensions of the element will be too big to fit the layer into memory as one piece. This means that when the element is animated, the GPU has to break up the layer manually into several manageable chunks. It’s much better to chunk it yourself. To do this, you trigger hardware acceleration on each chunk.

So you will change this:

.carousel {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

To this:

.carousel {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

.carousel-pane {
  -webkit-transform: translate3d(0,0,0);  /* or translateZ(0); */
}

Where carousel-pane represents each child element of the carousel.

More info: (slides) WebKit in Your Living Room (Matt Seeley, Netflix)

Also see HTML5 Techniques for Optimizing Mobile Performance.

HTTP Pipelining

HTTP Pipelining diagram

Take advantage of HTTP Pipelining, which is often overlooked, but has broad support on mobile. This virtually eliminates round trip times, for all but the first request.

Opera and Android support pipelining, and the newly released iOS 5 has added support for it.

Great! So how do you make sure your server is taking advantage of pipelining?

The first request to every server is sent by itself (only one request on the connection), and the browser looks for two properties in the response:

  1. Use of HTTP/1.1
  2. An explicit “Connection: Keep-Alive” header (required by Android)

-Blaze.io

If these criteria are met, subsequent requests will be pipelined. Sweet!

DNS Prefetching

Take advantage of DNS Prefetching. This theoretically speeds up load times, but there have been some issues, so be sure to test it.

To turn off DNS Prefetching, serve this meta tag:

<meta http-equiv="x-dns-prefetch-control" content="off">

You can also explicitly force a DNS lookup:

<link rel="dns-prefetch" href="http://www.example.com/">

Avoid library code bloat

If you’re developing for smart phones, you may be able to dramatically reduce the data over the wire by simply using new JavaScript APIs instead of a full-blown library or framework such as jQuery Mobile (which is rather a UI framework and depends on the desktop version of jQuery).

Among the things offered by newer browsers, which should reduce your dependence on a library:

  • querySelector/querySelectorAll: CSS selectors that replace the need for a selector engine
  • getElementsByClassName
  • classList – replaces the need for hasClass, addClass, removeClass helpers (available on iOS5, but not sure what else)
  • XMLHttpRequest – it’s probably time to learn how to do this natively instead of using a wrapper that takes care of IE’s old implementation that requests an ActiveX object. Cross-Origin Resource Sharing also means we can easily share resources across domains.
  • HTML5 history (pushState, etc) – though an older bug means you should maybe hold off for now (note: the fix is present in iOS 5, which is based on a newer WebKit)
  • HTML5 Form Validation reduces our need for JavaScript-based validation
  • HTML5 input types that reduce our need for custom UI controls (date, time, slider [range], etc)

You’ll want to consider using either a library that was optimized for the browsers you’re targeting (such as Zepto.js) or simply bring in a microlibrary for specific tasks you want to perform.

Clientside databases

Most mobile browsers currently support WebSQL, which is being phased out in support of indexedDB, which is not widely implemented. However, if you use a wrapper such as Lawnchair, the transition from one to the other is relatively painless.

TODO: testing

More

(to be expanded on later)

  • use standard desktop best practices (for smart phones)
  • requestAnimationFrame instead of setTimeout
  • setImmediate (where available) instead of setTimeout(fn(){},0)
  • Ajax parsing times: you might want to use Multipart XHR so you can yield the process as it’s going through big responses (to prevent the UI locking up)
  • avoid rgba, box shadows, text shadow, etc, as this greatly degrades performance (especially on animations)
  • high resolution screens may suffer from animation/transition slowness – it’s been shown that manipulating the viewport tag to scale the page will speed up canvas rendering, and high-dpi scaling concerns are also evident in CSS transforms
  • WebKit animations are faster than animations done via JavaScript
  • if you do use JavaScript-based animations, limit the UI updates to ~17ms, which is equivalent to 60fps, which is the fastest your display will refresh anyway, so it’s pointless to try to do anything faster than that. Keep track of the time since the last UI update and don’t do anything if 17ms haven’t passed.

On the horizon…

Tools

Mobile Perf bookmarklet
pcapperf (Packet Capture Web Performance Analyzer)
Blaze Mobitest – tests the loading time of your site on actual phones

More Resources

Mobile Web Application Best Practices
Mobile Performance (Steve Souders)
Strangeloop Web Performance Hub / Mobile

(odometer image via henrybloomfield on Flickr)