12 Comments

When to base64 encode images (and when not to)

August 28th, 2011

Introduction

Ever since Steve Souders started evangelizing web performance, it’s been pounded into our heads that extra HTTP requests add a lot of additional overhead, and that we should combine them if possible to dramatically decrease the load time of our web pages.

The practical implication of this has been to combine our JavaScript and CSS files, which is relatively easy and straightforward, but the harder question has been what to do with images.

Sprites

Image sprites are a concept taken from video games: the idea is to cram a ton of image assets into one file, and rearrange a “viewport” of sorts to view only specific pieces of that file at a time. For instance, a simple sprite that holds two images might have one viewport that only looks at the top half of the sprite (image #1), and another viewport that only looks at the bottom half (image #2).

On the web side of things, this means that those multiple requests have now been combined into one request. This is nice because it saves both the overhead of additional HTTP requests as well as the overhead of setting up an image’s file header each time.

But there’s a few drawbacks with using image sprites:

  • hard to maintain and update: without some tool to help, manually editing and putting together image sprites is quite a chore
  • increased memory consumption (possibly very dramatic): this is often overlooked. The time to deliver the images is decreased at the expense of a bigger memory and CPU footprint, especially for large sprites and sprites with a lot of whitespace.
  • bleedthrough: for sprites that don’t have much whitespace to separate images, there’s an increased chance of nearby images visibly bleeding through other elements

Data URIs and Base64 encoding

Data URIs (see this, this, and this) and Base64 encoding goes hand-in-hand. This method allows you to embed images right in your HTML, CSS, or JavaScript.

Just like sprites, you save HTTP requests, but there’s also some drawbacks:

  • base64 encoding makes file sizes roughly 33% larger than their original binary representations, which means more data down the wire (this might be exceptionally painful on mobile networks)
  • data URIs aren’t supported on IE6 or IE7
  • base64 encoded data may possibly take longer to process than binary data (anyone want to do a study on this?) (again, this might be exceptionally painful for mobile devices, which have more limited CPU and memory) (side note: CSS background-images seem to actually be faster than img tags)

The “33% larger” claim is generally accepted truth now, despite the fact that the figure varies wildly depending on the type of content. This is exactly what I wanted to test, albeit in a pretty limited and nonscientific way.

Before I tested, I wanted to keep in mind a few unverified intuitions (which aren’t entirely my own, but seem to be ideas that are floating around out there). Here’s a few questions I had before going to test:

  • Is base64 encoding with gzipping roughly equal to the original filesize of the binary file?
  • Is base64 encoding best for small images?
  • Is base64 encoding best for small and simple icons and not good for pictures and photos?
  • Is base64 encoding best when multiple files are merged together?

There’s something else I wanted to test: whether Gzipping binary image data made much difference. I know text compresses well, but is it even worth compressing JPEG files with Gzip, for instance?

I ran three tests: one with a set of small UI icons, one with a set of small photographs, and one with a set of the same photographs in a larger size. Though my tests were by no means extensive, they do show that care should be taken in making assumptions about base64.

Just a note about the tables: they are comparing the binary form (original png or jpeg) with the base64 form as it would appear in a CSS stylesheet, and comparing each of those with their gzipped form, which is most likely how they would be sent down the wire. The CSS representation has a few practical declarations and looks something like this:

.address-book--arrow {
	background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAp1JREFUeNqEU21IU1EYfu7unW5Ty6aBszYs6MeUjGVYokHYyH5E1B9rZWFEFPQnAwmy6Hc/oqhfJsRKSSZGH1JIIX3MNCsqLTD9o1Oj6ebnnDfvvefezrnbdCHhCw/n433P8z7nPe/hBEEAtX0U7hc164uwuvVSXKwZLoOmaRDim+7m9vZa0WiEKSUFFpNpCWlmMyypqTDRuYn6t3k8vmQ2gRDCxs0t9fW45F52aBTROJLtZl7nEZad2m+KtoQCQ0FBARyOCGRZ/q92I1WgqqXlfdd95VsrK8/pChIEqqpCkiQsiCII0aBQZZoWl8lzFDwsFjMl0DBLY8Lj41hBwK4jSQrWOIphL6xYyhwJDWGo6wFSaH1Y3PTCAsITE1oyAa8flhWkbSiCLX8vun11eiGIpiJ/z2nYdx5HqLdVV7elrOzsuqysL3rmBIGiKPizKCHHWY4PLVeQbnXAdegqdhy+hu8dDTBnbqQJZJ1A7u+vz7RaiymWCZgCRSF6Edk8b9cx+B/W6WuVxPaZnyiqXoPpyUmVYvkKTIFClHigEieKjYuSvETUllaF4GAUM1NT6ooaJDKx+aDfC9fByxj90REb+9ppmIoAscH/6leg8MS9DJXPAM9xHCM443K57C6biMjcHDaVVCHw9RmCA2/RGC5C00AqXk/m4p20HZK4CM/J3Zk9n0ecMBhDQnJHcrTisyMfdQXOilrdMfxcwoHq/fg5R59TiQV3hYGKo6X2J/c7LyQIjOx9GXhOw/zoJ8wEevRGyp53o/lGMNYsBgPtEwLecwov7/jGDKa1twT6o3KpL4MdZgGsWZLtfPr7f1q58k1JNHy7YYaM+J+K3Y2PmAIbRavX66229hrGVvvL5uzsHDEUvUu+NT1my78CDAAMK1a8/QaZCgAAAABJRU5ErkJggg==);
	width: 16px;
	height: 16px;
	background-repeat: no-repeat;
}

Ok, onto the tests!

Test #1: Five 16×16 icons from the Fugue Icon set (PNG)

File Binary Binary Gzipped CSS + Base64 CSS + Base64 Gzipped
abacus 1443 1179 2043 1395
acorn 1770 1522 2478 1728
address-book–arrow 763 810 1153 948
address-book–exclamation 795 848 1199 988
address-book–minus 734 781 1113 919
Total 5,505 5,140 7,986 5,978
Combined file (5,505) (4,128) 7,986 4,423

* All numbers are byte sizes
** Numbers in parenthesis represent actual but impractical data. Unfortunately, images cannot be combined and delivered together in their binary form.

Takeaways:

  • The binaries are always smaller.
  • Sometimes Gzipping makes the files larger.
  • Gzipping the base64 version brings the filesize close to the size of the original binary, but this ignores the fact that the binaries get Gzipped as well. The Gzipped binaries (how they would be delivered to the client) are always smaller than the Gzipped base64 images
  • Combining files together dramatically reduces filesizes.

Practically, the developer has two options: deliver 5,140 bytes to the user in 5 separate HTTP requests, or 4,423 bytes in one HTTP request (CSS with base64 encoded image data). Base64 is the clear winner here, and seems to confirm that small icons compress extremely well.

Test #2: Five Flickr 75×75 Pictures (JPEG)

File Binary Binary Gzipped CSS + Base64 CSS + Base64 Gzipped
1 6734 5557 9095 6010
2 5379 4417 7287 4781
3 25626 18387 34283 20103
4 7031 6399 9491 6702
5 5847 4655 7911 5077
Total 50,617 39,415 68,067 42,673
Combined file (50,617) (36,838) 68,067 40,312

Takeaways:

  • (some of the same takeaways as Test #1)
  • Separately, photos aren’t too much bigger when base64 encoded and Gzipped. It’s very much within reason.

Practically, the developer can deliver 39,415 bytes in 5 separate requests, or 40,312 in 1 request. Not much filesize difference here, but 1 request seems preferable when we’re talking about 40kb.

Test #3: Five Flickr 240×160 Pictures (JPEG)

File Binary Binary Gzipped CSS + Base64 CSS + Base64 Gzipped
1 24502 23403 32789 23982
2 20410 19466 27333 19954
3 43833 36729 58561 38539
4 31776 31180 42485 31686
5 21348 20208 28581 20761
Total 141,869 130,986 189,749 134,922
Combined file (141,869) (129,307) 189,749 133,615

Takeaways:

  • (some of the same takeaways as Test #1)
  • Larger photos seem to bring the Gzipped binary and Gzipped base64 filesizes MUCH closer together, making the difference very minimal

The developer must choose between delivering 130,986 bytes in 5 HTTP requests, or 133,615 bytes in one HTTP request. Any good Souders follower would opt for the one request, BUT I would be careful here…

Caution: things aren’t always as they seem

There’s a huge caveat here: it may actually be more beneficial for perceived performance to deliver the images in 5 separate requests.

Why? Because 133,615 bytes is a lot to deliver all in one package to an end user who will be staring at blank placeholders for the duration. If the 5 base64 images all come in one request, that request will have to complete before ANYTHING is shown on the screen. All 5 images go from blank placeholders to almost immediately decoded from base64 and displayed in place.

Compare this with 5 requests that are most likely made in parallel and actually give a visual indicator to the user that actual image content is being downloaded, by showing parts of the images as they’re downloaded (you can also try a throwback to progressive JPEGs – really anything will be better than just a blank screen). That’s why it might actually be beneficial for perceived performance to just load images in the good old fashioned way. They will most likely load in parallel anyway, so the extra HTTP requests may actually not really make a difference. Not to mention it will be easier to let the browser manage the cache for each file instead of having to make your JavaScript manage your cache and prevent you from downloading an image that’s already stored away in localStorage or sessionStorage.

This being said, it’s generally advisable to put your common UI icons in base64 in your CSS, then let that whole chunk get cached by the browser. Those are usually clean vector icons as well, which seem to get compressed quite well (see Test #1).

But for image content, where there is nothing to be saved but HTTP requests, you should definitely think twice about base64 encoding to save requests. Yes, you will save a few HTTP requests, you won’t really be saving bytes, and the user might actually think the experience is slower because they can’t see the image content as it’s being downloaded. Even if you shave off a few milliseconds of wait time, the perceived performance is what matters most.

(EDIT: changed the wording of the “unverified intuitions” section from “not verified” to actual questions, to make it clearer)

12 Comments

5 Comments

Different ways of defining functions in JavaScript (this is madness!)

June 24th, 2011

This is madness! This… is… JavaScript!

In JavaScript, there’s many different ways of doing something. This is both a good thing and a bad thing. To the newcomer this is definitely a bad thing, as it means not only more things to learn, but more little caveats and more places to go wrong. And so it is with declaring functions!

The aim of this is just an accessible tour of the landscape, just so you know what’s out there and what the basic differences are. Do be sure to check out the “further reading” section as well! Much of this is based on Juriy “kangax” Zaytsev’s article, which goes into more depth. But I found that there wasn’t just one reference to show all the different variable declarations.

How about ways to execute functions? That opens up another can of worms, and incidentally opens up the possibility for a future post on that topic. :)

Overview: different ways of declaring functions

function A(){};             // function declaration
var B = function(){};       // function expression
var C = (function(){});     // function expression with grouping operators
var D = function foo(){};   // named function expression
var E = (function(){        // immediately-invoked function expression (IIFE) that returns a function
  return function(){}
})();
var F = new Function();     // Function constructor
var G = new function(){};   // special case: object constructor

Function declarations: function A(){};

Function declarations are probably the most familiar and oldest way of doing things in JavaScript land. This creates a variable A which is accessible in the current scope. Scope is a separate topic, so we’ll do everything in the global scope for all these examples (something you want to avoid usually).

1. Hoisting

The interesting thing about these is that they are “hoisted” to the top of their scope, which means this code:

A();
function A(){
  console.log('foo');
};

Gets executed as this code:

function A(){
  console.log('foo');
};
A();

Which practically means that, yes, you can call the functions before they’re written in your code. It won’t matter, because the entire function gets hoisted to the top of its containing scope. (This is contrasted with variables, which only have their declaration hoisted, not their contents, as we’ll see in the next section).

2. No function declarations in If statements (or loops, etc)

You can’t define functions this way in expressions, for example if statements, which is common if we want to define different versions of a function for different circumstances, usually to address browser inconsistencies. Well, you can in some implementations, but the way the code is processed is inconsistent (kangax has documented the inconsistencies here). If you want to use this pattern, use function expressions instead.

3. Functions declarations must have names

This method doesn’t allow you to create anonymous functions, meaning that you always have to give it an identifier (in this case we’ve used “A”).

Function expressions: var B = function(){};

A function expression looks similar to function declarations, except that the function is assigned to a variable name. Though functions are not primitive values in JavaScript, this is the way they can be utilized to their full effect in this functional language. Functions are “first class“:

“[JavaScript] supports passing functions as arguments to other functions, returning them as the values from other functions, and assigning them to variables or storing them in data structures”

1. Anonymous functions (they don’t need names)

The function name is optional in function expressions, and we call these anonymous. Here we’re setting the variable B equal to an anonymous function:

var B = function(){};

2. Variable declaration hoisting

Variable declarations are hoisted to the top of their scope, somewhat similarly to function hoisting except the contents of the variable are not hoisted as well. This happens with all variables, and it means it’s now happening with our functions, now that we’re assigning them to variables.

This code:

var A = function(){};
var B = function(){};
var C = function(){};

Will be executed as this:

var A, B, C;  // variable declarations are hoisted
A = function(){};
B = function(){};
C = function(){};

Therefore the order of setting and calling this type of function is important:

// this works
var B = function(){};
B();

// this doesn't work
B2();  // TypeError (B2 is undefined)
var B2 = function(){};

The second example gives us an error because only the variable B2′s declaration is hoisted, but not its definition, thus the “undefined” error.

Function expressions with grouping operators: var C = (function(){});

These really aren’t different from plain old function expressions and aren’t really seen in the wild (so maybe they’re just good for JavaScript quizzes?). Recently this way of declaring functions was brought up in this article and confused some folks including myself.

Here’s a good way to see what’s happening:

function(){};  // SyntaxError
(function(){});

Why does one work and the other doesn’t? The first example is a function declaration, and we learned above that we can’t declare them anonymously – that is, they must have a name. That’s why we’re getting the syntax error.

The second example is using parenthesis – grouping operators – and is therefore evaluated differently, as a function expression. The grouping operators are the things we use to help show what should be evaluated first, as in mathematical problems. We’re saying “evaluate this first, then take the result and do something with it”:

(1 + 2) * 3;  // 9
1 + (2 * 3);  // 7

In the first example we’re saying “first add 1 and 2, then take the result and multiply by 3″, whereas in the second example we’re saying “first multiply 2 and 3, then take the result and add 1″.

Because functions are first class, we can use similar grouping operators. Here’s a facetious example, but it shows how we can essentially drop in a function in the same way:

(function(){} + 1);  // function(){}1

The result is a string (because toString is being called on the function, then added/appended with 1), but you get the idea I hope.

When the JavaScript engine encounters the opening parenthesis here, we’re essentially saying “ok, start grouping this together with something else”. Using our technical terms, we’re telling the engine that we’re not making a function declaration, but instead a function expression. And then we can assign the result to a variable:

(function(){});           // resulting function not assigned
var foo = (function(){}); // resulting function assigned to foo
var bar = function(){};   // resulting function assigned to bar

Here we can see that foo and bar are really just the same, because in foo we’re not grouping the function together with anything but itself.

Named function expression: var D = function foo(){};

Here we have our same old friend, the function expression. But instead of assigning the variable to an anonymous function, we’re assigning it to a named function (with the name foo).

1. The function name is only accessible within the function

We haven’t exposed the function name (foo) to the enclosing scope (in this case the global scope):

var D = function foo(){
  console.log(typeof foo);
};
D();                       // function
console.log(typeof foo);   // undefined

2. Useful for recursion

Because the function’s name is accessible in the function itself, this turns out to be useful for recursive functions, much more useful than the plain old anonymous function.

Here’s a trivial recursive function to illustrate calling itself from within the named function expression:

var countdown = function a(count){
  if(count > 0) {
    count--;
    return a(count);  // we can also do this: a(--count), which is less clear
  }
  console.log('end of recursive function');
}
countdown(5);

3. Useful for debugging

As a few have pointed out, giving previously anonymous functions names helps in debugging, since the function name shows up on the call stack.

4. Quirks: JScript’s bad implementation

kangax points out that named function expressions are basically poison to JScript, Internet Explorer’s implementation of JavaScript.

The named function becomes a global variable, is hoisted like a function declaration, and actually ends up creating multiple instances of the same function.

Immediately-invoked function expressions (IIFE): var E = (function(){return function(){}})();

“Execute this function, whose return value is another function, and assign that to the variable E”. This may seem like magic, but it’s actually quite simple, and the pattern is powerful and has useful applications, the most famous of which is the module pattern.

First we’ll use an example that doesn’t look like magic:

var foo = function(){
  return 'bar';
};
var output = foo();
console.log(output);  // 'bar'

We already learned about grouping operators above, so you should feel comfortable with saying this is equivalent:

var foo = function(){
  return 'bar';
};
var output = (foo)(); // note the extra grouping operators
console.log(output);  // 'bar'

Since foo is pointing to our function expression, we know that we can simply refrain from using the variable “foo” and drop in the entire function as an anonymous function (since functions are first class, after all!):

var output = (function(){
  return 'bar';
})();
console.log(output);  // 'bar'

Hey wait, we just arrived at the magical resulting function! It turns out to be not so magical after all, once we break it down and see it for what it is. It’s simply shorthand for the code we wrote originally, where we defined a function, executed it, and defined output to be its return value.

I’ve included this method on the list of declaring functions because we can assign the return value to itself be a function:

var E = (function(){
  return function(){}
})();

Applications

There are good applications for this, including information hiding using in the module pattern, (partial application, for example), and other clever uses of it. It’s definitely not a trivial pattern.

Function constructor: var F = new Function();

This method is extremely old and it’s not recommended to be used. You pass in an unlimited number of arguments in the front, then the actual function body appears as a string in the last argument (because it’s a string, it’s effectively the equivalent of eval(), and isn’t recommended).

1. Defining the function

You can create a function like this:

var F = new Function('arg1', 'arg2', 'console.log(arg1 + ", " + arg2)');
F('foo', 'bar');  // 'foo, bar'

2. You don’t need the new operator

You can simply write var F = Function(); to get the same result.

3. Quirks

The MDN docs have some good examples of the quirks, including the fact that functions declared with the Function constructor don’t inherit their current scope properly (i.e. a closure isn’t formed).

What this means is that they don’t have access to variables in their enclosing scope, which isn’t particularly useful:

function foo(){
  var bar = 'blah';

  var first = new Function('console.log(typeof bar)');
  first();   // undefined

  var second = function(){
    console.log(typeof bar);
  }
  second();  // string
}
foo();

In the function “first”, we’re using the Function constructor, so it doesn’t have access to the variable bar. However, if we use the function “second”, which is a function expression, it does in fact have access to variables defined in its enclosing scope (via closure).

In other words, don’t use the Function constructor.

Special case – object constructor: var G = new function foo(){};

I saved this for last because we’re not really defining a function, though we are using the function keyword, so it’s worth noting at least.

new function(){}; creates a new object and invokes the anonymous function as its constructor. If an object is returned from the function, that becomes the resulting object, otherwise a new object is created from scratch and function is executed in the context of that new function (let’s save the details for another post!).

It’s a bit unusual to see it in this form. Let’s do it the proper way:

var Person = function(){
  console.log(this);  // Person
}
var joe = new Person();

So really with the new operator, we are giving it a new ‘this’ context and then executing the given function with that new context. Much different than the function definitions we’ve been dealing with above! This does into a whole new topic, and we’ll save that for later!

Further reading

Named function expressions demystified (kangax)

Immediately-Invoked Function Expression (IIFE) (Ben Alman)

Functions and function scope (Mozilla Developer Network – MDN)

How does an anonymous function in JavaScript work? (StackOverflow)

Function Declarations vs. Function Expressions (JavaScript, JavaScript by Angus Croll)

JavaScript: The Definitive Guide (classic book by David Flanagan)

5 Comments

6 Comments

On professional self-worth

June 21st, 2011

Huge huge rant incoming, but all is not said and done, and much is open for debate of course. Counterpoints and criticisms are more than welcome in the comments. This is just my opinion based on my experience at this point in time.

Self-worth: the rich man and the poor man

For a long while there have been places in the world that view self-worth as largely tied to wealth.  No doubt there is more than just money, as even within the rich community there’s a distinction between the established rich and the “nouveau riche”, or the newly rich.  One is ranked higher than the other, but nonetheless the rich are themselves seen as being more important than the average person.

You might not believe me.  That means it’s time for a thought experiment!

One way to hammer home this point is to posit a classic thought experiment.  There are two people – one rich and one poor – and this is all the information we have about them (this is important).  Assuming that one of them will die in some horrific way and we can only choose to save one, most people would probably save the rich person (assuming they don’t have a bias or some psychological complex against rich folks).  Remember, we have no other information about them other than one is rich and one is poor.  The aim of the thought experiment, rather than trying to give a realistic scenario, is to simply try to show us that we give the rich person preference.  At least ij the case where we have no other information.

And it’s not even that we think we might receive a reward from the rich person!  Even if we figure that in, we still give preference to the rich person, probably in part because we think that they must be somewhat more important than the poor person.  They must have acquired that money some way, after all.  Possobly even through hard work.  The poor person, we assume, must be poor for a reason, and that might be through sheer laziness.  So we have a hard worker versus a lazy worker… which one is more worthy to live?  And it’s only those hasty assumptions that we have to deal with.

Professional worth: employees of famous and nonfamous companies

Laying aside issues of money for a moment, let’s perform a similar thought experiment, except with different people.  One person works at company X, which no one has really heard of, and the other person works at company Y, which at the time is a wildly famous and reputable company.  Because this industry is so fickle and subject to change, at the time of this writing this famous company might be Facebook, Google, or Twitter.  Or any other company which has the tech zeitgeist or mojo at the time.

Assuming we know nothing else but the people’s employers (and laying aside biases and hatred for certain companies), if we put these people on the line, the one employee of a no-name company and the other employee of a famous company, we would probably choose to save the latter.  For much of the same reasons as above.  For instance, this person got a job at Facebook, so they must be smart and a hard worker.  On the other hand, the person at the no-name company (i.e. the vast majority of people out there) is probably not as smart or not as hard-working.  After all, if they were, then they would’ve been at (famous company) by now!  Therefore they are undoubtedly less worthy professionally, and by hasty extension, less worthy as a human being (remember, we have no other information to base our conclusion off of).

So it is!  Death to the employee of the nonfamous company!

Work history, first impressions, and professional self-worth

We chose to save the rich person and the Facebook engineer. The good thing is that luckily almost no one thinks in these horrible black-and-white terms, and we always take into account other available factors.  For instance, as an example of a poor person we might have Mother Teresa and as a rich person we might have mean old Donald Trump.  No one in their right mind would send Mother Teresa off to the gallows while letting Trump off (regardless of what you think of Trump really).

The problem here is that although we do take into account different factors, professionally we (and recruiters) weigh our work history much too heavily in the grand scheme of things.  We take other factors into consideration, sure, but work history is a huge deciding factor as far as people are paid attention to, give off good first impressions, or are even given a single glance by recruiters.

This isn’t totally unwarranted, I should say.  Chances are, if someone worked at a few famous companies, there’s a good chance they are a better than average engineer, even though there’s no guarantee of this (there are always exceptions).  But that’s not the problem here.  The problem is because this is given a disproportionate amount of weight, people start to view this as THE deciding factor in whether to work for a company.  Name-brand recognition becomes a goal-in-itself.  Money always matters, sure, but professionally there is little that can beat having a reputable name brand on a resume.

So what happens in practice?  An engineer becomes employed at a famous big company, is put to work maintaining some existing platforms, and at the end of a year perhaps learns some things that make them a better engineer.  But the same person instead working for some no-name startup might in the same period of time learn many more technologies, solve many more problems, and have a bit more freedom and opportunity to create (though they won’t necessarily learn best practices or scaling issues that typically come with famous large companies, but there’s more to being a good engineer than that!).  Each gets an opportunity to get their hands dirty with different varieties of dirt.  At the end of the day, I would say the latter very well becomes a more well-rounded engineer, instead of a generic single-functioned cog in some big company machine.

Ok, now my bias is showing through brilliantly!

Let’s at least say, for the purposes of argument, that both engineers are equally skilled (though this is hard to quantify, though, obviously).  So who do the recruiters contact, the employee at the famous company or the employee at the nonfamous company?  Who gets more attention from people in general when they’re asked where they work?  In short, whose professional life is a priori more highly valued?  You already know the answer.

Final thoughts

We MUST become more confident of our skills (I need to work on this myself), and we must be tough enough and realistic enough to know our own professional worth.  Appearances count for much in this world, and too much at that.  But what better to maintain if we are authentic human beings: an appearance or something deeper?

Famous companies do look good on resumes in the same way Ivy Leagues colleges look good on the very same piece of paper.  But don’t make this a deciding factor when choosing an employer.  It’s true that it will help you find more jobs and get more respect in the long run, but then you are wasting your time doing something you don’t want to do for the duration, for a company that exists for its own ends, not the ends of its employees (this is true of most companies out there, to be fair).  If you are ok wasting your time for a cause such as this, then go for it. It seems some people are ok with that.  At the very least be aware of it.

So what happens to the passionate engineers who actually do want to work for a famous company for reasons other than brand recognition?  They might not always get the job.  But it’s definitely not the end of the world.  No one should be disheartened because Google turned them away for not being able to answer today’s technological equivalent of “how many angels can dance on the head of a pin” (or other mental masturbation puzzles).  In the end it’s you that makes you a worthy professional and a good engineer.  If your job doesn’t actually give you a chance to be one, you can still become one on your own.  I would say most people are capable of this if they invest the effort.

But working for a famous company, and for its own sake?  It doesn’t necessarily make you a good engineer, any more than wearing glasses and dressing in a suit gives more validity to your opinions (even though many people sadly tend to think otherwise).

(Note to self: I wrote this on the banks of the Kamogawa River in Kyoto on my iPad. I was enjoying the moment and obviously in a contemplative mood!)

6 Comments

78 Comments

New Mobile Safari stuff in iOS5: position:fixed, overflow:scroll, new input type support, web workers, ECMAScript 5

June 7th, 2011

(note: this is based on the first iOS5 beta [9A5220p] and is subject to change on final release)

It looks like there’s finally some major improvements in mobile Safari, some of which I’ve found below on my “first glance” after downloading the SDK. Chime in if you find anything yourself!

Bad news first

Much to my disappointment, browser-based file uploads are still not supported (input type=”file”), even though they have been supported on Android since version 2.2 (which they have continued to improve and refine). The best use case for this is uploading pictures and videos straight from the browser, but Apple seems more interested in making those functions possible directly from the operating system itself, supporting only big names like Twitter and YouTube. So much for the little guys.

There’s other stuff on my wishlist that hasn’t yet been implemented, such as WebGL and indexedDB, along with some HTML5 input types, but regardless it’s good to see some real progress in iOS 5. (update: it appears that some WebGL features are starting to be supported, although the demos I tried didn’t seem to work).

Now let’s move on to the awesome new stuff that IS implemented!

position:fixed

A point of contention since the original iPhone was released in 2007, position:fixed now works on iOS5! What this means to developers: it’s now very easy to create top and bottom toolbars for your web apps. (there’s a long history of hacks and workarounds which will still need to be implemented as a fallback for the time being, but I’ll save that for another post. I should also mention that position:fixed has worked on Android since version 2.2, and webOS seems to support a somewhat hacky implementation).

Here’s a video demo of position:fixed on iOS5:

(check out the demo for yourself)

overflow:scroll

This is a similar problem to position:fixed, but now it works on iOS5! For content that doesn’t fit in its container element, previously the content was completely cut off (similarly with iframes). On the desktop we have the option to show scrollbars instead, allowing the user to scroll to see the content, but iOS made this difficult by requiring two fingers to scroll. Now overflow:scroll works and the user can scroll through the container’s content with a single finger. Developers can use new CSS to enable native scrolling (with scrollbars and momentum): -webkit-overflow-scrolling: touch; (via Johan Brook).

It’s assumed that it’s up to the developer to implement some sort of visual indicator to let the user know they can interact with the element.

One criticism so far: there’s no inertia scroll on this element, so it doesn’t quite feel the same as scrolling the page.

Here’s a quick video demo of overflow:scroll on iOS 5.

(check out the demo for yourself)

New Input Types

HTML5 defined many new helpful input types, and mobile operating systems have been slowly implementing them and creating optimized keyboards for the inputs. iOS5 now supports these input types: date, datetime, month, time, range. Click on the images to see their respective demo pages:

iOS Date InputiOS Datetime InputiOS Month InputiOS Time InputiOS Range Input

And a fun one – my iPhone slider hack based on input type range, which now actually works on the iPhone itself (because input type=”range” is now supported)!

iOS Slider implemented with Range Input

Web Workers (JavaScript)

JavaScript is single-threaded, and it happens to run on the same thread as the UI, which means computationally expensive operations can lead to an unresponsive UI, and no one likes that. Web Workers help by offloading tasks to another thread and only returning output when it needs to. In other words, it’s a good thing for us web developers.

According to my own tests, Web Workers seemed to be briefly enabled on early versions of iOS3, but were soon disabled thereafter. Now with iOS5, they’re back again!

New ECMAScript 5 stuff (JavaScript)

(based on kangax’s ECMAScript 5 compatibility table)

Feature iOS 4.3.2 iOS 5 beta 1
Object.create Yes Yes
Object.defineProperty Yes Yes
Object.defineProperties Yes Yes
Object.getPrototypeOf Yes Yes
Object.keys Yes Yes
Object.seal No Yes
Object.freeze No Yes
Object.preventExtensions No Yes
Object.isSealed No Yes
Object.isFrozen No Yes
Object.isExtensible No Yes
Object.getOwnPropertyDescriptor Yes Yes
Object.getOwnPropertyNames Yes Yes
Date.prototype.toISOString Yes Yes
Date.prototype.toISOString Yes Yes
Date.prototype.toISOString Yes Yes
Date.now Yes Yes
JSON Yes Yes
Function.prototype.bind No No
String.prototype.trim Yes Yes
Array.prototype.indexOf Yes Yes
Array.prototype.lastIndexOf Yes Yes
Array.prototype.every Yes Yes
Array.prototype.some Yes Yes
Array.prototype.forEach Yes Yes
Array.prototype.map Yes Yes
Array.prototype.filter Yes Yes
Array.prototype.reduce Yes Yes
Array.prototype.reduceRight Yes Yes
Getter in property initializer Yes Yes
Setter in property initializer Yes Yes
Property access on strings Yes Yes
Reserved words as property names No Yes
Strict mode No Yes

HTML5Test score

iOS 4.3.2 scores 206 (7 bonus points) on the HTML5 Test, whereas iOS 5 beta scores 267 (9 bonus points). Much of these gains seem to be made in the “Forms” and “Parsing rules” sections, as well as the “Web Workers” section.

Miscellaneous new stuff

  • Inline SVG in text/html
  • float32array, int8array, int16array, int32array in JavaScript
  • window.matchmedia() (JavaScript)
  • onsearch Event (JavaScript) – it looks like this was just broken in later iOS 4.3 builds, but it’s back now in iOS 5
  • MathML elements recognized (based on html5test)
  • Improved speed of Canvas rendering (source)
  • UIWebView and home screen web apps now running on Nitro? (expected but not tested)
  • Newly supported CSS gradient syntax (-webkit-linear-gradient versus the older -webkit-gradient syntax)
  • classList (JavaScript)
  • A major WebKit HTML5 History (popstate) bug is now fixed
  • contentEditable now works properly

78 Comments

18 Comments

The incredibly painful way of getting an RSS feed of a Twitter list

June 5th, 2011

(why does Twitter make it so hard? What did RSS ever do to it?)

The skinny

The Twitter API appears to support two methods for getting RSS feeds of lists. Both of these methods are pretty buried and their API docs don’t help too much, but nonetheless both of these feeds work:

http://api.twitter.com/1/nathansmith/lists/javascript/statuses.atom

If you have the list ID (requires using some dev tools – see below), you can also use this format:

http://api.twitter.com/1/lists/statuses.atom?list_id=41981352

The explanation

As it turns out, getting an RSS feed of a Twitter list is no easy task, at least from what I’ve found. Twitter has largely replaced RSS for me, in terms of getting the latest and greatest information, and maybe Twitter wants to keep it that way by having me rely on their service instead of an RSS feed. That’s how it seems to me anyway (this is also evidenced by their killing off of making RSS feeds easily discoverable but still making the feed available as long as you can figure out your Twitter user id (which is a task in itself), for instance here’s my RSS feed: https://twitter.com/statuses/user_timeline/16521889.rss). I’d say this is simply a conspiracy against RSS feeds and developers in general, which is the direction Twitter is headed these days.

In any case, I digress! I want to get an RSS feed of a Twitter list. Great! All I should have to do is go to the page of the list and subscribe to the RSS feed. But.. there’s no RSS feed. Hmm.

Google is your friend… most of the time

A search for “twitter list rss” comes up with links to implementing an RSS feed via Yahoo Pipes as well as pages referring to an appspot service that now 404s (twiterlist2rss). In any case, they are both not what I’m looking for. Which means I have to get creative. Which means… my options at this point are screen scraping (made harder by the fact that Twitter content is now served with JavaScript) or.. gasp.. using their API!

Twitter REST API to the rescue

After digging through the API docs, I found just the page I was looking for, detailing the REST service for Twitter lists.

As it turns out, the basic structure for requesting list data is as follows:

http://api.twitter.com/version/lists/statuses.format

As a newbie Twitter developer, there were a couple of things wrong here. First, what’s the version of the API I’m supposed to use? From other examples on the web (outside of the documentation…) I just assumed I’d be using version 1, and that seemed to work. Ok, whatever, we’ll go with 1:

http://api.twitter.com/1/lists/statuses.format

Second step: this API is actually quite nice, and will return data in three different formats: xml, json, or atom (RSS!). Ok, now we’re making progress!

http://api.twitter.com/1/lists/statuses.atom

Where’s this list_id you speak of?

Now to pass in the info – I need to pass in the list ID in a key called “list_id”. The only problem is… where do I find the list ID? For the purposes of this example I’ll use Nathan Smith’s excellent JavaScript developers list. To normal human beings, this list might be called “nathansmith/javascript”, but a Twitter ID is numerical, so that’s not what we’re looking for.

So.. I assume at this point I can use some of the Twitter developer tools and fiddle around with them until I find an API call that will translate the human-readable list name into a numerical ID. But no.. I don’t want to download anything, I don’t want to investigate more of the API or go down any more rabbit holes.

It’s time to cut this rabbit hole short and do a bit of investigating. So I pulled up the list, opened up Web Inspector in Chrome, and took a look at the Network tab in the hopes of seeing an API call with an ID I was looking for…

…And I found it! Somewhere down the line, Twitter was calling statuses.json with this request:

https://api.twitter.com/1/lists/statuses.json?list_id=41981352&include_entities=1&include_available_features=1&contributor_details=true

There’s our list_id (and incidentally, the same API call [with json instead of atom].. which shows that Web Inspector is more useful than API docs sometimes)! After a short copy-paste, all the pieces of my Frankenstein API call have been put together:

http://api.twitter.com/1/lists/statuses.atom?list_id=41981352

And it works, without having to mess around with OAuth or authentication, registering an app, or whatever. Of course now we can refer back to the API docs and mess around with some parameters to get just the data I’m looking for. Maybe most useful to developers is the handy ability to get easily paginated results:

http://api.twitter.com/1/lists/statuses.atom?list_id=41981352&page=2

There’s also what would be a handy “count” variable, but it’s altogether useless as it appears to be ignored. The API always returned 20 results for me.

In any case, there you have it! The incredibly roundabout way to get an RSS feed of your favorite Twitter list.

UPDATE

Thanks to this site, it looks like there’s a more straightforward way of getting the RSS feed. Using the same list we were trying to get above, it looks like this also works:

http://api.twitter.com/1/nathansmith/lists/javascript/statuses.atom

18 Comments

Older Entries
Newer Entries