David Calhoun's Blog

(why does Twitter make it so hard? What did RSS ever do to it?)

The skinny

The Twitter API appears to support two methods for getting RSS feeds of lists. Both of these methods are pretty buried and their API docs don’t help too much, but nonetheless both of these feeds work:

http://api.twitter.com/1/nathansmith/lists/javascript/statuses.atom

If you have the list ID (requires using some dev tools – see below), you can also use this format:

http://api.twitter.com/1/lists/statuses.atom?list_id=41981352

The explanation

As it turns out, getting an RSS feed of a Twitter list is no easy task, at least from what I’ve found. Twitter has largely replaced RSS for me, in terms of getting the latest and greatest information, and maybe Twitter wants to keep it that way by having me rely on their service instead of an RSS feed. That’s how it seems to me anyway (this is also evidenced by their killing off of making RSS feeds easily discoverable but still making the feed available as long as you can figure out your Twitter user id (which is a task in itself), for instance here’s my RSS feed: https://twitter.com/statuses/user_timeline/16521889.rss). I’d say this is simply a conspiracy against RSS feeds and developers in general, which is the direction Twitter is headed these days.

In any case, I digress! I want to get an RSS feed of a Twitter list. Great! All I should have to do is go to the page of the list and subscribe to the RSS feed. But.. there’s no RSS feed. Hmm.

Google is your friend… most of the time

A search for “twitter list rss” comes up with links to implementing an RSS feed via Yahoo Pipes as well as pages referring to an appspot service that now 404s (twiterlist2rss). In any case, they are both not what I’m looking for. Which means I have to get creative. Which means… my options at this point are screen scraping (made harder by the fact that Twitter content is now served with JavaScript) or.. gasp.. using their API!

Twitter REST API to the rescue

After digging through the API docs, I found just the page I was looking for, detailing the REST service for Twitter lists.

As it turns out, the basic structure for requesting list data is as follows:

http://api.twitter.com/version/lists/statuses.format

As a newbie Twitter developer, there were a couple of things wrong here. First, what’s the version of the API I’m supposed to use? From other examples on the web (outside of the documentation…) I just assumed I’d be using version 1, and that seemed to work. Ok, whatever, we’ll go with 1:

http://api.twitter.com/1/lists/statuses.format

Second step: this API is actually quite nice, and will return data in three different formats: xml, json, or atom (RSS!). Ok, now we’re making progress!

http://api.twitter.com/1/lists/statuses.atom

Where’s this list_id you speak of?

Now to pass in the info – I need to pass in the list ID in a key called “list_id”. The only problem is… where do I find the list ID? For the purposes of this example I’ll use Nathan Smith’s excellent JavaScript developers list. To normal human beings, this list might be called “nathansmith/javascript”, but a Twitter ID is numerical, so that’s not what we’re looking for.

So.. I assume at this point I can use some of the Twitter developer tools and fiddle around with them until I find an API call that will translate the human-readable list name into a numerical ID. But no.. I don’t want to download anything, I don’t want to investigate more of the API or go down any more rabbit holes.

It’s time to cut this rabbit hole short and do a bit of investigating. So I pulled up the list, opened up Web Inspector in Chrome, and took a look at the Network tab in the hopes of seeing an API call with an ID I was looking for…

…And I found it! Somewhere down the line, Twitter was calling statuses.json with this request:

https://api.twitter.com/1/lists/statuses.json?list_id=41981352&include_entities=1&include_available_features=1&contributor_details=true

There’s our list_id (and incidentally, the same API call [with json instead of atom].. which shows that Web Inspector is more useful than API docs sometimes)! After a short copy-paste, all the pieces of my Frankenstein API call have been put together:

http://api.twitter.com/1/lists/statuses.atom?list_id=41981352

And it works, without having to mess around with OAuth or authentication, registering an app, or whatever. Of course now we can refer back to the API docs and mess around with some parameters to get just the data I’m looking for. Maybe most useful to developers is the handy ability to get easily paginated results:

http://api.twitter.com/1/lists/statuses.atom?list_id=41981352&page=2

There’s also what would be a handy “count” variable, but it’s altogether useless as it appears to be ignored. The API always returned 20 results for me.

In any case, there you have it! The incredibly roundabout way to get an RSS feed of your favorite Twitter list.

UPDATE

Thanks to this site, it looks like there’s a more straightforward way of getting the RSS feed. Using the same list we were trying to get above, it looks like this also works:

http://api.twitter.com/1/nathansmith/lists/javascript/statuses.atom

I fired up the newly released Android 3.0 SDK to run some tests and found they’ve implemented part of the long-awaited Device API (aka the Media Capture API). From your browser you can now upload pictures and videos from the camera as well as sounds from the microphone. The returned data should be available to manipulate via the File API (although I haven’t yet tested this).

I made a short video demo with some explanation:

Here’s the code so you can play around with it for yourself!

<form enctype="multipart/form-data" method="post">
  <h2>Regular file upload</h2>
  <input type="file"></input>

  <h2>capture=camera</h2>
  <input type="file" accept="image/*;capture=camera"></input>

  <h2>capture=camcorder</h2>
  <input type="file" accept="video/*;capture=camcorder"></input>

  <h2>capture=microphone</h2>
  <input type="file" accept="audio/*;capture=microphone"></input>
</form>

Introduction

Ok, so this post is only here in the hopes I’ll actually win something.

O’Reilly Books has a promotion where a random winner will receive up to $500 of books on their wishlist, provided they post the list somewhere online. You can probably see why this is here then!

The List

Programming the Mobile Web ($50)
Palm webOS ($45)
MongoDB: The Definitive Guide ($40)
JavaScript: The Definitive Guide: Rough Cuts Version, Sixth Edition ($50)
HTML5: Up and Running ($30)
Building Android Apps with HTML, CSS, and JavaScript ($30)
Building iPhone Apps with HTML, CSS, and JavaScript ($30)
HTML & CSS: The Good Parts ($35)
HTML5 Canvas ($40)
jQuery Cookbook ($35)
Learning Android ($35)
CSS Cookbook, Third Edition ($50)

(OR “The Post In Which I Conflate Programming and Philosophy” OR “The Programmer’s Existentialist Crisis”)

I woke up this morning to read Dmitry A. Soshnikov’s tweet:

While this seems like good advice for aspiring programmers, I can’t help but think what this means for those who will buy into this. Us programmers have a keen ability to sit and focus on problems for extended periods of time, usually to the detriment of everything else around us. From the perspective of “real life”, we might as well be sitting in a basement somewhere playing World of Warcraft. Sure, programming is more productive, profitable, and surely not brainless, but at the end of the day we could still only be sitting in a basement somewhere. What I mean to say here is that we’re not actually living our actual lives doing these things.

I realize Dmitry was only talking about learning another programming language for professional life, but for me and so many others, professional life is not easily separated from personal life. We don’t just work during the week during work hours. We can’t just turn off our minds and stop thinking about it when we go home. I end up thinking about programming or web design puzzles in my leisure time and on weekends, largely because I enjoy doing so. But when I see something questioning whether I should be devoting even more of this time to learning another programming language, I think about what I could be doing instead. Put another way, I think about what is missing and neglected in my life, and how those things will remain missing and neglected.

Socrates famously said “The unexamined life is not worth living”. He, along with those who followed in his immediate footsteps (famously, Plato and Aristotle) were concerned with how us mere mortals should live our lives, and more particularly how we should lead good lives. The sad fact about (western) philosophy is that it’s turned into a shadow of its former self, now plagued by millennia of self-doubt and failed grandiose systems. But it seems the original Socratic intention of finding how to lead a good life is still an admirable goal, even if we disagree on how exactly to achieve it.

Socrates also had a bit to say about those seeking to be experts in their field, and he couldn’t resist getting in a jab at his wife in the process:

It is the example of the rider who wishes to become an expert horseman: “None of your soft-mouthed, docile animals for me,” he says; “the horse for me to own must show some spirit” in the belief, no doubt, if he can manage such an animal, it will be easy enough to deal with every other horse besides. And that is just my case. I wish to deal with human beings, to associate with man in general; hence my choice of wife. I know full well, if I can tolerate her spirit, I can with ease attach myself to every human being else.

Yes, Socrates essentially just compared his wife with a horse. A wild horse, to be exact.

I’m unsure if this translates well into learning programming languages. Does this mean we should learn the worst languages first, so that the rest of the languages we learn will be comparably easy? I hope not, that seems like quite a lot of wasted time. But I certainly agree that the first languages we learn shouldn’t be the easiest. This might only make us frustrated in the long run. So perhaps we can learn something from Socrates’s quote after all, if taken with a grain of salt.

So should we still strive to become experts in our field? I believe that’s an admirable and acceptable goal, but it shouldn’t be our main goal. We should first be concerned with leading good lives. I myself have a LONG way to go on this point, believe me.

I’ll surely learn another programming language, but not for its own sake, unless my life is in reasonably good order.