I originally published this post to the Inside NPR.org blog on April 19, 2010

Daniel Jacobson’s Presentation on NPR API Usage and Metrics

It has been more than a year since my last post about API usage, so this is long overdue.  Needless to say, the NPR API has grown tremendously since its launch in 2008.  The biggest consumer of the API is still NPR by a long shot.  That said, member stations, partners and the general public have been making extensive use of the API. 

Below are metrics on the API followed by a suite of examples of how it is being used.  But first, here are a few qualifiers to the statistics:

– These statistics are about requests to the API, not necessarily what was consumed by or presented to users.

– These numbers were obtained through server logs.  Although we believe they are largely accurate, it is possible that these numbers are off by a couple percentage points in either direction.

– A request to the API can mean many things.  For example, on the NPR News iPhone app, the Top Stories section is populated by one request to the API – that one request returns many stories.  The Flash Player on NPR.org, when playing a single piece of audio, represents one request to the API for exactly one story.  Meanwhile, some requests (such as a story page on WBUR.org) get exactly one story, but because they get cached result in many page views for that one request.  As a result, metrics around requests are imperfect.  That said, they do offer some information about usage and consumption, even if not the whole story.

The big jump in total API requests from July to August are due to the launch of many new products in July.  Among them are the new NPR.org, the NPR.org Flash Player, the NPR News iPhone app, WBUR’s new web site, and Minnesota Public Radio’s new site. Since then  an increasing number of applications have been implemented on the API, including the NPR mobile site and other station sites (like KQED and KPCC), accounting for the steady growth over the last few months.

The total number of stories requested is a new metric that we started tracking in October, 2009.  This is the number of stories requested, not the number necessarily delivered or consumed by a user.  This metric is based on the numResults parameter (or lack thereof) in the API request.  And to be clear, in March, 2010, the API did deliver over 1.1 billion stories!  Since tracking this metric six months ago, the API has handled almost 5 billion story requests.

Overwhelmingly, NPRML is the dominant output format delivered by the API.  That said, most of the requests are from NPR and related products.  In the past, prior to NPR building out the new site, iPhone app, etc., the distribution was slightly less NPRML-heavy.  Those models had a distribution of NPRML at about 55% and RSS at about 25%, with MediaRSS, JS and HTML getting reasonable traffic but significantly less than RSS.

As you can see from this chart, the non-NPRML formats have grown a bit over time, especially the JavaScript Widget in recent months.  Prior to August, 2009, NPRML was still very dominant, but it its use was substantially closer to the use of the other formats.

Abstracting away NPRML, it is much clearer how the other formats have grown over time.  RSS and the JavaScript Widget have really grown tremendously over the last year.  Mix-Your-Own-Podcast launched in December, 2008 and took a while to start its growth. 

To get a sense as to what is creating all of this traffic, I have created a presentation that shows a range of examples on how the NPR API is getting used.  The following presentation is probably missing some interesting and heavily-accessed examples, but it should give you an idea as to how pervasive the API has become. 

Given the growth charts above, along with the introduction of the API to station sites and the launch of the iPad and other upcoming tablets and portable devices, we expect these numbers to continue to climb.

Finally, if you are aware of some interesting implementations using the NPR API, please let us know in the comments here or by emailing us at techcenter at npr dot org.

This presentation was first published to my Slideshare account on October 5, 2009 and was used in different forms to demonstrate the power of the COPE model in content management and API development. It shows the same NPR story displayed in a wide range of platforms. The content, through the principles of COPE, is pushed out to all of these destinations through the NPR API. Each destination, meanwhile, uses the appropriate content for that presentation layer.

This presentation was given at the Wolfram Data Summit in September of 2010, focusing on content management, APIs and COPE.

Derek Gottfrid Speaking at OSCON Thursday. Photo Courtesy James Duncan Davidson/O’Reilly Media Is the future of news in the hands of internet developers? News organizations New York Times and National Public Radio (NPR) think so. O’Reilly Open Source Convention (OSCON) this week offers the opportune time for NPR and New York Times programmers to discuss […]

O’Reilly Open Source Convention (OSCON) this week offers the opportune time for NPR and New York Times programmers to discuss the release of their news source Application Programming Interfaces (APIs).

NPR’s announcement came earlier in the week. NPR’s API introduces the ability to write applications surrounding public radio’s text and audio from most radio programs dating back to 1995. It was only a matter of days before Phoenix programmer John Tynan exemplified what one can do with the API by mashing up NPR headlines with a Simile Timeline visualization.

Likewise, New York Times programmer Derek Gottfrid is excited about his API. Officially on the menu: public-ready releases of some of the APIs they’ve used internally. First out of the gate later this year will be read-only APIs in distinct content segments, like movie reviews, restaurant reviews and wedding announcements.

Both APIs follow Reuter’s lead: The news agency released its API in May. If the APIs take off, soon all major global news organizations will be offering audiences ways to craft their own presentations of what the news is and what it looks like on the Web.

Wired.com took Dan Jacobson from NPR and Derek Gottfrid from the New York Times to a Portland restaurant to talk about their company’s news APIs.

Wired.com: What do you say when people ask you what an API is?

Dan Jacobson from NPR: We’ve been spending a lot of time in front of lots of people explaining what we’re doing. It is a challenging topic for people who don’t understand it. Basically we’ve been saying it’s like an implicit handshake between two applications, or two computer systems, or whatever.

Derek Gottfrid from The New York Times: It really is just another syndication mechanism for us. That data that you, an API user, have. Because it’s [in a] semi-structured form, it allows it to show up in different places – in applications, in different places around our website, in different places around the web in general.

Wired.com: So your people know what RSS syndication is. Can you leverage that to explain things?

Gottfrid: When we talk about syndication, it’s something that, for a newspaper, has been part of the business vernacular for a long time. So it’s distribution. It’s the same notion as distribution, or influence… Part of The New York Times mission is to create, collect and distribute high-quality news, information and entertainment. “Distribute” is analogous to syndication. I think the terms have a natural flow to people in the organization.

Wired.com: What makes your API special? How do you expect people to use it?

Gottfrid: What makes it special is the data it accesses, right? It’s not the format and whether it is XML or JSON or anything like that. For the format part, we want to just follow best practices of the community. It’s really access to all the interesting data, whether that’s all the recipes that have been in the paper, or all the news articles about particular topics, or weddings, or events or whatever data that we’ve accumulated over the years. I think that the data is really the interesting part, and that’s the unique part that we have. That is a differentiator.

Wired.com: So The New York Times API will be able to go back in time through the entire historical archives of the paper?

Gottfrid: We have the data, so creating APIs around it is done internally all the time. Making them so they’re consumable by the outside world requires additional effort, and that’s really where we are. We have all this data. We’re familiar with it. We’re trying to make it as palatable, and as easy as possible for outside folks to get at it is really the next step that we’re working on.

Jacobson: I agree completely. It’s all about the content. If you don’t have compelling content, then no matter how sweet the application is nobody’s going to want to come and get it. I think that NPR, like The New York Times, offers a rich, unique spin on the content we provide. In terms of the functionality of the API, one of the interesting things we’re offering is a very comprehensive way of slicing through the data. If you go to NPR.org, for example, you’re getting NPR’s presentation of our data. It’s our compilations and our topic structures. Through the API, users can come and slice the content however they want it, create their own custom feeds and we’ll leave it up to them to build exactly what they want. Things we couldn’t even envision.

Wired.com: NPR has an affiliate network that The New York Times doesn’t have. Does the API stand to affect the dynamic between National Public Radio and its affiliates?

Jacobson: There are two sweet spots for the API: It fits NPR’s public service mission to help people be better informed, enabling users to get our content in a variety of ways – however, they want it. For the stations, it lets us get local station content in and then feed it back out through the API, which we’re doing some of already. But it also enables the stations to represent to their communities whatever they want. They can mash up local and national content. Or their users can do the same in ways that, prior to the API, they couldn’t do.

Wired.com: Does The New York Times have intermediaries it’s looking to serve with your API?

Gottfrid: We’re geared to whoever is going to find the content interesting. Anyone that’s interested in it, we’re interested in making it accessible and having them use it. This isn’t something that’s driven off of market research or anything like that. This is fulfilling a basic gut-level instinct that this is how the internet works.

Wired.com: Where does that gut-level instinct come from? Is it a matter of transforming internal work processes and extending them?

Gottfrid: Yeah, it’s a natural outgrowth. As we’ve become more sophisticated, we’ve taken more of a platform and service architecture [approach] to a lot of the things we do so that we can re-use them and mix them and mash them for our own site. I think [The New York Times API] is the natural extension. It flows into a lot of things that we see in terms of opening up to the broader web. Really going from being “on the internet” to being “part of the internet” – intermingling our stuff with the full experience of things around the internet.

Wired.com: Where does NPR fall on the internal versus external utility of its API? Was it developed specifically with external users in mind?

Jacobson: The evolution of our API was pretty organic. We built an API to support NPR.org, and launched that in November of 2007. Our site’s been running on the API for that long. The natural next step was to say, ‘Wait a minute. Why can’t we just put this out there.’ What do we need to do in order to open this up and satisfy … users’ goals with YouTube and Google Maps and the way they’re able to reach new audiences. Then it became a policy question as we sat down with a range of stakeholders and figured out what we are allowed to do. Turns out we’re allowed to distribute through the API everything that we have the rights to (which isn’t everything you hear on NPR stations).

Gottfrid: The technology people sit at the nexus of [our audience] so we facilitate an interchange between [them]. Clearly we need to be able to do stuff with our content management to support the reporting efforts. Our end users, well we wouldn’t be here without the readers. It’s a continual balancing act, especially when online readers aren’t as remote as they are with the printed product. There’s a different relationship that we’re establishing.

Jacobson: I think Derek said something really interesting, implying that technology is also a stakeholder in the kinds of things that happen. This API project, for example, is something that we drove. We became a stakeholder because this is a project that we wanted to release, which is somewhat tied to the business goals. Making the case that we need to do this is convincing the business people that, yes, we need to do this.

[Interview by Brad Stenger]