{"id":494,"date":"2013-03-15T19:39:00","date_gmt":"2013-03-15T23:39:00","guid":{"rendered":"https:\/\/danieljacobson.net\/blog\/?p=494"},"modified":"2024-09-07T11:31:34","modified_gmt":"2024-09-07T15:31:34","slug":"insert-content-here-podcast-episode-11-nprs-cope-and-content-apis","status":"publish","type":"post","link":"https:\/\/danieljacobson.net\/blog\/494","title":{"rendered":"Interview : Insert Content Here Podcast, Episode 11: NPR&#8217;s COPE and Content APIs"},"content":{"rendered":"\n<p><a href=\"https:\/\/www.lullabot.com\/podcasts\/insert-content-here\/daniel-jacobson-on-nprs-cope-and-content-apis\" data-type=\"link\" data-id=\"https:\/\/www.lullabot.com\/podcasts\/insert-content-here\/daniel-jacobson-on-nprs-cope-and-content-apis\">This podcast episode was published on Insert Content Here with Jeff Eaton on March 15, 2013<\/a><\/p>\n\n\n\n<p>Daniel Jacobson recaps the history of NPR&#8217;s COPE, his work at Netflix, and the future of content APIs.<audio preload=\"none\"><\/audio><\/p>\n\n\n\n<p>Host: <a href=\"https:\/\/twitter.com\/eaton\">Jeff Eaton<\/a><\/p>\n\n\n\n<p>Guest: <a href=\"https:\/\/twitter.com\/daniel_jacobson\">Daniel Jacobson<\/a>, Netflix<\/p>\n\n\n\n<p>Mentioned in this episode:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/insertcontenthere.com\/episode-11\/APIs%20A%20Strategy%20Guide\">APIs A Strategy Guide<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/insertcontenthere.com\/episode-11\/7%20Ways%20to%20Improve%20Your%20API\">7 Ways to Improve Your API<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/insertcontenthere.com\/episode-11\/NPR%20API%20Reference\">NPR API Reference<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/insertcontenthere.com\/episode-11\/Create%20Once,%20Publish%20Everywhere\">Create Once, Publish Everywhere<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/insertcontenthere.com\/episode-11\/Netflix%20Tech%20Blog\">Netflix Tech Blog<\/a><\/li>\n<\/ul>\n\n\n\n<p>This episode&#8217;s transcript is in progress; the partial transcript below has been auto-generated by&nbsp;<a href=\"https:\/\/otter.ai\/\">Otter.ai<\/a>.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Hi, I\u2019m Jeff Eaton, senior architect at Lullabot. And your host for insert content here. today I\u2019m here with Daniel Jacobson, the illustrious Daniel Jacobson. He\u2019s not necessarily someone that you may have seen him in content circles or nerding out in the CMS world that much, but he\u2019s actually one of the movers and shakers.<\/p>\n\n\n\n<p>His role for quite a few years was Director of Application Development at NPR, where he oversaw the development of COPE \u2013 the Create Once Publish Everywhere infrastructure that made all kinds of waves in the content strategy and CMS world. He\u2019s the author of O\u2019Reilly and Associates\u2019 <em>APIs: A Strategy Guide<\/em>. And now, he\u2019s the director of engineering for the Netflix API. So, he is just all over the place when it comes to content, APIs, and reusable structured content. It\u2019s a pleasure to have you here, Daniel, thank you for joining us.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Thanks for having me, Jeff. Thanks for that very nice intro.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Well, you know, it\u2019s funny, I was actually we reading your blog a couple of weeks ago. And, you talked about a presentation that you had done at a conference on the architecture and you said, &#8220;wow&#8221;. You know, a bunch of people in the room knew about it and you know, they were familiar with it.<\/p>\n\n\n\n<p>And it seems like you were almost a little startled by that. Did you expect the work that you were doing on that project to become such a big discussion topic,<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: You know, I really didn\u2019t. We had been working &#8212; this goes back to the time of NPR &#8212; we\u2019d been working on these concepts and implementation since 2002.<\/p>\n\n\n\n<p>I can give you the history on that if you\u2019re interested, but we were just doing what we thought was right. You know, thinking about architecture, thinking about longevity of application development and knowing that there are going to be a host of things that we\u2019re not going to be aware of down the road, so trying to prepare for nimbleness.<\/p>\n\n\n\n<p>So then in 2009, that\u2019s when I first published that blog post on programmable web about COPE, I was just taking this step of, &#8220;okay, we\u2019ve done all this stuff in the spirit of sharing, let\u2019s share it&#8221;. And it was interesting to see how some people latched onto it.<\/p>\n\n\n\n<p>But what really has surprised me is we\u2019re in 2013, people are still talking about it.  It\u2019s mind boggling for me. And if you look at the blog post, over half of the, the comments were from 2012. I don\u2019t know, 70 or 80 comments. so what\u2019s really kind of staggering for me is the fact that, we published it in 2009.<\/p>\n\n\n\n<p>We\u2019ve been thinking of it for a number of years and implementing it for that time. But it\u2019s still, or it is now resonating. It\u2019s very weird.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Well, I\u2019ll back up for a second for anybody who might not be familiar, what is COPE?  You know, what\u2019s the idea behind it.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Okay. I\u2019ll give you a little bit of a history on how we arrived at COPE and some of the sensibilities that went into it.  COPE stands for Create Once Publish Everywhere. And the idea there is, you want to maximize the leverage of content creation and minimize the effort to distribute. The way that the whole thing really started, it goes back a little deeper. It was actually in 2002 when we had a series of different ways of publishing content to NPR.  We had HTML files. We had a Cold Fusion system with a SQL Server database that publish very thin radio pieces to the web.  Then I had build this other system, a Java based system with an Informix database to capture a little bit richer content and offer an opportunity for co-branding with local stations. In 2002, we looked at this and said, well, this is kind of a mess. We\u2019re spending all this energy publishing in three different ways and we have three different presentation layers.  What we really need to do is to collapse them into a system that really leverages our editorial time and gets us an opportunity to distribute to all these different places. <\/p>\n\n\n\n<p>So interestingly, part of the story is that I  pitched the idea with my boss at the time Robert Holt, who\u2019s now at Microsoft.  We pitched the idea to actually spend a bunch of time collapsing these systems and building an awesome system that takes care of all of this.  At the time the VP, her name was MJ Bear, she wanted us to do due diligence and explore all these different avenues, content management systems like TeamSite and vignette, which are all quite expensive.  So we did all that due diligence but I wasn\u2019t convinced that that was the right route. And in fact, I knew we could do a better job by building it in house.  The inflection point was that she quit. She left the job and, I basically said, &#8220;Great, let\u2019s go build it. There\u2019s no impediment anymore before things settle, let\u2019s get this built before the next VP.&#8221;  And, so we did that. We hunkered down and that\u2019s where we really started thinking about the philosophies of &#8220;separate content from display&#8221; and, and &#8220;content modularity&#8221; and things like that. <\/p>\n\n\n\n<p>So this was back in 2002 and it was partially driven by the idea that everything we\u2019re doing here of collapsing these data systems, we\u2019re also doing a redesign of the presentation layer.  If we\u2019re doing that, it\u2019s highly likely that in the future, we\u2019re going to have another presentation layer, either a new one to replace the old one or an additional one. And it\u2019s almost like this keeps happening.  It\u2019s cyclical.<\/p>\n\n\n\n<p>We need something great. so let\u2019s throw out the old, and so we said, &#8220;all right, this presentation layer is going to go away too. So we really need a decoupling. And that\u2019s where a lot of these COPE philosophies started to soak in. And actually we launched that CMS in November, 2002 and it\u2019s still the centerpiece of NPR today. <\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Holy cow. <\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah, the CMS has lasted 10 or 11 years and we see that as a kind of success. So we take a lot of pride in that.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>:  It seems like that that decoupling that you talked about from a pure software development and architecture standpoint, that feels like the right thing to do, but you know what you mentioned the idea of teasing apart the core business assets that are content that gets created and managed over time from the changing sort of ephemeral presentation, teasing those things apart. It feels like it makes good business sense too. It\u2019s not just about architectural purity. It\u2019s a way to make sure that you don\u2019t have to dig up the foundation every time the house needs to get painted.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Absolutely. And I will say it\u2019s not without some costs because there were certainly some cultural battles that went into those discussions. An example that I can offer is when we were doing this design, what we intentionally said is that the story is the atom of the system. That was another philosophy of what we were doing.  And so the atom the center of the NPR system universe. And we basically said the story is super thin. And generally speaking in NPR, people think of the story as being a radio piece. And we said, &#8220;no, radio pieces are not a necessary component of a story, it\u2019s an enhancing and enriching part.&#8221; But so are images. So is text, so are pull quotes. So is whatever else you want to imagine that you want to put in there, but they\u2019re not necessarily. The only parts that are necessary are basically the title, a unique ID, a date. And I think we said a teaser, that\u2019s it. That\u2019s all you need for a story.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Interesting.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: And from there, there\u2019s a hierarchical data model. Tell me if I\u2019m getting too geeky.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Oh, this is the kind of stuff I love.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: So we had hierchical data model where we basically had a story own what we called a &#8220;resource&#8221; and a resource was any enhancing component of a story. And a resource is generic. And then the sub hierarchy underneath that were the particulars, which would be text, images, videos, external links, internal references&#8230;<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: transcripts, I think just went live a little while ago.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Transcripts is another one for sure. Those were all enhancing enriching components, but not necessary to have a story on the site. And I know that you and others on this podcast have talked about blobs and chunks. So, we, we tried to be very surgical with this idea, we wanted to have everything be its own entity.  We don\u2019t want these blobs of data. In fact, every paragraph in the text are stored separately and very distinct fields. We really thought about how we can manage this stuff in modular ways so, as you were saying, we\u2019re teasing it out so that our business can gain value down the road.<\/p>\n\n\n\n<p>And the controversial part is, especially in 2002, NPR is a journalism company focused on broadcast. Saying that the radio or the audio piece is not essential was pretty controversial. And it takes a little, it took a little massaging inside to get people to understand the power of this model. And it took actually a number of years to really get everybody on board.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: I would imagine that some of the decisions that you made earlier, like the story is central, even if every single story has a radio component for the first several years, there\u2019s a stake in the ground, just in the nomenclature that you\u2019ve been able to get in there.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Sure. you know, we really talked about stories and lists. Those were the two core concepts and you start talking about it and eventually that becomes part of the vernacular and part of the culture. And really, every story can belong to any number of lists, any number of types of lists. Lists are really just aggregating mechanisms.  And that cut against the culture as well. And NPR, because you think about morning edition for March 13th, you think there\u2019s a rundown, there are 15 stories for that day. That\u2019s really just another list. And it\u2019s not really a program. It is, it happens to be that this list&#8217;s name is Morning Edition&#8230;<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: It\u2019s a particular branded aggregation of stories, not necessarily this thing that is the primary means that people approach and find content, right?<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: It\u2019s not a radio program anymore. It\u2019s a list, and you know, you can identify it by titles.  But you gotta think about it more generically and that\u2019s when we started introducing a broader set of topics, such as &#8220;News&#8221;, &#8220;Music&#8221;, and things like that. And it really created, tremendous opportunity for NPR actually.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: There\u2019ve been a couple of things that have been written about the actual longterm impact of it and what it\u2019s allowed NPR to do in terms of turning around new products that are targeted at emerging devices or platforms without having to go through the same sort of just profound, painful rearchitecting of everything that a lot of other companies are having to do. Do you think that it\u2019s interesting, do you think that this set up NPR for taking advantage of mobile, because of all the work that had already been in place.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Unquestionably, yes. Following the story of the content management system creation, I\u2019ve talked a bunch about not building a web publishing tool.  So, we felt like we were building a content management system and we thought about it in terms of having the content management editing tool be just another their presentation layer. And you can have any number of presentation layers. And so that actually gave a lot of freedom. If the editing tool is just one presentation layer and the website is one, we can have any number of these layers. We can just tack on new presentation layers, some of which have write capabilities and some don\u2019t.<\/p>\n\n\n\n<p>So as that started to mature a little bit back in 2004, RSS started to emerge. It was easy just to spin up another PHP file and have it render a RSS feeds and you pass in parameters and it\u2019s going to yield a topic or a program. And then shortly after that, podcasting started to emerge and we can just easily float an MP3 file in the RSS feeds.<\/p>\n\n\n\n<p>And, it was about 2007 when the NPR music site launched. We started to see the fact that we had this single point of failure, an Oracle database, which we were growing out of.  We actually need a different data redundancy model.  We needed a cluster of databases to be able to scale it. And NPR was a nonprofit, so we couldn&#8217;t afford a million Oracle\u2019s servers. So, there were a couple of things that had me thinking that we needed another level of abstraction. And that\u2019s when we introduced the API. That gave us an opportunity to basically put the NPR music development on a different trajectory, as well.<\/p>\n\n\n\n<p>Now that everything is, or at least was, we were moving in the direction of having our sites fueled from the API. We can more easily abstract away the database underneath the API and swap in a cluster of MySQL servers, multiple databases. And, so we started thinking of the API in those terms. And then sometime after that, we opened it up publicly.<\/p>\n\n\n\n<p>After we opened it up, we realized, &#8220;wow, we should be using that for other presentation layers like mobile sites and iPhones and iPads&#8221; and feeding into the Livio radio, radio devices that are feeding NPR content. Now it\u2019s going into cars. So, all of this strategy, which started early in 2002, I think was in retrospect, we were lucky with having that kind of architecture.<\/p>\n\n\n\n<p>It put us in a great position to just tack on more presentation layers and allow them all to feed off of one central distribution channel, which was the API.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: No, it definitely makes sense. I mean, although I think it is interesting that the real rise of mobile web traffic, especially apps, this idea is a given.<\/p>\n\n\n\n<p>Businesses that have content might need this cluster of different ways that people can get to their stuff. I think it feels like that\u2019s broken and a lot of existing sites and a lot of existing workflows and a lot of existing platforms that companies have built out. And I think that sort of corresponds to where the huge spike in interest in the work that\u2019s been done on COPE, really took off because suddenly everyone was feeling just such a tremendous amount of pain around that issue.<\/p>\n\n\n\n<p>What you\u2019re describing is interesting because it\u2019s not the same kind of story your team had been working on this for a long time based on deeper needs than simply we need to go mobile or something like that.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah, that\u2019s right. I think we were lucky to have had a series of challenges, like financial challenges, not being able to afford vignette.  We were lucky and that we were doing a series of things all at once. And we knew that we were redesigning and we were likely to have to redesign again. There\u2019s a confluence of things that got us thinking early about it. We had no idea the iPhone was going to go nuts. I mean, if we had that kind of foresight, I should have been betting on the stock.<\/p>\n\n\n\n<p>But we were very lucky and I think we actually made a series of decent decisions thereafter that put us in a good position. When I hear about folks who have been, who weren\u2019t quite as lucky to have that confluence of events and they have to go back and retrofit. Well, now the space is much more complicated and you\u2019re already embedded in your web publishing tool.  So their rearchitecture at that stage is actually much more expensive and much more painful. It just seems like we did it at the right time. Actually, things were very lucky.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: You mentioned that the actual editorial tools, the things that the actual content creation people use is just another presentation layer in that sort of approach.  How did that side of things evolve? Cause you know, the structured content approach that you\u2019re describing isn\u2019t necessarily a natural fit or a natural transition for people who aren\u2019t used to saying data modeling is their weekend hobby.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Data modeling is really at the center of it all.  I felt it was critically essential to start with a very clean data model. That that was the starting point. How do we, how do we imagine this content to be stored and thinking about it in those very teased out ways?  So yeah. Text is very different than a teaser, which is very different than images.<\/p>\n\n\n\n<p>And everything was really isolated. That\u2019s where the content modularity part comes in. So we, that\u2019s where we started. And if you have that well, that\u2019s just your data repository. And then any number of apps can hit against that. And we had the website hitting against that, but we also had this suite of tools that could write to it.  They can also access it, but then we started, over time experimenting with areas of the website, writing different user related things to the database. Basically, it starts with the database. That\u2019s all you have. And then every, everything else is either reading or writing or both to that database. And we just thought about it in those terms.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Was the user experience aspects of building out those tools, something that you were involved with or was that something that there was a different team of people trying to make the tools usable for this way of approaching things.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Okay. So I\u2019m actually really glad you brought that up because I kind of glossed over that. First I want to be clear and maybe this kind of gives people some hope. When you say the team of people, the total team that I can think of that executed on this entire CMS project back in 2002, it was about four people.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Oh, this is like learning how bacon gets made.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. I hope I don\u2019t disappoint anybody.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: I think that\u2019ll encourage a lot of people actually.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. I mean, it really started with this massive document that I put together that we used for due diligence to send out the vendors per the VPs request.<\/p>\n\n\n\n<p>So we had a vision that we kind of pieced together and that vision, both encompassed what the data structure\u2019s going to look like, what\u2019s it going to look like in terms of conceptually the CMS or the content capture part as well as, directions for the website.  We obviously then brought it in house.<\/p>\n\n\n\n<p>We didn\u2019t have a VP anymore. It was me and one other backend developer at the time. And there was one front end developer and there was a designer. And then my boss, so that was the team. We had a suite of editorial folks who contributed meaningfully to this. I don\u2019t want to discount them, but in terms of the engineering, that was it. <\/p>\n\n\n\n<p>So me and the other backend engineer, we wrote all the tools around the content management system. But before we did that, we were heavily informed by the designer. And I was pretty involved in this as well. We did a series of usability tests. We took data from both the usage patterns of our users online and  the <a href=\"http:\/\/npr.org\/\">npr.org<\/a> users. We took information about the three discreet ways that we publish. We had the Cold Fusion system, that Java system, the HTML files.  <\/p>\n\n\n\n<p>What are the kinds of things we\u2019re building?  We knew that actually, in the mode right now, at that moment too, we had a very limited sets of assets.  It was very audio focused, but we knew that things coming down the road were going to be much more expansive in terms of the available assets: images were probably to come later, and full text. And we were hiring editors too on the online side specifically to build out those stories.<\/p>\n\n\n\n<p>So we were informed by all of those things and did a whole series of mockups and clickable prototypes for the editors and sat down with them and said, &#8220;okay, well, what do you think of this? What do you think of that?&#8221; And then here\u2019s the interesting part. We took all that data and we had to discount fair amount of it because we thought that they were still thinking about it too much like NPR, right?<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Yeah. They were thinking, &#8220;Oh, you\u2019re building the new version of what we\u2019re used to.&#8221;<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yes. Right. So, we took on all of that. There were a lot of really great ideas and fundamental things that drove the direction of where we\u2019re going. But again, we had to think, we\u2019re thinking bigger than this. We\u2019re thinking there will be another design, there\u2019s this and that.<\/p>\n\n\n\n<p>So, we needed to discount a portion of our learnings. And, yeah, so all of that kind of boiled into the content management system. And I think you and Karen have talked about this fair amount, that you can\u2019t just build tools as an engineer would build a tool, right? You\u2019re building your website to have it be meaningfully useful to the website user.<\/p>\n\n\n\n<p>You need the same mindset when you\u2019re building the CMS. You want it to be infused with the sensibilities that will make them effective at what they\u2019re doing. So we tried to take all those sensibilities and make something that they would succeed with, and evolve it over time.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Especially with sites that have an existing infrastructure and stuff like that. It\u2019s very easy to see those kinds of things. And imagine it\u2019s just insurmountable, there\u2019s definitely a lot of hurdles, especially, it feels like every year that passes there\u2019s more weight that\u2019s being put on a lot of the web properties, the different companies, but the idea is it doesn\u2019t necessarily take an army to do this.  It seems like it\u2019s more of a wheel inside of the organization to take this kind of path.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. I have a couple of thoughts on that. First, generally I agree. I think the one area I would disagree is that the world is different now than 10 years ago. And I think, again, we were lucky to have gone down that route at the time that we did, because like you said, the weight is lighter in 2002 than in 2013. But in every other capacity, I agree. And you need to have commitment, and the commitment not only starts with the commitment to vision, but resourcing it appropriately. If you need these engineers to build this out, hire engineers, and I\u2019m a big believer in the idea that excellent engineers are going to be, some people say, 10 times more effective than average engineers. So having really smart people who are able to execute on these things, you can really tease out what you\u2019re going for. What\u2019s the best way to execute on it? That\u2019s going to pay way more dividends than just hiring a bunch of people and throwing money at it.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: What you were describing earlier was it was a very cross functional team, all working on these things. You were talking about working in close conjunction with the designers who are planning out what the new visual appearance of the site was going to be in how it was going to operate, working with the the content creators and the editorial team.<\/p>\n\n\n\n<p>It wasn\u2019t just a matter of wireframes that were thrown over the wall and you guys implemented it, which I think is one of the hardest scenarios for a lot of people who build an implement to find themselves in these days.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. It was kind of a scrum before scrum started taking off.  I mean, that\u2019s basically what it was. The only way it could have succeeded was very close collaboration with everybody on board with the message.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: So I guess coming back to that initial question, is it a little weird now to hear the stuff that you worked on turned into sort of the GoTo example slide in everyone\u2019s presentations about structured content and reuse?<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. It still freaks me out. It\u2019s great. I love seeing it. I love talking to people about it and if there\u2019s a way that I can help people, I love doing that. It is still a little surreal too, however, as I haven\u2019t even been at NPR for the last two and a half years. And, so it\u2019s interesting to still see it coming through the tweet stream once in a while.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: I can imagine. Well, you mentioned that you\u2019re now at Netflix actually, and you\u2019re the director of engineering for the Netflix API.  How is that different? I mean, it seems like it\u2019s an API and it deals with content, but it is really a big shift.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. It\u2019s a huge shift in certain categories and it\u2019s actually quite similar in others. The similarities are, we\u2019re both media companies.  But actually Netflix considers ourselves technology, company in media. I think NPR should be striving, if they are not already, for the same thing, as a technology company producing content for distribution, trying to get on multiple devices, reaching consumers with rich multimedia experiences.<\/p>\n\n\n\n<p>Those kinds of things are similar. The scale. is fundamentally different. NPR, I think is a reasonably decent scale operation. but again, by the time I left it, my team was, including some contractors, around 20, I think.  Here the engineering team is about 600.<\/p>\n\n\n\n<p>The scale of the APIs: I think the NPR API is however many millions of requests a day. Maybe it\u2019s a hundred, maybe it\u2019s 50. I don\u2019t remember the exact number. The API here does two and a half billion transactions a day. So what goes into those problems, you know, solving those problems?  It\u2019s a fundamentally different approach. And so contextually at NPR, it was even when I left, it was basically one team broken down into different groups, but focused on one pipeline and that pipeline was pretty interconnected. So, you have the content management system that publishes into a cluster of databases and the cluster of databases draw from an API, and the API distributes out to any number of destinations.  The NPR engineering team was building pretty much all of that. <\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Wow. 20 people or so! <\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Here it\u2019s highly distributed. It\u2019s an SOA model. lots of engineering teams focusing on specialized tasks.  And my team does not really store any data. We don\u2019t really have any editorial tools or anything like that. We\u2019re basically a broker that takes data from other people\u2019s systems and pass it across HTTP over to devices and people.  So the core responsibilities for this team is making sure that we have a solid distribution pipe, scaling the system effectively with the growth of the company and growth of the system and ensuring resiliency. Those are the three key responsibilities I played out for the team. Whereas NPR, it\u2019s building a lot of features and presentation layers, or managing a CMS. <\/p>\n\n\n\n<p>So yeah, the scale I think is really at the core fundamentally different, and that drives a lot of the differences in other categories.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Yeah. I think, at Netflix, you guys are responsible for what I think a double digit percentage of evening internet traffic or something, something like that.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. It\u2019s 33%, right?<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: That\u2019s definitely a statistic. Not many people could claim.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: I mean, there are all kinds of different ways that Netflix has massive scale.  That\u2019s one of them. The two and a half billion transactions a day, but we\u2019re also an 800 different device types. It\u2019s kind of mind boggling. When you think about some of these numbers.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: I am just blown away that there are that many kinds of devices to be on. I mean, I guess it makes sense, but it\u2019s just, it is staggering.<\/p>\n\n\n\n<p>When you think about that the device proliferation that we\u2019re seeing, is it really difficult to keep up with it?<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. And the scary part is it\u2019s not done. Your fridge is going to have a screen on it. Why not have Netflix there? Right. Basically this is the beginning of it, actually, in my view.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: So how has it been, how has it been different? Just philosophically, like at NPR? I think a lot of the case for the API is about very public distribution of a lot of different information. But with Netflix, it\u2019s different, it\u2019s basically you\u2019re API is part of a tool chain that allows you to provide a particular service to customers.<\/p>\n\n\n\n<p>Are there ripple effect differences in how the two get approached?<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: I actually think you\u2019ve mischaracterized NPR a little bit. Even by the time I left, I was starting to position it differently and I think that\u2019s still the case. So, Javaun Moradi, he\u2019s the PM there now for the API and the rest of the folks there.  I think they focus intently on internal consumption. They still have the public API and still use it as a distribution mechanism to the member stations, among other things. But an overwhelming percentage of the traffic to the API is from NPR properties. It\u2019s not from the general public domain. and then the next category would be the stations, and then the general public.<\/p>\n\n\n\n<p>So I think in that regard, it\u2019s very similar to Netflix. now that the percentages and the numbers are very different. So I think whatever their percentage is, 60, 70% at NPR.  For Netflix, 99.9 plus percent of the traffic is from Netflix-ready devices. And we still have a public API, but that\u2019s sees an incredibly small percentage of the traffic<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Then, I guess that is sort of a perception thing then.  I mean, I\u2019ve always heard a lot about the context of the public open API being there. And that\u2019s actually a question that I think a lot of people have as well, &#8220;if my business doesn\u2019t make sense for putting out a giant fire hose of all of my stuff to the world, how can I really leverage this stuff?&#8221; And I think that that\u2019s part of it. You don\u2019t have to think about it as just the all you can eat buffet of our stuff.  It can be used for internal purposes too.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Yeah. I\u2019d go a step further and I\u2019d say it&#8217;s the majority. In fact, the overwhelming majority of value from API use will be the internal consumption. and this is from my background at NPR and at Netflix, but also in talking to a lot of people in the space.<\/p>\n\n\n\n<p>I see other companies like Evernote and The Guardian and The New York Times. They all have similar pie charts where the overwhelming consumption is internal. So I actually think that we are seeing a shift in the marketplace towards internal consumption.  People are looking at their businesses differently.<\/p>\n\n\n\n<p>It\u2019s like, how can we get on all these different devices? Let\u2019s not worry as much about trying to piggyback on developers in their free time in their garage. And let\u2019s dedicate our resources to building these things on our own. How do we get there? Let\u2019s build an API so we can leverage the distribution rather than building one-offs all the time for either individuals, developers who are looking at dealing with the massive explosion of devices and channels.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: What would, what kind of advice would you have for them? What kind of major pitfalls would you tell them to steer clear of?<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Well, definitely steer clear of lots of one-off development and steer clear of thinking, depending on who you are and what your business is, steer clear thinking of yourself as not being a software company.<\/p>\n\n\n\n<p>I think that\u2019s, to me, that\u2019s the number one thing. If you can re-imagine whatever your business is and think of yourself as being a technology company, or at least partially a technology company, then you\u2019re going to dedicate the resources to that. And if you\u2019re dedicating resources to that, then you\u2019re gonna have smart people who are thinking about these problems in the right way.<\/p>\n\n\n\n<p>And I don\u2019t think there\u2019s a one size fits all approach for everybody, but I think if you have good people thinking about it, you\u2019re going to end up with highly leverageable content management or distribution channels. You\u2019re going to end up being much more nimble than you were otherwise.<\/p>\n\n\n\n<p>So it\u2019s probably sidestepping your question, but, I don\u2019t know how else to say it because even at Netflix, we have very clearly stepped away from trying to be one size fits all for reaching all of the platforms we\u2019re hitting.<\/p>\n\n\n\n<p>Our REST API used to be a one size fits all model that we use for quite a while, and it felt like the right thing to do.  But my view on that is that the API is a tool and when that tool runs its course, we need to move on to something that has greater pragmatic value.  So I wouldn\u2019t be beholden to any given technology. I\u2019d be beholden to smart technologists who you trust to make good decisions.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: And it sounds like an important part of that is having a really clear coherent grasp of what it is that you\u2019re trying to accomplish and what the longterm goals are too.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Absolutely. Yep. That goes right in with the commitment. If you\u2019re committed to this, then think about how this is going to play out in five years and start planning for that.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Well, I want to say thank you very much for joining us. it\u2019s been a pleasure and, I hope that we\u2019ll cross paths again in the future.<\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Absolutely. Thanks a lot, Jeff. I really appreciate it. And yeah, definitely.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Thanks for listening to insert content here. If you\u2019d like to catch up on our archives or keep up on our new episodes, visit us at dot com slash ideas slash podcasts slash insert content queue.<\/p>\n\n\n\n<p>And you can also visit us directly at <a href=\"https:\/\/www.insertcontenthere.com\" data-type=\"link\" data-id=\"https:\/\/www.insertcontenthere.com\">InsertContentHere.com<\/a>.<\/p>\n\n\n\n<p><strong>Extra Story: Daniel Jacobson<\/strong>: Me and another NPR engineer, when we were doing a modeling exercise, we really got caught up on what to call the story, the atom of the content management system, in the database. And we debated about this for way too long. And her stance was let\u2019s call it &#8220;page&#8221;, as in a web page or some page representation. And I wanted nothing to do with that because I thought that was too bound to a given presentation layer concept. I really wanted something way more generic. So I started throwing out ideas like &#8220;object&#8221; or something that really didn\u2019t have a whole lot of meaning, but was abstract.<\/p>\n\n\n\n<p><strong>Jeff Eaton<\/strong>: Totally abstract. <\/p>\n\n\n\n<p><strong>Daniel Jacobson<\/strong>: Exactly. Yeah. And we went around this debate time and time again, and ultimately what we decided on was &#8220;thing&#8221;. So the central table in the system, and I think it still is the case today is called &#8220;thing&#8221;. We did that specifically so that we could not be worried about if a story going to end up on a mobile app or on an IP radio, which don&#8217;t really have a concept of &#8220;page&#8221;.  It\u2019s just, here\u2019s this &#8220;thing&#8221;, we\u2019re distributing it out and that\u2019s it.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This podcast episode was published on Insert Content Here with Jeff Eaton on March 15, 2013 Daniel Jacobson recaps the history of NPR&#8217;s COPE, his work at Netflix, and the future of content APIs. Host: Jeff Eaton Guest: Daniel Jacobson, Netflix Mentioned in this episode: This episode&#8217;s transcript is in progress; the partial transcript below [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,33,65,60,81,34,62,64,61],"tags":[72,74],"class_list":["post-494","post","type-post","status-publish","format-standard","hentry","category-apis","category-content-management","category-create-once-publish-everywhere","category-engineering","category-interview","category-netflix","category-npr","category-org-culture","category-technology","tag-insert-content-here","tag-interview"],"_links":{"self":[{"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/posts\/494","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/comments?post=494"}],"version-history":[{"count":26,"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/posts\/494\/revisions"}],"predecessor-version":[{"id":603,"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/posts\/494\/revisions\/603"}],"wp:attachment":[{"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/media?parent=494"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/categories?post=494"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/danieljacobson.net\/blog\/wp-json\/wp\/v2\/tags?post=494"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}