Comments Off on Forbes : Why APIs Now Mean Business Transformation, Not Just Technology Infrastructure

By daniel_jacobson, March 23, 2016 12:12 pm

This article was originally written by Dan Woods and published on Forbes on March 23, 2016

Thirty-six years ago, my first consulting project was fixing an IBM 360 assembler language program that had broken because the behavior of one of the machine instructions had change subtly.  At that time, you could consider the definition of the IBM 360 assembler language an API to the hardware.

In the years that passed, the idea of the API has changed. We’ve moved from LPA libs, to JCL, to Microsoft DLLs, Java packages and Python libraries. The strict definition of the term “application programming interface” suggests that the API is a tool of abstraction and simplification. The application developer doesn’t want to want to have to worry about the details of how to send a text message from a mobile app or how to run an analytics routine in R or how to display a graphic using the D3 libraries.

Where is your API journey taking you?

Because of this legacy, APIs are most often seen as something deeply technical, and this impression lingers to this day. But as Google and Yahoo rose out of the wreckage of the dot com bust, as Facebook and Twitter were born, the technological essence of APIs is giving way to a more business-focused emphasis on enabling unfettered innovation.

The APIs of Google, Yahoo, Facebook, and Twitter in the mid and late 2000s were unusual in that they were public. These APIs did have a technological purpose. They abstracted the basic capabilities of publishers so that they became useful for developers. But the point and the value was unrestricted innovation. SAP had APIs decades before called BAPIs that allowed their massive ERP system to be controlled. The trick that the Internet darlings performed was to figure out that by making APIs public, by allowing self-service, a seething tribe of creative developers could put those APIs to use creating new ways to put the core services to work. John Musser, the founder of Programmable Web, was the prophet of this era.

For example, Google Maps was one of the first public APIs to be created, inspired in part because developers figured out how to use Google’s capabilities on their own in an unauthorized manner. Google quickly realized that this was a great idea, published APIs, and a flood of innovation and use of Google Maps followed. Yahoo, Twitter, Facebook, and Amazon followed the same pattern for many of their services.

So in this era, John Musser created a catalog of public APIs and Chet Kapoor of Apigee and other companies like Mashery, now part of Tibco, and others created products to allow everyone to get in the game of creating public APIs. Working with the CTO of Apigee, Greg Brail, and Daniel Jacobson, VP of Edge Engineering at Netflix, I co-authored APIs: A Strategy Guide (O’Reilly), a book about API strategy. (Disclosure: I have done work as an analyst and content marketer for Apigee, Intel, Tibco, Microsoft, and other companies who sell API technology. For a full list of my clients visit EvolvedMedia.com/clients.)

Forbes Daily: Join over 1 million Forbes Daily subscribers and get our best stories, exclusive reporting and essential analysis of the day’s news in your inbox every weekday.Sign Up

By signing up, you agree to receive this newsletter, other updates about Forbes and its affiliates’ offerings, our Terms of Service (including resolving disputes on an individual basis via arbitration), and you acknowledge our Privacy Statement. Forbes is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

The point of that book was to explain the notion of APIs from top to bottom, but also to explain how APIs had become far more than just technology infrastructure.

How APIs Have Become the Building Blocks of Business Transformation

Daniel Jacobson’s evolution of APIs at Netflix shows how APIs can become building blocks for business transformation. Jacobson is in charge of creating the API infrastructure to support product development. Netflix, like many other companies, had a set of “resource-based” APIs that exposed the core capabilities of the key operational systems. Using these APIs, you could get to the catalog information on Netflix and invoke services to play movies.

Jacobson and his team determined that the code they were writing to create Netflix clients for various families of devices such as iPhones, iPads, Android tablets, DVD players, and so on was far more complex than it needed to be. The problem was that resource-based APIs were built to expose the capabilities of key systems in a general way. That mean that the client code had to do all sorts of stuff like transform and combine data, connect information from a multitude of sources, and create data structures and navigation that matched the needs of the device.

Jacobson’s team realized that a new layer of APIs was needed, which they called experience-based APIs (see “How A Netflix Tech Innovation Can Unleash Creativity in Your Business”). This layer of APIs was created not by the teams in charge of the key operational systems, which created the general purpose resource-based APIs. Rather, the developers in charge of creating clients for all the devices used to access Netflix defined just the APIs needed for each family of devices. The experience-based APIs moved all the code for transforming and formatting data and adapting the resource-based APIs to the needs of the device into the experience-based API. Jacobson’s team found that this process could be accelerated by allowing the experience-based APIs to be created using a scripting language. Jacobson and his team packaged this idea as the Nicobar open source project.

APIs as a Unit of Digital Business Design

FinTech and other industries have taken the general pattern of experience-based APIs and adapted it to different circumstances. Consider these examples:

• Tradier has created a Brokerage-as-a-Service offering that is a combination of resource- and experience-based APIs. Tradier customers are able to embed the ability to research and trade stocks within their existing applications.
• Orchard Platforms has created a platform powered by APIs that normalizes access to lending marketplaces. Using this platform, large financial institutions can make loans on platforms like Prosper and Lending Club using automated underwriting.
• Xignite has created a set of APIs that normalizes access to a vast amount of financial data. Some of these APIs are resource-based, providing general access, and some are experience-based, focused on meeting a specific need.
• Bechtel has introduced huge amounts of efficiencies by creating a platform as well as resource- and experience-based APIs to enable the creation of mobile apps. On job sites around the world, instant access to information has dramatically reduced delays because workers don’t have to go to sheds to access computers.

The point to remember is that all of these APIs are both technical artifacts and units of digital business design. As technology infrastructure, both resource-based and experience-based APIs need to be supported by generic API gateway capabilities such as:

• Proxy support
• Authentication/Authorization
• SSL/TLS termination
• Encryption
• Logging
• Load balancing
• Routing
• Throttling
• Lightweight orchestration

But there is another business-focused process going on here as well: the process of developing the unit of business design. In other words, resource-based and especially experience-based APIs are not designed simply as technology artifacts, but as ways to enable a business purpose. The people involved in designing the desired digital business experience must be involved.

Enterprise-grade API management – the kind that can power digital transformation and drive a business – goes beyond an API gateway and supports an ecosystem of digital collaboration. Many types of software follow this path. The software starts out supporting specific, targeted functionality but ends up adding features because people want to collaborate and need supporting capabilities as the original function becomes more important.

When you are using APIs as a unit of digital business design, you need capabilities such as:

• Analytics about developers, operational performance, app performance, and business metrics
• A customized developer portal
• API monetization
• Multi-tenancy and support for high scale
• Global policy enforcement
• SDKs for all popular development environments to simplify the process of developing an API
• Support for adding application code and data to the API infrastructure to enable distribution of code and data
• A powerful transformation engine to speed the task of adapting resource-based APIs to experience-based APIs

It is perfectly possible to use APIs as a unit of business design without an enterprise-grade API management platform. The challenge then becomes creating enterprise capabilities when you need them and supporting them over time.

The API marketplace is evolving rapidly. Amazon recently released an API gateway product. Microsoft has one as well, and there are various types of open source toolkits and such. The functionality of many of these vendors’ products supports the notion that APIs are simply technology infrastructure.

But vendors like Apigee would argue that to really use APIs as a unit of digital business design, to accelerate the creation of business value that comes from apps that can be developed faster and that cost less to maintain because of practices like experience-based design, you need much more than an API gateway.

“APIs have gone well beyond just bits of technology – they are essentially the foundation for digital transformation,” said Apigee’s Kapoor. “We believe that businesses will need to use APIs and API management to support digital business initiatives or risk becoming increasingly irrelevant.”

The challenge in my view is the legacy that APIs have as a unit of technology infrastructure. The people who are buying solutions are often thinking small about APIs rather than thinking big. The challenge for anyone using APIs is to stop thinking of them only as technology and to start thinking about the results you want to achieve and how APIs can play a role in getting there faster.

Dan Woods is on a mission to help people find the technology they need to succeed. Users of technology should visit CITO Research, a publication where early adopters find technology that matters. Vendors should visit Evolved Media for advice about how to find the right buyers. See list of Dan’s clients on this page.

Comments Off on Interview : Daniel Jacobson on Ephemeral APIs and Continuous Innovation at Netflix

By daniel_jacobson, November 17, 2015 4:04 pm

This interview with Jerome Louvel originally appeared on InfoQ on November 17, 2015

Following his talk at the recent “I Love APIs” conference, InfoQ had the opportunity to interview Daniel Jacobson about ephemeral APIs, their relationship to experience-based APIs and when to consider them in your organization.

Daniel leads development of critical systems that are the front door of Netflix, servicing 1,000+ different device types and billions of requests per day. He also manages the Netflix playback experience which accounts for approximately one-third of Internet downstream traffic in North America during peak hours.

InfoQ: What is your current role at Netflix and your day-to-day responsibilities?

Daniel Jacobson: I run the edge engineering team which is responsible for handling all traffic for all devices around the world for signup, discovery and playback. On the playback side we are responsible for the functionality that supports the playback experience. The API side is responsible for handling the traffic directly from devices, fetching data from a broad set of mid-tier data services and then we broker the data back. Both teams are critical to success of Netflix because nobody can stream if playback is not available and nobody can stream if the API is not available.

InfoQ: Can you explain what Ephemeral APIs are all about and how different they are from the Experience APIs that you have proposed before?

DJ: Experience APIs are trying to handle an optimized response for a given requesting agent. That’s orthogonal to the ephemeral APIs. The experience API is more about the requesting pattern and the payload. Ephemeral API is more about the process of iterating and evolving the experience APIs.

Traditionally, APIs get set up to make it easier for the API provider to support, which results in one-size-fits-all APIs. The problem with that approach is that the API ends up being harder to use for a wide array of consumers. In other words, the optimization in that model is to make things easier for the few but harder for the many. For experience APIs, the goal is to focus on the needs of the individual requesters and optimize the APIs for each of them. It means that you are essentially running a wide array of different APIs. This results in a more challenging environment for the API provider to support because the variability is higher, but it allows the API consumers to develop what is best for them and for the performance of their clients. Ultimately, this should translate into a better customer experience.

Ephemerality is part of our story in how we develop our APIs, but not essential for the experience API model. Ephemeral APIs mean that the endpoints and payloads should be able to be terminated and created with ease and flexibility with the expectation that this can happen at any moment and potentially very frequently. If we can support ephemerality, then we can innovate faster and continuously to support the product needs without being a bottleneck.

To give an example, if we are running an A/B test to evaluate a new feature in our SmartTV experience, the UI team working on that feature can iterate on the client code and the APIs without the API team’s involvement. As they develop the test, they may realize that the data needs change or can be optimized, which would result in them killing the endpoints and create new ones. This can happen dozens of times over the course of the project and without the API team getting involved (as long as all of the data elements already exist in the pipeline).

InfoQ: What is the best way to find the right granularity for experience-based APIs? Is it mostly based on the device capabilities or on team organization?

DJ: I’ve written a detailed blog post on this topic in the past, which includes the recipe for when experience-based APIs might be a good choice. Basically, it is likely many companies don’t need to go this route because it’s a scale question.

So, if you have a wide array of different interaction models that are diverging and a close relationship with those who are consuming the APIs, those are good indicators that you might want to optimize for this. The proximity to the consumer of the API is key because you have a tighter feedback loop and more understanding of what their individual needs are.

The difference with generic resource-based APIs is that you don’t know who is going to consume the APIs and how they will be consumed. If the consumers are in your organization, and if you understand those nuances, you can create an architecture that is optimized for them all.

Within Netflix, we have created the architecture as a set of Java APIs and all these different device teams can build their own experience-based web APIs that are optimized for their clients. We like to call our system a platform for API development, more than an traditional API.

InfoQ: Do you have a separate API for Netflix mobile app on Android and on iOS?

DJ: In the construct of the platform, we have base Java APIs that are method calls within a JVM. Then, we have an adapter layer that sits on top of that where web APIs can be developed in a device-specific way. So, we have mobile teams developing their corresponding adapters, those are different endpoints, request patterns, payloads and maybe different protocols.

There used to be more overlap between iOS and Android, but now these experiences are indeed different. There are shared functions across all of this so we built a set of tools to allow for the shareability.

InfoQ: Do you rely on an API language to describe Netflix APIs?

DJ: Not at this point. This is something we discuss periodically, but have not pursued yet because of the challenges and costs in maintaining them. Most of the time, if you have language descriptors it means that you are trying to fix things in place, make them consistent for the API consumers. Because our web APIs are ephemeral, the descriptor would also need to be ephemeral, so using one would cost more and not be as helpful.

But another thing is you have many teams building these web APIs with different needs and those teams are iterating on their consumption of the web APIs. This iteration is happening continuously because we are always running A/B tests that require changes to the data being delivered. As the teams iterate, the same person or group is writing and consuming the web API and they are doing the development of both at the same time, which means they already know the nature of the interface, so there is no value.

Most of the discussion for description languages have been at the Java API level, but again, those APIs are changing frequently as well. If we can find a way to describe those APIs consistently at very low cost, we would like to add that to the system, but so far it seems as though the costs of maintenance exceed the benefit.

InfoQ: Do you rely on API tooling to accelerate the development of APIs by device teams?

DJ: We develop a suite of tools to allow people to manage, deploy, and view the health of their API scripts, and to determine which endpoints are active and not. We also have tools to support shareability of code around these scripts and we have tools to inspect the payloads. Also, there are tools that we still need to develop. For example, the difficulty in this world is debuggability and we need to improve in this area.

InfoQ: How does your move to Universal JavaScript for your main web site fit into the experience-based API platform?

DJ: The architecture and API for the web site team is different than most devices because they have a separate tier fronting their API calls. For typical devices, they call directly into the web API but for the web site, they call into their own cluster where they handle the traffic directly and then call into our API cluster to get the data. What’s happening in their cluster and above it is currently outside our view but they are still writing scripts in our adapter layer.

What’s interesting is that we are investigating now if we should apply similar constructs across the breadth of devices or some subsets, and evaluating the cost of doing this more broadly. Some things that we might gain in this approach would be process isolation and an easier path towards debuggability.

InfoQ: What is the place of Groovy and other scripting languages in the Netflix API platform?

DJ: Groovy is the only language in our API environment that people are writing adapter scripts with, but we are looking at other languages. The next one is likely going to be Node.js. Going to another JVM language would be easier, but there hasn’t been enough interest so far. If device teams want to use Scala or other languages, we would need to do more investigation and work to make it happen.

Node.js is not going to run integrated in the JVM so it’s an additional benefit of isolating that into another layer like we’ve done for the main web site.

InfoQ: How were the device teams able to adapt to such changes in their development flows?

DJ: The cultural change to the company was a lot harder than the technology changes. Even with teams willing to go to this route, there were some challenges in getting people to think and operate differently in the new environment. For example, it took some time for them to adapt to writing Groovy and to the functional programming paradigm. But looking back it is definitely a net win.

InfoQ: In your talk, you mentioned an ongoing project to introduce containers at the API adapter layer. Will that effort have impact on the Nicobar open source project?

DJ: As we are investigating containers for the web site layer, we are thinking about how it could be applied to other devices as well. For the container-based model, Nicobar would not be a central player for us. In fact, when we designed Nicobar and the scriptability, it was in part to deploy the scripts in an isolated way. Containers take our original intent to the next level and obviates away the need for Nicobar. That said, our system will continue to support the scripting and Nicobar for years to come, so we expect to continue to develop and evolve Nicobar for a while. As Nicobar evolves, it is likely that such changes will be made in the open source project as well.

InfoQ: The Netflix Falcor open source project was announced in August and its usage on Android recently explained. What does it offer and how does it relate to your broader API platform?

DJ: It helps us represent remote data sources as a single domain model through a virtual JSON graph. You code the same way no matter where the data is, whether in memory on the client or over the network on the server. Falcor also handles network communications between devices and servers, and can batch and deduplicate requests to make them more efficient.

Because Falcor is a more efficient data fetching mechanism between devices and servers, it’s going to continue to play a significant role in our platform even as our system evolves into a different architecture.

The main benefits we get out of Falcor are developer efficiency and improved application performance. We get the developer efficiency because the access patterns for the engineers writing the adapters is more consistent. That said, there is a steeper learning curve to use Falcor and it is a more challenging environment to debug.

InfoQ: What are the limitations that you found with AWS Auto Scaling Groups and how does Netflix Scryer help? Will it become open source?

DJ: AWS autoscaling is used widely at Netflix. It’s very useful and powerful. Amazon is responding to metrics like load average, determining that it’s time to add new servers when those metrics pass a certain threshold. Meanwhile, it can take 10 to 20 minutes to bring a new set of servers online. A lot of bad things can happen in a manner of minutes, so that adds risk to our availability profile.

That’s what prompted us to develop Scryer. What Scryer does is it looks at the historical data and incorporates a feedback loop of real-time data, evaluates what the needs will be in the near future for capacity, and then it adds servers in advance of that need. What we see is that response times and latencies are much more leveled with Scryer because load averages are not spiking and because the cluster can handle the traffic more effectively.

While we announced it via a blog post a couple of years ago, there is no plan right now to open source it.

InfoQ: Netflix Engineering is well known for its Chaos Monkey service. Can you tell more about other services that are part of your Simian Army?

DJ:There is a suite of monkeys that do different things. Here are some of these services:

• Latency Monkey has various degrees of utility and was designed to inject errors and latencies into a service to see how the failure would cascade. That has since evolved into FIT (Failure Injection Testing).
• Chaos Gorilla is similar to Chaos Monkey but instead of killing individual instances, it is killing AWS availability zones. The idea here is to test high availability across zones by redirecting traffic from a failed zone to a healthy one.
• Conformity Monkey and Security Monkey make sure that builds conform to certain operational and security guidelines and shuts down those that are not confirming.
• Janitor Monkey which will cleanup unhealthy or dead instances.
• Chaos Kong is a recent addition to the army, which simulates and outage in an entire AWS region and pushes traffic to a different region.

InfoQ: Over the years, Netflix has launched many open source projects. What is the best way to know what is available and actively maintained, to take advantage of these contributions?

DJ: As our OSS strategy has evolved, we’ve released around 60 projects in total across a diverse set of categories including UI, cloud and tools. Some of them are more actively managed than others and we try to partition them in our developer website. Supporting the APIs directly, there are a range of tools including Zuul, Nicobar, Histrix and RxJava.

InfoQ: Should a company new to APIs start with a one-size-fits-all API approach and progressively evolve like Netflix did, or start immediately with finer-grained ephemeral experience APIs?

DJ: If you are brand new to APIs, start with OSFA (one size fits all). There is a question of whether you will ever get to the scale needs that Netflix has. Experience APIs are more of a challenge. I believe that ephemerality should be part of the mindset of each company, regardless.

Going the experience based API route is a function of opportunity and cost. You are adding more overall cost, but the efficiency and the optimization gains might be worth it. If you only have a few devices or very small development team or if you have a wide range of external parties that consume APIs, the cost of operating this more variable environment would likely not be recovered.

You really need to have a tipping point where the development efficiency of the API consumers is hindered by the fact that they are fighting against the rigid API. In other words, if you have different device teams, that have to make inefficient API calls that are different from each other and they have to compensate by doing additional parsing, error handling, etc. then the cost of all of that added energy can potentially be obfuscated by creating an optimized interaction model. This benefit is only worth it if you have enough developers doing these inefficient activities.

InfoQ: In addition to developer efficiency, are there other benefits that you might be looking for with Experience APIs?

DJ: With an optimized set of APIs, you are building a solution to provide a better experience for the customer, such as improved system performance and improved velocity in getting changes into the product.

If you want to have this kind of ephemerality and optimization, you can’t set it up for public APIs. The experience APIs are excellent tactics but are geared towards private APIs because having a close relationship with a small set of developers allows you to have much more latitude in solving the needs of the API consumers.

InfoQ: What excites you the most right now about the API space?

DJ: We are most excited about things like containers, streaming data, HTTP 2.0, websocket and persistence connections, tooling and analytics behind supporting a massive scale API. So we are investigating in those kind of things and experimenting.

Other things are emerging in this space like microservices, continuous integration, continuous deployment, and we are already doing them. At Netflix, we have a distributed architecture with specific functions for each microservice. But successful microservices inevitably grow in scope, potentially causing it to become more of a monolith over time. At that point, it makes sense to start breaking things down again.

InfoQ: Finally, how does continuous deployment relate to ephemeral APIs?

DJ:I often describe my team as being the skinny part of the hourglass that’s pushing data back and forth between the two fat parts. In one of the fat parts is all of the API consumers, the UI and devices teams. On the other fat part we have all the distributed server-side microservices. Both of the fat parts are constantly changing (A/B testing, new features, new devices, etc.).

As those change, we need to ensure that data is flowing through the skinny part to support the product and any test that is being performed on the product. We need to change at a faster rate than the rest of the company because we need to handle the changes that many other teams make.

Several years ago we decided the only way to do this was to develop a fully automated deployment pipeline. From a continuous deployment perspective, it was important for us to be able to deploy rapidly, frequently, at low risk and with the high ability to quickly rollback. The goal behind all of that is that we should not be the bottleneck to getting product change to the customer.

Like other things my team does, continuous deployment is a means to an end. And the end is continuous innovation. Having an environment that can rapidly and constantly change to the need of the business and the customer ties back to our ephemerality mindset.

Comments Off on Review of Rush’s 40th Anniversary Show – San Jose

By daniel_jacobson, July 27, 2015 1:16 pm

Rush in San Jose

As my musical tastes have evolved over the years, one group has pervaded throughout: Rush.  My interest to them grew in parallel with my drumming, at least in my formative drumming years, and that attachment has persisted.  That said, as much as I love Rush, it has been a long time (Roll the Bones tour in 1991) since my last Rush show.  In fact, I didn’t really anticipate seeing them ever again, mainly because I tend to see live music at smaller, more intimate venues, but also because I just haven’t connected with anything they have done in the past 20 years (since Counterparts).  But this tour felt different to me.  This one, I needed to go to.  The Rush 40th Anniversary Tour is likely their last and one that celebrates an incredible run by this quietly influential band.  Moreover, their concert was going to cycle backwards through their copious collection of songs (roughly 170 in total, depending on how you count some of them), touching on Rush-nerd classics such as Natural Science and Jacob’s Ladder.  Again, this felt like a must attend event.

Show Duration

First of all, I do want to highlight the fact that the show had no opening band and lasted over three hours from start to finish.  Not bad for a trio of 60+ year olds, huh?

The show was scheduled to start at 7:30pm.  It actually kicked off at 7:45pm.  There was a fever pitch in the arena starting at about 7:20pm, lasting all the way until the opening video started off the show.  Any time the lights dimmed a little during the time leading up to the start of the show, the crowd noise would swell with anticipation, only to settle back down until the next false alarm.

Once they started, the first set lasted a little more than an hour, during which they played 10 songs, spanning the albums Clockwork Angels through Signals.  The band then took a “short break to rejuvinate” lasting about 20 minutes or so, after which they played another 16 songs (keep in mind that some of these songs lasted 10 or more minutes).  The show closed at 10:50pm.

Stage Design

Simple Stage Design for R40

Unlike previous Rush concerts that I have been to, this stage design was super clean and simple.  There we no big hats with rabbits in them, for example, bouncing around the stage at various points in time.  Most of the stage floor was open and flat, with just a big yellow R40 in the middle near the front.  There were some simple stage affects (e.g. popcorn machine, laundry machines, fake Marshall stacks, etc.) to either side of the drum set that we bordering on goofy, but I found them largely unoffensive.  It was also a bit entertaining to see the guys in orange R40 suits walking across the stage in the open to shift around those affects during the song.  I also appreciated the apparent obliviousness of the band members to these moments which were clearly designed to be there.

The biggest offense in the stage props came in the form of two crew members dressed in a horse suit who then marched across the stage holding a sign that read “Hey”, begging the audience to cheer the word in those empty spaces towards the end of the 2112 Overture.  I mean, seriously guys, this is your 40th anniversary tour.  The place is filled with Rush geeks who know what you expect.  A goofy horse is not necessary…

Behind the drum kit was a large screen flanked by two tall and narrow screens.  These screens were used to present a range of videos between and during songs.  More on this later, but I think it was classy, out of the way of the actual performance, and generally enhancing to the show.

Energy

This was the show of two sets.  In the first set, I will sum it up by saying that I felt like Geddy was thrilled to be there, Neil was thrilled to just be playing the drums, and Alex was just thrilled to be alive.

To be more specific, Geddy was bouncing around the stage, deftly managing his significant workload while occasionally doing his awkward duck walk.  His enthusiasm definitely could not have been mistaken.  Meanwhile, Neil was behind the kit with a typical stoic expression.  While he played pretty proficiently throughout the night, he just seemed hunched over the whole time like this tour was weighing him down.  I got the sense that he would be more happy just playing the drums back in their garage in Toronto.  Alex, well, he was very stationary for the first set and there was little evidence that he was even awake.  It really did feel like a two person band for most of the first set.  This is particularly surprising given that Alex is the youngest of the three – he should have the most energy.

Alex looking lifeless
(courtesy of Jeff Butsch)

Between sets, however, they took a 20-minute break.  I have no idea what they did back stage, but Alex was a different player afterwards.  Maybe he got an injection or perhaps he was uninspired by the more recent material (similar to the sentiment of the majority of the crowd, I presume)…  Whatever it was, the second set put the “life” back into Lifeson.  No change in energy or performance for the other guys, for the most part.

Neil

Neil definitely showed his age.  Don’t get me wrong, he is still a great rock drummer and he generally played well, showing the poise of someone who has nearly 50 years experience on his instrument.  I have two major critiques of him in this show though.  First, and more fundamentally, he continues to play most of the parts as he crafted them on the albums.  Granted, the parts are well-constructed.  But he has the ability and creativity to improvise or vary things up more.  It would be great to see the fills in Tom Sawyer and YYZ, for example, played differently than their first incarnation nearly 35 years ago.  The second issue is that he simply made a bunch of mistakes during the show.  I don’t mean mistakes like he played the wrong part.  I mean, he would occasionally stumble and miss the one.  The most obvious occasion of such a misstep was in the triplet-based fill leading into the guitar solo of Tom Sawyer – he must have played this a million times correctly in the past, yet this was a big flub.  Geddy and Alex played on and Neil caught up again, so no real harm.  I note this because it is really uncharacteristic of an otherwise very precise player repeating the parts he has always played.  To be clear, this didn’t really adversely affect my enjoyment of the show.  I just noted it as an oddity, one that is likely a result of an aging player who has been touring on and off for 40+ years.

Neil played a mini-solo in the first set.  He played a larger one in the second set which split the uprights between the beginning and the end of Cygnus X-1 (with the middle of the song getting excised completely).  I don’t have much to say about the solos aside from the fact that he played them well and that I am a little disappointed that the majority of both solos consist of parts that I have heard many times before.  There were definitely nuggets in both that were exceptional though.

Instruments

I loved Neil’s drum kit choices!  He ditched the circular drum set with all of the extraneous pieces in favor of his older and more traditional configurations.  Of course, the different eras have different percussive needs, so in exchange for the circulating kit Neil had his crew swap out kits during breaks so he can have the appropriate apparatus based on where the set was in the 40-year history.  For example, the kit he played after the big break included the chimes and bells for Xanadu and Closer to the Heart.  It was refreshing to see him get back to some of the roots and to match the simplicity of the stage design with the simplicity in his drum sets (with “simplicity in his drum sets” being a relative phrase).

Geddy with four instruments: vocals, keys, bass, guitar
(courtesy of Jeff Butsch)

The other really noteworthy detail about the instruments is that both Geddy and Alex pulled out their double-neck guitars for Xanadu.  Those things look great!

Song Choice

First, coming up with a set list for these shows must have been a very difficult task to take roughly 170 songs or so and boiling them down to a set list that would cover about two hours and forty five minutes worth of material.  There are the obvious songs that you would expect, like Spirit of Radio, Closer to the Heart, Tom Sawyer, and Subdivisions.  But I was thrilled that they pulled out some others that cater to the more committed fan, like 2112 and part of Hemispheres.  Songs that surprised me that they didn’t play include Limelight, Show Don’t Tell, and The Trees, I would have gladly forsaken any of these in favor of La Villa Strangiato, Vital Signs, and By-Tor.  All in all, a reasonable set list given the concept they were shooting for.  That said, if I were them, I probably would have trimmed the bookends and had the entire set be material from between Fly By Night and Roll the Bones, giving them more time to fill out the set with their stronger material.  Again, just my opinion.

Video Interludes

There were a bunch of videos that they played throughout the show.  Overall, I found them to be a lot of fun and a good break from the music.  It was great to see the cameos from a range of comedians like Paul Rudd, Jason Segal, Eugene Levy and Jerry Stiller.  I like how they didn’t take themselves too serious during the interludes – they were able to laugh at themselves a bit.  Good stuff.  The videos during the music were also interesting at times, but I found myself not paying attention to them during the actual performance.  Maybe this is because I was sitting off to the side more.

All in all, this was a great show.  With the ticket prices what they were, I feel like I did my part in sending this under-appreciated band out to pasture… but I don’t feel bad about spending that money at all.  The material is as great as ever and they played it well and with dignity.

Thanks for all the memories, Rush!

Comments Off on Forbes : How A Netflix Tech Innovation Can Unleash Creativity in Your Business

By daniel_jacobson, February 11, 2015 5:59 pm

This article was originally published by Dan Woods on Forbes on February 11, 2015

In the tech community, Netflix is a company that everyone learns from. But what is less widely celebrated are the broader business lessons of what Netflix has accomplished. I was bonked over the head by this point when I sat through a presentation about Experience-based APIs by Daniel Jacobson, VP of Edge Engineering for Netflix.

Jacobson and his team that builds and maintains the Netflix API platform have developed a way of working that solves a problem that is common in any company that is building its own technology: How do you best leverage the skills of everyone on a team so you can move as fast as possible? Jacobson and his team have figured this out for a general use case: Building many different applications based on a complex set of centralized resources. The structure of this solution, which relies on the idea of separation of concerns between different groups of developers, can help optimize the structure of many different types of teams.

For those who work on building software, Jacobson has embedded in the Nicobar open source project the technical capabilities needed to support for the separation of concerns and rapid innovation. I will describe briefly the Nicobar project, which was released yesterday, at the end of this article.

What are Experience-based APIs?

To understand the beauty of Experience-based APIs, we first must understand the challenge facing Netflix with respect to supporting many devices. Netflix client developers write software applications that allow Netflix to deliver its content over thousands of different devices types. Netflix has many teams of client developers, each of which specialize in creating applications for different types of devices. The goal of Jacobson’s team and the client development teams is to deliver a great user experience and make sure that Netflix works perfectly on any device you might use, from an iPad, to an Android phone, to a DVD player or smart TV.

In the Resource-based API platform the work gets done this way:

• The central API team creates APIs that make it easy to get to all of the data and resources of a company. These APIs are all about creating a general way for programs to get access to customer data, data about films, recommendations, and so on. The resources that a client program may need include data and services for:
  • User info
  • Movie metadata
  • Movie Ratings
  • Similar movies
  • My list
• The central team implements the APIs to deliver scalability and excellent performance.
• The client team takes these APIs and creates applications for a variety of devices. The problem is that the code for handing errors and adapting the data to the needs of the device all resides in the client application code.

The Resource-based API paradigm works just fine and is used all over the world. But Jacobson and his team wanted to find a way to move faster, create better software, and enable more optimized experiences for Netflix subscribers. They realized that different devices potentially needed a different set of data to support the user experience. The best way to make the client software simpler would be to have a new API that allowed as much as possible of the data translation, error handing, and other utility functions to happen on the server, not on the client.

But if you asked the central API team to create versions of APIs for all the different families of devices, the team would be overwhelmed. In addition, it would be hard for each of the client teams to actually ask for the right design for the APIs they would need. It would require trial and error and that would be hard to accomplish with many client teams competing for the attention of the central team.

Jacobson and his team realized they needed to create a new layer of API they called an Experience-based API. The idea is that the client team should design the perfect API to serve the needs of the device on which the client application will run.

For example, when writing the application code to create a page on the client using the resource-based approach, the developer may have to call a dozen APIs, handle errors from all of them, and then format the data for use on the page. All of this communication takes place over HTTP, which is less efficient than the networking methods used on the server.

But if the client developers could create their own API to support the needs of that page, and move all of the code for data access from the client to the server, lots of good things would happen such as:

• Client code would get simpler.
• More code could be shared across families of devices.
• New device families could be supported faster in a way that best fit the needs of the device.
• Performance could improve because network communication is optimized, reducing round trips from the device to the server.

So Jacobson and his team set out to create a new layer to enable client developers to create their own APIs. Here’s how it works:

• Client developers analyze the application that they want to build for a particular client and then decide on the APIs that they will need to support them.
• They define the endpoints of the APIs and protocols to handle the requests and responses that are optimized for their client
• The APIs are implemented by the client developers using the Groovy scripting language which is used to call Java APIs that will gather the data needed to handle the request. Groovy is easier for client developers to work with than Java but still is scalable enough.

This architecture, in the case of Netflix powered by Nicobar, allows the client teams to move faster and reduce the technical debt by making the client programs simpler. The code enabled by Nicobar is more shareable and easier to maintain.

“There are some significant engineering benefits to this structure, but the real win is increasing the velocity of innovation,” said Jacobson. “Because Experience-based APIs properly separate concerns across the two teams, the work gets done faster. The client teams control their own velocity and the server teams can focus on building the platform that supports them.”

Lessons for the Rest of Us

Jacobson, Greg Brail, Chief Architect of Apigee, and I were co-authors of “APIs: A Strategy Guide”. During the time we wrote the book in 2010, the world was focused on the impact that public APIs were having. Google, Facebook, Twitter, and Amazon all harnessed massive amounts of attention and energy as people put their public APIs to work.

At that time Jacobson argued that internal use of APIs would have the far larger impact. He saw that few companies, even Netflix, would be able to get a large amount of value from public innovation. Instead, the real victory would come as internal APIs increased efficiency, productivity, and the velocity of innovation. Experience-based APIs are one major proof point for Jacobson’s way of thinking.

I believe this two-tier structure that allows those closest to the problem to translate the Resource-based APIs to a more suitable form and create APIs that are adapted to their needs can be used in all sorts of companies who how have bottlenecks in their IT departments.

To make this work you need a front office IT staff, one that works with the business. These are the equivalent of the client teams at Netflix. The back office IT provides Resource-based APIs to the front office teams who then use simple tools, scripting languages, model driven development, content management systems, and so on to adapt the resources to the needs of the business. In large Wall Street Firms it is common to have highly skilled front office IT teams. I think that many companies could benefit from this model if they choose the right tools and create the right separation of concerns.

Nicobar 

For companies who create -multiple applications and are committed to the Java ecosystem, Nicobar is definitely worth a look. This library provides the core functionality to implement the Experience-based API program. Right now the Groovy language is the way that the adapter code to assemble the data needed for the Experience-based is written, but in the future other JVM languages may be supported.

The key capabilities of Nicobar are:

• Rapid application delivery via dynamic updates of components
• Ability to add support for multiple JVM languages

To learn more go to Nicobar: Dynamic Scripting Library for Java.

And, for the truly nerdy, here’s what the project wiki says about why the project was called Nicobar:

“Nicobar refers to a remote, archipelagic island chain in the Eastern Indian Ocean. It is aUNESCO World Biosphere Reserve, and like many other remote volcanic islands, it has rich biodiversity, and various endemic species are found here. Taken together with the surrounding Andaman group of islands, six different human tribes occupy these islands, each with its own spoken language. We felt that these isolated islands, evolving independently are a good metaphor for the polyglot, modular runtime framework we were setting out to build.”

Dan Woods is on a mission to help people find the technology they need to succeed. Users of technology should visit CITO Research, a publication where early adopters find technology that matters. Vendors should visit Evolved Media for advice about how to find the right buyers. See list of Dan’s clients on this page.

Comments Off on Presentation : Maintaining the Netflix Front Door – Intuit Meetup

By daniel_jacobson, May 23, 2014 8:10 pm

This presentation was for a meetup at Intuit on May 23, 2024

This presentation goes into detail on the key principles behind the Netflix API, including design, resiliency, scaling, and deployment. Among other things, I discuss our migration from our REST API to what we call our Experienced-Based API design. It also shares several of our open source efforts such as Zuul, Scryer, Hystrix, RxJava and the Simian Army.