This article was originally written by Dan Woods and published on Forbes on March 23, 2016

Thirty-six years ago, my first consulting project was fixing an IBM 360 assembler language program that had broken because the behavior of one of the machine instructions had change subtly.  At that time, you could consider the definition of the IBM 360 assembler language an API to the hardware.

In the years that passed, the idea of the API has changed. We’ve moved from LPA libs, to JCL, to Microsoft DLLs, Java packages and Python libraries. The strict definition of the term “application programming interface” suggests that the API is a tool of abstraction and simplification. The application developer doesn’t want to want to have to worry about the details of how to send a text message from a mobile app or how to run an analytics routine in R or how to display a graphic using the D3 libraries.

Where is your API journey taking you?

Because of this legacy, APIs are most often seen as something deeply technical, and this impression lingers to this day. But as Google and Yahoo rose out of the wreckage of the dot com bust, as Facebook and Twitter were born, the technological essence of APIs is giving way to a more business-focused emphasis on enabling unfettered innovation.

The APIs of Google, Yahoo, Facebook, and Twitter in the mid and late 2000s were unusual in that they were public. These APIs did have a technological purpose. They abstracted the basic capabilities of publishers so that they became useful for developers. But the point and the value was unrestricted innovation. SAP had APIs decades before called BAPIs that allowed their massive ERP system to be controlled. The trick that the Internet darlings performed was to figure out that by making APIs public, by allowing self-service, a seething tribe of creative developers could put those APIs to use creating new ways to put the core services to work. John Musser, the founder of Programmable Web, was the prophet of this era.

For example, Google Maps was one of the first public APIs to be created, inspired in part because developers figured out how to use Google’s capabilities on their own in an unauthorized manner. Google quickly realized that this was a great idea, published APIs, and a flood of innovation and use of Google Maps followed. Yahoo, Twitter, Facebook, and Amazon followed the same pattern for many of their services.

So in this era, John Musser created a catalog of public APIs and Chet Kapoor of Apigee and other companies like Mashery, now part of Tibco, and others created products to allow everyone to get in the game of creating public APIs. Working with the CTO of Apigee, Greg Brail, and Daniel Jacobson, VP of Edge Engineering at Netflix, I co-authored APIs: A Strategy Guide (O’Reilly), a book about API strategy. (Disclosure: I have done work as an analyst and content marketer for Apigee, Intel, Tibco, Microsoft, and other companies who sell API technology. For a full list of my clients visit EvolvedMedia.com/clients.)

https://www.google.com/recaptcha/api2/anchor?ar=1&k=6LeLC2wpAAAAAHxdEQ59hxseRouCtGrCo22PUjuu&co=aHR0cHM6Ly93d3cuZm9yYmVzLmNvbTo0NDM.&hl=en&v=WV-mUKO4xoWKy9M4ZzRyNrP_&size=invisible&cb=fkbl3y6x75i2

Forbes Daily: Join over 1 million Forbes Daily subscribers and get our best stories, exclusive reporting and essential analysis of the day’s news in your inbox every weekday.Sign Up

By signing up, you agree to receive this newsletter, other updates about Forbes and its affiliates’ offerings, our Terms of Service (including resolving disputes on an individual basis via arbitration), and you acknowledge our Privacy Statement. Forbes is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

The point of that book was to explain the notion of APIs from top to bottom, but also to explain how APIs had become far more than just technology infrastructure.

How APIs Have Become the Building Blocks of Business Transformation

Daniel Jacobson’s evolution of APIs at Netflix shows how APIs can become building blocks for business transformation. Jacobson is in charge of creating the API infrastructure to support product development. Netflix, like many other companies, had a set of “resource-based” APIs that exposed the core capabilities of the key operational systems. Using these APIs, you could get to the catalog information on Netflix and invoke services to play movies.

Jacobson and his team determined that the code they were writing to create Netflix clients for various families of devices such as iPhones, iPads, Android tablets, DVD players, and so on was far more complex than it needed to be. The problem was that resource-based APIs were built to expose the capabilities of key systems in a general way. That mean that the client code had to do all sorts of stuff like transform and combine data, connect information from a multitude of sources, and create data structures and navigation that matched the needs of the device.

Jacobson’s team realized that a new layer of APIs was needed, which they called experience-based APIs (see “How A Netflix Tech Innovation Can Unleash Creativity in Your Business”). This layer of APIs was created not by the teams in charge of the key operational systems, which created the general purpose resource-based APIs. Rather, the developers in charge of creating clients for all the devices used to access Netflix defined just the APIs needed for each family of devices. The experience-based APIs moved all the code for transforming and formatting data and adapting the resource-based APIs to the needs of the device into the experience-based API. Jacobson’s team found that this process could be accelerated by allowing the experience-based APIs to be created using a scripting language. Jacobson and his team packaged this idea as the Nicobar open source project.

APIs as a Unit of Digital Business Design

FinTech and other industries have taken the general pattern of experience-based APIs and adapted it to different circumstances. Consider these examples:

• Tradier has created a Brokerage-as-a-Service offering that is a combination of resource- and experience-based APIs. Tradier customers are able to embed the ability to research and trade stocks within their existing applications.
• Orchard Platforms has created a platform powered by APIs that normalizes access to lending marketplaces. Using this platform, large financial institutions can make loans on platforms like Prosper and Lending Club using automated underwriting.
• Xignite has created a set of APIs that normalizes access to a vast amount of financial data. Some of these APIs are resource-based, providing general access, and some are experience-based, focused on meeting a specific need.
• Bechtel has introduced huge amounts of efficiencies by creating a platform as well as resource- and experience-based APIs to enable the creation of mobile apps. On job sites around the world, instant access to information has dramatically reduced delays because workers don’t have to go to sheds to access computers.

The point to remember is that all of these APIs are both technical artifacts and units of digital business design. As technology infrastructure, both resource-based and experience-based APIs need to be supported by generic API gateway capabilities such as:

• Proxy support
• Authentication/Authorization
• SSL/TLS termination
• Encryption
• Logging
• Load balancing
• Routing
• Throttling
• Lightweight orchestration

But there is another business-focused process going on here as well: the process of developing the unit of business design. In other words, resource-based and especially experience-based APIs are not designed simply as technology artifacts, but as ways to enable a business purpose. The people involved in designing the desired digital business experience must be involved.

Enterprise-grade API management – the kind that can power digital transformation and drive a business – goes beyond an API gateway and supports an ecosystem of digital collaboration. Many types of software follow this path. The software starts out supporting specific, targeted functionality but ends up adding features because people want to collaborate and need supporting capabilities as the original function becomes more important.

When you are using APIs as a unit of digital business design, you need capabilities such as:

• Analytics about developers, operational performance, app performance, and business metrics
• A customized developer portal
• API monetization
• Multi-tenancy and support for high scale
• Global policy enforcement
• SDKs for all popular development environments to simplify the process of developing an API
• Support for adding application code and data to the API infrastructure to enable distribution of code and data
• A powerful transformation engine to speed the task of adapting resource-based APIs to experience-based APIs

It is perfectly possible to use APIs as a unit of business design without an enterprise-grade API management platform. The challenge then becomes creating enterprise capabilities when you need them and supporting them over time.

The API marketplace is evolving rapidly. Amazon recently released an API gateway product. Microsoft has one as well, and there are various types of open source toolkits and such. The functionality of many of these vendors’ products supports the notion that APIs are simply technology infrastructure.

But vendors like Apigee would argue that to really use APIs as a unit of digital business design, to accelerate the creation of business value that comes from apps that can be developed faster and that cost less to maintain because of practices like experience-based design, you need much more than an API gateway.

“APIs have gone well beyond just bits of technology – they are essentially the foundation for digital transformation,” said Apigee’s Kapoor. “We believe that businesses will need to use APIs and API management to support digital business initiatives or risk becoming increasingly irrelevant.”

The challenge in my view is the legacy that APIs have as a unit of technology infrastructure. The people who are buying solutions are often thinking small about APIs rather than thinking big. The challenge for anyone using APIs is to stop thinking of them only as technology and to start thinking about the results you want to achieve and how APIs can play a role in getting there faster.

Dan Woods is on a mission to help people find the technology they need to succeed. Users of technology should visit CITO Research, a publication where early adopters find technology that matters. Vendors should visit Evolved Media for advice about how to find the right buyers. See list of Dan’s clients on this page.

This article was originally published by Dan Woods on Forbes on February 11, 2015

In the tech community, Netflix is a company that everyone learns from. But what is less widely celebrated are the broader business lessons of what Netflix has accomplished. I was bonked over the head by this point when I sat through a presentation about Experience-based APIs by Daniel Jacobson, VP of Edge Engineering for Netflix.

Jacobson and his team that builds and maintains the Netflix API platform have developed a way of working that solves a problem that is common in any company that is building its own technology: How do you best leverage the skills of everyone on a team so you can move as fast as possible? Jacobson and his team have figured this out for a general use case: Building many different applications based on a complex set of centralized resources. The structure of this solution, which relies on the idea of separation of concerns between different groups of developers, can help optimize the structure of many different types of teams.

For those who work on building software, Jacobson has embedded in the Nicobar open source project the technical capabilities needed to support for the separation of concerns and rapid innovation. I will describe briefly the Nicobar project, which was released yesterday, at the end of this article.

What are Experience-based APIs?

To understand the beauty of Experience-based APIs, we first must understand the challenge facing Netflix with respect to supporting many devices. Netflix client developers write software applications that allow Netflix to deliver its content over thousands of different devices types. Netflix has many teams of client developers, each of which specialize in creating applications for different types of devices. The goal of Jacobson’s team and the client development teams is to deliver a great user experience and make sure that Netflix works perfectly on any device you might use, from an iPad, to an Android phone, to a DVD player or smart TV.

In the Resource-based API platform the work gets done this way:

• The central API team creates APIs that make it easy to get to all of the data and resources of a company. These APIs are all about creating a general way for programs to get access to customer data, data about films, recommendations, and so on. The resources that a client program may need include data and services for:
  • User info
  • Movie metadata
  • Movie Ratings
  • Similar movies
  • My list
• The central team implements the APIs to deliver scalability and excellent performance.
• The client team takes these APIs and creates applications for a variety of devices. The problem is that the code for handing errors and adapting the data to the needs of the device all resides in the client application code.

The Resource-based API paradigm works just fine and is used all over the world. But Jacobson and his team wanted to find a way to move faster, create better software, and enable more optimized experiences for Netflix subscribers. They realized that different devices potentially needed a different set of data to support the user experience. The best way to make the client software simpler would be to have a new API that allowed as much as possible of the data translation, error handing, and other utility functions to happen on the server, not on the client.

But if you asked the central API team to create versions of APIs for all the different families of devices, the team would be overwhelmed. In addition, it would be hard for each of the client teams to actually ask for the right design for the APIs they would need. It would require trial and error and that would be hard to accomplish with many client teams competing for the attention of the central team.

Jacobson and his team realized they needed to create a new layer of API they called an Experience-based API. The idea is that the client team should design the perfect API to serve the needs of the device on which the client application will run.

For example, when writing the application code to create a page on the client using the resource-based approach, the developer may have to call a dozen APIs, handle errors from all of them, and then format the data for use on the page. All of this communication takes place over HTTP, which is less efficient than the networking methods used on the server.

But if the client developers could create their own API to support the needs of that page, and move all of the code for data access from the client to the server, lots of good things would happen such as:

• Client code would get simpler.
• More code could be shared across families of devices.
• New device families could be supported faster in a way that best fit the needs of the device.
• Performance could improve because network communication is optimized, reducing round trips from the device to the server.

So Jacobson and his team set out to create a new layer to enable client developers to create their own APIs. Here’s how it works:

• Client developers analyze the application that they want to build for a particular client and then decide on the APIs that they will need to support them.
• They define the endpoints of the APIs and protocols to handle the requests and responses that are optimized for their client
• The APIs are implemented by the client developers using the Groovy scripting language which is used to call Java APIs that will gather the data needed to handle the request. Groovy is easier for client developers to work with than Java but still is scalable enough.

This architecture, in the case of Netflix powered by Nicobar, allows the client teams to move faster and reduce the technical debt by making the client programs simpler. The code enabled by Nicobar is more shareable and easier to maintain.

“There are some significant engineering benefits to this structure, but the real win is increasing the velocity of innovation,” said Jacobson. “Because Experience-based APIs properly separate concerns across the two teams, the work gets done faster. The client teams control their own velocity and the server teams can focus on building the platform that supports them.”

Lessons for the Rest of Us

Jacobson, Greg Brail, Chief Architect of Apigee, and I were co-authors of “APIs: A Strategy Guide”. During the time we wrote the book in 2010, the world was focused on the impact that public APIs were having. Google, Facebook, Twitter, and Amazon all harnessed massive amounts of attention and energy as people put their public APIs to work.

At that time Jacobson argued that internal use of APIs would have the far larger impact. He saw that few companies, even Netflix, would be able to get a large amount of value from public innovation. Instead, the real victory would come as internal APIs increased efficiency, productivity, and the velocity of innovation. Experience-based APIs are one major proof point for Jacobson’s way of thinking.

I believe this two-tier structure that allows those closest to the problem to translate the Resource-based APIs to a more suitable form and create APIs that are adapted to their needs can be used in all sorts of companies who how have bottlenecks in their IT departments.

To make this work you need a front office IT staff, one that works with the business. These are the equivalent of the client teams at Netflix. The back office IT provides Resource-based APIs to the front office teams who then use simple tools, scripting languages, model driven development, content management systems, and so on to adapt the resources to the needs of the business. In large Wall Street Firms it is common to have highly skilled front office IT teams. I think that many companies could benefit from this model if they choose the right tools and create the right separation of concerns.

Nicobar 

For companies who create -multiple applications and are committed to the Java ecosystem, Nicobar is definitely worth a look. This library provides the core functionality to implement the Experience-based API program. Right now the Groovy language is the way that the adapter code to assemble the data needed for the Experience-based is written, but in the future other JVM languages may be supported.

The key capabilities of Nicobar are:

• Rapid application delivery via dynamic updates of components
• Ability to add support for multiple JVM languages

To learn more go to Nicobar: Dynamic Scripting Library for Java.

And, for the truly nerdy, here’s what the project wiki says about why the project was called Nicobar:

“Nicobar refers to a remote, archipelagic island chain in the Eastern Indian Ocean. It is aUNESCO World Biosphere Reserve, and like many other remote volcanic islands, it has rich biodiversity, and various endemic species are found here. Taken together with the surrounding Andaman group of islands, six different human tribes occupy these islands, each with its own spoken language. We felt that these isolated islands, evolving independently are a good metaphor for the polyglot, modular runtime framework we were setting out to build.”

Dan Woods is on a mission to help people find the technology they need to succeed. Users of technology should visit CITO Research, a publication where early adopters find technology that matters. Vendors should visit Evolved Media for advice about how to find the right buyers. See list of Dan’s clients on this page.

This article was first published on The Next Web on March 25, 2014

Daniel Jacobson (LinkedIn) is the VP of Edge Engineering for the Netflix API. Prior to Netflix, Daniel ran application development for NPR where, among other things, he created the NPR API. He is also the co-author of APIs: A Strategy Guide.

“Get busy living, or get busy dying” – Shawshank Redemption

Building great engineering teams is difficult, but it is also increasingly important as the world in which we live is more than ever driven by software. Because of this growing importance, it is essential for engineering leaders to maintain a culture of innovation within their teams to ensure high performance and to keep the company ahead of the curve.

In high performance cultures like at Netflix, there are basically two outcomes that will play out over time for engineering teams. Either the team will enjoy an upward spiral established by a strong culture of innovation or it will spiral in the downward direction, resulting in an inevitable decay of the team and its products.

Here are my experiences as an engineering leader and how I’ve worked to build a culture around innovation for my teams, virtually at all costs.

The downward spiral

For most engineering teams, it is easy to enter a steady state of development and maintenance as systems get off the ground and mature.

Accordingly, managers often slow or halt hiring as the amount of work is relatively well-understood. As a result, the engineers on the team enter a daily or weekly (or perhaps monthly) ritual of incremental improvements, responding to requests, and fixing bugs.

As engineers churn through task lists, however, they become bored, uninspired, and complacent, resulting in degradation in velocity and/or quality. That degradation will result in more churn around testing and/or support issues, which will further frustrate and bore the engineers while generating more potential for system failures that will increase the churn.

The more churn, the more turnover in staff; the more turnover in staff, the more additional churn. This downward spiral can play out very quickly or it can take quite a while.

In either case, there is a clear direction, it is inevitable, and it has a bad ending.

Upward spiral

The way out of the downward spiral is to make some very difficult decisions that have short-term ramifications for the benefit of the long term. I call this “taking your lumps.”

If you take your lumps now by deferring non-essential work, it frees the team up to think about the long-term and to seek patterns in their work, systems, and operations. Through these patterns, the team can potentially program away a class of work that otherwise would occupy the team’s time on an ongoing basis.

Eliminating a class of work enables the team have more available time in the future to seek other such patterns or opportunities, which will create even more available time.

With the available time, not only is the team further alleviated from the daily churn of reacting to external needs, they are also able to pursue higher order projects that allow the team to make transformative leaps forward rather than churning to keep up or making minor incremental improvements.

Repeated enough, this will eventually become part of the team’s culture, resulting in higher quality work and greater velocity. Unlike the downward spiral, there will positivity around the team that will be infectious and will create a breeding ground for attracting new talent.

Virtually every engineering team will find itself in one of the two aforementioned trajectories. It might not be obvious which way things are headed, but there will be a trend one way or the other.

It is the job of the engineering leader to ensure that the spiral is upward. Here are my 10 philosophies and approaches that I employ with my teams to strive for the upward trajectory:

1. Establish a strong identity

Be very clear on the identity of the team and establish a set of philosophies against which the team can operate. Be stubborn about adhering to the identity. The more that identity gets compromised by one-off requests, the more the architecture weakens, the more churn the team will have to deal with, and the more likely morale will suffer.

Be clear on what you will and won’t do and make sure the team knows these boundaries, lives them, and communicates them to others.

2. Important vs. Urgent

In “The 7 Habits of Highly Effective People,” Stephen R. Covey talks about the difference between urgent and important. Engineering organizations can very easily fall into the trap of being highly reactionary to externally imposed requests.

While many of these externally imposed requests are very important (and in fact, even if they are not), they tend to team’s attention as both urgent and important. But there are many other tasks or efforts that are very important despite the fact that they are internally driven and elective.

Understanding this distinction and being able to distinguish which tasks fall into which category is paramount in getting out of the churn and enabling that first critical step: introspection.

3. Introspection

Introspection is the key to innovation. Handling requests from a range of external (or even internal) stakeholders is the natural, easy thing for a team to do. Taking a step back from those requests and looking for patterns across them while imagining what they might look like in the future will give a broader and more impactful perspective.

If the system gets refactored in some other way, will that eliminate a class of requests in the future?  Given how the industry is evolving, can you anticipate weaknesses in the system’s architecture that should be examined now? These are examples of important questions that can help springboard your team out of their everyday churn of satisfying urgent requests.

4. Don’t throw good money at bad

During the introspection process, it is important to be future-oriented. Your team has a lot of functioning code and other system-oriented assets which should be considered.

That said, they should only be considered after evaluating the long-term needs of the team and its relationship to its constituents. Imagine starting from scratch and target that as your outcome. From there, it is much easier to see how, if at all, existing assets can play a role in that future state (or in the transition to get there).

5. Hire beyond your needs

The most important resource to enable introspection is time. Many companies and hiring managers work towards “right-sizing” their teams. That is, they project what the incoming requests will be for the team and attempt to staff the team based on those expectations.

This is perhaps the biggest flaw that a team manager can make when building and operating an innovation team because that will ultimately limit the amount of available time for introspection.

Instead, hiring managers should staff beyond the bandwidth needed for known tasks. This will give the team the ability to swell and contract its focus on such work while continually maintaining a reasonable amount of time towards introspection and innovation.

6. Great engineers NEED to be challenged

If staffing is such that your great engineers are spending the majority of their time handling very tactical work, they will slowly but surely lose interest in the job and eventually leave.

Of course, doing that kind of work is a necessary part of every engineering job, but there needs to be a balance for great engineers to remain happy and excited about their work. Engineers need to also have deep architectural challenges that allow them to think, to stretch their minds, and to have a greater value to the company than just keeping the lights on.

In fact, most of them want to have the freedom to identify and pursue these challenges in a way that help them feel empowered and impactful. That is why engineers get into this field in the first place and if that is not available in their current job for too long, they will find those opportunities elsewhere.

7. Instill a culture of (good) laziness

There are two kinds of “lazy” in engineering: bad laziness and good laziness. Bad laziness is allowing yourself to repeat the same tasks over and over because that is easier than stepping back, looking for patterns, and spending the up-front time to program those tasks away. Manual deployment pipelines or manual tests are great examples. But ultimately, if a human can do it, a computer can (and should) do it too.

This is where good laziness comes in. Great engineers will ultimately be fed up with the arduous nature of the repeated task and seek to eliminate that work from his/her docket.

8. Innovation breeds innovation

Once an initial innovation occurs that liberates the team from some encumbering set of repeated tasks, the team now has some newly available time. That time can be used in any number of ways, but to maximize its utility the team should use that time for even more introspection which paves the way for the upward spiral.

The more such innovations that the team can yield, the more likely the team can yield more innovations. This is the case, not only because of the growth in available time, but also because it eventually becomes part of the team’s culture.

9. Don’t treat your systems like your baby

Many people in the engineering world grow very attached to the systems that they build. It is easy to establish that loyalty as engineers spend a lot of time working on a specific system. In fact, I have often heard people call their systems their baby (I may have been guilty of that in my past as well).

There is a value in growing so attached to the systems in that is does strengthen the bond and builds pride for the team as they strive for excellence with that system. That said, there is a long-term detriment to this as well.

Systems, like virtually any piece of technology, have a limited shelf life. At some point, the system will hit its limit and will need to be overhauled or replaced.

Loyalty to that system clouds one’s objectivity about what is best. We need to be able to treat our solutions as tactics towards a broader goal and if the tactic is no longer effective we need to abandon it.

10. There’s no such thing as maintenance mode

If a system is to go into maintenance mode, it really means one of two things: It is either not an important system anymore (which begs the question as to whether or not it should just be retired outright) or the business function is still important to the company even though the company no longer wants to invest in the system that supports it.

As part of the team’s culture, it is important to aspire to eliminate the idea of maintenance mode from the team’s vernacular.

Maintenance mode has two main detriments. First, it adversely affects the team’s morale and goes against the spirit of great engineers, which is to constantly be challenged. Second, most maintenance systems conflate the idea of supporting a legacy system with supporting its business function.

In fact, the latter is the real goal and an innovative team will seek ways to retire legacy systems in favor of future-oriented systems that still supports the required business function. This is not always easy or feasible, but you should always be seeking opportunities to move on from the legacy system.  Sometimes executing on that migration work is of equal or greater value to pursuing new innovations.

External risks

Ultimately, all of these principles depend on having excellent talent on the team. No amount of leadership can offset the challenges introduced by having the wrong skills or people.

Another risk is that many engineers like to chase the shiny new objects. There is a balance that needs to be maintained between enabling great engineers to experiment, innovate, and identify and pursue challenges with their propensity to play with emerging technologies.

It is also worth noting that there are often external forces that prevent some organizations and/or leaders from achieving the above philosophies. For example, not all companies have enough available resources to staff beyond the needs or they may have a legacy of disparate and unrelated technologies that make it inherently more difficult to find a path out of the churn.

As a result, these philosophies require a strong company-level culture that puts leaders and teams in a position to achieve greatness. If the culture is there, however, these 10 philosophies, if truly embraced, will help springboard your team to being innovative and non-reactionary.

This article was originally published to The Next Web on December 17, 2013

The digital world is expanding at an amazing rate, giving us access to applications and content on myriad connected devices in your homes, offices, cars, pockets and on even on your body. The glue that allows all of this to happen, that connect the companies who provide these services to the devices that you use, is the API.

Because APIs have such a huge responsibility for so many people and companies, it is natural that API design is often one of the industry’s liveliest discussions, touching on a range of topics including resource modeling, payload format, how to version the system, and security.

While these are likely important areas to explore when designing virtually any API, the reality is that a much larger decision needs to be made first. That decision is based on a fundamental question: who are the primary audiences for this API and how can we optimize for those audiences?

This seems like an innocuous enough question, but don’t underestimate its importance or complexity in the growing world of APIs.

Years ago, this question was much simpler

At that time, many emerging APIs were being built as open or public APIs, targeting a large set of unknown developers (LSUDs) external to the providing company.

Because of the (hopefully) vast numbers of external developers using the API representing different use cases, the most sensible way to design the API for this audience is to have the providing API team design it in a very clean, concise, and resource-oriented way that closely represents the data model and/or features of its source(s).

In a previous post, I referred to these as OSFA APIs (or one-size-fits-all APIs). Allowing for such granularity in the modeling means that any developer who wishes to use the API can mix and match the elements in whatever way they choose to satisfy their application without further API team involvement.

The resource-model approach to designing an API can be very powerful, especially for this type of audience. The problem with this approach, however, is that the way that many companies use APIs today is different than described above. While many are still supporting the use cases of LSUDs, more are using their APIs to support a growing mobile or device strategy.

For some of these implementations, the engagement with the developers is different. The audience is a small set of known developers (SSKDs). They may be engineers down the hall from the API team, a contracted company hired to develop an iPhone app, or an engineering team in a partnering company. In all of these cases, however, the API team knows who these people are (at least in the abstract sense).

More importantly, however, the API team and the providing company care about the success of these implementations in a different way than they might care about the applications developed by the LSUDs. In fact, the success of the SSKDs may very well be paramount to the success of the business as a whole, a model that is becoming increasingly more pervasive.

Because of this change in audience and the deep interest in their success, there is great opportunity to change the API design.

For the SSKDs, having granular resource-based APIs that closely represent the data model works, but it just isn’t as optimal as it could be. This is especially the case when you consider the growing number of device types in the world and the fact that more and more companies’ business strategies are dependent on providing value to customers on such devices.

So, all it takes is a couple of devices with diverging needs and/or capabilities, each of great import to the company, for the resource-based API to start to show some warts. Making the API better, more optimized, for each of these target applications is the next logical, and most critical, step.

Enter the Orchestration Layer

An API Orchestration Layer (OL) is an abstraction layer that takes generically-modeled data elements and/or features and prepares them in a more specific way for a targeted developer or application

To address this opportunity, more companies are employing orchestration layers into their API infrastructure. While there are many ways in which to implement this architectural construct, the concept remains the same across all of them.

Below, I will describe a few of the more common patterns that I have seen (and/or been involved in implementing). But first, here are a few key principles that need to be considered when building an OL:

1. Most APIs are designed by the API provider with the goal of maintaining data model purity. When building an OL, be prepared to sometimes abandon purity in favor of optimizations and/or performance.

2. Many APIs are designed by API teams to make it easier for the API team to support. When building an OL, be prepared to potentially add complexity for the API team (or other teams, depending on the way it is implemented).

While this sounds undesirable, the goal here is to dramatically improve efficiency and/or simplicity for other people at some mild cost to the API team. Also keep in mind that such costs can potentially be programmed away over time.

3. It is important to understand the breadth of the audiences for the API.Depending on those constituents, you may only need the OL. In other cases, you may need the OSFA foundation in addition to the OL.

Here are a few examples of how some OLs have been approached:

Device-specific wrappers

This is the most common pattern that I have seen because most companies that are experiencing the distress referenced above already have APIs that they still use, continue to support, and invest in. The result is to continue to offer the granular resources as they always have, but to offer a wrapper tier on top of them – with new endpoints that are tailored to specific developers, devices or device clusters.

In this model, the API team will work more closely with, for example, the iPhone team to write a custom wrapper that handles specific requests and deliver specific payloads that are optimized for the iPhone app. In this model, most often the team to build the endpoints and the wrappers is the API team although that doesn’t have to be the case.

Query-based APIs

In this model, the API team is putting the power in the hands of the requesting developer, although that power is limited. The goal here is to create a more flexible way in which the requester can make requests and tailor payloads without putting additional ongoing burden on the API team, as could be the case with the Device-Specific Wrappers.

This is achieved by breaking down the resource-based APIs and allowing them to be queried against like a database through flexible parameters and payloads that can contract, expand and possibly morph based on what is needed. The benefit here is that once the query language is set, the API team does not need to keep writing wrappers as new implementations are needed for different devices.

The detriment, however, is that the query-based API is still a set of rules to which the developer needs to adhere, although these rules are much more flexible than the resource-based API model.

Experienced-based APIs

This is the model that Netflix has implemented, which in some ways is a blend of the two above. In this model, we basically have device-specific wrappers but they are designed, implemented and owned by the device teams.

A key concept here is that we have put the API team in the position of gathering the data in a generic, reusable way while putting the device teams in the position of owning the data formatting and delivery. After all, the formatting needs evolve in concert with the UI changes so putting that effort in the hands of those closest to the changes eliminates additional steps.

(For more details on how this system operates, see the links at the bottom of the post.)

As I noted, the range of implementations is potentially much more diverse than these three, although these are some of the most consistent and interesting patterns that I have seen. Regardless of how this is achieved, however, the key is for the API team to stop supporting the API as a service that is designed independent of those SSKDs who consume it.

Rather, the API team needs to view the SSKDs as partners in the design with an interest in making the products as great as possible so the end-users can get the best experience possible. The API team has the opportunity to build services that help developers to be better at developing by focusing on optimizing for the developers’ needs rather than how to optimize the time spent supporting the API.

Given the opportunity ahead with the potential number and diversity of connected devices, the effort to provide such optimizations is a small price to pay for the massive upside.

This article originally appeared on The Next Web on September 15, 2013.

“Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat.” – Sun Tzu, The Art of War

Over the course of 2013, the API industry has matured a great deal. Not only have we seen many of the major vendors (ie. Apigee, Mashery, 3Scale, Layer7, etc) get acquired and/or receive large rounds of funding, we are also seeing an uptick in new players, new tools and services, new publications, and even a series of API-focused conferences.

Meanwhile, according to a recent survey by Layer7, more than 85% of companies expect to have an “API program” within the next five years. All of this is evidence that the appetite for tools and information about APIs is robust. Accordingly, there is no shortage of people and companies attempting to satisfy that hunger. The question I have amidst this growth, however, is whether the concepts around API strategies being served by some is the right meal for those who are eager to feast on APIs.

The problem: “API Strategy”

The majority of the non-technical conversations in the API industry seem to be focusing on terms like “API strategy” and “API economy.” In fact, I even co-wrote a book called APIs: A Strategy Guide a couple of years ago, further facilitating the use of those words in the API vernacular. There is absolutely a strong case to be made for needing an API strategy for certain situations. But how many companies should really be thinking about their API in that way?

Before continuing, it is worth being clear on what I mean by the terms “strategy” and “tactic”. Bobby Ghoshal puts it nicely in his post, Greeks Gave us Strategy vs. Tactics: Now Understand the Difference where he says, “A strategy is a grand plan, a tactic is a specific measure implemented to push the grand plan forward.” Applied to APIs, if there is an API strategy then it means that API is the product in-and-of-itself. In other words, the API is the target of a distinct business and opportunity (with its own metrics), which will then have a range of tactics to support it.

There are certainly cases where APIs are businesses and where a strategy is appropriate. The most common example of an API strategy is around companies who aspire to build a developer community as a new revenue source or as the foundation of their business. Twilio is an interesting example of such a company. Twilio’s strategy is to offer APIs that tap into their backend services to allow developers to build apps supporting their communication initiatives.

In this case, the API is a strategy, one that is fundamental to the business as a whole. Accordingly, Twilio invests heavily in the API, supporting documentation for it, fostering the developer community, and all of the other things one would expect such a company to do for their public API (and some would suggest that they do this as well as or better than anyone else). Twilio should invest heavily in this — a significant portion of the opportunity is predicated on the success of the API program.

The reality: “API as a tactic”

But most companies should not be trying to set up distinct businesses with their APIs as the focal point. They should not be trying to generate new revenue streams or reach new audiences through such programs. Instead, most companies should be focusing on their core business and then designing APIs that support larger strategy.

In pursuing that route, most companies should not be discussing their “API Strategy,” they should be talking about their API as a tactic in support of their broader business strategy and objectives.

An example that I am very close to is Netflix. In the early days of the Netflix API when the program was targeted exclusively to public developers, the API had its own metrics and its own objectives, all of which were designed to support the primary goals of the company.

In this sense, the Netflix API was a product designed to offer incentives to developers to motivate them to build applications around the Netflix experience. These applications would hopefully reach new audiences to generate new subscribers and/or create new user experiences for existing subscribers that would increase their satisfaction with our service. Although the API was treated as a new product within Netflix, it was still operating under the company’s larger business objectives.

While the original vision was incrementally valuable to the company, the results were not as transformative as originally expected. As a result, we pursued a new approach with the API, using it to drive the larger strategy of device proliferation for our growing streaming business. In this sense, the API was transitioning from a product to a tactic.

Today, Netflix can be watched on more than 1,000 different device types, the vast majority of which are developed by Netflix-employed UI Engineers. The API served as an excellent engineering tactic that allowed us to quickly get on more devices, which in turn allows us to create a better overall experience for our customers.

More changes have since been made with the API. Most recently, the Netflix API team, which used to provide traditional REST APIs to the Netflix UI teams, is now providing content distribution platforms that enable data to be pushed from our AWS backend systems to the devices in people’s homes and pockets. We are no longer truly an API team, we are a team that embraces the differences of the different devices and empowers the UI teams to customize and optimize the request/response models needed for their specific device. In other words, we are now a platform for API development.

All of these pivots within Netflix further demonstrate that our API is nothing more than a tactic to achieve our broader goals. There are no allegiances to a tactical solution. Tactics can (and should) be modified, discarded and replaced as appropriate. Strategies, on the other hand, should have longer shelf-lives, evolving over time but less frequently overhauled.

The majority of companies that are considering API implementations, based on my conversations and experience in the industry, are more like Netflix than Twilio. There are countless examples of companies who have made similar pivots to refocus their API attention towards supporting the company’s primary business objectives. These examples range from media companies (NPR, The New York Times, The Guardian) to financial institutions (PayPal, E-Trade) to social media sites (Twitter, LinkedIn).

Even service companies like Amazon and Salesforce, whose systems are differentiated in part by their APIs, use them as a tactic to provide increased value for the primary business, which is providing robust services supporting cloud computing and CRM respectively.

The bottom line

The key to a successful API program is to know your audience. Your audience is defined by your business opportunity. So, be very thoughtful about the opportunity and then define your API accordingly. In some cases, pursuing the public developer opportunity is absolutely the right thing to do and it may have a tremendous upside (although realizing that upside is quite rare).

However, if your opportunity is truly to support a broader business objective, then launching a public developer program is not likely to yield large dividends. It is more likely to come with increased costs and risks that will weigh down your returns, dilute your resources for the larger opportunity, and distract you from the real prize. Instead, focus all of your energy on building a great system that helps you optimize for the larger goal. And don’t be married to that system as it is nothing more than a tactic.

Ultimately, if you know your audience, you can define a strategy and then design the right tactics. Otherwise, brace yourself for a very slow route to victory or, more likely, a noisy defeat.

This post originally appeared on The Next Web on September 15, 2013.