Everything You Know about REST Is Mostly Confusing

Leave a comment.

At the time that I write this it’s been fifteen years since Roy Fielding’s doctoral dissertation Architectural Styles and the Design of Network-based Software Architectures was published. It’s the seminal work on describing the Representational State Transfer style of architecture so indelibly ingrained within the specifications of the World Wide Web. Let’s face facts. You’ve likely not read it.

Just like the REST acronym you have good intentions though. You know the dissertation is in the computer science canon, and if it was in published book form it would hold a special place between your Code Complete and The Mythical Man Month book ends. However, therein lays the fundamental problem. The road to hell is paved with good intentions.

Beyond your obviously shameful lack of commitment to your field there is also the issue of sifting through fifteen years of misinformation. So frustrating was this that Dr. Fielding himself needed to clarify his position on so-called RESTful APIs in a post entitled REST APIs must be hypertext driven. In it are some clear-cut requirements.

At this point it’s no wonder that most everything anyone knows about REST is not necessarily right or wrong. It’s just confusing.

Where’s REST?

Since we’re talking about things you’ve read, have you ever read the Where’s Waldo? books? According to Wikipedia,

The books consist of a series of detailed double-page spread illustrations depicting dozens or more people doing a variety of amusing things at a given location. Readers are challenged to find a character named Wally hidden in the group. Wally’s distinctive red-and-white-striped shirt, bobble hat, and glasses make him slightly easier to recognize, but many illustrations contain red herrings involving deceptive use of red-and-white striped objects.

I loved these books as a kid, but there were days when I wondered if the publisher was playing a cruel joke. Sometimes after an hour of searching I was convinced I received the only book with a page where there was no Waldo. I get that same feeling with REST. There are lots of blog posts that claim to have found REST. I must have a defective book.

The reality is REST doesn’t need to be found, or defined, or expounded upon. REST is whatever Dr. Fielding said it was fifteen years ago. That might sound disingenuous, but it’s the hard to swallow truth. The reason I can say this with confidence is because REST is not a protocol. Nor is it even a standard or a specification. It’s an abstraction of already defined standards, specifications and protocols.

That doesn’t make it matter any less. It’s just a different matter altogether. That’s the first reason why understanding REST can be so confusing. You can’t really aspire to it. You either are, or you are not REST.

REST Is Potential Energy

If I can use an analogy to drive this point home, REST is potential energy. According to Wikipedia,

In physics, potential energy is the energy that an object has due to its position in a force field or that a system has due to the configuration of its parts.

I like to think about REST the same way. It is potential energy as a system due to the configuration of its parts. The most prevalent is the HTTP specification, but there is more to it than that. In some sense Dr. Fielding describes these parts in chapter six of his dissertation based upon his own experiences,

Since 1994, the REST architectural style has been used to guide the design and development of the architecture for the modern Web. This chapter describes the experience and lessons learned from applying REST while authoring the Internet standards for the Hypertext Transfer Protocol (HTTP) and Uniform Resource Identifiers (URI), the two specifications that define the generic interface used by all component interactions on the Web, as well as from the deployment of these technologies in the form of the libwww-perl client library, the Apache HTTP Server Project, and other implementations of the protocol standards.

Of course if REST is potential energy, then wouldn’t it make sense that someone needs to turn it into kinetic energy through an active implementation of the abstraction? Why shouldn’t you or anyone else be the one to release all that force? Well, mostly because that’s where the analogy goes to die. The ultimate goal of REST isn’t an idealized final form. Dr. Fielding speaks about this in some of his conclusions when he says,

The modern Web is one instance of REST-style architecture. Although Web-based applications can include access to other styles of interaction, the central focus of its protocol and performance concerns is distributed hypermedia. REST elaborates only those portions of the architecture that are considered essential for Internet-scale distributed hypermedia interaction. Areas for improvement of the Web architecture can be seen where existing protocols fail to express all of the potential semantics for component interaction, and where the details of syntax can be replaced with more efficient forms without changing the architecture capabilities. Likewise, proposed extensions can be compared to REST to see if they fit within the architecture; if not, it is more efficient to redirect that functionality to a system running in parallel with a more applicable architectural style.

So while REST may be an architectural style existing as a layer within your API, it does not mean all interactions between interested parties (e.g. – client and server) within that API can necessarily be classified as RESTful. This is a very important distinction. Depending upon your own needs this could be a benefit. This is another reason why REST can be confusing.

The Uninformed URI

Fundamentally there are a number of issues to consider when designing an API. However, if you read what most engineers have to say about the defining characteristics of REST it would become abundantly clear the format of URIs within an API is of utmost importance. Would you be surprised to learn this has nothing to do with REST, and that so much focus on the format of the URI actually detracts from core REST principles?

There are three underlying aspects of this format that get discussed most, and that is with regard to the plurality of nouns, the use of verbs, and the use of query string parameters. Let’s start with a simple example resource – users. Assume anything surrounded by curly braces in the examples is replaced with an appropriate value. A well defined URI endpoint in an API might look something like this:

/users
/users/{id}

Now I’ll extend it a little further to get a user’s order history on an ecommerce website. No consideration for method request types (GET, POST, PUT, etc.) is even being considered at this point.

/users/{id}/orders
/users/{id}/orders/{id}

Not bad. You might choose to do it a little differently, but for the most part it follows a normalized convention as seen elsewhere. Now what if I told you the following format can be just as valid within REST style architecture:

Are you raging right now? It’s okay. Take a deep breath and realize you have been misled like so many others before. I’m going to explain, but first let’s hear from Dr. Fielding again under subsection 5.1.5 Uniform Interface,

In order to obtain a uniform interface, multiple architectural constraints are needed to guide the behavior of components. REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state.

The last portion of that, hypermedia as the engine of application state was so important it eventually got its own acronym, HATEOAS. A small taste of what that means can be found later under subsection 5.2.1 Data Elements,

Unlike the distributed object style, where all data is encapsulated within and hidden by the processing components, the nature and state of an architecture’s data elements is a key aspect of REST. The rationale for this design can be seen in the nature of distributed hypermedia. When a link is selected, information needs to be moved from the location where it is stored to the location where it will be used by, in most cases, a human reader.

Think of any anchor link on the web today. In your browser you click a link of interest, and you’re magically transported to a location where your data is supposedly located. Most often this data is HTML, which is one classification of hypermedia. There is even a convention for helping a browser understand when this data has moved. However, you could care less about that. All you did was click a link. It was a named resource you expected to receive. Where it lives is inconsequential.

Hang on to that thought.

HATEOAS

HATEOAS is one of the most controversial aspects of REST because at least in the context of a well-designed API some engineers (me included) are still in disagreement with regard to its usefulness. Yet, this doesn’t change the fact that as far as REST is concerned, it is foundational. Furthermore, total discoverability in HATEOAS has been given too much emphasis. The point is not to discover all of the possible operations within an API. HATEOS can be articulated instead as follows, “I’ve just done something. What can I do now?”

Now I’ll finish the thought from earlier. Think about a simple HTML page. It has nothing but text and hyperlinks. How does the browser (client), and therefore you, know what operations can be performed next? Ignoring anything related to JavaScript or CSS, you know that you can click a link because it’s underlined, and within the context of the page you know it will perform some operation as described. It might be a link to a news story, or it might be an instruction to load an image of babies hugging kittens.

The possibilities made available to you after clicking the link are completely transparent to the client. This is REST by design, but as it concerns APIs there remains confusion. Why? It’s primarily because there is no counterpart to the HTTP and URI specifications that can be layered on top to emulate the purity of this design. Everyone chooses to do it differently. However, some are trying to lay the foundations, like the JSON Hypertext Application Language.

Even Dr. Fielding appears to realize that there will be web applications in the future that will not necessarily need to align with the REST model for hypermedia when he says in subsection 6.1 Standardizing the Web,

REST is not intended to capture all possible uses of the Web protocol standards. There are applications of HTTP and URI that do not match the application model of a distributed hypermedia system. The important point, however, is that REST does capture all of those aspects of a distributed hypermedia system that are considered central to the behavioral and performance requirements of the Web, such that optimizing behavior within the model will result in optimum behavior within the deployed Web architecture. In other words, REST is optimized for the common case so that the constraints it applies to the Web architecture will also be optimized for the common case.

The question then becomes, how important is distributed hypermedia for your API?

According to the subsection 6.2.5 REST Mismatches in the URI, it certainly shouldn’t be the most important consideration,

Although the URI design matches REST’s architectural notion of identifiers, syntax alone is insufficient to force naming authorities to define their own URI according to the resource model. One form of abuse is to include information that identifies the current user within all of the URI referenced by a hypermedia response representation. Such embedded user-ids can be used to maintain session state on the server, track user behavior by logging their actions, or carry user preferences across multiple actions (e.g., Hyper-G’s gateways). However, by violating REST’s constraints, these systems also cause shared caching to become ineffective, reduce server scalability, and result in undesirable effects when a user shares those references with others.

Of course that’s just my assessment. I don’t know the needs of your API, and the design of URIs and the elements of hypermedia may be crucial.

Idempotence: The Other Weakness

Another point of confusion with REST is idempotent methods in HTTP. This is the most baffling aspect of REST discussions because method definitions are so clearly characterized. Problems arise primarily because that clarity comes from the Hypertext Transfer Protocol – HTTP/1.1 specification, and not Dr. Fielding’s dissertation. The specification even provides concrete real-world examples of when to use certain methods when performing a request.

Yet, there are those still trying to map one-to-one CRUD functions of persistent storage to HTTP method verbs. The conditions for considering what method to use go beyond the resource entity itself. Misuse of any of the constraints outlined within the HTTP specification is not really a matter of being more or less RESTful in an API. It’s simply a mistake (or conscious choice) in the implementation of the API.

Arguing about that particular implementation doesn’t change the fact that a very intentional use of request methods is a constraint ascribed to REST.

Conclusions

I understand there are instances where further clarification is required when dealing with certain concepts in technology, especially when they appear more philosophical in nature. Semantics are very important when moving from abstractions or even protocols to implementations. However, emulation of an architecture style can be difficult when the implementation exists in form and substance already in the vastness of the Internet – like in the case of REST.

In the end, calling your API RESTful may do more harm than good. It’s better to communicate to users instead that it meets a need, and does so with well defined constraints. When you’re done you can perform the Duck test. If it’s not REST, then it’s simply a duck by a different name.