Building robust APIs in R using plumber

I must admit that it is another intense week in Arla Foods. As we are getting closer to the end our another PI and we need to deliver the MVP of a tool I am working on in one of the projects, the pace of working is really high. Especially when it comes to the application deployment and you are doing various things in plentiful environments for the first time. But… usually the right path is not the easiest one, isn’t it?

A common saying is that the first pancake is always spoiled – hopefully it doesn’t correspond to the software! Just joking. Although it stresses me sometimes that I have still so much to learn as a Junior, I know that everything comes with the experience and if we combine the great teamwork, patience and self determination (which all are just in place!), everything is doable with the right amount of hard work.

Coming back to the main topic, I have learned another useful R package this week. In this blog post, I will show you how to translate a simple R script into a REST API with the R package Plumber. But before I dive deeper into the package and tell you about its application and benefits, let’s talk about APIs.

APIs

So, what is an API? As per Wikipedia’s Definition of API: In computer programming, an application programming interface (API) is a set of subroutine definitions, protocols, and tools for building software and applications.

What’s interesting is that even if you may think that we’re not familiar with API – if you use the {tidy verse}, you already have some experience! The tidy verse package has a very opinionated API on how you interact with data right and so the pipe and the way in which the functions operate, there’s a set of guidelines around those to make them fit in the the tidy verse paradigm and that could be considered an API, a way of dealing with and interaction so that there’s kind of this commonality that exists across all interfaces.

Web API as the name suggests, is an API over the web which can be accessed using HTTP protocol. It is a concept and not a technology. You can build Web API using different technologies such as Java, .NET etc.

Source: Tutorialsteacher

REST

Let’s briefly talk also about REST, which stands for Representational State Transfer. There is a lot of research and you can read a lot about its idea but essentially it’s just a kind of a further specification of how web interfaces should interact with one another with the emphasis on performance and scalability. It’s achieved by maintaining what’s referred to as a stateless API and the idea is that the server shouldn’t contain any information about the state of a client. Everything what’s important for the server to know should be contained in the request itself. Such idea makes it easy for us to scale out horizontally with additional servers to meet demand without worrying about making sure that what’s on client side stays on the client side and is communicated as needed.

Plumber executes R code in response to incoming HTTP requests, so that it’s important to understand how incoming HTTP requests get translated into the execution of R functions. An incoming HTTP request must be “routed” to one or more R functions.

The annotations that generate an endpoint include:

  • @get
  • @post
  • @put
  • @delete
  • @head

These map to the HTTP methods that an API client might send along with a request. Usually when you open a page in a web browser, it sends a GET request to the API. But you can use other API clients to form HTTP requests using the other methods listed here.

When a request arrives, Plumber begins by passing that request through its filters. After the request has been processed by all the filters, the router will look for an endpoint that can satisfy the incoming request. If it finds one, we will receive the responce to the incoming request using the value the endpoint returned. In case no endpoint matches the request, then a 404 Not Found error will be returned.

But… why should I care?

Okay, so now we have got familiar with some theory but you may ask me- why should I really even care about this as a simple R user? Let me explain. As R user you probably at least once came across below data science process graph from Hadley Wickham’s book:

See the source image

As a data scientist you spend the majority of your time on data preparation, then another amount of time on the little cycle in the middle where you’re just iterating and trying to find the truth in your data so that you could transform it according to the business use case and derive insights. Once you finish the process and you’re happy with your results, there comes a time when you’d like to communicate them. Imagine – you have done all of this work… for what? To have it in a one-page PDF? Or e-mail? It can be so demotivating if you don’t find a good way to prolong this cruel shelf life on what you’ve done.

Let me guess – have you just screamed “What about Shiny!?!?” after reading those lines? Yes, you’re right but often it might not be enough. And here comes the {plumber}

When we put plumber at the end of the graph as a way to transform our work into API endpoints, we open up the entire new world where we can provide access to our work. Just imagine, if you expose your work through those API endpoints, you’re not limited to the R ecosystem anymore. You can cooperate with your colleagues working on different environments such as JavaScript for example. It’s pretty obvious that they wouldn’t like to learn R language to figure out what you’ve done to re-implement it to other languages.

With HTTP you’re not limited to R and you can communicate your work to anyone. You don’t have to worry that someone rejects your code due to the lack language knowledge. Especially that R syntax is a little bit different and can seem “Chinese” comparing to other technologies.

In our case, we have used {plumber} in the project to separate the process of reading and cleaning the data (which due to its volume took a lot of time) from the possibility to use Shiny dashboard.

Plumber

Getting back to the main hero of this article, fortunately, plumber package has a pretty simple interface and you can turn any kind of function into a REST API endpoint just with some special comments. You don’t need to learn new kind of paradigm of coding or any bunch of methods. Transforming an R script into a REST API is quite easy. All you need, in addition to R and RStudio, is the package Plumber and optionally Docker (which won’t be in scope of this article).

Plumber: Getting R ready for production environments? - Data Scientists

Short plumber tutorial

  1. Package installation

You can install plumber just in a same way you install other well-known packages. Plumber uses Swagger, which provides kind of a nice out-of-the-box way for you to interact with and see your API without any command-line utility.

install.packages(plumber)
library(plumber)

2. Writing functions

It’s extremely important that for the request-response format of the API, we have to translate all objects different than strings convert into JavaScript Object Notation (JSON) or Extensible Markup Language  (XML). The decision for one or the other will most likely depend on which additional R packages you want to use or on which format the API’s clients are predominantly using. The R packages that are used to handle REST Requests in my example API are {jsonlite} and {httr}.

Just for the purpose of this post, we’ll use a very simple function from {stringr} package so that no conversion is needed. Let’s assume we would like to simply replace “a” letter with “e” in our strings.

Create the “plumber.R” file and write your first function.

library(plumber)
library(stringr)

replace_a_with_e = function(txt) {
 input = txt
 output = str_replace_all(txt, "a", "e")

 return(output)
 }

This function takes a character (string) value, executes the str_replace_all() function and then returns the results of str_replace_all(). We can run this code and then try something like replace_a_with_e("Sandra") to make sure it works.

3. Adding special Plumber comments to the function

Plumber uses special comments and decorators:

  • #’ – plumber comments –
  • @ – endpoint attributes/details
#* @param txt A string value, the text to be corrected with e letter
#* @get /a_replacement

The @param tag will define a parameter input for the function and the @get tag will define the http method used. So our file would now look like this:

library(plumber)
library(stringr)

* @txt Replace a letter with e.
 * @get /a_replacement
 replace_a_with_e = function(txt) {
 input = txt
 output = str_replace_all(txt, "a", "e")

 return(output)
 }

4. Try out the API!

To serve the API locally on a machine, you should now open a new script file, e.g. plumberRunner.R and save it in the parent directory of your project. Below few commands that will set up a local server and run the API:

library(plumber)
 pr("plumber.R") %>%
   pr_run(port=8000)

You will see some output in the console something like this and Swagger window should pop up:

Starting server to listen on port 8000 
Running the swagger UI at http://127.0.0.1:8000/__swagger__/

You can also copy the URL http://127.0.0.1:8000/__swagger__/ into your browser and have your API running with a nice swagger UI to test it with. If you expand the /a_replacement box and click ‘Try it out’ you will get a box that you can type into. Type a string in and click execute to see some example output.

Summary

In this blog post, I showed you how to translate a simple R function into a REST API with the R package Plumber. I hope you enjoyed the read and learned something about operationalizing R scripts. Due to lack of more advanced data transformation and formulas, you can treat this post as some kind of a theoretical teaser. I will definitely come back to this topic with some JSON data transformation and another more complex actions bringing us closer to the micro-services and application deployment.

Leave a Reply

Your email address will not be published. Required fields are marked *