Remote Service API Design

With the longer weekend (Friday Nov 11 is a public holiday in Belgium, to remember the end of WW I) took some time to watch some sessions from the Devoxx conference that has just finished. The talks are already available on Youtube! Great content: Microservices, Reactive, Netty, Kafka messaging, Docker, ... With this conference almost in my backyard, will have to free up some time next year to attend again myself.

One talk immediately drew my attention: "Effective Service API Design" by Elliott Rusty Harold. I know Elliott an XML guru. Remember being at the speaker's table with him at XML DevCon in London back in 2001 where I presented "Understanding SOAP".

Some thinks I picked up:

• "Contract first" >> "Documentation first"
• Start small (MVP Minimum Viable Product, YAGNE You Ain't Gonna Need It)
• Leverage the URL, not everything 
• Prefer idempotency
• Use standards: UTF-8, standard data/date formats, standard decimals with 2 digits 
• Avoid required data elements
• Deprecation policy, how long to keep the old API, 2 year notice is good
• Plan for versioning: easier without schema's but with optional fields
• No assumption about use of client code/library, developers will often not use those 
• If you build client code/library, develop it by hand and do not generate it 
• Authentication and authorization remain a challenge
• Performance!

Although a good talk, would have hoped for some more specific guidelines.

 
•  1:00:41

  Effective Service API Design by Elliotte Rusty Harold

At the start of the presentation, Elliott clearly makes the difference between the local "library API" (e.g. the JDBC API with the java.sql package) and "Remote API" or "Service API". Great definition by the way: a network service (almost always a server) by which programs communicate with a bundle of funcitonality provided by code owned by somone else, running on their computer (not yours).

Thanks to the Devoxx team and Stephan Janssen to publish this great content for free!

Reactive programming, the old way

Reactive programming is the new hit: applications work with a non-blocking, asynchronous programming model. Use of multi-threading is none or very limited. The reactive program responds to messages or events that come in via callbacks. These messages must be handled as quickly as possible, never blocking themselves while being processed.

Made me think of a fun project I was involved with in 1999 and 2000: "GSL", the Generic Service Layer" at Rabobank. We developed an integration layer (by Gartner defined as a "Super API") based on the message passing product Netweave. And this product was completely based on callbacks and a reactive programming model.

Pre-dating XML and with maximum message buffer size of 32K on CICS, we defined our own message format. The messages had a tree structure in a proprietary, textual format. The GSL API allowed the creation, sending, receiving and parsing messages.

All the main platforms were supported: IBM CICS, Tandem (Guardian), AIX Windows NT and AIX. Communication was supported in all directions. So yes, a COBOL CICS program could invoke an ActiveX component on Windows. Code was written in C (and bit of C++). Most often the IBM mainframe and Tandem would be the actual servers.

An old code snippet from those days. This piece of C code writes a buffer back to a client application.

void loclSend(client_t * pClient, size_t szBuf, char * pBuf)
{
  NWDS_ERRNO      retcode  ;
  NWDS_CALL_BACK  complete ;

  /* write data to client process */
  complete.procedure = loclSendComplete ;
  complete.context   = (NWDS_CONTEXT) pClient ;
  retcode = nwds_ipc_write(pClient->hLocl
                          ,(NWDS_SIZE) szBuf
                          ,(void *)    pBuf
                          ,NULL
                          ,&complete) ;
  if ((retcode == NWDS_PENDING) ||
      (retcode == NWDS_SUCCESSFUL))
    lcTraceHdr('I', 'S', pBuf, szBuf) ;

  if (retcode != NWDS_PENDING)
    loclSleepCallback(&complete,retcode) ;
}

The write of the buffer is executed with ndws_ipc_write. This write could return immediately, whereby NDWS_SUCCESSFUL would be returned. But most often, the return code would be NWDS_PENDING. Then the callback function loclSendComplete would be registered and executed when the write was finished. The complete.procedure element contains a pointer to the loclSendComplete() function.