Getting Started

In this chapter we'll develop, deploy, and exercise a simple CSIP temperature conversion service. The service will receive a value in Celsius and will provide it in Fahrenheit, a truly simple operation. We will go through the steps from creating the source, compiling the service package, deploying it and executing the service with example data.

To simplify this process, Docker engine is used to develop and execute a CSIP service in this chapter. Docker (https://docker.com) is a lightweight, container-based virtualization environment, which is available for all operating systems. CSIP Docker images are available on DockerHub (https://hub.docker.com). Instead of installing and configuring several software packages, we use two Docker image that are already setup and ready to be used.

The following sections will use the bash command line syntax (Linux, OS X, Windows-Cygwin) to walk you through the process. You should be able to copy and paste the commands (without the '$') to reproduce the development workflow. Follow the steps below to develop the example CSIP service with minimal effort.

First, let's install Docker.

Install Docker

The Docker engine is supported on Linux, Cloud, Windows, and OS X. Install Docker according to the installations instructions found on: https://docs.docker.com/engine/installation. Once installed you should be able to execute the docker command.

 $ docker -v                                                                    
 Docker version 1.10.3, build 20f81dd

The CSIP Docker images was created with this version of docker, however it should work with earlier versions.

Create a Workspace

First, we need a workspace that stores the source files and also the compiled packages. Create the workspace work/java in your home folder.

 $ mkdir -p $HOME/work/java/m/conv

Note: The '-p' option creates all sub-folders if not present.

We also created the folder m/conv, a Java package folder within the source structure.

Create a Service

The source code of the temperature conversion service is shown below. It will be stored as V1_0.java in the m/conv folder in java.

 $ cat > $HOME/work/java/m/conv/V1_0.java << END
 package m.conv;
 import javax.ws.rs.Path;
 import oms3.annotations.*;
 import csip.annotations.*;

 @Name("Temperature conversion.")
 @Path("m/conv/1.0")
 @Resource(type = ResourceType.OMS_COMP)
 public class V1_0 extends csip.ModelDataService {

     @Description("Temperature in C")
     @Unit("C")
     @In public double temp_c = 45.2;

     @Description("Temperature in F")
     @Unit("F")
     @Out public double temp_f;
     
     @Execute public void run() {
         temp_f = temp_c * 9 / 5 + 32;
     }
 }
END

Note: This cat command copies all content between the two END symbols into the file.

The service is using the JAX-RS, CSIP, and OMS3 API to mostly describe the service resources and elements with annotations and implement the conversion code. The @Path annotation (JAX-RS) specifies the service endpoint. The @Resource annotation (CSIP) indicates that this service is implemented as an OMS3 type component. The CSIP manual discusses also other @ResourceType definitions for external native executables, database connections, static data, archives, and other resources common to most environmental data and model services. An OMS3 component ResourceType is the most concise implementation approach for this simple example.

The service must subclass ModelDataService (CSIP) from the csip package. The class fields are representing the input and output data, annotated with @In and @Out according to the OMS3 specification. The temp_c input does have a default value of 45.2, which is arbitrary and will be used to generate a service request template in addition to other field annotations. Finally, the OMS3 annotated run() method implements the logic, a simple temperature conversion transforming temp_c from Celsius to temp_f in Fahrenheit.

Build the Service

DockerHub stores the image olafdavid/csip_dev (https://hub.docker.com/r/olafdavid/csip_dev) to build a CSIP service. It bundles all required software such as the correct JDK 8, Tomcat 8, Ant 1.9, JAX-RS, and the CSIP core libraries in order to compile and build a deployable service package, a *.war file. This Docker image lets you create a deployable CSIP service from the source above in one single step.

The docker command below runs the docker image olafdavid/csip_dev. If invoked for the first time, it will download the image from DockerHub and will then start the container. It compiles the sources found in $HOME/work/java and builds the war file in $HOME/work/dist. No other setup is needed, quite simple.

 $ sudo docker run --rm -it -v $HOME/work:/work olafdavid/csip_dev

Note: The option '-v $HOME/work:/work' makes the previously created local workspace available to the container. The '-it' option run the container interactively, '--rm' will terminate the container at the end of the build.

Now, we should have our service package available for deployment.

 $ ls $HOME/work/dist 
 csip.war

The default package name is 'csip.war', which can be renamed to something more meaningful if needed.

Deploy the Service

The created service package can now be deployed to an application server. Of course, we are using Docker again. This time it is a different, vanilla Tomcat 8 image. The tomcat:8-jre8 image on DockerHub is sufficient for our job[1]. The command below starts a container of this image and will download it first if needed.

 $ sudo docker run -d --name csip_rt -p 8080:8080 tomcat:8-jre8

Note: The container will run as a daemon through the '-d' option. The command will return once the container is started. The container is named 'csip_rt', we need to further interact with it using other commands. Having a name makes it easier. Its internal tomcat port 8080 is mapped to the host's port 8080 with the '-p 8080:8080' option so we can access the service from the host.

Next, we need to deploy our war file to the running container. Docker provides a copy (cp) command that allows to transfer files from the host to the container.

 $ sudo docker cp $HOME/work/dist/csip.war csip_rt:/usr/local/tomcat/webapps

The csip.war file is copied from the workspace dist folder to the csip-rt container's webapps folder. Tomcat will pickup that change and automatically loads and deploys the service.

Query the Service

Let's check if the csip.war file with our service is deployed correctly. CSIP offers an internal registry that lists all provided services within a given context. The listing is provided as a JSON array.

 $ curl http://localhost:8080/csip
 [{
   "name": "Temperature conversion.",
   "description": "Temperature conversion.",
   "url": "http://localhost:8080/csip/m/conv/1.0"
 }]

Note: The curl command is used to issue a HTTP/GET to fetch the service registry. The URL is constructed as localhost, the exposed port is 8080, and the name of the service package, csip. The resulting URL is http://localhost:8080/csip.

Our temperature conversion service descriptor is returned, the only one in this package. Name and description are obtained "on-the-fly" from CSIP annotations in the service and the service URL is provided, too. If the package contains more than one service endpoint, they would all be listed in this JSON array.

Now we like to know how to call the conversion service. The service parameter and input signature can be queried by invoking a HTTP/GET against the service endpoint:

 $ curl http://localhost:8080/csip/m/conv/1.0
 {
   "metainfo": {},
   "parameter": [{
     "name": "temp_c",
     "value": "45.2",
     "unit": "C",
     "description": "Temperature in C"
   }]
 }

This call returns a JSON template that shows a valid service request for our conversion service. Where does this information come from? Now, service source annotations come to play. CSIP and OMS3 introspect the service annotations and construct an example JSON input payload. You recognize the mapping between the JSON elements and the annotations in the service source. CSIP services are self-describing. The Web Processing Services (WPS) multi-level approach of cataloging services and service signature exploration is also used in CSIP as seen in this little example. Those steps are optional but help to check the service availability.

Now we are ready get to the third level, finally executing the service.

Execute the Service

We can simply store the request template by redirecting the curl output of the previous command to a file 'req.json'.

 $ curl -s http://localhost:8080/csip/m/conv/1.0 >req.json

Note: The '-s' option puts curl into silent mode.

Since we have captured a valid request, we can just change the temperature input value in req.json to '5.123'. The stream editor (sed) allows this with a simple search and replace command.

 $ sed -i 's/45.2/5.123/' req.json

Note: The sed '-i' option will replace the numbers in-place. You can use any other text editor to achieve the same result.

A CSIP service will always be invoked with a HTTP/POST request, where the request JSON is the HTTP body. It will execute the service on the server (localhost) and return the result as JSON payload. The curl POST command below passes the content of the req.json file to the service URL as 'application/json'. The curl response is shown.

 $ curl -X POST -H "content-type:application/json" \
       -d @req.json http://localhost:8080/csip/m/conv/1.0 
 {
  "metainfo": {
    "status": "Finished",
    "suid": "63eff3f9-ebeb-11e5-bb63-658e82f5a7a4",
    "tstamp": "2016-03-16 20:53:13",
    "service_url": "http://localhost:8080/csip-example/m/conv/1.0",
    "request_ip": "127.0.0.1",
    "csip.version": "$version: 2.1.1 a0d47e2e9e7d 2016-03-16 $",
    "cpu_time": 12,
    "expiration_date": "2016-03-16 20:53:33"
  },
 "parameter": [{
    "name": "temp_c",
    "value": "5.123",
    "unit": "C",
    "description": "Temperature in C"
  }],
  "result": [{
    "name": "temp_f",
    "value": "41.2214",
    "unit": "F",
    "description": "Temperature in F"
  }]
 }

Since this is all RESTful, the response JSON contains the input parameter as well as the result, the converted temperature. CSIP also used the annotations of the the temp_f field in the source to provide meaningful result info. The response contains also metainfo about the service with status information about the execution. All those JSON elements are further explained in the manual.

A client may now parse this JSON response and extract the result value for temp_f, the number 41.2214. The manual also shows this in various languages and platforms. Here, we leave it with the curl output for us to look at.

Once we are done with testing, the service can be shut down.

 $ sudo docker stop csip_rt

This will stop the running container and the deployed service with it. The container still exists and can be restarted later with the conversion service intact (sudo docker restart csip_rt).

Finally, removing the service container from the system is done by executing the 'docker rm' command, the images remain on the machine ready to be use used for the next development session.

 $ sudo docker rm csip_rt

Summary

This brief tutorial demonstrated the development, deployment and use of a simple CSIP service. Docker helped making this a painless experience, mostly with respect to the build process. For serious CSIP development, one might want to setup their own software stack on a development machine and download and install the required software.

Temperature conversion is a simple one-line equation. This example shows the concise approach of combined annotations in JAX-RS, CSIP, and OMS3 to easily put together a modeling service. CSIP is designed from the ground up to serve complex models in a scalable setup. It was tested on hundreds of service nodes. It takes into account failover and supports responsiveness of applications by providing asynchronous access to services of long running models.

CSIP is currently being used to serve (legacy) models for erosion, water quantity, water quality, rangeland hydrology, curve number hydrology, water supply forecasting, watershed delineation, detrended kriging, natural resource analysis and assessment, and more . CSIP services are also providing data for soils, climate, and agricultural land management information, etc. A comprehensive listing of available CSIP services can be found under https://alm.engr.colostate.edu/cb/project/csip.



[1] Using different images for development and deployment makes good use of the 'separation of concerns' principle and follows best practices for isolating aspects. Development environments always differ from deployment infrastructures. Other tools such as Fig or Ansible may be used later for real world deployment.