The leap from the original CyREST implementation to address the broader scope of the Cytoscape Automation initiative required new features and upgraded approaches in a number of technical areas:
-
❶
New CyREST access to Cytoscape apps
-
❷
New CyREST access to Cytoscape Command script operations
-
❸
Improved documentation infrastructure and content standards
-
❹
New interactive CyREST call prototyping
-
❺
Consistent mechanisms for calling CyREST and receiving return values
-
❻
Improved coverage of core Cytoscape functionality
Figure 2 shows the relationship between the Cytoscape Desktop and Cytoscape Automation workflows. The Cytoscape Desktop includes both the Cytoscape core (including CyCommands and CyREST) and apps sourced from the Cytoscape App store. Automation workflows execute outside of Cytoscape but use CyREST to leverage Cytoscape features. Figure 2 is annotated to show the components important in each facet of the Cytoscape Automation design, which are described in this section.
Note that for a workflow to access Cytoscape functionality, Cytoscape must be running and accessible via HTTP calls from the workflow execution environment.
Note that calling a CyREST endpoint requires the use of REST interface functions found in most modern languages. While all CyREST endpoints are accessible in this manner, we have created harmonization libraries for R and Python (described in the “Implementation” section) to enable quick and easy access to common Cytoscape Automation features. However, direct CyREST calls are required for all other endpoints, including those supplied by Cytoscape apps—see the “Python and R Harmonization Libraries” section for details.
New CyREST access to Cytoscape apps ❶
A large part of Cytoscape’s utility to researchers is provided by apps—their inclusion in Cytoscape Automation, facilitated by CyREST, greatly expands the functionality that can be leveraged via scripting workflows. Apps that support automation (called Automation Apps) can expose functionality either via a Function or Command interface [15].
A Function interface enables a script to pass complex parameters and receive return results of arbitrary length and complexity. While Functions can be called from scripting languages such as Python, R, and Javascript, they cannot be called from the Cytoscape Command Tool [16].
To create a Function in an existing app, the app author must add a new function that defines a CyREST endpoint using JAX-RS [17] annotations and which executes app-related code—most likely code that implements existing app functionality. The JAX-RS annotations define the endpoint name, the HTTP protocol [18], and the parameters to be passed. For example:
This defines the diffuse_with_options endpoint that accepts three parameters (networkSUID, networkViewSUID, and diffusionParameters) and returns a CIResponse structure. The @PUT annotation defines the HTTP method (as PUT), and the @Path annotation defines the endpoint path (/diffusion/v1/{networkSUID}/views/{networkViewSUID}/diffuse_with_options), which the client appends to CyREST’s base URL (http://localhost:1234) when calling diffuse_with_options. The @Produces and @Consumes annotations define the PUT payload and response as JSON [19].
An actual CyREST URL that calls diffuse_with_options might appear as http://localhost:1234/diffusion/v1/53/views/744/diffuse_with_options and would include JSON corresponding to the DiffusionParameters class as the HTTP PUT payload. For diffuse_with_options, a sample DiffusionParameters payload is:
As shown in Fig. 2, at runtime, CyREST’s JAX-RS connector parses the URL to extract the networkSUID and networkViewSUID values and parses the PUT payload to create a DiffusionParameters instance. JAX-RS calls the diffuse_with_options function with these values, which performs a diffusion operation and returns a CIResponse instance. Finally, JAX-RS encodes the CIResponse into JSON and returns it to the caller.
The process for exposing app features as Commands is different, as explained below.
New CyREST access to Cytoscape Command script operations ❷
A Command interface enables a script to execute Cytoscape Commands analogous to commands executed within a Unix or Windows terminal, and they offer similar argument structure and execution. Command executions can pass simple parameters and can return results of predefined length.
Users can execute Commands as single lines (via Cytoscape’s Command Tool [16]) or as scripts (either via Cytoscape Tools → Execute Command File menu or on the Cytoscape command line via the -S parameter). Scripting languages such as Python, R, and Javascript can execute them via CyREST using an HTTP POST operation and passing Command parameters as a JSON object. The endpoint path begins with /v1/commands and is followed by the Command namespace and the command name. A fully formed URL and POST payload for the diffuse_advanced Command is:
http://localhost:1234/v1/commands/diffusion/diffuse_advanced
As shown in Fig. 2, the Commands system leverages the Cytoscape Tunable/Task system [20] (i.e., CyCommands Manager and Cytoscape Core) originally defined to collect execution parameters via a dialog box and then execute a Java function. The function consumes the parameters, performs the Command operation, and possibly returns a result as a fetchable task state.
To create a Command in an existing app, the app author must first register the Command’s namespace and name in a TaskFactory via the app’s CyActivator. The name must be the name of a public function within the app, and the app author must add @Tunable annotations to the function to define Command parameters.
If an app already exposes a function as a Tunable/Task, enabling the function to be called as a Command can be as simple as registering the TaskFactory with a suitable namespace and name. If an app does not use Tunable/Tasks, it may be easier to expose app features as Functions (as described above).
Note that in CyREST previous to Cytoscape v3.6, Commands were available via an HTTP GET operation, where parameters passed on the URL (e.g., ?time = 0.1) and the result form and content was not JSON, and they varied with the Command. The GET form has been deprecated in favor of POST to allow more parameters and to enable JSON-structured parameters and return results.
Improved documentation infrastructure and content standards ❸
In the process of implementing Python and R support libraries and providing support for researchers producing scripting workflows, we found that the coverage and quality of CyREST’s Miredot-based [21] API documentation was a major impediment to productivity. We replaced Miredot with the popular Swagger [11] framework, which organizes CyREST endpoints by category, provides for more complete documentation, and presents an easy click-to-run web interface. This allowed us to more rigorously define and enforce the documentation standards that define an endpoint contract, including the context, purpose, caveats, parameters, and return results for each endpoint. Swagger also facilitates the documentation of structures (called models) pertinent to parameters and return results.
To access Swagger for Functions, use Cytoscape’s Help → Automation → CyREST API menu. For Commands, use Help → Automation → CyREST Command API.
For each Function and Command implemented in Cytoscape Core, we audited the documentation to verify that it contained meaningful and actionable content for each Swagger section according to best practices. Similarly, Automation App authors wrote their Swagger page documentation to the same standards.
For Functions, the CyREST Swagger Connector (see Fig. 2) synthesizes an endpoint’s Swagger documentation from text embedded in annotations attached to endpoint code. For Functions, a basic contract is defined by the @ApiOperation and @ApiParam annotations, which describe the endpoint generally and its parameters specifically. For the diffuse_with_options Function, these annotations might appear as follows:
In the @ApiOperation annotation, the value attribute contains a short description; the notes attribute contains the context, purpose, and caveats, and the response attribute identifies the model (i.e., class) for the return result. The @ApiParam annotation applies to each parameter, whether it is part of the URL (e.g., networkSUID and networkViewSUID) or the PUT payload (e.g., diffusionParameters). The value attribute describes the parameter, while the required attribute indicates whether the parameter must be present. Additional annotations describe possible results and models.
Figure 3 shows a sample Swagger page corresponding to the diffusion_with_options Function above.
For Commands, the CyREST Swagger and Commands Connectors (see Fig. 2) synthesize a similar page from OSGi properties and annotated fields within a TaskFactory. Command-level descriptions, for example, are synthesized from attributes supplied in the TaskFactory properties when the task is created in CyActivator:
Parameter-level descriptions are synthesized using @Tunable annotations applied to variables within each TaskFactory:
New interactive CyREST call prototyping ❹
A significant cost in most workflow authoring processes is experimentation with library functions to determine what types of calls achieve workflow goals. The Swagger documentation system addresses this in an innovative way by enabling a user to formulate and submit a CyREST endpoint call directly from the endpoint’s Swagger page.
Using the example in Fig. 3, once the user fills the endpoint’s parameter values, provided by the included Example Value, clicking on the Try it out! button results in a well-formed diffuse_with_options call to Cytoscape, which performs a diffusion and returns a result (as shown in Fig. 4). If the diffusion fails, an error result is returned. By experimentation, and without any programming skills, a user can quickly understand and productively use a CyREST endpoint, which informs the correct composition of a REST call using the workflow language’s REST interface.
Note that parameters for some endpoints are references to Cytoscape objects represented by SUIDs (e.g., POST /diffusion/v1/{networkSUID}/views/{networkViewSUID}/diffuse_with_options). A user can discover Cytoscape SUIDs by using Swagger to execute query endpoints (e.g., GET /v1/networks/currentNetwork).
Consistent mechanisms for calling CyREST and receiving return values ❺
To improve workflow author productivity, we created conventions for the data returned by CyREST endpoints and we revitalized the Python and R harmonization libraries (called py2cytoscape and RCy3).
But for minor exceptions, all CyREST endpoints now return their results in a standard JSON data structure called CIResponse [22], which has two main elements: data and errors. If the endpoint is successful, the endpoint returns its result in data and leaves errors empty—the exact result is endpoint-dependent and is described in the endpoint’s Swagger page. For example:
If the endpoint fails, it leaves data empty and returns errors, where errors[0] describes the endpoint error, and subsequent errors entries describe failures in any nested services that caused the endpoint failure, similar to a Java stack trace. For example:
The status contains an HTTP status describing the error. Type contains a URN unique to the endpoint and error (shortened here for readability, but actually containing “urn:cytoscape:ci:diffusion-app:v1:diffuse_with_options:2”), and the message describes the error in prose. If the caller needs to take action for one type of error as compared to another, it should compare the type value, not the message content. The link value is returned but is not used.
The separation of data and errors enables callers to centralize their CyREST calling code, thereby easing coding and maintenance burden on workflow and harmonization library authors. A centralized CyREST caller should return the data value and throw an exception if an error is received.
Improved coverage of core Cytoscape functionality ❻
Under Cytoscape Automation, the exposure of Cytoscape’s API via CyREST expanded from 113 available operations to 157. These new operations, as well as the data they consume and produce, are consistent with previous implementations. This API consistency follows the same Semantic Versioning [23] best practices laid out for Cytoscape core development.