Chapter 7. JanusGraph Server

JanusGraph uses the Gremlin Server engine as the server component to process and answer client queries. When packaged in JanusGraph, Gremlin Server is called JanusGraph Server.

JanusGraph Server must be started manually in order to use it. JanusGraph Server provides a way to remotely execute Gremlin scripts against one or more JanusGraph instances hosted within it. This section will describe how to use the WebSockets configuration, as well as describe how to configure JanusGraph Server to handle REST-style interactions.

7.1. Getting Started

7.1.1. Using the Pre-Packaged Distribution

The JanusGraph release comes pre-configured to run JanusGraph Server out of the box leveraging a sample Cassandra and ElasticSearch configuration to allow users to get started quickly with JanusGraph Server. This configuration defaults to client applications that can connect to JanusGraph Server via WebSockets with a custom subprotocol. There are a number of clients developed in different languages to help support the subprotocol. The most familiar client to use the WebSocket interface is the gremlin.sh shell. The quick-start bundle is not intended to be representative of a production installation, but does provide a way to perform development with JanusGraph Server, run tests and see how the components are wired together. To use this default configuration:

  • Download a copy of the current janusgraph-$VERSION.zip file from the Releases page
  • Unzip it and enter the janusgraph-$VERSION directory
  • Run bin/janusgraph.sh start. This step will start Gremlin Server with Cassandra/ES forked into a separate process.
$ bin/janusgraph.sh start
Forking Cassandra...
Running `nodetool statusthrift`.. OK (returned exit status 0 and printed string "running").
Forking Elasticsearch...
Connecting to Elasticsearch (127.0.0.1:9300)... OK (connected to 127.0.0.1:9300).
Forking Gremlin-Server...
Connecting to Gremlin-Server (127.0.0.1:8182)... OK (connected to 127.0.0.1:8182).
Run gremlin.sh to connect.

7.1.1.1. Connecting to Gremlin Server

After running janusgraph.sh, Gremlin Server will be ready to listen for WebSocket connections. The easiest way to test the connection is with Gremlin Console.

Start Gremlin Console with bin/gremlin.sh and use the :remote and :> commands to issue Gremlin to Gremlin Server:

$  bin/gremlin.sh
         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----
plugin activated: tinkerpop.server
plugin activated: tinkerpop.hadoop
plugin activated: tinkerpop.utilities
plugin activated: janusgraph.imports
plugin activated: tinkerpop.tinkergraph
gremlin> :remote connect tinkerpop.server conf/remote.yaml
==>Connected - localhost/127.0.0.1:8182
gremlin> :> graph.addVertex("name", "stephen")
==>v[256]
gremlin> :> g.V().values('name')
==>stephen

The :remote command tells the console to configure a remote connection to Gremlin Server using the conf/remote.yaml file to connect. That file points to a Gremlin Server instance running on localhost. The :> is the "submit" command which sends the Gremlin on that line to the currently active remote.

7.2. JanusGraph Server as a WebSocket Endpoint

The default configuration described in Section 7.1, “Getting Started” is already a WebSocket configuration. If you want to alter the default configuration to work with your own Cassandra or HBase environment rather than use the quick start environment, follow these steps:

To Configure JanusGraph Server For WebSockets

  1. Test a local connection to a JanusGraph database first. This step applies whether using the Gremlin Console to test the connection, or whether connecting from a program. Make appropriate changes in a properties file in the ./conf directory for your environment. For example, edit ./conf/janusgraph-hbase.properties and make sure the storage.backend, storage.hostname and storage.hbase.table parameters are specified correctly. For more information on configuring JanusGraph for various storage backends, see Part III, “Storage Backends”. Make sure the properties file contains the following line:

    gremlin.graph=org.janusgraph.core.JanusGraphFactory
  2. Once a local configuration is tested and you have a working properties file, copy the properties file from the ./conf directory to the ./conf/gremlin-server directory.

    cp conf/janusgraph-hbase.properties conf/gremlin-server/socket-janusgraph-hbase-server.properties
  3. Copy ./conf/gremlin-server/gremlin-server.yaml to a new file called socket-gremlin-server.yaml. Do this in case you need to refer to the original version of the file

    cp conf/gremlin-server/gremlin-server.yaml conf/gremlin-server/socket-gremlin-server.yaml
  4. Edit the socket-gremlin-server.yaml file and make the following updates:

    1. If you are planning to connect to JanusGraph Server from something other than localhost, update the ip address for host:

      host: 10.10.10.100
    2. Update the graphs section to point to your new properties file so the JanusGraph Server can find and connect to your JanusGraph instance:

      graphs: {
        graph: conf/gremlin-server/socket-janusgraph-hbase-server.properties}
  5. Start the JanusGraph Server, specifying the yaml file you just configured:

    bin/gremlin-server.sh ./conf/gremlin-server/socket-gremlin-server.yaml
    [Important]Important

    Do not use bin/janusgraph.sh. That starts the default configuration, which starts a separate Cassandra/ElasticSearch environment.

  6. The JanusGraph Server should now be running in WebSocket mode and can be tested by following the instructions in Section 7.1.1.1, “Connecting to Gremlin Server”

7.3. JanusGraph Server as a REST-style Endpoint

The default configuration described in Section 7.1, “Getting Started” is a WebSocket configuration. If you want to alter the default configuration in order to use JanusGraph Server as a REST endpoint for your JanusGraph database, follow these steps:

To Configure JanusGraph Server For Rest

  1. Test a local connection to a JanusGraph database first. This step applies whether using the Gremlin Console to test the connection, or whether connecting from a program. Make appropriate changes in a properties file in the ./conf directory for your environment. For example, edit ./conf/janusgraph-hbase.properties and make sure the storage.backend, storage.hostname and storage.hbase.table parameters are specified correctly. For more information on configuring JanusGraph for various storage backends, see Part III, “Storage Backends”. Make sure the properties file contains the following line:

    gremlin.graph=org.janusgraph.core.JanusGraphFactory
  2. Once a local configuration is tested and you have a working properties file, copy the properties file from the ./conf directory to the ./conf/gremlin-server directory.

    cp conf/janusgraph-hbase.properties conf/gremlin-server/rest-janusgraph-hbase-server.properties
  3. Copy ./conf/gremlin-server/gremlin-server.yaml to a new file called rest-gremlin-server.yaml. Do this in case you need to refer to the original version of the file

    cp conf/gremlin-server/gremlin-server.yaml conf/gremlin-server/rest-gremlin-server.yaml
  4. Edit the rest-gremlin-server.yaml file and make the following updates:

    1. If you are planning to connect to JanusGraph Server from something other than localhost, update the ip address for host:

      host: 10.10.10.100
    2. Update the channelizer setting to specify the HttpChannelizer:

      channelizer: org.apache.tinkerpop.gremlin.server.channel.HttpChannelizer
    3. Update the graphs section to point to your new properties file so the JanusGraph Server can find and connect to your JanusGraph instance:

      graphs: {
        graph: conf/gremlin-server/rest-janusgraph-hbase-server.properties}
  5. Start the JanusGraph Server, specifying the yaml file you just configured:

    bin/gremlin-server.sh ./conf/gremlin-server/rest-gremlin-server.yaml
  6. The JanusGraph Server should now be running in REST mode and available for testing. curl can be used to verify the server is working:

    curl -XPOST -Hcontent-type:application/json -d '{"gremlin":"g.V().count()"}' http://[IP for JanusGraph server host]:8182

7.4. Advanced JanusGraph Server Configurations

7.4.1. WebSocket versus REST

JanusGraph Server must be run with a configuration for either WebSockets (for gremlin shell or other WebSocket programs) or re-configured for REST to work with REST clients.

There is, currently, no way to configure a single running instance of JanusGraph Server to communicate simultaneously with clients talking WebSockets and REST. However, it is possible to create configuration files (following the steps in this chapter) for both WebSockets and REST and start two instances of the JanusGraph Server by pointing it to the various configurations when it is started. In this way, the same JanusGraph database instance can be reached through different JanusGraph Servers through both WebSocket and REST calls. Make sure the port parameter in each yaml configuration is different if starting more than one JanusGraph Server on the same machine:

port: 8182

In the future, the option to have one running instance of JanusGraph Server supporting multiple protocols may be possible.

7.4.2. Using TinkerPop Gremlin Server with JanusGraph

Since JanusGraph Server is a TinkerPop Gremlin Server packaged with configuration files for JanusGraph, a version compatible TinkerPop Gremlin Server can be downloaded separately and used with JanusGraph. Get started by downloading the appropriate version of Gremlin Server, which needs to match a version supported by the JanusGraph version in use.

[Important]Important

Any references to file paths in this section refer to paths under a TinkerPop distribution for Gremlin Server and not a JanusGraph distribution with the JanusGraph Server, unless specifically noted.

Configuring a standalone Gremlin Server to work with JanusGraph is similar to configuring the packaged JanusGraph Server. You should be familiar with graph configuration. Basically, the Gremlin Server yaml file points to graph-specific configuration files that are used to instantiate JanusGraph instances that it will then host. In order to instantiate these Graph instances, Gremlin Server requires that the appropriate libraries and dependencies for the JanusGraph be available on its classpath.

For purposes of demonstration, these instructions will outline how to configure the BerkeleyDB backend for JanusGraph in Gremlin Server. As stated earlier, Gremlin Server needs JanusGraph dependencies on its classpath. Invoke the following command replacing $VERSION with the version of JanusGraph to use:

bin/gremlin-server.sh -i org.janusgraph janusgraph-all $VERSION

When this process completes, Gremlin Server should now have all the JanusGraph dependencies available to it and will thus be able to instantiate JanusGraph objects.

[Important]Important

The above command uses Groovy Grape and if it is not configured properly download errors may ensue. Please refer to this section of the TinkerPop documentation for more information around setting up ~/.groovy/grapeConfig.xml.

Create a file called GREMLIN_SERVER_HOME/conf/janusgraph.properties with the following contents:

gremlin.graph=org.janusgraph.core.JanusGraphFactory
storage.backend=berkeleyje
storage.directory=db/berkeley

Configuration of other backends is similar. See Part III, “Storage Backends”. If using Cassandra, then use Cassandra configuration options in the janusgraph.properties file. The only important piece to leave unchanged is the gremlin.graph setting which should always use JanusGraphFactory. This setting tells Gremlin Server how to instantiate a JanusGraph instance.

Next create a file called GREMLIN_SERVER_HOME/conf/gremlin-server-janusgraph.yaml that has the following contents:

host: localhost
port: 8182
graphs: {
  graph: conf/janusgraph.properties}
plugins:
  - janusgraph.imports
scriptEngines: {
  gremlin-groovy: {
    scripts: [scripts/janusgraph.groovy]}}
serializers:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { useMapperFromGraph: graph }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
metrics: {
  slf4jReporter: {enabled: true, interval: 180000}}

There are several important parts to this configuration file as they relate to JanusGraph.

  1. In the graphs map, there is a key called graph and its value is conf/janusgraph.properties. This tells Gremlin Server to instantiate a Graph instance called "graph" and use the conf/janusgraph.properties file to configure it. The "graph" key becomes the unique name for the Graph instance in Gremlin Server and it can be referenced as such in the scripts submitted to it.
  2. In the plugins list, there is a reference to janusgraph.imports, which tells Gremlin Server to initialize the "JanusGraph Plugin". The "JanusGraph Plugin" will auto-import JanusGraph specific classes for usage in scripts.
  3. Note the scripts key and the reference to scripts/janusgraph.groovy. This Groovy file is an initialization script for Gremlin Server and that particular ScriptEngine. Create scripts/janusgraph.groovy with the following contents:
def globals = [:]
globals << [g : graph.traversal()]

The above script creates a Map called globals and assigns to it a key/value pair. The key is g and its value is a TraversalSource generated from graph, which was configured for Gremlin Server in its configuration file. At this point, there are now two global variables available to scripts provided to Gremlin Server - graph and g.

At this point, Gremlin Server is configured and can be used to connect to a new or existing JanusGraph database. To start the server:

$ bin/gremlin-server.sh conf/gremlin-server-janusgraph.yaml
[INFO] GremlinServer -
         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----

[INFO] GremlinServer - Configuring Gremlin Server from conf/gremlin-server-janusgraph.yaml
[INFO] MetricManager - Configured Metrics Slf4jReporter configured with interval=180000ms and loggerName=org.apache.tinkerpop.gremlin.server.Settings$Slf4jReporterMetrics
[INFO] GraphDatabaseConfiguration - Set default timestamp provider MICRO
[INFO] GraphDatabaseConfiguration - Generated unique-instance-id=7f0000016240-ubuntu1
[INFO] Backend - Initiated backend operations thread pool of size 8
[INFO] KCVSLog$MessagePuller - Loaded unidentified ReadMarker start time 2015-10-02T12:28:24.411Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@35399441
[INFO] GraphManager - Graph [graph] was successfully configured via [conf/janusgraph.properties].
[INFO] ServerGremlinExecutor - Initialized Gremlin thread pool.  Threads in pool named with pattern gremlin-*
[INFO] ScriptEngines - Loaded gremlin-groovy ScriptEngine
[INFO] GremlinExecutor - Initialized gremlin-groovy ScriptEngine with scripts/janusgraph.groovy
[INFO] ServerGremlinExecutor - Initialized GremlinExecutor and configured ScriptEngines.
[INFO] ServerGremlinExecutor - A GraphTraversalSource is now bound to [g] with graphtraversalsource[standardjanusgraph[berkeleyje:db/berkeley], standard]
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-stringd with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0
[INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 8 and boss thread pool of 1.
[INFO] GremlinServer$1 - Channel started at port 8182.

The following section explains how to connect to the running server.

7.4.2.1. Connecting to JanusGraph via Gremlin Server

Gremlin Server will be ready to listen for WebSocket connections when it is started. The easiest way to test the connection is with Gremlin Console.

Follow the instructions here Section 7.1.1.1, “Connecting to Gremlin Server” to verify the Gremlin Server is working.

[Important]Important

A difference you should understand is that when working with JanusGraph Server, the Gremlin Console is started from underneath the JanusGraph distribution and when following the test instructions here for a standalone Gremlin Server, the Gremlin Console is started from under the TinkerPop distribution.

GryoMapper mapper = GryoMapper.build().addRegistry(JanusGraphIoRegistry.INSTANCE).create();
Cluster cluster = Cluster.build().serializer(new GryoMessageSerializerV1d0(mapper)).create();
Client client = cluster.connect();
client.submit("g.V()").all().get();

By adding the JanusGraphIoRegistry to the org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, the driver will know how to properly deserialize custom data types returned by JanusGraph.