A Scala library to load STIX-2 objects to a Neo4j graph database
Neo4j "is a highly scalable native graph database that leverages data relationships as first-class entities, helping enterprises build intelligent applications to meet today’s evolving data challenges." In essence, a graph database and processing engine that is used here for storing Stix objects and their relationships.
stixtoneolib provides a set of classes and methods to convert STIX-2 domain objects (SDO) and relationships (SRO) to Neo4j nodes and relations using the Java Neo4j API. The library adds the converted STIX-2 objects and bundles to an existing Neo4j graph database or creates a new one. The library includes methods for processing text files containing STIX-2 bundles in JSON format and zip files.
Installation and packaging
stixtoneolib is a library for use in Scala (2.13.3) applications to convert and load STIX-2 objects into a Neo4j graph database. To use the latest release add the following dependency to your build.sbt:
libraryDependencies += "com.github.workingDog" %% "stixtoneolib" % "0.6"
The current source code version is 0.7-SNAPSHOT using scala 2.13.3.
To compile and package stixtoneolib from source use SBT. To compile and generate a jar file from source:
This will produce stixtoneolib-0.7-SNAPSHOT.jar in the ./target/scala-2.13 directory. See the build.sbt file for the required dependencies.
To publish the library to your local (Ivy) repository, simply type:
Then put this in your Scala application build.sbt file:
libraryDependencies += "com.github.workingDog" %% "stixtoneolib" % "0.7-SNAPSHOT"
The main class for loading STIX-2 objects into a Neo4j database is Neo4jLoader. This class constructor requires a Neo4j database directory name, which is used to connect to an existing database or create a new database. In addition a Logger can be passed-in implicitly to log the loading progress, defaults to no Logger if absent. Neo4jLoader has two main methods:
loadIntoNeo4j(bundle) to load a bundle of STIX-2 objects into a Neo4j database loadIntoNeo4j(stix) to load a STIX-2 object into a Neo4j database
Neo4jLoader delegates the creation of Neo4j nodes and relations to NodesMaker and RelationsMaker classes respectively. Typically, nodes are created first followed by the relations.
In addition the Neo4jFileLoader helper class has four methods for processing files of STIX-2 bundles:
loadBundleFile(infile) to load a file containing a bundle of STIX-2 objects loadBundleZipFile(infile) to load a zip file containing files each having a bundle of STIX-2 objects loadLargeTextFile(infile) to load one line at a time a file containing a STIX-2 object on one line loadLargeZipTextFile(infile) to load one line at a time a zip file containing files each having a STIX-2 object on one line
An example use of stixtoneolib is StixToNeoDB which loads files of STIX-2 data into a Neo4j database.
Dependencies and requirements
See also the build.sbt file.
Neo4j Community Edition should be installed to process the results.
work in progress.