Apache Bahir provides extensions to distributed analytics platforms such as Apache Spark & Apache Flink.
Apache Bahir origins
The Initial Bahir source code (see issue BAHIR-1) containing the source for the Apache Spark streaming connectors for akka, mqtt, twitter, zeromq extracted from Apache Spark revision 8301fad (before the deletion of the streaming connectors akka, mqtt, twitter, zeromq).
Source code structure
Source code folder structure:
- streaming-akka - examples/src/main/... - src/main/... - streaming-mqtt - examples - src - python - ...
Bahir is built using Apache Maven. To build Bahir and its example programs, run:
mvn -DskipTests clean install
Testing first requires building Bahir. Once Bahir is built, tests can be run using:
Each extension currently available in Apache Bahir has an example application located under the "examples" folder.
Currently, each submodule has its own README.md, with information on example usages and API.
- SQL Cloudant
- SQL Streaming Akka
- SQL Streaming MQTT
- Streaming Akka
- Streaming MQTT
- Streaming PubNub
- Streaming Google Pub/Sub
- Streaming Twitter
- Streaming ZeroMQ
Furthermore, to generate scaladocs for each module:
$ mvn package
Scaladocs is generated in,
MODULE_NAME/target/site/scaladocs/index.html. __ Where
MODULE_NAME is one of,
A note about Apache Spark integration
Currently, each module in Bahir is available through spark packages. Please follow linking sub section in module specific README.md for more details.