gRPC to REST gateway generates a reverse-proxy server which converts REST HTTP API into gRPC. This project is based on work done by grpcgateway.
This project has two main modules:
-
Runtime library that serves a bridge between HTTP request and gRPC server and responsible for converting to and from HTTP to gRPC.
-
Code generation module which generates runtime handler(s) from given
protobuf
definition.
There are three main classes in each runtime implementations along with server classe(s) to build REST server:
-
GrpcGatewayHandler
— base abstract class / trait responsible for dispatching given HTTP request to corresponding function in gRPC service. -
SwaggerHandler
— responsible for Open API yaml files and generate Swagger UI. -
GatewayServer
— responsible for starting HTTP server at the given host and port.
Three different implementations are provided for runtime library:
-
Netty based runtime library —
grpc-rest-gateway-runtime-netty
. -
Apache Pekko based runtime library —
grpc-rest-gateway-runtime-pekko
. -
Akka based runtime library —
grpc-rest-gateway-runtime-akka
.
gRPC-REST gateway has built in mapping between gRPC and HTTP status codes. Following is the mappings between two systems:
gRPC status code | HTTP status code |
---|---|
OK |
OK (200) |
DATA_LOSS |
Partial Content (206) |
INVALID_ARGUMENT, OUT_OF_RANGE |
Bad Request (400) |
UNAUTHENTICATED |
Unauthorized(401) |
PERMISSION_DENIED |
Forbidden (403) |
NOT_FOUND, UNKNOWN |
Not Found (404) |
UNAVAILABLE |
Not Acceptable (406) |
ALREADY_EXISTS |
Conflict (409) |
ABORTED, CANCELLED |
Gone (410) |
FAILED_PRECONDITION |
Precondition Failed (412) |
INTERNAL |
Internal Server Error (500) |
UNIMPLEMENTED |
Not Implemented (501) |
DEADLINE_EXCEEDED |
Gateway Timeout (504) |
RESOURCE_EXHAUSTED |
Insufficient Storage (507) |
Note: Any unmapped code will be mapped to Internal Server Error (500)
.
Build io.grpc.StatusRuntimeException
using io.grpc.protobuf.StatusProto
to set corresponding status code and message.
import com.google.rpc.{Code, Status}
import io.grpc.protobuf.StatusProto
import scala.concurrent.Future
// handle bad request
Future.failed(StatusProto.toStatusRuntimeException(
Status
.newBuilder()
.setCode(Code.INVALID_ARGUMENT_VALUE)
.setMessage("Invalid argument")
.build())
)
// not found
Future.failed(StatusProto.toStatusRuntimeException(
Status
.newBuilder()
.setCode(Code.NOT_FOUND_VALUE)
.setMessage("Not found")
.build())
)
Following is how Protobuf to REST mapping will work as described in the documentation.
Given following Protobuf definition:
service Messaging {
rpc GetMessage(GetMessageRequest) returns (Message) {
option (google.api.http) = {
get: "/v1/messages/{message_id}/{sub.subfield}"
additional_bindings {
get: "/v1/messages/{message_id}"
}
};
}
rpc PostMessage(GetMessageRequest) returns (Message) {
option (google.api.http) = {
put: "/v1/messages/{message_id}"
body: "sub"
};
}
rpc PostMessage(GetMessageRequest) returns (Message) {
option (google.api.http) = {
post: "/v1/messages"
body: "*"
};
}
}
message GetMessageRequest {
message SubMessage {
string subfield = 1;
}
string message_id = 1;
SubMessage sub = 2;
}
message Message {
string text = 1;
}
Following mapping defines how HTTP request supposed to be constructed.
HTTP method: GET
Path: /v1/messages/{message_id}/{sub.subfield}
HTTP request: http://localhost:7070/v1/messages/xyz/abc
Mapping: Both message_id
and sub.subfield
are mapped as path variables
HTTP method: GET
Path: /v1/messages/{message_id}
HTTP request: http://localhost:7070/v1/messages/xyz?sub.subfield=abc
Mapping: message_id
is mapped as path variable while sub.subfield
is mapped as query parameter
HTTP method: PUT
Path: |http://localhost:7070/v1/messages/xyz
HTTP request: http://localhost:7070/v1/messages/xyz?sub.subfield=abc [body
: {"subfield": "sub"}]
Mapping: message_id
is mapped as path variable while sub
is mapped as body payload
HTTP method: POST
Path: /v1/messages
HTTP request: http://localhost:7070/v1/messages
Mapping: entire message is mapped as body payload
Code generation library is responsible for reading given Protobuf files and generating corresponding implementation of GrpcGatewayHandler
based on its runtime library. The runtime handler can be generated by passing implementationType
parameter:
There are three different plugins to generate runtime handlers, namely:
-
grpc_rest_gateway.gatewayGen(implementationType = "netty")
for Netty based implementation -
grpc_rest_gateway.gatewayGen(implementationType = "pekko")
for Pekko based implementation -
grpc_rest_gateway.gatewayGen(implementationType = "akka")
for Akka based implementation
Warning
|
Akka implementation hasn’t been tested yet due version dependency eviction in e2e testing module.
|
For example, for following Protobuf definition:
syntax = "proto3";
package rest_gateway_test.api;
import "scalapb/scalapb.proto";
import "google/api/annotations.proto";
import "common.proto";
option java_multiple_files = false;
option java_package = "rest_gateway_test.api.java_api";
option java_outer_classname = "TestServiceBProto";
option objc_class_prefix = "TS2P";
option (scalapb.options) = {
single_file: true
lenses: true
retain_source_code_info: true
preserve_unknown_fields: false
flat_package: true
package_name: "rest_gateway_test.api.scala_api"
};
// Test service B
service TestServiceB {
rpc GetRequest (rest_gateway_test.api.model.TestRequestB) returns (rest_gateway_test.api.model.TestResponseB) {
option (google.api.http) = {
get: "/restgateway/test/testserviceb"
};
}
rpc Process (rest_gateway_test.api.model.TestRequestB) returns (rest_gateway_test.api.model.TestResponseB) {
option (google.api.http) = {
post: "/restgateway/test/testserviceb"
body: "*"
};
}
}
swagger: '2.0'
info:
version: 3.1.0
description: 'REST API generated from TestServiceB.proto'
title: 'TestServiceB.proto'
tags:
- name: TestServiceB
description: Test service B
schemes:
- http
- https
consumes:
- 'application/json'
produces:
- 'application/json'
paths:
/restgateway/test/testserviceb:
get:
tags:
- TestServiceB
summary:
'GetRequest'
description:
'Generated from rest_gateway_test.api.TestServiceB.GetRequest'
produces:
['application/json']
responses:
200:
description: 'Normal response'
schema:
$ref: "#/definitions/TestResponseB"
parameters:
- name: requestId
in: query
type: integer
format: int64
post:
tags:
- TestServiceB
summary:
'Process'
description:
'Generated from rest_gateway_test.api.TestServiceB.Process'
produces:
['application/json']
responses:
200:
description: 'Normal response'
schema:
$ref: "#/definitions/TestResponseB"
parameters:
- in: 'body'
name: body
schema:
$ref: "#/definitions/TestRequestB"
definitions:
TestRequestB:
type: object
properties:
requestId:
type: integer
format: int64
TestResponseB:
type: object
properties:
success:
type: boolean
request_id:
type: integer
format: int64
result:
type: string
/*
* Generated by GRPC-REST gateway compiler. DO NOT EDIT.
*/
package rest_gateway_test.api.scala_api
import scalapb.GeneratedMessage
import io.grpc.ManagedChannel
import io.netty.handler.codec.http.{HttpMethod, QueryStringDecoder}
import com.improving.grpc_rest_gateway.runtime
import runtime.core.*
import runtime.handlers.*
import rest_gateway_test.api.model.TestRequestB
import scala.concurrent.{ExecutionContext, Future}
import scala.util.Try
object TestServiceBGatewayHandler {
private val GetGetRequestPath = "/restgateway/test/testserviceb"
private val PostProcessPath = "/restgateway/test/testserviceb"
def apply(channel: ManagedChannel)(implicit ec: ExecutionContext): TestServiceBGatewayHandler =
new TestServiceBGatewayHandler(channel)
}
class TestServiceBGatewayHandler(channel: ManagedChannel)(implicit ec: ExecutionContext)
extends GrpcGatewayHandler(channel)(ec) {
import TestServiceBGatewayHandler.*
override val serviceName: String = "TestServiceB"
override val specificationName: String = "TestServiceB"
private val client = TestServiceBGrpc.stub(channel)
override protected val httpMethodsToUrisMap: Map[String, Seq[String]] = Map(
"GET" -> Seq(
GetGetRequestPath
),
"POST" -> Seq(
PostProcessPath
)
)
override protected def dispatchCall(method: HttpMethod, uri: String, body: String): Future[GeneratedMessage] = {
val queryString = new QueryStringDecoder(uri)
val path = queryString.path
val methodName = method.name
if (isSupportedCall(HttpMethod.GET.name, GetGetRequestPath, methodName, path))
dispatchGetRequest(mergeParameters(GetGetRequestPath, queryString))
else if (isSupportedCall(HttpMethod.POST.name, PostProcessPath, methodName, path))
dispatchProcess(body)
else Future.failed(GatewayException.toInvalidArgument(s"No route defined for $methodName($path)"))
}
private def dispatchGetRequest(parameters: Map[String, Seq[String]]) = {
val input = Try {
val requestId = parameters.toLongValue("requestId")
TestRequestB(requestId = requestId)
}
toResponse(input, client.getRequest)
}
private def dispatchProcess(body: String) = {
val input = parseBody[TestRequestB](body)
toResponse(input, client.process)
}
}
/*
* Generated by GRPC-REST gateway compiler. DO NOT EDIT.
*/
package rest_gateway_test.api.scala_api
import com.improving.grpc_rest_gateway.runtime
import runtime.core._
import runtime.handlers.GrpcGatewayHandler
import rest_gateway_test.api.model.TestRequestB
import org.apache.pekko
import pekko.grpc.GrpcClientSettings
import pekko.actor.ClassicActorSystemProvider
import pekko.http.scaladsl.server.Route
import pekko.http.scaladsl.server.Directives._
import scala.concurrent.ExecutionContext
import scala.util.Try
class TestServiceBGatewayHandler(settings: GrpcClientSettings)(implicit sys: ClassicActorSystemProvider)
extends GrpcGatewayHandler {
private implicit val ec: ExecutionContext = sys.classicSystem.dispatcher
private val client = TestServiceBClient(settings)
override val specificationName: String = "TestServiceB"
override val route: Route = handleExceptions(exceptionHandler) {
pathPrefix("restgateway") {
pathPrefix("test") {
pathPrefix("testserviceb") {
pathEnd {
concat(
get {
parameterMultiMap { queryParameters =>
dispatchGetRequest(queryParameters)
}
},
post {
entity(as[String]) { body =>
dispatchProcess(body)
}
}
)
}
}
}
}
}
private def dispatchGetRequest(parameters: Map[String, Seq[String]]) = {
val input = Try {
val requestId = parameters.toLongValue("requestId")
TestRequestB(requestId = requestId)
}
completeResponse(input, client.getRequest)
}
private def dispatchProcess(body: String) = {
val input = parseBody[TestRequestB](body)
completeResponse(input, client.process)
}
}
object TestServiceBGatewayHandler {
def apply(settings: GrpcClientSettings)(implicit sys: ClassicActorSystemProvider): GrpcGatewayHandler =
new TestServiceBGatewayHandler(settings)
def apply(clientName: String)(implicit sys: ClassicActorSystemProvider): GrpcGatewayHandler =
TestServiceBGatewayHandler(GrpcClientSettings.fromConfig(clientName))
}
To generate Scala classes for gateway handler and Swagger documentation add following in plugin.sbt
:
addSbtPlugin("com.thesamet" % "sbt-protoc" % "1.0.7")
libraryDependencies ++= Seq(
"com.thesamet.scalapb" %% "compilerplugin" % "0.11.17",
"io.github.sfali23" % "grpc-rest-gateway-code-gen" % "0.8.1"
)
And following in the build.sbt
:
Compile / PB.targets := Seq(
scalapb.gen() -> (Compile / sourceManaged).value / "scalapb",
grpc_rest_gateway.gatewayGen(implementationType = "netty") -> (Compile / sourceManaged).value / "scalapb",
grpc_rest_gateway.swaggerGen() -> (Compile / resourceDirectory).value / "scalapb"
)
// generate code for Scala 3
Compile / PB.targets := Seq(
scalapb.gen(scala3Sources = true) -> (Compile / sourceManaged).value / "scalapb",
grpc_rest_gateway.gatewayGen(scala3Sources = true, implementationType = "netty") -> (Compile / sourceManaged).value / "scalapb",
grpc_rest_gateway.swaggerGen() -> (Compile / resourceDirectory).value / "scalapb"
)
// TODO: generate resource files in target and add resource path
// add `**/resources/specs/**/*.yml` in your .gitignore to ignore generated yml files
Add following dependencies
val ScalaPb: String = scalapb.compiler.Version.scalapbVersion
val GrpcJava: String = scalapb.compiler.Version.grpcJavaVersion
libraryDependencies ++= Seq(
"io.github.sfali23" %% "grpc-rest-gateway-runtime-netty" % "0.8.1",
"com.thesamet.scalapb" %% "compilerplugin" % V.ScalaPb % "compile;protobuf",
"com.thesamet.scalapb" %% "scalapb-runtime" % V.ScalaPb % "compile;protobuf",
"com.thesamet.scalapb" %% "scalapb-runtime-grpc" % V.ScalaPb,
"io.grpc" % "grpc-netty" % V.GrpcJava,
"com.thesamet.scalapb" %% "scalapb-json4s" % V.ScalaPbJson,
"com.thesamet.scalapb.common-protos" %% "proto-google-common-protos-scalapb_0.11" % "2.9.6-0" % "compile,protobuf"
)
Add following dependencies:
val ScalaPb: String = scalapb.compiler.Version.scalapbVersion
val GrpcJava: String = scalapb.compiler.Version.grpcJavaVersion
libraryDependencies ++= Seq(
"io.github.sfali23" %% "grpc-rest-gateway-runtime-pekko" % "0.8.1",
"org.apache.pekko" %% "pekko-actor" % "1.1.2",
"org.apache.pekko" %% "pekko-actor-typed" % "1.1.2",
"org.apache.pekko" %% "pekko-stream-typed" % "1.1.2",
"org.apache.pekko" %% "pekko-http" % "1.1.0",
"org.apache.pekko" %% "pekko-grpc-runtime" % "1.1.1",
"com.thesamet.scalapb.common-protos" %% "proto-google-common-protos-scalapb_0.11" % "2.9.6-0" % "compile,protobuf"
)
Note
|
Your project should be configured at least to generate Pekko gRPC client code. |
Implement your gRPC services as per your need and run gRPC server. Gateway server can be build and run as follows:
import com.improving.grpc_rest_gateway.runtime.server.GatewayServer
import rest_gateway_test.api.scala_api.TestServiceB.TestServiceBGatewayHandler
import scala.concurrent.ExecutionContext
implicit val ex: ExecutionContext = ??? // provide ExecutionContext
val server = GatewayServer(
serviceHost = "localhost",
servicePort = 8080, // assuming gRPC server is running on port 8080
gatewayPort = 7070, // REST end point is running at port 7070
toHandlers = channel => Seq(TestServiceBGatewayHandler(channel)),
executor = None, // Executor is useful if you want to allocate different thread pool for REST endpoint
usePlainText = true
)
server.start()
// stop server once done
server.stop()
// via Typesafe config
val mainConfig = ConfigFactory.load()
val server = GatewayServer(
config = mainConfig.getConfig("rest-gateway"),
toHandlers = channel => Seq(TestServiceBGatewayHandler(channel)),
executor = None
)
Alternatively serviceHost
, servicePort
, gatewayPort
, usePlainText
can be overriden via environment variables GRPC_HOST
, GRPC_SERVICE_PORT
, REST_GATEWAY_PORT
, and GRPC_USE_PLAIN_TEXT
respectively.
// rest-gateway config is defined as follows:
rest-gateway {
host = "0.0.0.0"
host = ${?GRPC_HOST}
service-port = 8080
service-port = ${?GRPC_SERVICE_PORT}
gateway-port = 7070
gateway-port = ${?REST_GATEWAY_PORT}
use-plain-text = "true"
use-plain-text = ${?GRPC_USE_PLAIN_TEXT}
}
Providing Pekko gRPC client configuration is defined as follows:
pekko {
grpc {
client {
pekko-gateway {
host = "0.0.0.0" // gRPC host
port = 8080 // grPC port
use-tls = false
}
}
}
}
// rest gateway config
rest-gateway {
host = "0.0.0.0"
host = ${?REST_GATEWAY_HOST}
port = 7070
port = ${?REST_GATEWAY_PORT}
}
Gateway server can be initialized as follows:
implicit val system: ActorSystem[?] = ActorSystem[Nothing](Behaviors.empty, "grpc-rest-gateway-pekko")
val settings = GrpcClientSettings.fromConfig("pekko-gateway")
val config = system.settings.config
GatewayServer(
config.getConfig("rest-gateway"),
TestServiceBGatewayHandler(settings)
).run()
Providing Akka gRPC client configuration is defined as follows:
akka {
grpc {
client {
pekko-gateway {
host = "0.0.0.0" // gRPC host
port = 8080 // grPC port
use-tls = false
}
}
}
}
// rest gateway config
rest-gateway {
host = "0.0.0.0"
port = 7070 // Gateway port
}
Gateway server can be initialized as follows:
implicit val system: ActorSystem[?] = ActorSystem[Nothing](Behaviors.empty, "grpc-rest-gateway-pekko")
val settings = GrpcClientSettings.fromConfig("pekko-gateway")
val config = system.settings.config
GatewayServer(
config.getConfig("rest-gateway"),
TestServiceBGatewayHandler(settings)
).run()
e2e
module contains test code and a sample app.
Tests can be run as follows:
sbt "nettyJVM212Test"
sbt "nettyJVM213Test"
sbt "nettyJVM3Test"
sbt "pekkoJVM212Test"
sbt "pekkoJVM213Test"
sbt "pekkoJVM3Test"
Sample app can be run as follows:
# For Scala 2.12
sbt "nettyJVM212Run"
sbt "pekkoJVM212Run"
# # For Scala 2.13
sbt "nettyJVM213Run"
sbt "pekkoJVM213Run"
# For Scala 3
sbt "nettyJVM3Run"
sbt "pekkoJVM3Run"
Open browser and paste following URL in address bar http://localhost:7070
, you should see Open API specification for service.
A reference implementation of Swagger petstore is attempted here.
Run app as follows:
sbt "service / run local"
Open browser and paste following URL in address bar http://localhost:7070
, you should see Open API specification for petstore service.
Following is corresponding proto file.