Consider adopting gRPC to replace custom async client library

[beauremus]

https://grpc.io/docs/languages/python/

With DPM's adoption of gRPC, DPM clients can now adopt gRPC. This alleviates the need for a custom async client library or multi-threaded library for performance reasons.

Are there any concerns around this adoption?

Here's a simple synchronous example, https://grpc.io/docs/languages/python/quickstart/#update-the-client.

Here's the AsyncIO documentation, https://grpc.github.io/grpc/python/grpc_asyncio.html.

[rneswold]

Some Background

The historical interface to DPM has been sending messages via ACNET. The messages are defined using the protocol compiler. Our client APIs closely mimic the underlying protocol.

Due to some UDP issues (ACNET uses UDP), we created a TCP channel for clients to use. It wasn't too much of a step to say why should we be maintaining a protocol compiler when Google's protobufs do the same thing? And why come up with a custom TCP interface when there are many standard ways to make reliable connections? gRPCs already do what we're trying to do, and more.

Benefits of gRPC

• Messages are serialized using Google's protobuf tool. This tool inspired our protocol compiler in that it provides a way to define protocol messages in a type-safe way. But it does a little more:
  • Protobufs makes it easier to update a protocol while maintaining backward compatibility.
  • Protobufs has a oneof field type which can greatly reduce the number of messages in a protocol
  • Protobufs supports a map<> type for messages
• We no longer are in the business of adding language support for the protocol compiler. Most language communities have already provided support. Google provides official support for Java, C++, Python, and JavaScript.
• gRPC uses http/2 as its transport layer so it can provide the following service types:
  • Single-request, single reply -- This is the same as our ACNET "one-shot" requests
  • Single-request, streaming reply -- This is similar to our ACNET "request/multiple-reply
  • Streaming requests, single reply -- We have no corresponding want to do this
  • Streaming-request, streaming reply -- We have no corresponding way to do this, either
• Google provides Python users a synchronous and asynchronous set of APIs so you can use whichever style of programming you prefer.

gRPC Issues

• Not all client languages have a complete http/2 library
  • Those that do don't necessarily support all four communication types. Usually, the multiple-request types are not supported.