Topics

0MQ Alternatives - Tuesday's meeting

James.White2@...
 

At the center of our Go Lang project meeting on Tuesday, we will be a discussion on ZeroMQ alternatives and which to use (if any) for the upcoming California release.

 

In advance, I looked at the products in the table below and did some experimentation.  These are the products I’ve heard from community members most.  Below are some notes to kick off our discussions.  From my research, I am most intrigued by Nats.  Nanomsg appears dead (and with many indicating other problems like speed) and Thrift and gRPC are, well, RPC.  On the negative side, Nats does require a server, but it is already available on multiple platforms and is already Dockerized.   I found the Nats API/client libs and docs seem to be the cleanest.  If we are dead set to go brokerless, Thrift would be my pick – although the compiler/generator part of Thrift doesn’t thrill me.

 

Look forward to the discussion.

 

 

Product

Native Language Bindings, OS support, etc.

Brokerless

License

Used By

Thrift

C, C++, Go, Python, Java,

Yes

Apache 2

Cloudera, Evernote, Facebook, Siemens, Uber

Nats

Go, Node, Ruby, Java, C, C#

No - server

MIT

Baidu, Siemens, HTC, Pivotal, VMWare, Ericsson

gRPC

C, Go, Java, Python, Node

Yes

Apache 2

Square, Netflix, Cisco, CoreOS

Nanomsg

C, Go, Java, Python, Rust, Node

Yes

MIT

???

 

Thrift: 

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

Thrift does require a “compiler” to create/generate a thrift file to use by the implementations.  

Nats:

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

Server binary is available for Linux (x86, x86_64, ARM), Windows (x86, x86_64), and macOS.  Server image is available in Docker image. Server is 2.4 MB in size

gRPC:

A high performance, open-source universal RPC framework

a Cloud Native Computing Foundation project.

Nanomsg:

A socket library that provides several common communication patterns. It aims to make the networking layer fast, scalable, and easy to use. Implemented in C, it works on a wide range of operating systems with no further dependencies.

There is now also an implementation of nanomsg in pure Go: mangos.

http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/

 

 

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 

Dellabianca, Alberto
 

Great summary Jim.

Any thoughts on having in the list also MQTT? It would be a broker option like NATS, it is lightweight and widely used in the community.

Great implementations are Mosquitto Broker and Paho Client (in different languages), not sure if the license is compatible with LF requirements though. We are already using Paho Client in EdgeX however.

Also I understand that core-data is already somewhat capable to subscribe to MQTT topics?

Thanks

Alberto

 

 

 

 

From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of James.White2@...
Sent: Friday, March 16, 2018 5:18 PM
To: edgex-golang@...
Subject: [EXTERNAL] [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

 

At the center of our Go Lang project meeting on Tuesday, we will be a discussion on ZeroMQ alternatives and which to use (if any) for the upcoming California release.

 

In advance, I looked at the products in the table below and did some experimentation.  These are the products I’ve heard from community members most.  Below are some notes to kick off our discussions.  From my research, I am most intrigued by Nats.  Nanomsg appears dead (and with many indicating other problems like speed) and Thrift and gRPC are, well, RPC.  On the negative side, Nats does require a server, but it is already available on multiple platforms and is already Dockerized.   I found the Nats API/client libs and docs seem to be the cleanest.  If we are dead set to go brokerless, Thrift would be my pick – although the compiler/generator part of Thrift doesn’t thrill me.

 

Look forward to the discussion.

 

 

Product

Native Language Bindings, OS support, etc.

Brokerless

License

Used By

Thrift

C, C++, Go, Python, Java,

Yes

Apache 2

Cloudera, Evernote, Facebook, Siemens, Uber

Nats

Go, Node, Ruby, Java, C, C#

No - server

MIT

Baidu, Siemens, HTC, Pivotal, VMWare, Ericsson

gRPC

C, Go, Java, Python, Node

Yes

Apache 2

Square, Netflix, Cisco, CoreOS

Nanomsg

C, Go, Java, Python, Rust, Node

Yes

MIT

???

 

Thrift: 

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

Thrift does require a “compiler” to create/generate a thrift file to use by the implementations.  

Nats:

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

Server binary is available for Linux (x86, x86_64, ARM), Windows (x86, x86_64), and macOS.  Server image is available in Docker image. Server is 2.4 MB in size

gRPC:

A high performance, open-source universal RPC framework

a Cloud Native Computing Foundation project.

Nanomsg:

A socket library that provides several common communication patterns. It aims to make the networking layer fast, scalable, and easy to use. Implemented in C, it works on a wide range of operating systems with no further dependencies.

There is now also an implementation of nanomsg in pure Go: mangos.

http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/

 

 

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 

vinoyang
 


---------- Forwarded message ----------
From: vino yang <yanghua1127@...>
Date: 2018-03-17 11:32 GMT+08:00
Subject: Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting
To: James.White2@...


Hi Jim White,

I tend to choose from Thrift and gRPC.  No broker, point to point, binary protocol, better performance, decentration, mature stability and so on....
Though a message broker also have some good advantage but I think the light-weight, less dependency, simplify the communication program model would be better for IoT scenario. 
However if we need topic scenario(or means publish / subscribe) , borker has more advantage than RPC framework.

Just some opinion.

Vino yang.
Thanks.


2018-03-17 6:18 GMT+08:00 <James.White2@...>:

At the center of our Go Lang project meeting on Tuesday, we will be a discussion on ZeroMQ alternatives and which to use (if any) for the upcoming California release.

 

In advance, I looked at the products in the table below and did some experimentation.  These are the products I’ve heard from community members most.  Below are some notes to kick off our discussions.  From my research, I am most intrigued by Nats.  Nanomsg appears dead (and with many indicating other problems like speed) and Thrift and gRPC are, well, RPC.  On the negative side, Nats does require a server, but it is already available on multiple platforms and is already Dockerized.   I found the Nats API/client libs and docs seem to be the cleanest.  If we are dead set to go brokerless, Thrift would be my pick – although the compiler/generator part of Thrift doesn’t thrill me.

 

Look forward to the discussion.

 

 

Product

Native Language Bindings, OS support, etc.

Brokerless

License

Used By

Thrift

C, C++, Go, Python, Java,

Yes

Apache 2

Cloudera, Evernote, Facebook, Siemens, Uber

Nats

Go, Node, Ruby, Java, C, C#

No - server

MIT

Baidu, Siemens, HTC, Pivotal, VMWare, Ericsson

gRPC

C, Go, Java, Python, Node

Yes

Apache 2

Square, Netflix, Cisco, CoreOS

Nanomsg

C, Go, Java, Python, Rust, Node

Yes

MIT

???

 

Thrift: 

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

Thrift does require a “compiler” to create/generate a thrift file to use by the implementations.  

Nats:

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

Server binary is available for Linux (x86, x86_64, ARM), Windows (x86, x86_64), and macOS.  Server image is available in Docker image. Server is 2.4 MB in size

gRPC:

A high performance, open-source universal RPC framework

a Cloud Native Computing Foundation project.

Nanomsg:

A socket library that provides several common communication patterns. It aims to make the networking layer fast, scalable, and easy to use. Implemented in C, it works on a wide range of operating systems with no further dependencies.

There is now also an implementation of nanomsg in pure Go: mangos.

http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/

 

 

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...y.org
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang



Drasko DRASKOVIC <drasko@...>
 

On Sat, Mar 17, 2018 at 2:59 AM, Dellabianca, Alberto
<Alberto.Dellabianca@...> wrote:

Great summary Jim.

Any thoughts on having in the list also MQTT? It would be a broker option like NATS, it is lightweight
NATS is more lightweight then majority MQTT brokers out there

and widely used in the community.
Not for the purpose of microservice inter-communication.


Great implementations are Mosquitto Broker
Not scalable (can not cluster easily like NATS for HA/FT). Plus
written in C, so we'll have to recompile for ARM wih C compiler- bring
us back to the problem because of which we are changing ZMQ. Plus I am
not sure it works on Windows (probably not).

and Paho Client (in different languages), not sure if the license is compatible with LF requirements though. We are already using Paho Client in EdgeX however.
For device interface, and this is OK.


Also I understand that core-data is already somewhat capable to subscribe to MQTT topics?
Export data, only as a MQTT client for the cloud (not a broker).

MQTT has several other problems - like that for simplicity of the
protocol it does not keep origin (publisher) info, so it is impossible
to prevent loopbacks when you publish and subscribe on the same topic.
Finally, and probably most importantly - besides pure PUB/SUB:
https://nats.io/documentation/concepts/nats-pub-sub/ - NATS offers
other patterns, like REQ/RSP and Queueing that are very useful and can
be eventually achieved with MQTT only with additional work.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Steve Osselton
 

Hi Guys,

Although currently only for evaluation is also worth taking a look at Intel DPS for IoT https://github.com/intel/dps-for-iot
This is similar in usage to MQTT but is brokerless and also supports RPC. 

Also having used Thrift, it has a couple of nice features in that it supports both pluggable transports and data encodings,
and has the concept of both required and optional RPC arguments, which gives some level of API flexibility in that
optional arguments can be added and removed from a call definition, while maintaining backwards compatibility. I actually
like the fact that there is an interface definition that defines the contract between client and server and that there is a code
generator that takes care of all boiler plate essentially allowing a developer to simply concentrate on implementation
logic.

Probably to evaluate the various solutions we need some prioritised criteria.

Cheers Steve.

On 16 March 2018 at 22:18, <James.White2@...> wrote:

At the center of our Go Lang project meeting on Tuesday, we will be a discussion on ZeroMQ alternatives and which to use (if any) for the upcoming California release.

 

In advance, I looked at the products in the table below and did some experimentation.  These are the products I’ve heard from community members most.  Below are some notes to kick off our discussions.  From my research, I am most intrigued by Nats.  Nanomsg appears dead (and with many indicating other problems like speed) and Thrift and gRPC are, well, RPC.  On the negative side, Nats does require a server, but it is already available on multiple platforms and is already Dockerized.   I found the Nats API/client libs and docs seem to be the cleanest.  If we are dead set to go brokerless, Thrift would be my pick – although the compiler/generator part of Thrift doesn’t thrill me.

 

Look forward to the discussion.

 

 

Product

Native Language Bindings, OS support, etc.

Brokerless

License

Used By

Thrift

C, C++, Go, Python, Java,

Yes

Apache 2

Cloudera, Evernote, Facebook, Siemens, Uber

Nats

Go, Node, Ruby, Java, C, C#

No - server

MIT

Baidu, Siemens, HTC, Pivotal, VMWare, Ericsson

gRPC

C, Go, Java, Python, Node

Yes

Apache 2

Square, Netflix, Cisco, CoreOS

Nanomsg

C, Go, Java, Python, Rust, Node

Yes

MIT

???

 

Thrift: 

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

Thrift does require a “compiler” to create/generate a thrift file to use by the implementations.  

Nats:

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

Server binary is available for Linux (x86, x86_64, ARM), Windows (x86, x86_64), and macOS.  Server image is available in Docker image. Server is 2.4 MB in size

gRPC:

A high performance, open-source universal RPC framework

a Cloud Native Computing Foundation project.

Nanomsg:

A socket library that provides several common communication patterns. It aims to make the networking layer fast, scalable, and easy to use. Implemented in C, it works on a wide range of operating systems with no further dependencies.

There is now also an implementation of nanomsg in pure Go: mangos.

http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/

 

 

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@lists.edgexfoundry.org
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang




--
Technical Director
IOTech Systems Ltd.

vinoyang
 

Hi guys,

What's the result of this topic? Or it's discussing?

Recently, I have talked with a friend works in CMCC(the Chinese biggest mobile network service provider). He said they paid attention to EdgeX and the have a full IoT ecosystem and used gRPC as the communication framework.

James.White2@...
 

Dell - Internal Use - Confidential

Hi Vino

You’ll find the short write up here:  https://wiki.edgexfoundry.org/display/FA/Architecture+Issues+and+Decisions#ArchitectureIssuesandDecisions-Messagingtechnology

 

Basically, we decided to hold on any changes yet as there didn’t appear to be any perfect solution and the impact too great to handle as part of California.  gRPC was an option we looked at, but didn’t appear to be a great alternative at this time.  We’ll be relooking at this longer term.  Your participation in exploring alternates, inputs and feedback welcome at that time.

Jim

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Sunday, May 06, 2018 3:07 AM
To: EdgeX-GoLang@...
Subject: Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

 

Hi guys,

What's the result of this topic? Or it's discussing?

Recently, I have talked with a friend works in CMCC(the Chinese biggest mobile network service provider). He said they paid attention to EdgeX and the have a full IoT ecosystem and used gRPC as the communication framework.

Zeng, Tingyu
 

Hey Vino,

 

I don’t think the community has come to a conclusion yet and this is still under discussion. The weekly meeting will have some update every on the progress and it is really great to attend such meeting to know updated information.

 

Thanks,

Tingyu

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Sunday, May 6, 2018 4:07 AM
To: EdgeX-GoLang@...
Subject: Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

 

Hi guys,

What's the result of this topic? Or it's discussing?

Recently, I have talked with a friend works in CMCC(the Chinese biggest mobile network service provider). He said they paid attention to EdgeX and the have a full IoT ecosystem and used gRPC as the communication framework.