Topics

Export Go - for preview

Drasko DRASKOVIC <drasko@...>
 

On Tue, Feb 13, 2018 at 11:54 AM, Fede Claramonte
<fclaramonte@...> wrote:
In this branch[1] on my repo I changed the distro dockerfile to use zeromq
to receive events. It will also be available in export-go when the PR is
merged [2]

Fede

[1] https://github.com/feclare/export-go/tree/dockerZMQ
[2] https://github.com/edgexfoundry/export-go/pull/25
Who are the maintainers of this repo now?

Please review and merge.

BTW, we should add MAINTAINERS file, similar to this:
https://github.com/torvalds/linux/blob/master/MAINTAINERS or this:
https://github.com/moby/moby/blob/master/MAINTAINERS, so that we know
who is responsible. I am waiting for edgex-go monorepo PR to be merged
to propose the changes like this.

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

James.White2@...
 

Dell - Internal Use - Confidential

Thanks for this Fede. For our Go Lang project call today, I have as one of the agenda items to discuss this a bit. Is it time, perhaps, to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go cross compiling? Or do we just say that for this release (California), you have to run containerized on a linux based container for core-data and export-distro and all development of those has to happen on linux to be able to do the 0MQ required compilation?

I think this will be a good discussion.

-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@...>; White2, James <James_White2@...>; dejan.mjc <dejan@...>
Cc: chencho <smunoz@...>; manuel@...; darko@...; Garcia, Gorka <Gorka.Garcia@...>; Janko Isidorovic <janko@...>; Jeremy Phelps <jphelps@...>
Subject: Re: Export Go - for preview

Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to another architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go binaries have a dependency with the glibc used in compilation, from what I can understand from the log of the compilation (pasted below)

Fede

feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-go$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags '-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread -static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data # github.com/edgexfoundry/edgex-go/cmd/core-data
HEADER = -H4 -T0x401000 -D0x0 -R0x1000
searching for runtime.a in $WORK/runtime.a searching for runtime.a in /home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
 0.00 deadcode
 0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
 0.11 dodata
 0.14 dwarf
 0.16 symsize = 0
 0.20 reloc
 0.22 asmb
 0.22 codeblk
 0.23 datblk
 0.24 sym
 0.24 symsize = 222144
 0.25 symsize = 224088
 0.25 dwarf
 0.28 headr
 0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/github.com/edgexfoundry/edgex-go/cmd/core-data/_obj/exe/a.out"
"-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):
In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):
In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):
In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):
In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
 1.46 cpu time
96145 symbols
88588 liveness data



On 19/02/18 20:36, Drasko DRASKOVIC wrote:
On Mon, Feb 19, 2018 at 5:11 PM, <James.White2@...> wrote:
Dell - Internal Use - Confidential

I think this is the 0MQ stuff right Fede??
I am sure Fede will find the way to compile ZMQ. It is a popular and
widely used lib, I doubt that we can not handle the compilation...

Fede,
can you please post the logs if you run into the trouble and we can
debug together.

Something we need to eventually revisit - replacing 0MQ with
something like Nats or gRPC so that we don't have to do any C
compiling and everything is Go. Always more work guys :)
We have to decide if we want centralized broker or decentralized async
comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot, as
every service will just send the events to a topic to a known broker
(no need for complicated service discovery), and then whichever
service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols/,
and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice than
ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-throughpu
t.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/

I propose that we make everything work with ZMQ for now, to have a
stable MVP. We can experiment later in separate experimental branches
in replacing the communication mechanisms and see how thi affects
performance and improves/simplifies the system design.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Dellabianca, Alberto
 

Hi all,

My 2 cents on the Message Bus discussion.

 

There are pros and cons related to broker / brokerless... trying to combine simplicity, latency requirements and single point of failure concerns.

There is no perfect solution. In critical control environments we use DDS but that would be an overcomplication for a system like EdgeX.

 

In a Go world, a proposal could be a phased approach with a simple broker infrastructure as first implementation and a brokerless option in a second release.

 

Release I --> NATS

Release II --> NATS (default), gRPC (alternative option)

 

 

In a NATS world, Device Services would publish messages to the broker. Subscribers could be core-data, rules-engine and export-distribution. Rules-engine could be a publisher as well since the results of a calculation could be written back to NATS as a new topic.

 

NATS:

 

 

 

 

In a gRPC world, all microservices could talk to each other with a point-to-point approach opening channels. This is similar to REST but would support streaming and higher throughputs.

 

gRPC:

 

 

 

 

Alberto

 

 

 

 

 

 

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of James.White2@...
Sent: Tuesday, February 20, 2018 9:07 AM
To: fclaramonte@...; drasko@...; dejan@...; edgex-golang@...
Cc: manuel@...; Gorka.Garcia@...
Subject: [EXTERNAL] Re: [Edgex-golang] Export Go - for preview

 

Dell - Internal Use - Confidential 

 

 

Thanks for this Fede.  For our Go Lang project call today, I have as one of the agenda items to discuss this a bit.  Is it time, perhaps, to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go cross compiling?  Or do we just say that for this release (California), you have to run containerized on a linux based container for core-data and export-distro and all development of those has to happen on linux to be able to do the 0MQ required compilation?

 

 

I think this will be a good discussion.

 

 

-----Original Message-----

From: Fede Claramonte [mailto:fclaramonte@...]

Sent: Tuesday, February 20, 2018 8:36 AM

To: Drasko DRASKOVIC <drasko@...>; White2, James <James_White2@...>; dejan.mjc <dejan@...>

Cc: chencho <smunoz@...>; manuel@...; darko@...; Garcia, Gorka <Gorka.Garcia@...>; Janko Isidorovic <janko@...>; Jeremy Phelps <jphelps@...>

Subject: Re: Export Go - for preview

 

 

Right now the problem with having dependencies using cgo are:

- Cross compilation, there is no simple way to crosscompile to another architecture when using cgo

- Use 'from scratch' dockers. When compiling with cgo enabled go binaries have a dependency with the glibc used in compilation, from what I can understand from the log of the compilation (pasted below)

 

 

Fede

 

 

feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-go$

make cmd/core-data/core-data

CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags '-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread -static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data # https://urldefense.proofpoint.com/v2/url?u=http-3A__github.com_edgexfoundry_edgex-2Dgo_cmd_core-2Ddata&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=NF4JujIr4XOYdRglzPPHCIVT4853mbe0ZsOvWoCBvaY&e=

HEADER = -H4 -T0x401000 -D0x0 -R0x1000

searching for runtime.a in $WORK/runtime.a searching for runtime.a in /home/feclare/projects/gopath/pkg/linux_amd64/runtime.a

searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a

 0.00 deadcode

 0.08 pclntab=1263809 bytes, funcdata total 172301 bytes

 0.11 dodata

 0.14 dwarf

 0.16 symsize = 0

 0.20 reloc

 0.22 asmb

 0.22 codeblk

 0.23 datblk

 0.24 sym

 0.24 symsize = 222144

 0.25 symsize = 224088

 0.25 dwarf

 0.28 headr

 0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"

"/tmp/go-build995971213/https://urldefense.proofpoint.com/v2/url?u=http-3A__github.com_edgexfoundry_edgex-2Dgo_cmd_core-2Ddata_-5Fobj_exe_a.out&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=JeJMMOEEPLU3_tIMkzaT2YWAlwMERKWO5OZRUtE1w0o&e="

"-static" "/tmp/go-link-683737096/go.o"

"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"

"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"

"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"

"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"

/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):

In function `zmq::ipc_listener_t::filter(int)':

(.text+0xd60): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

(.text+0xd43): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

/tmp/go-link-683737096/000000.o: In function

`_cgo_cb4050e05860_C2func_getaddrinfo':

/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:

warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):

In function `pgm_getnodeaddr':

(.text+0x28a): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):

In function `pgm_getnetbyname':

(.text+0x321): warning: Using 'getnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

(.text+0x31c): warning: Using 'setnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

(.text+0x482): warning: Using 'endnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):

In function `_pgm_native_getprotobyname':

(.text+0x42): warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

 1.46 cpu time

96145 symbols

88588 liveness data

 

 

 

 

 

 

On 19/02/18 20:36, Drasko DRASKOVIC wrote:

> On Mon, Feb 19, 2018 at 5:11 PM,  <James.White2@...> wrote:

>> Dell - Internal Use - Confidential

>> 

 

>> I think this is the 0MQ stuff right Fede??

> I am sure Fede will find the way to compile ZMQ. It is a popular and

> widely used lib, I doubt that we can not handle the compilation...

 

> Fede,

> can you please post the logs if you run into the trouble and we can

> debug together.

 

>> Something we need to eventually revisit - replacing 0MQ with

>> something like Nats or gRPC so that we don't have to do any C

>> compiling and everything is Go.  Always more work guys  :)

> We have to decide if we want centralized broker or decentralized async

> comm like it is today. Both approaches have benfits and drawbacks.

> Centralized broker (NATS) would probably simplify the design a lot, as

> every service will just send the events to a topic to a known broker

> (no need for complicated service discovery), and then whichever

> service needs to consume these events will just subscribe to a topic.

> On the other hand - it is a SPOF, but this can be solved with FT

> deployment.

 

> ZMQ is a good choice, although that I would personally select nanomsg:

> https://urldefense.proofpoint.com/v2/url?u=http-3A__nanomsg.org_&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=BmS0utrggPir5iXsIXapgZ9DTaO7K6RdX6iiOHPPRkI&e=, which is a successor of ZMQ (same author). Some

> interesting info:

> https://urldefense.proofpoint.com/v2/url?u=https-3A__bravenewgeek.com_a-2Dlook-2Dat-2Dnanomsg-2Dand-2Dscalability-2Dprotocols_&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=QDYiIWFFMO7cXtQn32Laqn65ycyVv-sRUkjymnrALCk&e=,

> and there is also Go-native implementation:

> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_go-2Dmangos_mangos&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=sQyL3CvmJNT4YvUxu1OR8d_YEPZW_XmDQwod2RTkmmE&e=.

 

> gRPC would be probably easier to implement and maintain, but it is

> HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with

> it. Here is a nice comparison:

> https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_39350681_grpc-2Dand-2Dzeromq-2Dcomparsion&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=oKq2ryhDyhiVYfFeMtaASx0-BduELexSpPwpI2-DRMk&e=.

> I am not sure that in our lightway approach gRPC is better choice than

> ZMQ/nanomsg. I expect higher throughput with ZMQ

> (https://urldefense.proofpoint.com/v2/url?u=https-3A__bbengfort.github.io_observations_2017_09_04_message-2Dthroughpu&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=sAPFVuNDEwzlOte5Ngo-taDM2g3_aenAWHlGDaJ6bmU&e=

> t.html)

> - although not much higher, since there is just a few clients

> connected. I also expect high throughput with NATS, but pure TCP

> socket that ZMQ/nanomsg uses is practically unbeatable:

> https://urldefense.proofpoint.com/v2/url?u=https-3A__bravenewgeek.com_dissecting-2Dmessage-2Dqueues_&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=n_JKT2l8uzHGYDg6ZIu9xgX0RHYG1SnLcirLtyhAfv0&e=

 

> I propose that we make everything work with ZMQ for now, to have a

> stable MVP. We can experiment later in separate experimental branches

> in replacing the communication mechanisms and see how thi affects

> performance and improves/simplifies the system design.

 

> Best regards,

> Drasko DRASKOVIC

> Mainflux Author and Technical Advisor

 

> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mainflux.com&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=tU3el7exroVgyVBSDozGtlcM-CTXHPFGdhH5Nm2nT-0&e=   |  Industrial IoT Cloud

> -------------------------------------------------------------------

> Engineering Division |   Paris, France

 

> LinkedIn: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_in_draskodraskovic&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=0w0xPRHHCQo39UVt4jn1ZqtFWXLtA7fH7OCd2tm0ldw&e=

> Twitter: @draskodraskovic

 

 

 

_______________________________________________

EdgeX-GoLang mailing list

EdgeX-GoLang@...

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.edgexfoundry.org_mailman_listinfo_edgex-2Dgolang&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=GRfZIthq58xKEy6k4JYMjEjQhP_OQX517NOqZh9l7Bk&e=

espy
 

On 02/20/2018 10:06 AM, James.White2@... wrote:
Dell - Internal Use - Confidential
Thanks for this Fede. For our Go Lang project call today, I have as one of the agenda items to discuss this a bit. Is it time, perhaps, to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go cross compiling?
Just to clarify, the reason we think Nats or gRPC would be better is that there's support for both which doesn't require cgo (i.e. linking C & Go)?

Or do we just say that for this release (California), you have to run containerized on a linux based container for core-data and export-distro and all development of those has to happen on linux to be able to do the 0MQ required compilation?
Or we alternatively run them native on a Linux system? ;)-

I think this will be a good discussion.
-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@...>; White2, James <James_White2@...>; dejan.mjc <dejan@...>
Cc: chencho <smunoz@...>; manuel@...; darko@...; Garcia, Gorka <Gorka.Garcia@...>; Janko Isidorovic <janko@...>; Jeremy Phelps <jphelps@...>
Subject: Re: Export Go - for preview
Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to another architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go binaries have a dependency with the glibc used in compilation, from what I can understand from the log of the compilation (pasted below)
It looks like lzmq uses libc, and libc can't be statically linked:

https://github.com/zeromq/libzmq/issues/2849

So yes, you need the same version of libc and supporting shared libraries that you linked against on the target system.

Regards,
/tony


Fede
feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-go$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags '-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread -static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data # github.com/edgexfoundry/edgex-go/cmd/core-data
HEADER = -H4 -T0x401000 -D0x0 -R0x1000
searching for runtime.a in $WORK/runtime.a searching for runtime.a in /home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
 0.00 deadcode
 0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
 0.11 dodata
 0.14 dwarf
 0.16 symsize = 0
 0.20 reloc
 0.22 asmb
 0.22 codeblk
 0.23 datblk
 0.24 sym
 0.24 symsize = 222144
 0.25 symsize = 224088
 0.25 dwarf
 0.28 headr
 0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/github.com/edgexfoundry/edgex-go/cmd/core-data/_obj/exe/a.out"
"-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):
In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):
In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):
In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):
In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
 1.46 cpu time
96145 symbols
88588 liveness data
On 19/02/18 20:36, Drasko DRASKOVIC wrote:
On Mon, Feb 19, 2018 at 5:11 PM, <James.White2@...> wrote:
Dell - Internal Use - Confidential

I think this is the 0MQ stuff right Fede??
I am sure Fede will find the way to compile ZMQ. It is a popular and
widely used lib, I doubt that we can not handle the compilation...

Fede,
can you please post the logs if you run into the trouble and we can
debug together.

Something we need to eventually revisit - replacing 0MQ with
something like Nats or gRPC so that we don't have to do any C
compiling and everything is Go. Always more work guys :)
We have to decide if we want centralized broker or decentralized async
comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot, as
every service will just send the events to a topic to a known broker
(no need for complicated service discovery), and then whichever
service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols/,
and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice than
ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-throughpu
t.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/

I propose that we make everything work with ZMQ for now, to have a
stable MVP. We can experiment later in separate experimental branches
in replacing the communication mechanisms and see how thi affects
performance and improves/simplifies the system design.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

espy
 

On 02/20/2018 06:53 PM, Dellabianca, Alberto wrote:
Hi all,
My 2 cents on the Message Bus discussion.
There are pros and cons related to broker / brokerless... trying to combine simplicity, latency requirements and single point of failure concerns.
There is no perfect solution. In critical control environments we use DDS but that would be an overcomplication for a system like EdgeX.
In a Go world, a proposal could be a phased approach with a simple broker infrastructure as first implementation and a brokerless option in a second release.
We actually have a polyglot world, not just Go.

Release I --> NATS
Release II --> NATS (default), gRPC (alternative option)
In a NATS world, Device Services would publish messages to the broker. Subscribers could be core-data, rules-engine and export-distribution. Rules-engine could be a publisher as well since the results of a calculation could be written back to NATS as a new topic.
Also, as I understand it, for this release, the problem to be solved isn't what's a good alternative messaging bus for "all services". There's a tactical problem to be solved as Core Data and Export Distro use zeromq, which for Go requires using cgo, which in turn requires dynamic linking to libc (and its supporting libraries), which means our binaries for both services would have runtime dependencies on those share libraries.

With more of a long term view, we should certainly survey the whole gamut of bus solutions.

Regards,
/tony

L
-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of James.White2@...
Sent: Tuesday, February 20, 2018 9:07 AM
To: fclaramonte@...; drasko@...; dejan@...; edgex-golang@...
Cc: manuel@...; Gorka.Garcia@...
Subject: [EXTERNAL] Re: [Edgex-golang] Export Go - for preview
Dell - Internal Use - Confidential
Thanks for this Fede.  For our Go Lang project call today, I have as one of the agenda items to discuss this a bit.  Is it time, perhaps, to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go cross compiling?  Or do we just say that for this release (California), you have to run containerized on a linux based container for core-data and export-distro and all development of those has to happen on linux to be able to do the 0MQ required compilation?
I think this will be a good discussion.
-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@... <mailto:drasko@...>>; White2, James <James_White2@... <mailto:James_White2@...>>; dejan.mjc <dejan@... <mailto:dejan@...>>
Cc: chencho <smunoz@... <mailto:smunoz@...>>; manuel@... <mailto:manuel@...>; darko@... <mailto:darko@...>; Garcia, Gorka <Gorka.Garcia@... <mailto:Gorka.Garcia@...>>; Janko Isidorovic <janko@... <mailto:janko@...>>; Jeremy Phelps <jphelps@... <mailto:jphelps@...>>
Subject: Re: Export Go - for preview
Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to another architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go binaries have a dependency with the glibc used in compilation, from what I can understand from the log of the compilation (pasted below)
Fede
feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-go$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags '-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread -static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data # https://urldefense.proofpoint.com/v2/url?u=http-3A__github.com_edgexfoundry_edgex-2Dgo_cmd_core-2Ddata&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=NF4JujIr4XOYdRglzPPHCIVT4853mbe0ZsOvWoCBvaY&e=
HEADER = -H4 -T0x401000 -D0x0 -R0x1000
searching for runtime.a in $WORK/runtime.a searching for runtime.a in /home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
 0.00 deadcode
 0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
 0.11 dodata
 0.14 dwarf
 0.16 symsize = 0
 0.20 reloc
 0.22 asmb
 0.22 codeblk
 0.23 datblk
 0.24 sym
 0.24 symsize = 222144
 0.25 symsize = 224088
 0.25 dwarf
 0.28 headr
 0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/https://urldefense.proofpoint.com/v2/url?u=http-3A__github.com_edgexfoundry_edgex-2Dgo_cmd_core-2Ddata_-5Fobj_exe_a.out&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=JeJMMOEEPLU3_tIMkzaT2YWAlwMERKWO5OZRUtE1w0o&e=" "-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o): In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o): In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o): In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o): In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
 1.46 cpu time
96145 symbols
88588 liveness data
On 19/02/18 20:36, Drasko DRASKOVIC wrote:

> On Mon, Feb 19, 2018 at 5:11 PM,  <James.White2@...
<mailto:James.White2@...>> wrote:

>> Dell - Internal Use - Confidential

>>

>> I think this is the 0MQ stuff right Fede??

> I am sure Fede will find the way to compile ZMQ. It is a popular and

> widely used lib, I doubt that we can not handle the compilation...

>

> Fede,

> can you please post the logs if you run into the trouble and we can

> debug together.

>

>> Something we need to eventually revisit - replacing 0MQ with

>> something like Nats or gRPC so that we don't have to do any C

>> compiling and everything is Go.  Always more work guys  :)

> We have to decide if we want centralized broker or decentralized async

> comm like it is today. Both approaches have benfits and drawbacks.

> Centralized broker (NATS) would probably simplify the design a lot, as

> every service will just send the events to a topic to a known broker

> (no need for complicated service discovery), and then whichever

> service needs to consume these events will just subscribe to a topic.

> On the other hand - it is a SPOF, but this can be solved with FT

> deployment.

>

> ZMQ is a good choice, although that I would personally select nanomsg:

>
https://urldefense.proofpoint.com/v2/url?u=http-3A__nanomsg.org_&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=BmS0utrggPir5iXsIXapgZ9DTaO7K6RdX6iiOHPPRkI&e=, which is a successor of ZMQ (same author). Some

> interesting info:

>
https://urldefense.proofpoint.com/v2/url?u=https-3A__bravenewgeek.com_a-2Dlook-2Dat-2Dnanomsg-2Dand-2Dscalability-2Dprotocols_&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=QDYiIWFFMO7cXtQn32Laqn65ycyVv-sRUkjymnrALCk&e=,

> and there is also Go-native implementation:

>
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_go-2Dmangos_mangos&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=sQyL3CvmJNT4YvUxu1OR8d_YEPZW_XmDQwod2RTkmmE&e=.

>

> gRPC would be probably easier to implement and maintain, but it is

> HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with

> it. Here is a nice comparison:

>
https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_39350681_grpc-2Dand-2Dzeromq-2Dcomparsion&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=oKq2ryhDyhiVYfFeMtaASx0-BduELexSpPwpI2-DRMk&e=.

> I am not sure that in our lightway approach gRPC is better choice than

> ZMQ/nanomsg. I expect higher throughput with ZMQ

>
(https://urldefense.proofpoint.com/v2/url?u=https-3A__bbengfort.github.io_observations_2017_09_04_message-2Dthroughpu&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=sAPFVuNDEwzlOte5Ngo-taDM2g3_aenAWHlGDaJ6bmU&e=

> t.html)

> - although not much higher, since there is just a few clients

> connected. I also expect high throughput with NATS, but pure TCP

> socket that ZMQ/nanomsg uses is practically unbeatable:

>
https://urldefense.proofpoint.com/v2/url?u=https-3A__bravenewgeek.com_dissecting-2Dmessage-2Dqueues_&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=n_JKT2l8uzHGYDg6ZIu9xgX0RHYG1SnLcirLtyhAfv0&e=

>

> I propose that we make everything work with ZMQ for now, to have a

> stable MVP. We can experiment later in separate experimental branches

> in replacing the communication mechanisms and see how thi affects

> performance and improves/simplifies the system design.

>

> Best regards,

> Drasko DRASKOVIC

> Mainflux Author and Technical Advisor

>

>
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mainflux.com&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=tU3el7exroVgyVBSDozGtlcM-CTXHPFGdhH5Nm2nT-0&e= |  Industrial IoT Cloud

> -------------------------------------------------------------------

> Engineering Division |   Paris, France

>

> LinkedIn:
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_in_draskodraskovic&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=0w0xPRHHCQo39UVt4jn1ZqtFWXLtA7fH7OCd2tm0ldw&e=

> Twitter: @draskodraskovic

>
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@... <mailto:EdgeX-GoLang@...>
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.edgexfoundry.org_mailman_listinfo_edgex-2Dgolang&d=DwIGaQ&c=oTZJxWjXA97U5efbDJZGEg&r=ng_tkMHyi7CGJozkvpxBfnH8oKpxLQBNTCY3LrRYwLU&m=wCsr92DqHd_iQG0VP-eIfYIy0eGsx2jmEPo_KUtdOmA&s=GRfZIthq58xKEy6k4JYMjEjQhP_OQX517NOqZh9l7Bk&e=
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

espy
 

On 02/22/2018 07:50 PM, espy wrote:
On 02/20/2018 10:06 AM, James.White2@... wrote:
Dell - Internal Use - Confidential

Thanks for this Fede.  For our Go Lang project call today, I have as one of the agenda items to discuss this a bit.  Is it time, perhaps, to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go cross compiling?
Just to clarify, the reason we think Nats or gRPC would be better is that there's support for both which doesn't require cgo (i.e. linking C & Go)?
Maybe I have this wrong, but the Java versions of Core Data and Export Distro use a native Java version of ZMQ, correct?

Do we know why there isn't a native Go version?

Also, if there really isn't, Drasko's suggestion to use nanomsg merits some consideration.

Regards,
/tony




Or do we just say that for this release (California), you have to run containerized on a linux based container for core-data and export-distro and all development of those has to happen on linux to be able to do the 0MQ required compilation?
Or we alternatively run them native on a Linux system?  ;)-

I think this will be a good discussion.

-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@...>; White2, James <James_White2@...>; dejan.mjc <dejan@...>
Cc: chencho <smunoz@...>; manuel@...; darko@...; Garcia, Gorka <Gorka.Garcia@...>; Janko Isidorovic <janko@...>; Jeremy Phelps <jphelps@...>
Subject: Re: Export Go - for preview

Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to another architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go binaries have a dependency with the glibc used in compilation, from what I can understand from the log of the compilation (pasted below)
It looks like lzmq uses libc, and libc can't be statically linked:
https://github.com/zeromq/libzmq/issues/2849
So yes, you need the same version of libc and supporting shared libraries that you linked against on the target system.
Regards,
/tony


Fede

feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-go$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags '-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread -static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data # github.com/edgexfoundry/edgex-go/cmd/core-data
HEADER = -H4 -T0x401000 -D0x0 -R0x1000
searching for runtime.a in $WORK/runtime.a searching for runtime.a in /home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
   0.00 deadcode
   0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
   0.11 dodata
   0.14 dwarf
   0.16 symsize = 0
   0.20 reloc
   0.22 asmb
   0.22 codeblk
   0.23 datblk
   0.24 sym
   0.24 symsize = 222144
   0.25 symsize = 224088
   0.25 dwarf
   0.28 headr
   0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/github.com/edgexfoundry/edgex-go/cmd/core-data/_obj/exe/a.out"
"-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):
In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):
In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):
In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):
In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
   1.46 cpu time
96145 symbols
88588 liveness data



On 19/02/18 20:36, Drasko DRASKOVIC wrote:
On Mon, Feb 19, 2018 at 5:11 PM,  <James.White2@...> wrote:
Dell - Internal Use - Confidential

I think this is the 0MQ stuff right Fede??
I am sure Fede will find the way to compile ZMQ. It is a popular and
widely used lib, I doubt that we can not handle the compilation...

Fede,
can you please post the logs if you run into the trouble and we can
debug together.

Something we need to eventually revisit - replacing 0MQ with
something like Nats or gRPC so that we don't have to do any C
compiling and everything is Go.  Always more work guys  :)
We have to decide if we want centralized broker or decentralized async
comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot, as
every service will just send the events to a topic to a known broker
(no need for complicated service discovery), and then whichever
service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols/,
and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice than
ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-throughpu
t.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/

I propose that we make everything work with ZMQ for now, to have a
stable MVP. We can experiment later in separate experimental branches
in replacing the communication mechanisms and see how thi affects
performance and improves/simplifies the system design.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com   |  Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division |   Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

James.White2@...
 

Dell - Internal Use - Confidential

Tony,
Correct - we believe other message engines like Nats and gRPC would have Go code which could be used (and cross compiled) without the need for cgo for each platform (even though we all love Linux - Canonical's form especially - we want to support other OS :) ) .

-----Original Message-----
From: espy [mailto:espy@...]
Sent: Thursday, February 22, 2018 6:50 PM
To: White2, James <James_White2@...>; fclaramonte@...; drasko@...; dejan@...; edgex-golang@...
Cc: manuel@...; Gorka.Garcia@...
Subject: Re: [Edgex-golang] Export Go - for preview

On 02/20/2018 10:06 AM, James.White2@... wrote:
Dell - Internal Use - Confidential

Thanks for this Fede. For our Go Lang project call today, I have as one of the agenda items to discuss this a bit. Is it time, perhaps, to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go cross compiling?
Just to clarify, the reason we think Nats or gRPC would be better is that there's support for both which doesn't require cgo (i.e. linking C & Go)?

Or do we just say that for this release (California), you have to run containerized on a linux based container for core-data and export-distro and all development of those has to happen on linux to be able to do the 0MQ required compilation?
Or we alternatively run them native on a Linux system? ;)-

I think this will be a good discussion.

-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@...>; White2, James
<James_White2@...>; dejan.mjc <dejan@...>
Cc: chencho <smunoz@...>; manuel@...;
darko@...; Garcia, Gorka <Gorka.Garcia@...>;
Janko Isidorovic <janko@...>; Jeremy Phelps
<jphelps@...>
Subject: Re: Export Go - for preview

Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to another
architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go
binaries have a dependency with the glibc used in compilation, from
what I can understand from the log of the compilation (pasted below)
It looks like lzmq uses libc, and libc can't be statically linked:

https://github.com/zeromq/libzmq/issues/2849

So yes, you need the same version of libc and supporting shared libraries that you linked against on the target system.

Regards,
/tony



Fede

feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-go
$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags
'-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread
-static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data #
github.com/edgexfoundry/edgex-go/cmd/core-data
HEADER = -H4 -T0x401000 -D0x0 -R0x1000 searching for runtime.a in
$WORK/runtime.a searching for runtime.a in
/home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
 0.00 deadcode
 0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
 0.11 dodata
 0.14 dwarf
 0.16 symsize = 0
 0.20 reloc
 0.22 asmb
 0.22 codeblk
 0.23 datblk
 0.24 sym
 0.24 symsize = 222144
 0.25 symsize = 224088
 0.25 dwarf
 0.28 headr
 0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/github.com/edgexfoundry/edgex-go/cmd/core-data/_obj/exe/a.out"
"-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):
In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications
requires at runtime the shared libraries from the glibc version used
for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):
In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):
In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):
In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
 1.46 cpu time
96145 symbols
88588 liveness data



On 19/02/18 20:36, Drasko DRASKOVIC wrote:
On Mon, Feb 19, 2018 at 5:11 PM, <James.White2@...> wrote:
Dell - Internal Use - Confidential

I think this is the 0MQ stuff right Fede??
I am sure Fede will find the way to compile ZMQ. It is a popular and
widely used lib, I doubt that we can not handle the compilation...

Fede,
can you please post the logs if you run into the trouble and we can
debug together.

Something we need to eventually revisit - replacing 0MQ with
something like Nats or gRPC so that we don't have to do any C
compiling and everything is Go. Always more work guys :)
We have to decide if we want centralized broker or decentralized
async comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot,
as every service will just send the events to a topic to a known
broker (no need for complicated service discovery), and then
whichever service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols/
, and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice
than ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-throughp
u
t.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/

I propose that we make everything work with ZMQ for now, to have a
stable MVP. We can experiment later in separate experimental branches
in replacing the communication mechanisms and see how thi affects
performance and improves/simplifies the system design.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

James.White2@...
 

Dell - Internal Use - Confidential

Hi Tony,
We don't know exactly why the pebbe/zmq4 is not entirely Go native. If there is an all native Go library for ZMQ, my guys could not find one that works when we looked a few months back. Might be worth a relook - I'll add it to my list to research.

Since we are in need of a different solution to make life better for us in Go, I am ok with looking at alternatives (nanomsg, Nats, gRPC, etc.). I do think we still want to find something that is supported by Java as we haven't totally abandoned the Java code yet.

Drasko - Mainflux introduced us to Nats and we had them give a presentation, do you now feel nanomsg is better? If so, can you provide a comparison from your perspective?

Other opinions?
Jim

-----Original Message-----
From: espy [mailto:espy@...]
Sent: Thursday, February 22, 2018 7:08 PM
To: White2, James <James_White2@...>; fclaramonte@...; drasko@...; dejan@...; edgex-golang@...
Cc: manuel@...; Gorka.Garcia@...
Subject: Re: [Edgex-golang] Export Go - for preview

On 02/22/2018 07:50 PM, espy wrote:
On 02/20/2018 10:06 AM, James.White2@... wrote:
Dell - Internal Use - Confidential

Thanks for this Fede.  For our Go Lang project call today, I have as
one of the agenda items to discuss this a bit.  Is it time, perhaps,
to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go
cross compiling?
Just to clarify, the reason we think Nats or gRPC would be better is
that there's support for both which doesn't require cgo (i.e. linking
C & Go)?
Maybe I have this wrong, but the Java versions of Core Data and Export Distro use a native Java version of ZMQ, correct?

Do we know why there isn't a native Go version?

Also, if there really isn't, Drasko's suggestion to use nanomsg merits some consideration.

Regards,
/tony




Or do we just say that for this release (California), you have to run
containerized on a linux based container for core-data and
export-distro and all development of those has to happen on linux to
be able to do the 0MQ required compilation?
Or we alternatively run them native on a Linux system?  ;)-

I think this will be a good discussion.

-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@...>; White2, James
<James_White2@...>; dejan.mjc <dejan@...>
Cc: chencho <smunoz@...>; manuel@...;
darko@...; Garcia, Gorka <Gorka.Garcia@...>;
Janko Isidorovic <janko@...>; Jeremy Phelps
<jphelps@...>
Subject: Re: Export Go - for preview

Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to
another architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go
binaries have a dependency with the glibc used in compilation, from
what I can understand from the log of the compilation (pasted below)
It looks like lzmq uses libc, and libc can't be statically linked:

https://github.com/zeromq/libzmq/issues/2849

So yes, you need the same version of libc and supporting shared
libraries that you linked against on the target system.

Regards,
/tony



Fede

feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-g
o$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags
'-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread
-static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data
# github.com/edgexfoundry/edgex-go/cmd/core-data
HEADER = -H4 -T0x401000 -D0x0 -R0x1000 searching for runtime.a in
$WORK/runtime.a searching for runtime.a in
/home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
   0.00 deadcode
   0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
   0.11 dodata
   0.14 dwarf
   0.16 symsize = 0
   0.20 reloc
   0.22 asmb
   0.22 codeblk
   0.23 datblk
   0.24 sym
   0.24 symsize = 222144
   0.25 symsize = 224088
   0.25 dwarf
   0.28 headr
   0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/github.com/edgexfoundry/edgex-go/cmd/core-data/_obj/exe/a.out"

"-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):

In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications
requires at runtime the shared libraries from the glibc version used
for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):

In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):

In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):

In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
   1.46 cpu time
96145 symbols
88588 liveness data



On 19/02/18 20:36, Drasko DRASKOVIC wrote:
On Mon, Feb 19, 2018 at 5:11 PM,  <James.White2@...> wrote:
Dell - Internal Use - Confidential

I think this is the 0MQ stuff right Fede??
I am sure Fede will find the way to compile ZMQ. It is a popular and
widely used lib, I doubt that we can not handle the compilation...

Fede,
can you please post the logs if you run into the trouble and we can
debug together.

Something we need to eventually revisit - replacing 0MQ with
something like Nats or gRPC so that we don't have to do any C
compiling and everything is Go.  Always more work guys  :)
We have to decide if we want centralized broker or decentralized
async comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot,
as every service will just send the events to a topic to a known
broker (no need for complicated service discovery), and then
whichever service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols
/, and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice
than ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-through
pu
t.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/

I propose that we make everything work with ZMQ for now, to have a
stable MVP. We can experiment later in separate experimental
branches in replacing the communication mechanisms and see how thi
affects performance and improves/simplifies the system design.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com   |  Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division |   Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Trevor.Conn@...
 

Dell - Internal Use - Confidential

" If there is an all native Go library for ZMQ, my guys could not find one that works when we looked a few months back."

From what I can see NanoMsg is pitched as the next-gen of ZeroMQ.
http://nanomsg.org/documentation-zeromq.html

There is a Go-native implementation of the NanoMsg scalability protocols called "Mangos"
https://github.com/go-mangos/mangos
https://godoc.org/github.com/go-mangos/mangos

It appears NanoMsg has clients in a wide variety of languages, though I can't speak to their maturity.
http://nanomsg.org/documentation.html

NATS is interesting too because it's Go native but it would require a server running in a container/natively which either way is another process to manage.

I personally don't even see gRPC as a player in the discussion because at the end of the day it's an RPC framework, which seems more suitable if we're talking about replacing REST than a decoupled messaging solution.

Trevor

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of White2, James
Sent: Thursday, February 22, 2018 11:15 PM
To: espy@...; fclaramonte@...; drasko@...; dejan@...; edgex-golang@...
Cc: manuel@...; Gorka.Garcia@...
Subject: Re: [Edgex-golang] Export Go - for preview

Dell - Internal Use - Confidential

Hi Tony,
We don't know exactly why the pebbe/zmq4 is not entirely Go native. If there is an all native Go library for ZMQ, my guys could not find one that works when we looked a few months back. Might be worth a relook - I'll add it to my list to research.

Since we are in need of a different solution to make life better for us in Go, I am ok with looking at alternatives (nanomsg, Nats, gRPC, etc.). I do think we still want to find something that is supported by Java as we haven't totally abandoned the Java code yet.

Drasko - Mainflux introduced us to Nats and we had them give a presentation, do you now feel nanomsg is better? If so, can you provide a comparison from your perspective?

Other opinions?
Jim

-----Original Message-----
From: espy [mailto:espy@...]
Sent: Thursday, February 22, 2018 7:08 PM
To: White2, James <James_White2@...>; fclaramonte@...; drasko@...; dejan@...; edgex-golang@...
Cc: manuel@...; Gorka.Garcia@...
Subject: Re: [Edgex-golang] Export Go - for preview

On 02/22/2018 07:50 PM, espy wrote:
On 02/20/2018 10:06 AM, James.White2@... wrote:
Dell - Internal Use - Confidential

Thanks for this Fede.  For our Go Lang project call today, I have as
one of the agenda items to discuss this a bit.  Is it time, perhaps,
to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go
cross compiling?
Just to clarify, the reason we think Nats or gRPC would be better is
that there's support for both which doesn't require cgo (i.e. linking
C & Go)?
Maybe I have this wrong, but the Java versions of Core Data and Export Distro use a native Java version of ZMQ, correct?

Do we know why there isn't a native Go version?

Also, if there really isn't, Drasko's suggestion to use nanomsg merits some consideration.

Regards,
/tony




Or do we just say that for this release (California), you have to run
containerized on a linux based container for core-data and
export-distro and all development of those has to happen on linux to
be able to do the 0MQ required compilation?
Or we alternatively run them native on a Linux system?  ;)-

I think this will be a good discussion.

-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@...>; White2, James
<James_White2@...>; dejan.mjc <dejan@...>
Cc: chencho <smunoz@...>; manuel@...;
darko@...; Garcia, Gorka <Gorka.Garcia@...>;
Janko Isidorovic <janko@...>; Jeremy Phelps
<jphelps@...>
Subject: Re: Export Go - for preview

Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to
another architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go
binaries have a dependency with the glibc used in compilation, from
what I can understand from the log of the compilation (pasted below)
It looks like lzmq uses libc, and libc can't be statically linked:

https://github.com/zeromq/libzmq/issues/2849

So yes, you need the same version of libc and supporting shared
libraries that you linked against on the target system.

Regards,
/tony



Fede

feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-g
o$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags
'-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread
-static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data
# github.com/edgexfoundry/edgex-go/cmd/core-data
HEADER = -H4 -T0x401000 -D0x0 -R0x1000 searching for runtime.a in
$WORK/runtime.a searching for runtime.a in
/home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
   0.00 deadcode
   0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
   0.11 dodata
   0.14 dwarf
   0.16 symsize = 0
   0.20 reloc
   0.22 asmb
   0.22 codeblk
   0.23 datblk
   0.24 sym
   0.24 symsize = 222144
   0.25 symsize = 224088
   0.25 dwarf
   0.28 headr
   0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/github.com/edgexfoundry/edgex-go/cmd/core-data/_obj/exe/a.out"

"-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):

In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications
requires at runtime the shared libraries from the glibc version used
for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):

In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):

In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):

In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
   1.46 cpu time
96145 symbols
88588 liveness data



On 19/02/18 20:36, Drasko DRASKOVIC wrote:
On Mon, Feb 19, 2018 at 5:11 PM,  <James.White2@...> wrote:
Dell - Internal Use - Confidential

I think this is the 0MQ stuff right Fede??
I am sure Fede will find the way to compile ZMQ. It is a popular and
widely used lib, I doubt that we can not handle the compilation...

Fede,
can you please post the logs if you run into the trouble and we can
debug together.

Something we need to eventually revisit - replacing 0MQ with
something like Nats or gRPC so that we don't have to do any C
compiling and everything is Go.  Always more work guys  :)
We have to decide if we want centralized broker or decentralized
async comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot,
as every service will just send the events to a topic to a known
broker (no need for complicated service discovery), and then
whichever service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols
/, and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice
than ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-through
pu
t.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/

I propose that we make everything work with ZMQ for now, to have a
stable MVP. We can experiment later in separate experimental
branches in replacing the communication mechanisms and see how thi
affects performance and improves/simplifies the system design.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com   |  Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division |   Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Drasko DRASKOVIC <drasko@...>
 

Hi Trevor,

On Fri, Feb 23, 2018 at 5:15 PM, <Trevor.Conn@...> wrote:
Dell - Internal Use - Confidential

" If there is an all native Go library for ZMQ, my guys could not find one that works when we looked a few months back."

From what I can see NanoMsg is pitched as the next-gen of ZeroMQ.
http://nanomsg.org/documentation-zeromq.html

There is a Go-native implementation of the NanoMsg scalability protocols called "Mangos"
https://github.com/go-mangos/mangos
https://godoc.org/github.com/go-mangos/mangos

It appears NanoMsg has clients in a wide variety of languages, though I can't speak to their maturity.
http://nanomsg.org/documentation.html
Agreed for Nanomsg, this is a good candidate (btw, I have mentioned
this 6 months ago :)).


NATS is interesting too because it's Go native but it would require a server running in a container/natively which either way is another process to manage.
Simplicity. Whole system will be highly simplified, IMHO


I personally don't even see gRPC as a player in the discussion because at the end of the day it's an RPC framework, which seems more suitable if we're talking about replacing REST than a decoupled messaging solution.
Async bi-directional event sending. Protobuf included. Real TLS
(because we have HTTP).

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

James.White2@...
 

Dell - Internal Use - Confidential

Thanks for this Trevor and Drasko,

Good stuff!!!

My top 3 requirements would be:
-Broker-less (we've done MQTT and broker before. It's not a problem, but additional broker requires more setup/containers/etc. and can be large footprint depending on implementation. We can and should still allow MQTT as alternative option for those that want it - like we have been offering in Java side to date - but the default should be brokerless).
-Go native (but also provides libraries for Java, C and possibly other languages). As we look at messaging use between other services in the future, support for polyglot is integral to our tenets and needs of the future.
-Lightweight, ease of use, etc.

What I am seeing on this thread is that people feel Nanomsg may be leading candidate?? Would like to hear from others that have an opinions or questions. I'd like to hold this as a major part of our discussion in the Go project meeting (once we have the mono repo work done).

-----Original Message-----
From: Drasko DRASKOVIC [mailto:drasko@...]
Sent: Friday, February 23, 2018 10:23 AM
To: Conn, Trevor <Trevor_Conn@...>
Cc: White2, James <James_White2@...>; espy <espy@...>; Fede <fclaramonte@...>; dejan.mjc <dejan@...>; edgex-golang@...; manuel@...; Garcia, Gorka <Gorka.Garcia@...>
Subject: Re: [Edgex-golang] Export Go - for preview

Hi Trevor,

On Fri, Feb 23, 2018 at 5:15 PM, <Trevor.Conn@...> wrote:
Dell - Internal Use - Confidential

" If there is an all native Go library for ZMQ, my guys could not find one that works when we looked a few months back."

From what I can see NanoMsg is pitched as the next-gen of ZeroMQ.
http://nanomsg.org/documentation-zeromq.html

There is a Go-native implementation of the NanoMsg scalability protocols called "Mangos"
https://github.com/go-mangos/mangos
https://godoc.org/github.com/go-mangos/mangos

It appears NanoMsg has clients in a wide variety of languages, though I can't speak to their maturity.
http://nanomsg.org/documentation.html
Agreed for Nanomsg, this is a good candidate (btw, I have mentioned this 6 months ago :)).


NATS is interesting too because it's Go native but it would require a server running in a container/natively which either way is another process to manage.
Simplicity. Whole system will be highly simplified, IMHO


I personally don't even see gRPC as a player in the discussion because at the end of the day it's an RPC framework, which seems more suitable if we're talking about replacing REST than a decoupled messaging solution.
Async bi-directional event sending. Protobuf included. Real TLS (because we have HTTP).

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Drasko DRASKOVIC <drasko@...>
 

Hi Jim,

On Fri, Feb 23, 2018 at 5:51 PM, <James.White2@...> wrote:
Dell - Internal Use - Confidential

Thanks for this Trevor and Drasko,

Good stuff!!!

My top 3 requirements would be:
-Broker-less (we've done MQTT and broker before. It's not a problem, but additional broker requires more setup/containers/etc. and can be large footprint depending on implementation. We can and should still allow MQTT as alternative option for those that want it - like we have been offering in Java side to date - but the default should be brokerless).
-Go native (but also provides libraries for Java, C and possibly other languages). As we look at messaging use between other services in the future, support for polyglot is integral to our tenets and needs of the future.
-Lightweight, ease of use, etc.

What I am seeing on this thread is that people feel Nanomsg may be leading candidate?? Would like to hear from others that have an opinions or questions. I'd like to hold this as a major part of our discussion in the Go project meeting (once we have the mono repo work done).
To clarify, my personal order of preference would be:
- NATS - for MVP, because of simplification of the system
- gRPC - because Google is behind it and it is extremely well
maintained and documented framework, used in industrial products and
with several additional benefits (protobuf, TLS, rate limiting,
circuit breaker, ...)
- Nanomsg - because it is a pure TCP and probably very performant

MQTT broker is not adequate for micro-service comm, I think. It is not
built for this purpose (like NATS, AMQP, RabbitMQ, Kafka, ...). In any
case - there is no native Go implementation of MQTT broker, which
would bring us to similar problem of cross-compilation.

Both of the 3 technologies are very good, and if you think that
brokerless design is needed - then I would rather investigate gRPC,
unless there is some special argument to use TCP-based messaging:
overhead, performance, etc...

I can also advise you to make a small PoC of several microservice
communication with NATS, just to get a feel of how simple it is, so
you can make an estimation would this actually simplify the system, or
make i more complex (extra Docker, etc...).

In any case - it would be hard to say for me which technology is the
best for particular EdgeX use-case: all 3 technologies look great, and
I think we will need to do research and make a comparison matrix to
select the best candidate. Maybe also try to engage a wider dev
community out of our edgex-golang group (for example ask on edgex-dev
later).

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Drasko DRASKOVIC <drasko@...>
 

Opened in October 2017: https://github.com/drasko/edgex-export/issues/10

Mentioned in October on ML:
https://lists.edgexfoundry.org/pipermail/edgex-golang/2017-October/000006.html,
https://lists.edgexfoundry.org/pipermail/edgex-golang/2017-October/000004.html

Written in this same thread before (unfortunately not sent to the
list) on Feb 19 (5 days ago):

Something we need to eventually revisit - replacing 0MQ with something like Nats or gRPC so that we don't have to do any C compiling and everything is Go. Always more work guys :)
"We have to decide if we want centralized broker or decentralized async
comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot, as
every service will just send the events to a topic to a known broker
(no need for complicated service discovery), and then whichever
service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols/,
and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice than
ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-throughput.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/
"

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

James.White2@...
 

Dell - Internal Use - Confidential

Good review and good suggestions Drasko. Thanks. Completely agree on the POC. We will crawl before we walk.
jim

-----Original Message-----
From: Drasko DRASKOVIC [mailto:drasko@...]
Sent: Friday, February 23, 2018 4:57 PM
To: White2, James <James_White2@...>
Cc: Conn, Trevor <Trevor_Conn@...>; manuel@...; edgex-golang@...; dejan.mjc <dejan@...>; Garcia, Gorka <Gorka.Garcia@...>
Subject: Re: [Edgex-golang] Export Go - for preview

Hi Jim,

On Fri, Feb 23, 2018 at 5:51 PM, <James.White2@...> wrote:
Dell - Internal Use - Confidential

Thanks for this Trevor and Drasko,

Good stuff!!!

My top 3 requirements would be:
-Broker-less (we've done MQTT and broker before. It's not a problem, but additional broker requires more setup/containers/etc. and can be large footprint depending on implementation. We can and should still allow MQTT as alternative option for those that want it - like we have been offering in Java side to date - but the default should be brokerless).
-Go native (but also provides libraries for Java, C and possibly other languages). As we look at messaging use between other services in the future, support for polyglot is integral to our tenets and needs of the future.
-Lightweight, ease of use, etc.

What I am seeing on this thread is that people feel Nanomsg may be leading candidate?? Would like to hear from others that have an opinions or questions. I'd like to hold this as a major part of our discussion in the Go project meeting (once we have the mono repo work done).
To clarify, my personal order of preference would be:
- NATS - for MVP, because of simplification of the system
- gRPC - because Google is behind it and it is extremely well maintained and documented framework, used in industrial products and with several additional benefits (protobuf, TLS, rate limiting, circuit breaker, ...)
- Nanomsg - because it is a pure TCP and probably very performant

MQTT broker is not adequate for micro-service comm, I think. It is not built for this purpose (like NATS, AMQP, RabbitMQ, Kafka, ...). In any case - there is no native Go implementation of MQTT broker, which would bring us to similar problem of cross-compilation.

Both of the 3 technologies are very good, and if you think that brokerless design is needed - then I would rather investigate gRPC, unless there is some special argument to use TCP-based messaging:
overhead, performance, etc...

I can also advise you to make a small PoC of several microservice communication with NATS, just to get a feel of how simple it is, so you can make an estimation would this actually simplify the system, or make i more complex (extra Docker, etc...).

In any case - it would be hard to say for me which technology is the best for particular EdgeX use-case: all 3 technologies look great, and I think we will need to do research and make a comparison matrix to select the best candidate. Maybe also try to engage a wider dev community out of our edgex-golang group (for example ask on edgex-dev later).

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic