Re: Export Go - for preview

espy
 

On 02/22/2018 07:50 PM, espy wrote:
On 02/20/2018 10:06 AM, James.White2@... wrote:
Dell - Internal Use - Confidential

Thanks for this Fede.  For our Go Lang project call today, I have as one of the agenda items to discuss this a bit.  Is it time, perhaps, to discuss replacing 0MQ and if so with Nats, gRPC, etc. that is Go cross compiling?
Just to clarify, the reason we think Nats or gRPC would be better is that there's support for both which doesn't require cgo (i.e. linking C & Go)?
Maybe I have this wrong, but the Java versions of Core Data and Export Distro use a native Java version of ZMQ, correct?

Do we know why there isn't a native Go version?

Also, if there really isn't, Drasko's suggestion to use nanomsg merits some consideration.

Regards,
/tony




Or do we just say that for this release (California), you have to run containerized on a linux based container for core-data and export-distro and all development of those has to happen on linux to be able to do the 0MQ required compilation?
Or we alternatively run them native on a Linux system?  ;)-

I think this will be a good discussion.

-----Original Message-----
From: Fede Claramonte [mailto:fclaramonte@...]
Sent: Tuesday, February 20, 2018 8:36 AM
To: Drasko DRASKOVIC <drasko@...>; White2, James <James_White2@...>; dejan.mjc <dejan@...>
Cc: chencho <smunoz@...>; manuel@...; darko@...; Garcia, Gorka <Gorka.Garcia@...>; Janko Isidorovic <janko@...>; Jeremy Phelps <jphelps@...>
Subject: Re: Export Go - for preview

Right now the problem with having dependencies using cgo are:
- Cross compilation, there is no simple way to crosscompile to another architecture when using cgo
- Use 'from scratch' dockers. When compiling with cgo enabled go binaries have a dependency with the glibc used in compilation, from what I can understand from the log of the compilation (pasted below)
It looks like lzmq uses libc, and libc can't be statically linked:
https://github.com/zeromq/libzmq/issues/2849
So yes, you need the same version of libc and supporting shared libraries that you linked against on the target system.
Regards,
/tony


Fede

feclare@serrano:~/projects/gopath/src/github.com/edgexfoundry/edgex-go$
make cmd/core-data/core-data
CGO_ENABLED=1 go build -ldflags "-X main.version=0.5.0 -extldflags '-lstdc++ -static -lsodium -static -lzmq -lpgm -static -lpthread -static -lm -static' -v " -o cmd/core-data/core-data ./cmd/core-data # github.com/edgexfoundry/edgex-go/cmd/core-data
HEADER = -H4 -T0x401000 -D0x0 -R0x1000
searching for runtime.a in $WORK/runtime.a searching for runtime.a in /home/feclare/projects/gopath/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/lib/go-1.8/pkg/linux_amd64/runtime.a
   0.00 deadcode
   0.08 pclntab=1263809 bytes, funcdata total 172301 bytes
   0.11 dodata
   0.14 dwarf
   0.16 symsize = 0
   0.20 reloc
   0.22 asmb
   0.22 codeblk
   0.23 datblk
   0.24 sym
   0.24 symsize = 222144
   0.25 symsize = 224088
   0.25 dwarf
   0.28 headr
   0.30 host link: "gcc" "-m64" "-gdwarf-2" "-o"
"/tmp/go-build995971213/github.com/edgexfoundry/edgex-go/cmd/core-data/_obj/exe/a.out"
"-static" "/tmp/go-link-683737096/go.o"
"/tmp/go-link-683737096/000000.o" "/tmp/go-link-683737096/000001.o"
"/tmp/go-link-683737096/000002.o" "-g" "-O2" "-g" "-O2" "-lpthread" "-g"
"-O2" "-lzmq" "-no-pie" "-lstdc++" "-static" "-lsodium" "-static"
"-lzmq" "-lpgm" "-static" "-lpthread" "-static" "-lm" "-static"
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libzmq.a(src_libzmq_la-ipc_listener.o):
In function `zmq::ipc_listener_t::filter(int)':
(.text+0xd60): warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0xd43): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-683737096/000000.o: In function
`_cgo_cb4050e05860_C2func_getaddrinfo':
/build/golang-1.8-GaFns3/golang-1.8-1.8.3/src/net/cgo-gcc-prolog:46:
warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnodeaddr.o):
In function `pgm_getnodeaddr':
(.text+0x28a): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getnetbyname.o):
In function `pgm_getnetbyname':
(.text+0x321): warning: Using 'getnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x31c): warning: Using 'setnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
(.text+0x482): warning: Using 'endnetent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libpgm.a(libpgm_noinst_la-getprotobyname.o):
In function `_pgm_native_getprotobyname':
(.text+0x42): warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
   1.46 cpu time
96145 symbols
88588 liveness data



On 19/02/18 20:36, Drasko DRASKOVIC wrote:
On Mon, Feb 19, 2018 at 5:11 PM,  <James.White2@...> wrote:
Dell - Internal Use - Confidential

I think this is the 0MQ stuff right Fede??
I am sure Fede will find the way to compile ZMQ. It is a popular and
widely used lib, I doubt that we can not handle the compilation...

Fede,
can you please post the logs if you run into the trouble and we can
debug together.

Something we need to eventually revisit - replacing 0MQ with
something like Nats or gRPC so that we don't have to do any C
compiling and everything is Go.  Always more work guys  :)
We have to decide if we want centralized broker or decentralized async
comm like it is today. Both approaches have benfits and drawbacks.
Centralized broker (NATS) would probably simplify the design a lot, as
every service will just send the events to a topic to a known broker
(no need for complicated service discovery), and then whichever
service needs to consume these events will just subscribe to a topic.
On the other hand - it is a SPOF, but this can be solved with FT
deployment.

ZMQ is a good choice, although that I would personally select nanomsg:
http://nanomsg.org/, which is a successor of ZMQ (same author). Some
interesting info:
https://bravenewgeek.com/a-look-at-nanomsg-and-scalability-protocols/,
and there is also Go-native implementation:
https://github.com/go-mangos/mangos.

gRPC would be probably easier to implement and maintain, but it is
HTTP-based protocol (not TCP, like ZMQ), so some overhead comes with
it. Here is a nice comparison:
https://stackoverflow.com/questions/39350681/grpc-and-zeromq-comparsion.
I am not sure that in our lightway approach gRPC is better choice than
ZMQ/nanomsg. I expect higher throughput with ZMQ
(https://bbengfort.github.io/observations/2017/09/04/message-throughpu
t.html)
- although not much higher, since there is just a few clients
connected. I also expect high throughput with NATS, but pure TCP
socket that ZMQ/nanomsg uses is practically unbeatable:
https://bravenewgeek.com/dissecting-message-queues/

I propose that we make everything work with ZMQ for now, to have a
stable MVP. We can experiment later in separate experimental branches
in replacing the communication mechanisms and see how thi affects
performance and improves/simplifies the system design.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com   |  Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division |   Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Join EdgeX-GoLang@lists.edgexfoundry.org to automatically receive all group messages.