Date   
Re: 0MQ Alternatives - Tuesday's meeting

Dellabianca, Alberto
 

Great summary Jim.

Any thoughts on having in the list also MQTT? It would be a broker option like NATS, it is lightweight and widely used in the community.

Great implementations are Mosquitto Broker and Paho Client (in different languages), not sure if the license is compatible with LF requirements though. We are already using Paho Client in EdgeX however.

Also I understand that core-data is already somewhat capable to subscribe to MQTT topics?

Thanks

Alberto

 

 

 

 

From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of James.White2@...
Sent: Friday, March 16, 2018 5:18 PM
To: edgex-golang@...
Subject: [EXTERNAL] [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

 

At the center of our Go Lang project meeting on Tuesday, we will be a discussion on ZeroMQ alternatives and which to use (if any) for the upcoming California release.

 

In advance, I looked at the products in the table below and did some experimentation.  These are the products I’ve heard from community members most.  Below are some notes to kick off our discussions.  From my research, I am most intrigued by Nats.  Nanomsg appears dead (and with many indicating other problems like speed) and Thrift and gRPC are, well, RPC.  On the negative side, Nats does require a server, but it is already available on multiple platforms and is already Dockerized.   I found the Nats API/client libs and docs seem to be the cleanest.  If we are dead set to go brokerless, Thrift would be my pick – although the compiler/generator part of Thrift doesn’t thrill me.

 

Look forward to the discussion.

 

 

Product

Native Language Bindings, OS support, etc.

Brokerless

License

Used By

Thrift

C, C++, Go, Python, Java,

Yes

Apache 2

Cloudera, Evernote, Facebook, Siemens, Uber

Nats

Go, Node, Ruby, Java, C, C#

No - server

MIT

Baidu, Siemens, HTC, Pivotal, VMWare, Ericsson

gRPC

C, Go, Java, Python, Node

Yes

Apache 2

Square, Netflix, Cisco, CoreOS

Nanomsg

C, Go, Java, Python, Rust, Node

Yes

MIT

???

 

Thrift: 

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

Thrift does require a “compiler” to create/generate a thrift file to use by the implementations.  

Nats:

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

Server binary is available for Linux (x86, x86_64, ARM), Windows (x86, x86_64), and macOS.  Server image is available in Docker image. Server is 2.4 MB in size

gRPC:

A high performance, open-source universal RPC framework

a Cloud Native Computing Foundation project.

Nanomsg:

A socket library that provides several common communication patterns. It aims to make the networking layer fast, scalable, and easy to use. Implemented in C, it works on a wide range of operating systems with no further dependencies.

There is now also an implementation of nanomsg in pure Go: mangos.

http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/

 

 

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 

0MQ Alternatives - Tuesday's meeting

vinoyang
 


---------- Forwarded message ----------
From: vino yang <yanghua1127@...>
Date: 2018-03-17 11:32 GMT+08:00
Subject: Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting
To: James.White2@...


Hi Jim White,

I tend to choose from Thrift and gRPC.  No broker, point to point, binary protocol, better performance, decentration, mature stability and so on....
Though a message broker also have some good advantage but I think the light-weight, less dependency, simplify the communication program model would be better for IoT scenario. 
However if we need topic scenario(or means publish / subscribe) , borker has more advantage than RPC framework.

Just some opinion.

Vino yang.
Thanks.


2018-03-17 6:18 GMT+08:00 <James.White2@...>:

At the center of our Go Lang project meeting on Tuesday, we will be a discussion on ZeroMQ alternatives and which to use (if any) for the upcoming California release.

 

In advance, I looked at the products in the table below and did some experimentation.  These are the products I’ve heard from community members most.  Below are some notes to kick off our discussions.  From my research, I am most intrigued by Nats.  Nanomsg appears dead (and with many indicating other problems like speed) and Thrift and gRPC are, well, RPC.  On the negative side, Nats does require a server, but it is already available on multiple platforms and is already Dockerized.   I found the Nats API/client libs and docs seem to be the cleanest.  If we are dead set to go brokerless, Thrift would be my pick – although the compiler/generator part of Thrift doesn’t thrill me.

 

Look forward to the discussion.

 

 

Product

Native Language Bindings, OS support, etc.

Brokerless

License

Used By

Thrift

C, C++, Go, Python, Java,

Yes

Apache 2

Cloudera, Evernote, Facebook, Siemens, Uber

Nats

Go, Node, Ruby, Java, C, C#

No - server

MIT

Baidu, Siemens, HTC, Pivotal, VMWare, Ericsson

gRPC

C, Go, Java, Python, Node

Yes

Apache 2

Square, Netflix, Cisco, CoreOS

Nanomsg

C, Go, Java, Python, Rust, Node

Yes

MIT

???

 

Thrift: 

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

Thrift does require a “compiler” to create/generate a thrift file to use by the implementations.  

Nats:

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

Server binary is available for Linux (x86, x86_64, ARM), Windows (x86, x86_64), and macOS.  Server image is available in Docker image. Server is 2.4 MB in size

gRPC:

A high performance, open-source universal RPC framework

a Cloud Native Computing Foundation project.

Nanomsg:

A socket library that provides several common communication patterns. It aims to make the networking layer fast, scalable, and easy to use. Implemented in C, it works on a wide range of operating systems with no further dependencies.

There is now also an implementation of nanomsg in pure Go: mangos.

http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/

 

 

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...y.org
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang



Re: 0MQ Alternatives - Tuesday's meeting

Drasko DRASKOVIC <drasko@...>
 

On Sat, Mar 17, 2018 at 2:59 AM, Dellabianca, Alberto
<Alberto.Dellabianca@...> wrote:

Great summary Jim.

Any thoughts on having in the list also MQTT? It would be a broker option like NATS, it is lightweight
NATS is more lightweight then majority MQTT brokers out there

and widely used in the community.
Not for the purpose of microservice inter-communication.


Great implementations are Mosquitto Broker
Not scalable (can not cluster easily like NATS for HA/FT). Plus
written in C, so we'll have to recompile for ARM wih C compiler- bring
us back to the problem because of which we are changing ZMQ. Plus I am
not sure it works on Windows (probably not).

and Paho Client (in different languages), not sure if the license is compatible with LF requirements though. We are already using Paho Client in EdgeX however.
For device interface, and this is OK.


Also I understand that core-data is already somewhat capable to subscribe to MQTT topics?
Export data, only as a MQTT client for the cloud (not a broker).

MQTT has several other problems - like that for simplicity of the
protocol it does not keep origin (publisher) info, so it is impossible
to prevent loopbacks when you publish and subscribe on the same topic.
Finally, and probably most importantly - besides pure PUB/SUB:
https://nats.io/documentation/concepts/nats-pub-sub/ - NATS offers
other patterns, like REQ/RSP and Queueing that are very useful and can
be eventually achieved with MQTT only with additional work.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter:@draskodraskovic

Re: [EXTERNAL] Re: 0MQ Alternatives - Tuesday's meeting

Dellabianca, Alberto
 

Drasko,

Thanks for your reply, I understand your points.

There are also MQTT broker implementations in Golang (e.g. https://github.com/VolantMQ/volantmq) although I never tested them.
Just curious: do you know on a Raspberry Pi how many messages/second can NATS process? (assuming 1 publisher and 1 subscriber for simplicity).

Thx

Alberto

 

 

From: Drasko DRASKOVIC [mailto:drasko@...]
Sent: Friday, March 16, 2018 11:38 PM
To: Dellabianca, Alberto <Alberto.Dellabianca@...>
Cc: James.White2@...; edgex-golang@...
Subject: [EXTERNAL] Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

 

On Sat, Mar 17, 2018 at 2:59 AM, Dellabianca, Alberto
<Alberto.Dellabianca@...> wrote:
>
> Great summary Jim.
>
> Any thoughts on having in the list also MQTT? It would be a broker option like NATS, it is lightweight
NATS is more lightweight then majority MQTT brokers out there

> and widely used in the community.

Not for the purpose of microservice inter-communication.

>
> Great implementations are Mosquitto Broker

Not scalable (can not cluster easily like NATS for HA/FT). Plus
written in C, so we'll have to recompile for ARM wih C compiler- bring
us back to the problem because of which we are changing ZMQ. Plus I am
not sure it works on Windows (probably not).

> and Paho Client (in different languages), not sure if the license is compatible with LF requirements though. We are already using Paho Client in EdgeX however.

For device interface, and this is OK.

>
> Also I understand that core-data is already somewhat capable to subscribe to MQTT topics?

Export data, only as a MQTT client for the cloud (not a broker).

MQTT has several other problems - like that for simplicity of the
protocol it does not keep origin (publisher) info, so it is impossible
to prevent loopbacks when you publish and subscribe on the same topic.
Finally, and probably most importantly - besides pure PUB/SUB:
https://nats.io/documentation/concepts/nats-pub-sub/ - NATS offers
other patterns, like REQ/RSP and Queueing that are very useful and can
be eventually achieved with MQTT only with additional work.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Re: [EXTERNAL] Re: 0MQ Alternatives - Tuesday's meeting

Drasko DRASKOVIC <drasko@...>
 

On Sat, Mar 17, 2018 at 6:11 AM, Dellabianca, Alberto
<Alberto.Dellabianca@...> wrote:
Drasko,

Thanks for your reply, I understand your points.

There are also MQTT broker implementations in Golang (e.g.
https://github.com/VolantMQ/volantmq) although I never tested them.
Just curious: do you know on a Raspberry Pi how many messages/second can
NATS process? (assuming 1 publisher and 1 subscriber for simplicity).
That depends on your RAM, but NATS is a monster:
https://seroter.wordpress.com/2016/05/16/modern-open-source-messaging-apache-kafka-rabbitmq-and-nats-in-action/
"NATS was originally built with Ruby and achieved a respectable 150k
messages per second. The team rewrote it in Go, and now you can do an
absurd 8-11 million messages per second. It’s tiny, just a 3MB Docker
image"

Mosquitto can achieve ~130k on a serious machine, if really pushed to
the edge. MQTT aedes can go ~200k:
https://www.nearform.com/blog/performance-reaching-ludicrous-speed/

I would not be surprised that NATS goes beyond 200k mps on RPim but I
never did the benchmarks - you must check with NATS community.

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter:@draskodraskovic

Re: [EXTERNAL] Re: 0MQ Alternatives - Tuesday's meeting

Dellabianca, Alberto
 

NATS seems to scale up very well indeed...
A good start would be if a EdgeX could sustain just 1% of what you mentioned about Mosquitto (130k —> 1.3k mps) on a RPi...
Thanks for sharing the stats.
Alberto

________________________________________
From: Drasko DRASKOVIC <drasko@...>
Sent: Saturday, March 17, 2018 12:24:58 AM
To: Dellabianca, Alberto
Cc: James.White2@...; edgex-golang@...
Subject: Re: [EXTERNAL] Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

On Sat, Mar 17, 2018 at 6:11 AM, Dellabianca, Alberto
<Alberto.Dellabianca@...> wrote:
Drasko,

Thanks for your reply, I understand your points.

There are also MQTT broker implementations in Golang (e.g.
https://github.com/VolantMQ/volantmq<https://github.com/VolantMQ/volantmq>) although I never tested them.
Just curious: do you know on a Raspberry Pi how many messages/second can
NATS process? (assuming 1 publisher and 1 subscriber for simplicity).
That depends on your RAM, but NATS is a monster:
https://seroter.wordpress.com/2016/05/16/modern-open-source-messaging-apache-kafka-rabbitmq-and-nats-in-action/<https://seroter.wordpress.com/2016/05/16/modern-open-source-messaging-apache-kafka-rabbitmq-and-nats-in-action/>
"NATS was originally built with Ruby and achieved a respectable 150k
messages per second. The team rewrote it in Go, and now you can do an
absurd 8-11 million messages per second. It’s tiny, just a 3MB Docker
image"

Mosquitto can achieve ~130k on a serious machine, if really pushed to
the edge. MQTT aedes can go ~200k:
https://www.nearform.com/blog/performance-reaching-ludicrous-speed/<https://www.nearform.com/blog/performance-reaching-ludicrous-speed/>

I would not be surprised that NATS goes beyond 200k mps on RPim but I
never did the benchmarks - you must check with NATS community.

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com<http://www.mainflux.com> | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic<https://www.linkedin.com/in/draskodraskovic>
Twitter:@draskodraskovic

Re: [EXTERNAL] Re: 0MQ Alternatives - Tuesday's meeting

James.White2@...
 

Dell - Internal Use - Confidential

Alberto,
In the code base of the Java core data and export distro, there were replacement classes and commented-out configuration for use of MQTT in place of ZeroMQ. We initially used MQTT (Paho client - ActiveMQ broker) in Project Fuse. But we found that just too heavy for some of the smaller environments.

Having said that, as we go forward and pick a "reference implementation" alternate to ZeroMQ, I would also like to again allow for those with larger platforms and use cases that need it to replace whatever we choose (Nats, etc.) with their own implementation of message infrastructure (like MQTT or even one of the rejected alternatives if it is more appropriate to their needs). We need to isolate the messaging function from the rest of the micro service so that alternatives are more easily dropped in place. That's the goal.
Jim

-----Original Message-----
From: Dellabianca, Alberto [mailto:Alberto.Dellabianca@...]
Sent: Saturday, March 17, 2018 7:01 AM
To: Drasko DRASKOVIC <drasko@...>
Cc: White2, James <James_White2@...>; edgex-golang@...
Subject: Re: [EXTERNAL] Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

NATS seems to scale up very well indeed...
A good start would be if a EdgeX could sustain just 1% of what you mentioned about Mosquitto (130k -> 1.3k mps) on a RPi...
Thanks for sharing the stats.
Alberto

________________________________________
From: Drasko DRASKOVIC <drasko@...>
Sent: Saturday, March 17, 2018 12:24:58 AM
To: Dellabianca, Alberto
Cc: James.White2@...; edgex-golang@...
Subject: Re: [EXTERNAL] Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

On Sat, Mar 17, 2018 at 6:11 AM, Dellabianca, Alberto <Alberto.Dellabianca@...> wrote:
Drasko,

Thanks for your reply, I understand your points.

There are also MQTT broker implementations in Golang (e.g.
https://github.com/VolantMQ/volantmq<https://github.com/VolantMQ/volantmq>) although I never tested them.
Just curious: do you know on a Raspberry Pi how many messages/second
can NATS process? (assuming 1 publisher and 1 subscriber for simplicity).
That depends on your RAM, but NATS is a monster:
https://seroter.wordpress.com/2016/05/16/modern-open-source-messaging-apache-kafka-rabbitmq-and-nats-in-action/<https://seroter.wordpress.com/2016/05/16/modern-open-source-messaging-apache-kafka-rabbitmq-and-nats-in-action/>
"NATS was originally built with Ruby and achieved a respectable 150k messages per second. The team rewrote it in Go, and now you can do an absurd 8-11 million messages per second. It's tiny, just a 3MB Docker image"

Mosquitto can achieve ~130k on a serious machine, if really pushed to the edge. MQTT aedes can go ~200k:
https://www.nearform.com/blog/performance-reaching-ludicrous-speed/<https://www.nearform.com/blog/performance-reaching-ludicrous-speed/>

I would not be surprised that NATS goes beyond 200k mps on RPim but I never did the benchmarks - you must check with NATS community.

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com<http://www.mainflux.com> | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic<https://www.linkedin.com/in/draskodraskovic>
Twitter:@draskodraskovic

[SECURITY] EdgeX Auth Service in Go

Drasko DRASKOVIC <drasko@...>
 

Hi all,
I have advanced with my Auth service: https://github.com/drasko/edgex-auth

Currently:
- HTTPS (TLS v1.2) is working
- NginX is forwarding all requests to Auth service via standard
feature `auth_request`:
http://nginx.org/en/docs/http/ngx_http_auth_request_module.html

In progress:
- Consul auto-discovery support (NginX can read Consul)
- Traefik support (Traefik also has `auth_request` forwarding feature)

At this point I think that code has basic functionality and can be
contributed to EdgeX official codebase.

It will bring:
- User creation and management
- User login via JWT token
- Authorization (access control) to all API endpoints if user is not logged in
- TLS encryption

If you are interested I can present the service on one of the
following TSC meetings.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter:@draskodraskovic

Re: [SECURITY] EdgeX Auth Service in Go

James.White2@...
 

Dell - Internal Use - Confidential

Drasko,
Thanks for this work!
Because this is a security feature, if you don't mind, let's work it through the security working group first. This WG has been working on the reverse proxy as well as AA and data protection (through Vault). I'd like to make sure their work and yours is merged appropriately. I can send a note to Doug Gardner (WG chair) tomorrow and ask that he get it on the schedule. After that conversation, we can move the work into the temp repo and work it through the new contribution process.

Thanks again Drasko.
Jim

-----Original Message-----
From: Drasko DRASKOVIC [mailto:drasko@...]
Sent: Sunday, March 18, 2018 9:26 PM
To: edgex-golang@...; edgex-tsc@...; edgex-devel@...; edgex-tsc-security@...; Janko Isidorovic <janko@...>; dejan.mjc <dejan@...>; Nikola Marcetic <nikola@...>; manuel@...; White2, James <James_White2@...>
Subject: [SECURITY] EdgeX Auth Service in Go

Hi all,
I have advanced with my Auth service: https://github.com/drasko/edgex-auth

Currently:
- HTTPS (TLS v1.2) is working
- NginX is forwarding all requests to Auth service via standard feature `auth_request`:
http://nginx.org/en/docs/http/ngx_http_auth_request_module.html

In progress:
- Consul auto-discovery support (NginX can read Consul)
- Traefik support (Traefik also has `auth_request` forwarding feature)

At this point I think that code has basic functionality and can be contributed to EdgeX official codebase.

It will bring:
- User creation and management
- User login via JWT token
- Authorization (access control) to all API endpoints if user is not logged in
- TLS encryption

If you are interested I can present the service on one of the following TSC meetings.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter:@draskodraskovic

Re: 0MQ Alternatives - Tuesday's meeting

Steve Osselton
 

Hi Guys,

Although currently only for evaluation is also worth taking a look at Intel DPS for IoT https://github.com/intel/dps-for-iot
This is similar in usage to MQTT but is brokerless and also supports RPC. 

Also having used Thrift, it has a couple of nice features in that it supports both pluggable transports and data encodings,
and has the concept of both required and optional RPC arguments, which gives some level of API flexibility in that
optional arguments can be added and removed from a call definition, while maintaining backwards compatibility. I actually
like the fact that there is an interface definition that defines the contract between client and server and that there is a code
generator that takes care of all boiler plate essentially allowing a developer to simply concentrate on implementation
logic.

Probably to evaluate the various solutions we need some prioritised criteria.

Cheers Steve.

On 16 March 2018 at 22:18, <James.White2@...> wrote:

At the center of our Go Lang project meeting on Tuesday, we will be a discussion on ZeroMQ alternatives and which to use (if any) for the upcoming California release.

 

In advance, I looked at the products in the table below and did some experimentation.  These are the products I’ve heard from community members most.  Below are some notes to kick off our discussions.  From my research, I am most intrigued by Nats.  Nanomsg appears dead (and with many indicating other problems like speed) and Thrift and gRPC are, well, RPC.  On the negative side, Nats does require a server, but it is already available on multiple platforms and is already Dockerized.   I found the Nats API/client libs and docs seem to be the cleanest.  If we are dead set to go brokerless, Thrift would be my pick – although the compiler/generator part of Thrift doesn’t thrill me.

 

Look forward to the discussion.

 

 

Product

Native Language Bindings, OS support, etc.

Brokerless

License

Used By

Thrift

C, C++, Go, Python, Java,

Yes

Apache 2

Cloudera, Evernote, Facebook, Siemens, Uber

Nats

Go, Node, Ruby, Java, C, C#

No - server

MIT

Baidu, Siemens, HTC, Pivotal, VMWare, Ericsson

gRPC

C, Go, Java, Python, Node

Yes

Apache 2

Square, Netflix, Cisco, CoreOS

Nanomsg

C, Go, Java, Python, Rust, Node

Yes

MIT

???

 

Thrift: 

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

Thrift does require a “compiler” to create/generate a thrift file to use by the implementations.  

Nats:

NATS Server is a simple, high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

Server binary is available for Linux (x86, x86_64, ARM), Windows (x86, x86_64), and macOS.  Server image is available in Docker image. Server is 2.4 MB in size

gRPC:

A high performance, open-source universal RPC framework

a Cloud Native Computing Foundation project.

Nanomsg:

A socket library that provides several common communication patterns. It aims to make the networking layer fast, scalable, and easy to use. Implemented in C, it works on a wide range of operating systems with no further dependencies.

There is now also an implementation of nanomsg in pure Go: mangos.

http://sealedabstract.com/rants/nanomsg-postmortem-and-other-stories/

 

 

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@lists.edgexfoundry.org
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang




--
Technical Director
IOTech Systems Ltd.

Re: [SECURITY] EdgeX Auth Service in Go

Drasko DRASKOVIC <drasko@...>
 

Hi Jim,

On Mon, Mar 19, 2018 at 4:27 AM, <James.White2@...> wrote:
Dell - Internal Use - Confidential

Drasko,
Thanks for this work!
Because this is a security feature, if you don't mind, let's work it through the security working group first.
Sure - this is why mail was sent to TSC Security ML.

This WG has been working on the reverse proxy as well as AA and data protection (through Vault). I'd like to make sure their work and yours is merged appropriately. I can send a note to Doug Gardner (WG chair) tomorrow and ask that he get it on the schedule. After that conversation, we can move the work into the temp repo and work it through the new contribution process.
It should be noted that my work is in PoC phase. It implements a
service behind the reverse proxy to which proxy sends each request for
auth. This service is practically a "replacement" for "config file"
approach - the philosophy is the same, just that NginX does not read
config file, but consults Auth service instead.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter:@draskodraskovic

Go Lang Project WG Meeting - last one!!

James.White2@...
 

The last of our Go Lang Project Meetings will be tomorrow at 2pm CDT. The main topic of discussion will be Zero MQ replacement. See https://wiki.edgexfoundry.org/display/FA/Go+Lang+Microservices+Project+Group for meeting agenda and conferencing information. The Go Lang topics will move to Thursday’s Core Working Group meetings after Tuesday.

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 

EdgeX Auth Service Proposal - PPT

Drasko DRASKOVIC <drasko@...>
 

Hi all,
please find attached the slides I presented for Auth Service proposal.

GitHub repo: https://github.com/drasko/edgex-auth

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter:@draskodraskovic

Re: Go Branching/Merging

Fede Claramonte
 

On 03/16/2018 05:03 PM, Trevor.Conn@... wrote:

Dell Customer Communication

There’s been a lot of talk recently around how we manage commits and commit messages in GitHub to make the history more clean. While in Portland earlier this week, I described to Tony and Jim a challenge in my workflow which will prevent the desired cadence around git merge check-ins. I have attached a diagram showing how my feature work progresses.

 

Note that I am following the recommended procedure of creating my own fork from the edgexfoundry/edgex-go/master and not doing my feature work in a branch of the master. This means that if I make a change in the following file, for example:

 

$GOROOT/src/github.com/tsconn23/edgex-go/core/data/init.go

 

In order to expose that new functionality, I then have to change the import path in $GOROOT/src/github.com/tsconn23/edgex-go/cmd/core-data/main.go like so:

 

From: import “github.com/edgexfoundry/edgex-go/core/data”

To: import “github.com/tsconn23/edgex-go/core/data”

 

If I don’t do this, my new functionality is unavailable. Rather than manage this on a file by file basis, my first commit always consists of a mass find/replace of the given paths in ALL files so that my fork is internally consistent. The same is true in the reverse, prior to commit back to master. However since my new changes are not in

 

$GOROOT/src/github.com/edgexfoundry/edgex-go

 

I cannot actually build locally before committing and creating a PR. I have to rely on CI/CD to ensure the build is good.

 

Given the above process, and assuming everyone is working from a fork, I don’t see how we will avoid these find/replace commits. If you see something I’m doing incorrectly or otherwise outside of best practices, let me know.

 

Thanks.

 

Trevor Conn

Senior Principal Software Engineer

Dell Technologies | IoT DellTech

Trevor.Conn@...

Round Rock, TX USA

 



_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: [disscuss] github issue standardization

vinoyang
 

Hi guys:

Anyone has more suggestions to continue this topic? How about using JIRA to manage the Edgex Project?

Vino yang
Thanks.

2018-03-16 10:15 GMT+08:00 vino yang <yanghua1127@...>:

Hi guys:

I make a suggestion : shall we standardize the github issue's title and detail?

Some point : 

  • one PR one issue
  • mark some key info in issue title with tag, such as `[core] [task] xxxx` , `[support] [improvement] xxx` and so on... to let users see more valuable information without looking details
  • how to split a big task into some sub taks, it's useful for interacting or cooperating with others and reviewing PR.
  • the commit message contains the issue id , like this pattern '#123 xxxx' , if contributor sends this PR the commit will be linked to the issue with id 123
  • mark issue with labels
  • ...
just some opinion, if some rules be wrote into contribution guidance document, it looks good to the edgex community.

Actually, I really think the JIRA is a better choice.

Vino yang.
Thanks

Re: [disscuss] github issue standardization

James.White2@...
 

Dell - Internal Use - Confidential

Hi Vino,

While I understand your sentiment, Jira would be an added expense to the project and given the number of potential users in the community, could be a rather hefty expense.  So at this time, the community is choosing to use tools more in line with project budgets.

Jim

 

From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of vino yang
Sent: Friday, March 23, 2018 12:02 AM
To: edgex-golang@...
Subject: Re: [Edgex-golang] [disscuss] github issue standardization

 

Hi guys:

 

Anyone has more suggestions to continue this topic? How about using JIRA to manage the Edgex Project?

 

Vino yang

Thanks.

 

2018-03-16 10:15 GMT+08:00 vino yang <yanghua1127@...>:

Hi guys:

 

I make a suggestion : shall we standardize the github issue's title and detail?

 

Some point : 

 

  • one PR one issue
  • mark some key info in issue title with tag, such as `[core] [task] xxxx` , `[support] [improvement] xxx` and so on... to let users see more valuable information without looking details
  • how to split a big task into some sub taks, it's useful for interacting or cooperating with others and reviewing PR.
  • the commit message contains the issue id , like this pattern '#123 xxxx' , if contributor sends this PR the commit will be linked to the issue with id 123
  • mark issue with labels
  • ...

just some opinion, if some rules be wrote into contribution guidance document, it looks good to the edgex community.

 

Actually, I really think the JIRA is a better choice.

 

Vino yang.

Thanks

 

EdgeX: TSC Call (Wednesday, March 28)

Brett Preston
 

Members of the EdgeX Technical Community,

The focus for this week's TSC call will be to plan out the California release and identify resource gaps or issues for delivery.

Full agenda will be sent to the TSC mail list prior to the call, but in general, the schedule will be as follows:

9:00-9:30CST: Core, Supporting, Application work planning

9:30-9:55: Device Service work planning

9:55-10:05: break

10:05-10:30: Security work planning

10:30-11:00: System management work planning


For those on the Security mail list, please note that your call has been canceled for this week, as Security topics will be covered as part of the extended TSC meeting.

To be added to the meeting invite for Wednesday, please subscribe to the TSC mail list at https://lists.edgexfoundry.org/mailman/listinfo/edgex-tsc, or send me an email and I can add you directly to the mail list and/or meeting invite.

Dial-in information has also been provided below for easy reference:

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/983155298

Or iPhone one-tap (US Toll): +14086380968,983155298# or +16465588656,983155298#

Or Telephone:
    Dial: +1 408 638 0968 (US Toll) or +1 646 558 8656 (US Toll)
    +1 855 880 1246 (US Toll Free)
    +1 877 369 0926 (US Toll Free)
    Meeting ID: 983 155 298
    International numbers available: https://zoom.us/zoomconference?m=mkFexUxEcqHlvXHw53PqScTDRvS48PiQ


Thank you,


Brett

--
Brett Preston
The Linux Foundation
+1 (971) 303-9030
bpreston@...

Google Talk: bpreston@...
Skype: bprestoncf

Upcoming Go Versioning Enhancements

Trevor.Conn@...
 

Dell Customer Communication

In short, it is long past time to add versions to the working vocabulary of both Go developers and our tools, and this proposal describes a way to do that.”

 

https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md

 

And a little more background to how the proposal came about:

https://blog.golang.org/versioning-proposal

 

Trevor Conn

Senior Principal Software Engineer

Dell Technologies | IoT DellTech

Trevor.Conn@...

Round Rock, TX USA

 

Re: EdgeX: TSC Call (Wednesday, March 28)

Brett Preston
 

TSC Meeting: Special Release Planning Meeting for CA call beginning in 30 minutes --

Slides:
https://docs.google.com/presentation/d/1ojeq8CtuPdAQyXtdb_Aw2xuBhLpPaJANFoum1gMUFMg/edit#slide=id.p3

Time: Wednesday, March 28, 7-9 AM (PDT)

Phone/Internet:
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/983155298

Or iPhone one-tap (US Toll):  +14086380968,983155298# or +16465588656,983155298#

Or Telephone:
    Dial: +1 408 638 0968 (US Toll) or +1 646 558 8656 (US Toll)
    +1 855 880 1246 (US Toll Free)
    +1 877 369 0926 (US Toll Free)
    Meeting ID: 983 155 298


Thanks,


Brett


On Mon, Mar 26, 2018 at 12:26 PM, Brett Preston <bpreston@...> wrote:
Members of the EdgeX Technical Community,

The focus for this week's TSC call will be to plan out the California release and identify resource gaps or issues for delivery.

Full agenda will be sent to the TSC mail list prior to the call, but in general, the schedule will be as follows:

9:00-9:30CST: Core, Supporting, Application work planning

9:30-9:55: Device Service work planning

9:55-10:05: break

10:05-10:30: Security work planning

10:30-11:00: System management work planning


For those on the Security mail list, please note that your call has been canceled for this week, as Security topics will be covered as part of the extended TSC meeting.

To be added to the meeting invite for Wednesday, please subscribe to the TSC mail list at https://lists.edgexfoundry.org/mailman/listinfo/edgex-tsc, or send me an email and I can add you directly to the mail list and/or meeting invite.

Dial-in information has also been provided below for easy reference:

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/983155298

Or iPhone one-tap (US Toll): +14086380968,983155298# or +16465588656,983155298#

Or Telephone:
    Dial: +1 408 638 0968 (US Toll) or +1 646 558 8656 (US Toll)
    +1 855 880 1246 (US Toll Free)
    +1 877 369 0926 (US Toll Free)
    Meeting ID: 983 155 298
    International numbers available: https://zoom.us/zoomconference?m=mkFexUxEcqHlvXHw53PqScTDRvS48PiQ


Thank you,


Brett

--
Brett Preston
The Linux Foundation

Skype: bprestoncf



--
Brett Preston
The Linux Foundation
+1 (971) 303-9030
bpreston@...

Google Talk: bpreston@...
Skype: bprestoncf

Reminder - Core WG meeting tomorrow at 10am CDT

James.White2@...
 

Dell - Internal Use - Confidential

The Core Working Group meeting tomorrow is at 10am CDT.  Topics for tomorrow's meeting include discussion on configuration standardization and Consul integration improvements for the Go code.  Find the agenda and connection information here:  https://wiki.edgexfoundry.org/display/FA/Core+Working+Group

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...