Date   
Re: Regarding Base Functionality in Core Services

James.White2@...
 

Dell - Internal Use - Confidential

Sorry Drasko - poor choice of words on my part when I wrote "common utilities." I meant the common elements as Trevor defined.

-----Original Message-----
From: Drasko DRASKOVIC [mailto:drasko@...]
Sent: Friday, March 09, 2018 3:38 PM
To: White2, James <James_White2@...>
Cc: Conn, Trevor <Trevor_Conn@...>; Steve Osselton <steve@...>; edgex-golang@...; Dejan Mijic <dejan.mijic@...>
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

Hi Jim,

On Fri, Mar 9, 2018 at 6:17 PM, <James.White2@...> wrote:

Is there anyone that completely objects to the use of pkg as the place for these common utilities for now?
What do you consider under "utilities"? Libraries shared/reused between system packages (like some common structures that you do not want to repeat)? These are not utilities, and should go in project root or at least in `common` or `shared`.

If it is something else, I am not against using in `pkg`.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Re: Regarding Base Functionality in Core Services

Drasko DRASKOVIC <drasko@...>
 

Hi Jim,

On Fri, Mar 9, 2018 at 6:17 PM, <James.White2@...> wrote:

Is there anyone that completely objects to the use of pkg as the place for these common utilities for now?
What do you consider under "utilities"? Libraries shared/reused
between system packages (like some common structures that you do not
want to repeat)? These are not utilities, and should go in project
root or at least in `common` or `shared`.

If it is something else, I am not against using in `pkg`.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Re: Regarding Base Functionality in Core Services

James.White2@...
 

Dell Customer Communication

Good comments and research from everyone. Thanks all - especially Trevor, Drasko, Fede, and Steve!

From the How to Write Go Code "bible" (https://golang.org/doc/code.html), they suggest following structure:

src contains Go source files,
pkg contains package objects, and
bin contains executable commands.
The go tool builds source packages and installs the resulting binaries to the pkg and bin directories.

So, that would, at least based on this supposed authoritative guide, suggest that pkg is not used in the same way we are intending.
However, I don't think Moby, Kubernetes, Prometheus, Fleet, and others I looked at are using it in this way either (at least I don't see them as places for binaries). Through deeper research, I found projects like these...
Prometheus https://github.com/prometheus/prometheus have both a pkg and a "shared"
Ethereum https://github.com/ethereum/go-ethereum has a common (no pkg)
Packer https://github.com/hashicorp/packer uses "common" (no pkg)
Drone https://github.com/drone/drone uses "shared" (no pkg)
Consul https://github.com/hashicorp/consul & Dropbox Go https://github.com/dropbox/godropbox uses none of the above (and don't bother to have a src folder either).

So, my take away is that there doesn't appear to be any real consistency yet. We'll have to take our best shot at providing a name (and location) that people can agree upon - at least for now, until the organization of Go code becomes a little more mature.

Is there anyone that completely objects to the use of pkg as the place for these common utilities for now?

Jim

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of Conn, Trevor
Sent: Friday, March 09, 2018 10:22 AM
To: drasko@...; steve@...
Cc: edgex-golang@...; dejan.mijic@...
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

Dell Customer Communication

Taking the Docker/Moby tree as an example, if you look in "pkg" you'll find packages used throughout the application that don't contain any of the primary functionality of the application. When I think of Docker, I think about OS virtualization and not "parser", "directory", "fsutils" and so on. For example, a container is a primary entity of Docker:

https://github.com/moby/moby/blob/master/container/container.go

The above file references many smaller utility packages in a "pkg" folder. These utilities are used throughout the system but are not primary application entities. They facilitate the primary application entities, and that's how I understood the comment from the ReadMe.

Trevor

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of Drasko DRASKOVIC
Sent: Friday, March 9, 2018 10:08 AM
To: Steve Osselton <steve@...>
Cc: Conn, Trevor <Trevor_Conn@...>; edgex-golang@...; Dejan Mijic <dejan.mijic@...>
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

"pkg/ is a collection of utility packages used by the Moby project **without being specific to its internals**."

I was convinced that we are speaking about exactly opposite thing - where to put the "common" files that are part of the system internals and used across many/all system packages.

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic


On Fri, Mar 9, 2018 at 5:02 PM, Steve Osselton <steve@...> wrote:
+1

On 9 March 2018 at 15:58, <Trevor.Conn@...> wrote:

Dell Customer Communication

Here's something I found in the Docker tree structure

https://github.com/moby/moby/tree/master/pkg

From the readme:

"pkg/ is a collection of utility packages used by the Moby project
without being specific to its internals."

Kubernetes does the same:
https://github.com/kubernetes/kubernetes/tree/master/pkg

This is exactly what I'm talking about. We don't know how many of
these packages we'll wind up having and to just put them in the root
will dilute the overall structure of the project. We need to group
them somewhere. I think "pkg" is good based on the rather mature and
widely adopted projects referenced above.

Trevor

-----Original Message-----
From: edgex-golang-bounces@...
[mailto:edgex-golang-bounces@...] On Behalf Of
Drasko DRASKOVIC
Sent: Friday, March 9, 2018 8:02 AM
To: Fede Claramonte <fclaramonte@...>; Dejan Mijic
<dejan.mijic@...>
Cc: edgex-golang@...
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core
Services

On Fri, Mar 9, 2018 at 2:11 PM, Fede Claramonte
<fclaramonte@...> wrote:
What about putting that files in the root?
+1 for root. These common files should belong to `edgex` package
living in the root.

You can see example here:
https://github.com/mijicd/mainflux/tree/mainflux-159 - for example
`messages.go` that is used in other packages sits in the project
root, and belongs to "central" `mainflux` package:
https://github.com/mijicd/mainflux/blob/mainflux-159/messages.go#L1

We did similar for export services previously:
https://github.com/edgexfoundry/export-go

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang



--
Technical Director
IOTech Systems Ltd.
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Regarding Base Functionality in Core Services

Trevor.Conn@...
 

Dell Customer Communication

Taking the Docker/Moby tree as an example, if you look in "pkg" you'll find packages used throughout the application that don't contain any of the primary functionality of the application. When I think of Docker, I think about OS virtualization and not "parser", "directory", "fsutils" and so on. For example, a container is a primary entity of Docker:

https://github.com/moby/moby/blob/master/container/container.go

The above file references many smaller utility packages in a "pkg" folder. These utilities are used throughout the system but are not primary application entities. They facilitate the primary application entities, and that's how I understood the comment from the ReadMe.

Trevor

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of Drasko DRASKOVIC
Sent: Friday, March 9, 2018 10:08 AM
To: Steve Osselton <steve@...>
Cc: Conn, Trevor <Trevor_Conn@...>; edgex-golang@...; Dejan Mijic <dejan.mijic@...>
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

"pkg/ is a collection of utility packages used by the Moby project **without being specific to its internals**."

I was convinced that we are speaking about exactly opposite thing - where to put the "common" files that are part of the system internals and used across many/all system packages.

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic


On Fri, Mar 9, 2018 at 5:02 PM, Steve Osselton <steve@...> wrote:
+1

On 9 March 2018 at 15:58, <Trevor.Conn@...> wrote:

Dell Customer Communication

Here's something I found in the Docker tree structure

https://github.com/moby/moby/tree/master/pkg

From the readme:

"pkg/ is a collection of utility packages used by the Moby project
without being specific to its internals."

Kubernetes does the same:
https://github.com/kubernetes/kubernetes/tree/master/pkg

This is exactly what I'm talking about. We don't know how many of
these packages we'll wind up having and to just put them in the root
will dilute the overall structure of the project. We need to group
them somewhere. I think "pkg" is good based on the rather mature and
widely adopted projects referenced above.

Trevor

-----Original Message-----
From: edgex-golang-bounces@...
[mailto:edgex-golang-bounces@...] On Behalf Of
Drasko DRASKOVIC
Sent: Friday, March 9, 2018 8:02 AM
To: Fede Claramonte <fclaramonte@...>; Dejan Mijic
<dejan.mijic@...>
Cc: edgex-golang@...
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core
Services

On Fri, Mar 9, 2018 at 2:11 PM, Fede Claramonte
<fclaramonte@...> wrote:
What about putting that files in the root?
+1 for root. These common files should belong to `edgex` package
living in the root.

You can see example here:
https://github.com/mijicd/mainflux/tree/mainflux-159 - for example
`messages.go` that is used in other packages sits in the project
root, and belongs to "central" `mainflux` package:
https://github.com/mijicd/mainflux/blob/mainflux-159/messages.go#L1

We did similar for export services previously:
https://github.com/edgexfoundry/export-go

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang



--
Technical Director
IOTech Systems Ltd.
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Regarding Base Functionality in Core Services

Drasko DRASKOVIC <drasko@...>
 

"pkg/ is a collection of utility packages used by the Moby project
**without being specific to its internals**."

I was convinced that we are speaking about exactly opposite thing -
where to put the "common" files that are part of the system internals
and used across many/all system packages.

BR,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

On Fri, Mar 9, 2018 at 5:02 PM, Steve Osselton <steve@...> wrote:
+1

On 9 March 2018 at 15:58, <Trevor.Conn@...> wrote:

Dell Customer Communication

Here's something I found in the Docker tree structure

https://github.com/moby/moby/tree/master/pkg

From the readme:

"pkg/ is a collection of utility packages used by the Moby project without
being specific to its internals."

Kubernetes does the same:
https://github.com/kubernetes/kubernetes/tree/master/pkg

This is exactly what I'm talking about. We don't know how many of these
packages we'll wind up having and to just put them in the root will dilute
the overall structure of the project. We need to group them somewhere. I
think "pkg" is good based on the rather mature and widely adopted projects
referenced above.

Trevor

-----Original Message-----
From: edgex-golang-bounces@...
[mailto:edgex-golang-bounces@...] On Behalf Of Drasko
DRASKOVIC
Sent: Friday, March 9, 2018 8:02 AM
To: Fede Claramonte <fclaramonte@...>; Dejan Mijic
<dejan.mijic@...>
Cc: edgex-golang@...
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

On Fri, Mar 9, 2018 at 2:11 PM, Fede Claramonte
<fclaramonte@...> wrote:
What about putting that files in the root?
+1 for root. These common files should belong to `edgex` package
living in the root.

You can see example here:
https://github.com/mijicd/mainflux/tree/mainflux-159 - for example
`messages.go` that is used in other packages sits in the project root, and
belongs to "central" `mainflux` package:
https://github.com/mijicd/mainflux/blob/mainflux-159/messages.go#L1

We did similar for export services previously:
https://github.com/edgexfoundry/export-go

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang



--
Technical Director
IOTech Systems Ltd.

Re: Regarding Base Functionality in Core Services

Steve Osselton
 

+1

On 9 March 2018 at 15:58, <Trevor.Conn@...> wrote:
Dell Customer Communication

Here's something I found in the Docker tree structure

https://github.com/moby/moby/tree/master/pkg

From the readme:

"pkg/ is a collection of utility packages used by the Moby project without being specific to its internals."

Kubernetes does the same:
https://github.com/kubernetes/kubernetes/tree/master/pkg

This is exactly what I'm talking about. We don't know how many of these packages we'll wind up having and to just put them in the root will dilute the overall structure of the project. We need to group them somewhere. I think "pkg" is good based on the rather mature and widely adopted projects referenced above.

Trevor

-----Original Message-----
From: edgex-golang-bounces@lists.edgexfoundry.org [mailto:edgex-golang-bounces@lists.edgexfoundry.org] On Behalf Of Drasko DRASKOVIC
Sent: Friday, March 9, 2018 8:02 AM
To: Fede Claramonte <fclaramonte@caviumnetworks.com>; Dejan Mijic <dejan.mijic@...>
Cc: edgex-golang@lists.edgexfoundry.org
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

On Fri, Mar 9, 2018 at 2:11 PM, Fede Claramonte <fclaramonte@caviumnetworks.com> wrote:
> What about putting that files in the root?

+1 for root. These common files should belong to `edgex` package
living in the root.

You can see example here:
https://github.com/mijicd/mainflux/tree/mainflux-159 - for example `messages.go` that is used in other packages sits in the project root, and belongs to "central" `mainflux` package:
https://github.com/mijicd/mainflux/blob/mainflux-159/messages.go#L1

We did similar for export services previously:
https://github.com/edgexfoundry/export-go

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com   |  Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division |   Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@lists.edgexfoundry.org
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@lists.edgexfoundry.org
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang



--
Technical Director
IOTech Systems Ltd.

Re: Regarding Base Functionality in Core Services

James.White2@...
 

Dell Customer Communication

+1
Following other industry leaders in this is a great idea

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of Conn, Trevor
Sent: Friday, March 09, 2018 9:58 AM
To: drasko@...; fclaramonte@...; dejan.mijic@...
Cc: edgex-golang@...
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

Dell Customer Communication

Here's something I found in the Docker tree structure

https://github.com/moby/moby/tree/master/pkg

From the readme:

"pkg/ is a collection of utility packages used by the Moby project without being specific to its internals."

Kubernetes does the same:
https://github.com/kubernetes/kubernetes/tree/master/pkg

This is exactly what I'm talking about. We don't know how many of these packages we'll wind up having and to just put them in the root will dilute the overall structure of the project. We need to group them somewhere. I think "pkg" is good based on the rather mature and widely adopted projects referenced above.

Trevor

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of Drasko DRASKOVIC
Sent: Friday, March 9, 2018 8:02 AM
To: Fede Claramonte <fclaramonte@...>; Dejan Mijic <dejan.mijic@...>
Cc: edgex-golang@...
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

On Fri, Mar 9, 2018 at 2:11 PM, Fede Claramonte <fclaramonte@...> wrote:
What about putting that files in the root?
+1 for root. These common files should belong to `edgex` package
living in the root.

You can see example here:
https://github.com/mijicd/mainflux/tree/mainflux-159 - for example `messages.go` that is used in other packages sits in the project root, and belongs to "central" `mainflux` package:
https://github.com/mijicd/mainflux/blob/mainflux-159/messages.go#L1

We did similar for export services previously:
https://github.com/edgexfoundry/export-go

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Regarding Base Functionality in Core Services

Trevor.Conn@...
 

Dell Customer Communication

Here's something I found in the Docker tree structure

https://github.com/moby/moby/tree/master/pkg

From the readme:

"pkg/ is a collection of utility packages used by the Moby project without being specific to its internals."

Kubernetes does the same:
https://github.com/kubernetes/kubernetes/tree/master/pkg

This is exactly what I'm talking about. We don't know how many of these packages we'll wind up having and to just put them in the root will dilute the overall structure of the project. We need to group them somewhere. I think "pkg" is good based on the rather mature and widely adopted projects referenced above.

Trevor

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of Drasko DRASKOVIC
Sent: Friday, March 9, 2018 8:02 AM
To: Fede Claramonte <fclaramonte@...>; Dejan Mijic <dejan.mijic@...>
Cc: edgex-golang@...
Subject: Re: [Edgex-golang] Regarding Base Functionality in Core Services

On Fri, Mar 9, 2018 at 2:11 PM, Fede Claramonte <fclaramonte@...> wrote:
What about putting that files in the root?
+1 for root. These common files should belong to `edgex` package
living in the root.

You can see example here:
https://github.com/mijicd/mainflux/tree/mainflux-159 - for example `messages.go` that is used in other packages sits in the project root, and belongs to "central" `mainflux` package:
https://github.com/mijicd/mainflux/blob/mainflux-159/messages.go#L1

We did similar for export services previously:
https://github.com/edgexfoundry/export-go

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Regarding Base Functionality in Core Services

Drasko DRASKOVIC <drasko@...>
 

On Fri, Mar 9, 2018 at 2:11 PM, Fede Claramonte
<fclaramonte@...> wrote:
What about putting that files in the root?
+1 for root. These common files should belong to `edgex` package
living in the root.

You can see example here:
https://github.com/mijicd/mainflux/tree/mainflux-159 - for example
`messages.go` that is used in other packages sits in the project root,
and belongs to "central" `mainflux` package:
https://github.com/mijicd/mainflux/blob/mainflux-159/messages.go#L1

We did similar for export services previously:
https://github.com/edgexfoundry/export-go

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Re: Regarding Base Functionality in Core Services

Fede Claramonte
 

What about putting that files in the root?


On 03/09/2018 01:35 PM, Steve Osselton wrote:
I'm good with common :)

On 9 March 2018 at 12:30, Drasko DRASKOVIC <drasko.draskovic@...> wrote:
On Fri, Mar 9, 2018 at 8:05 AM, Steve Osselton <steve@...> wrote:
> He Trevor,
>
> Sounds good how about "util" for a package?

I see `util` more for binary utilities, scripts, tools...

I would go with `common`... I would also think about the option to put
the files in the project root and make them a part of general `edgex`
package.

Best regards,
Drasko



--
Technical Director
IOTech Systems Ltd.


_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Regarding Base Functionality in Core Services

Steve Osselton
 

I'm good with common :)

On 9 March 2018 at 12:30, Drasko DRASKOVIC <drasko.draskovic@...> wrote:
On Fri, Mar 9, 2018 at 8:05 AM, Steve Osselton <steve@...> wrote:
> He Trevor,
>
> Sounds good how about "util" for a package?

I see `util` more for binary utilities, scripts, tools...

I would go with `common`... I would also think about the option to put
the files in the project root and make them a part of general `edgex`
package.

Best regards,
Drasko



--
Technical Director
IOTech Systems Ltd.

Re: Regarding Base Functionality in Core Services

Drasko DRASKOVIC <drasko.draskovic@...>
 

On Fri, Mar 9, 2018 at 8:05 AM, Steve Osselton <steve@...> wrote:
He Trevor,

Sounds good how about "util" for a package?
I see `util` more for binary utilities, scripts, tools...

I would go with `common`... I would also think about the option to put
the files in the project root and make them a part of general `edgex`
package.

Best regards,
Drasko

Re: EdgeX in embbeded device

Joan Duran
 

Hello Drasko,

BoltDB is a pure Go implementation, so there is no need to use CGO and cross-compile (as you know, we want to avoid cross-compile). That said, I didn't checked BaggerDB, which seems to be much faster that BoltDB in write operations, but slightly slower in read operations:

https://medium.com/kongkow-it-medan/badger-vs-boltdb-persistent-key-value-store-written-in-go-e94ada87048f
https://blog.dgraph.io/post/badger-lmdb-boltdb/

Regarding about using Key/Value or typical SQL/NoSQL database, as most of the readings are performed using the ID, using a Key/Value data base doesn't seems to be a bad option.

Anyway, we will perform a more exhaustive check and let you know our conclusions.

Finally, participate in Go meetings, it's quite hard for us because usually they are in our dinner time...

Best regards,
Joan
________________________________________
De: Drasko DRASKOVIC <drasko@...>
Enviat el: dijous, 8 / març / 2018 22:04
Per a: Joan Duran
A/c: edgex-golang@...
Tema: Re: [Edgex-golang] EdgeX in embbeded device

Hello Joan,
during our last TSC meeting Go enthusiasts promoted the idea of EdgeX
running on RPi-like targets. We believe that this can be possible now,
thanks to Go footprint.

We are also aware that we'll have to advance on points of ZMQ and DB.

On Thu, Mar 8, 2018 at 4:52 PM, Joan Duran <jduran@...> wrote:
Hello!


We are a very interested in use EdgeX in a ARM device with low resources
(RAM should be 256MB and NAND Flash 512MB). Also we will manage few devices
(usually less than 10).


In our first tests, we detected two main issues with the current
implementation of EdgeX using Go:

It's painful to crosscompile 0MQ, so for us use another option like
nanomsg/mangos looks like a good option.

Yes - this has been discussed on mailing lists and Go IG meetings. We
are currently evaluating:
- Nanomsg (mangos)
- GRPC
- NATS

All approaches have benefits and drawbacks. I expressed my personal
preferences, you can find the thread in the ML archives.

We can't use MongoDB, as it use a minimum of 100MB of RAM, and just creating
the database uses around 600MB.
Depends. MongDB can be very small, and we need to see where data size
comes from.


So we are looking for a MongoDB alternative, and we tried those options:

Tideot [1]. Is a Go documental database, which can be embedded. We discarded
this option because also uses a huge amount of Flash.
SQlite3 [2][3]. A very known SQL database. We discarded this option because
we want to avoid to crosscompile
Bolt [4]. Is an embedded Go key/value database. We modified some parts of
core-metadata and it's working fine.

What do you think about using Bolt with device with low resources as
alternative of MongoDB? Do you know another options?

BoltDB is not an alternative to classical DB - it is in-RAM key-value
storage. It is industry-class product, but can not fully replace
SQL/NoSQL DBs.
Additionally, to avoid same class of CGO and cross-compilation
problems related to ZMQ, I would personally look into BadgerDB:
https://blog.dgraph.io/post/badger/, which is pure Go implementation
from dgraph team.

I would advise first analyzing the solution (how DB is used today in
EdgeX), and then proposing replacement. Bolt/Badger can be good for
PoC, however I expect that you will run in into the various problems:
- Size of available RAM
- Dropped and lost messages
- No qerying
- etc...

In this case I would personally lean towards SQL DB, probably start
with LightSQL, but code defensively and generically so that the same
code can tomorrow use Postgres or CockroachDB. This can be achieved
via gorm (https://github.com/jinzhu/gorm) as an abstraction layer.

Overall, my suggestions would be following:
- Actively participate in the community and join Go group meetings
- Investigate `nanomsg` vs `gRPC` vs `NATS` problem and give your
inputs to the community
- Once community reach this consensus and does the changes go to the
next step: storage
- Design (on the architectural level) abstraction layer around DB
storage, so that we can have detachable and replaceable DBs (i.e. stay
DB agnostic as much as possible - gorm is one of the ideas in this
direction)
- Implement Bolt/Badger PoC
- Implement gorm-based PoC


Of course, if everything goes fine, we will commit this changes and help
with other things.
Of course, I will not be helping unless I see your work incrementally
develop in a public repo under Apache-2.0 license. Throwing at us a
huge lump of code changes at one point out of nowhere is probably not
a good idea.

Best regards,
Drasko DRASKOVIC
Mainflux Author and Technical Advisor

www.mainflux.com | Industrial IoT Cloud
-------------------------------------------------------------------
Engineering Division | Paris, France

LinkedIn: https://www.linkedin.com/in/draskodraskovic
Twitter: @draskodraskovic

Re: EdgeX in embbeded device

Fede Claramonte
 

Hi Joan,


replacing 0MQ has been already discussed, for the same reasons you describe. Another proposed lib is nats.


Regards,

Fede


On 03/08/2018 04:52 PM, Joan Duran wrote:

Hello!


We are a very interested in use EdgeX in a ARM device with low resources (RAM should be 256MB and NAND Flash 512MB). Also we will manage few devices (usually less than 10).


In our first tests, we detected two main issues with the current implementation of EdgeX using Go:

  1. It's painful to crosscompile 0MQ, so for us use another option like nanomsg/mangos looks like a good option.
  2. We can't use MongoDB, as it use a minimum of 100MB of RAM, and just creating the database uses around 600MB.

So we are looking for a MongoDB alternative, and we tried those options:

  1. Tideot [1]. Is a Go documental database, which can be embedded. We discarded this option because also uses a huge amount of Flash.
  2. SQlite3 [2][3]. A very known SQL database. We discarded this option because we want to avoid to crosscompile
  3. Bolt [4]. Is an embedded Go key/value database. We modified some parts of core-metadata and it's working fine.

What do you think about using Bolt with device with low resources as alternative of MongoDB? Do you know another options?


Of course, if everything goes fine, we will commit this changes and help with other things.


Best regards,

Joan


[1] https://github.com/HouzuoGuo/tiedot

[2] https://www.sqlite.org/index.html

[3] https://github.com/mattn/go-sqlite3

[4] https://github.com/coreos/bbolt





_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Regarding Base Functionality in Core Services

Steve Osselton
 

He Trevor,

Sounds good how about "util" for a package?

Cheers Steve


On 9 Mar 2018 00:36, <Trevor.Conn@...> wrote:

Hi folks -- With my most recent PR in the edgex-go repo I've started to consolidate some of the code repetition involved in bootstrapping the core services. For this particular PR, the target is the heartbeat functionality. As you can see, I moved the logic into a support/heartbeat package with is then called via one line in the main.go of each service.


heartbeat.Start(configuration.HeartBeatMsg, configuration.HeartBeatTime, loggingClient)


I don't necessarily think this is the best place for it, but it will do for now. We don't particularly have a good place in the package structure for common, base functionality required by all services in EdgeX. As Jim explained it to me, "support" is meant for optional services (like logging and notifications) so to me, the support package should be concerned solely with the implementation of these services (like "core" or "export"). So if we have functionality that cuts across all services, like a heartbeat or perhaps configuration, where should it go?


We should probably avoid overly broad terms like "common" or "libs". I don't know. "Service-base" or "aspects"? I'm concerned that "support" will become a dumping ground for miscellaneous stuff.


If anyone has any thoughts, let me know.


Also, please note that LoggingClient is now an interface. The heartbeat implementation receives this interface as a parameter and the unit test has its own mock implementation. I'll be emphasizing decoupling and unit testing as I make more changes.


Thanks,

Trevor



_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@lists.edgexfoundry.org
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang


EdgeX: Move Go Project discussions under Core WG? [Poll]

Brett Preston
 

Members of the Go Project group,

Per discussions in the Core WG meeting on Thursday, March 9, a proposal was risen to end the Go Lang Project Group after the March 20 meeting, and address all Go related work/questions/issues in the Core Working Group moving forward.

Before any final decisions are made, the group would like to poll the community to gage preferences.

Please complete the poll below by 5pm PDT on Wednesday, March 14


Thank you,


Brett

--
Brett Preston
The Linux Foundation
+1 (971) 303-9030
bpreston@...

Google Talk: bpreston@...
Skype: bprestoncf

Regarding Base Functionality in Core Services

Trevor.Conn@...
 

Hi folks -- With my most recent PR in the edgex-go repo I've started to consolidate some of the code repetition involved in bootstrapping the core services. For this particular PR, the target is the heartbeat functionality. As you can see, I moved the logic into a support/heartbeat package with is then called via one line in the main.go of each service.


heartbeat.Start(configuration.HeartBeatMsg, configuration.HeartBeatTime, loggingClient)


I don't necessarily think this is the best place for it, but it will do for now. We don't particularly have a good place in the package structure for common, base functionality required by all services in EdgeX. As Jim explained it to me, "support" is meant for optional services (like logging and notifications) so to me, the support package should be concerned solely with the implementation of these services (like "core" or "export"). So if we have functionality that cuts across all services, like a heartbeat or perhaps configuration, where should it go?


We should probably avoid overly broad terms like "common" or "libs". I don't know. "Service-base" or "aspects"? I'm concerned that "support" will become a dumping ground for miscellaneous stuff.


If anyone has any thoughts, let me know.


Also, please note that LoggingClient is now an interface. The heartbeat implementation receives this interface as a parameter and the unit test has its own mock implementation. I'll be emphasizing decoupling and unit testing as I make more changes.


Thanks,

Trevor


Re: Extra merge commits & commit messages...

James.White2@...
 

Dell - Internal Use - Confidential

Thanks Drasko. I like this.

I also just sent a note to Tony in that I think we need a video or tutorial that helps new developers to our community - a place to go to learn about the dos and don'ts of working in our community and with GitHub, etc.
I will admit, that I have a lot to learn in this area as well, and I suspect that while I am not the brightest bulb on the Christmas tree, if I am questioning and not sure about some of this, others are also. So we need to offer some education - and the more (video, .md doc, etc.) the better.

I will volunteer to help put some of this together if the experts can help me with the content.
Jim

-----Original Message-----
From: edgex-golang-bounces@... [mailto:edgex-golang-bounces@...] On Behalf Of Drasko DRASKOVIC
Sent: Thursday, March 08, 2018 4:44 PM
To: Tony Espy <espy@...>
Cc: edgex-golang@...
Subject: Re: [Edgex-golang] Extra merge commits & commit messages...

I think we're missing something like this:
https://github.com/mainflux/mainflux/blob/master/.github/CONTRIBUTING.md

If somebody (Tony?) can craft a CONTRIBUTING.md doc, that would be great, IMHO.

Also, issue template in .github would be good idea:
https://github.com/mainflux/mainflux/blob/master/.github/ISSUE_TEMPLATE.md

BR,
Drasko

On Thu, Mar 8, 2018 at 11:38 PM, Tony Espy <espy@...> wrote:
A couple of suggestions about our git workflow, after poking around
with git log...

1) Since we're using the github flow to merge our PRs, there are a lot
of merge commits, however I'm also seeing quite a few merge commits
that look like this:

Merge branch 'master' into <feature branch>

This happens when a feature branch sits around for awhile, and then
has to be re-merged with HEAD. Now, I know it's easy to filter merge
commits using 'git log --no-merges', but it's also easy to rebase your
feature branch before pushing, or using 'git stash; git pull; git
stash apply' to perform a similar workflow. Both techniques get rid of
these merge commits, making our git history much cleaner.

2) Please make sure you follow our commit style guide! There are
bunch of commits that landed recently that are one long line of
description. The commit summary should be no more than 50 chars, and
then feel free to write more detail in the message body. 'git log --oneline' makes these obvious.

Regards,
/tony




_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang
_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Extra merge commits & commit messages...

espy
 

On 3/8/18 5:43 PM, Drasko DRASKOVIC wrote:

I think we're missing something like this:
https://github.com/mainflux/mainflux/blob/master/.github/CONTRIBUTING.md

If somebody (Tony?) can craft a CONTRIBUTING.md doc, that would be great, IMHO.
Well as mentioned earlier, Andy did a great job writing up our code contribution guidelines on the wiki, but that said, I'm not opposed to a file in the repo either:

https://wiki.edgexfoundry.org/display/FA/Committing+Code+Guidelines

That said, maybe we just add a Contributing.md that points to our wiki page?

Regards,
/tony







Also, issue template in .github would be good idea:
https://github.com/mainflux/mainflux/blob/master/.github/ISSUE_TEMPLATE.md

BR,
Drasko

On Thu, Mar 8, 2018 at 11:38 PM, Tony Espy <espy@...> wrote:
A couple of suggestions about our git workflow, after poking around with git
log...

1) Since we're using the github flow to merge our PRs, there are a lot of
merge commits, however I'm also seeing quite a few merge commits that look
like this:

Merge branch 'master' into <feature branch>

This happens when a feature branch sits around for awhile, and then has to
be re-merged with HEAD. Now, I know it's easy to filter merge commits using
'git log --no-merges', but it's also easy to rebase your feature branch
before pushing, or using 'git stash; git pull; git stash apply' to perform a
similar workflow. Both techniques get rid of these merge commits, making our
git history much cleaner.

2) Please make sure you follow our commit style guide!  There are bunch of
commits that landed recently that are one long line of description.  The
commit summary should be no more than 50 chars, and then feel free to write
more detail in the message body.  'git log --oneline' makes these obvious.

Regards,
/tony




_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang

Re: Extra merge commits & commit messages...

Drasko DRASKOVIC <drasko.draskovic@...>
 

I think we're missing something like this:
https://github.com/mainflux/mainflux/blob/master/.github/CONTRIBUTING.md

If somebody (Tony?) can craft a CONTRIBUTING.md doc, that would be great, IMHO.

Also, issue template in .github would be good idea:
https://github.com/mainflux/mainflux/blob/master/.github/ISSUE_TEMPLATE.md

BR,
Drasko

On Thu, Mar 8, 2018 at 11:38 PM, Tony Espy <espy@...> wrote:
A couple of suggestions about our git workflow, after poking around with git
log...

1) Since we're using the github flow to merge our PRs, there are a lot of
merge commits, however I'm also seeing quite a few merge commits that look
like this:

Merge branch 'master' into <feature branch>

This happens when a feature branch sits around for awhile, and then has to
be re-merged with HEAD. Now, I know it's easy to filter merge commits using
'git log --no-merges', but it's also easy to rebase your feature branch
before pushing, or using 'git stash; git pull; git stash apply' to perform a
similar workflow. Both techniques get rid of these merge commits, making our
git history much cleaner.

2) Please make sure you follow our commit style guide! There are bunch of
commits that landed recently that are one long line of description. The
commit summary should be no more than 50 chars, and then feel free to write
more detail in the message body. 'git log --oneline' makes these obvious.

Regards,
/tony




_______________________________________________
EdgeX-GoLang mailing list
EdgeX-GoLang@...
https://lists.edgexfoundry.org/mailman/listinfo/edgex-golang