Date   
Re: 0MQ Alternatives - Tuesday's meeting

James.White2@...
 

Dell - Internal Use - Confidential

Hi Vino

You’ll find the short write up here:  https://wiki.edgexfoundry.org/display/FA/Architecture+Issues+and+Decisions#ArchitectureIssuesandDecisions-Messagingtechnology

 

Basically, we decided to hold on any changes yet as there didn’t appear to be any perfect solution and the impact too great to handle as part of California.  gRPC was an option we looked at, but didn’t appear to be a great alternative at this time.  We’ll be relooking at this longer term.  Your participation in exploring alternates, inputs and feedback welcome at that time.

Jim

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Sunday, May 06, 2018 3:07 AM
To: EdgeX-GoLang@...
Subject: Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

 

Hi guys,

What's the result of this topic? Or it's discussing?

Recently, I have talked with a friend works in CMCC(the Chinese biggest mobile network service provider). He said they paid attention to EdgeX and the have a full IoT ecosystem and used gRPC as the communication framework.

Re: 0MQ Alternatives - Tuesday's meeting

Zeng, Tingyu
 

Hey Vino,

 

I don’t think the community has come to a conclusion yet and this is still under discussion. The weekly meeting will have some update every on the progress and it is really great to attend such meeting to know updated information.

 

Thanks,

Tingyu

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Sunday, May 6, 2018 4:07 AM
To: EdgeX-GoLang@...
Subject: Re: [Edgex-golang] 0MQ Alternatives - Tuesday's meeting

 

Hi guys,

What's the result of this topic? Or it's discussing?

Recently, I have talked with a friend works in CMCC(the Chinese biggest mobile network service provider). He said they paid attention to EdgeX and the have a full IoT ecosystem and used gRPC as the communication framework.

[discuss] Introduce Codeclimate to improve code quality

vinoyang
 

Copy from Github issue #286

based on the discussion in issue #280 , I find a tool name Codeclimate which may be suitable for us.

It contains many engines(plugins), one of the plugins is golint .

I have two basic requirement :

  • a CLI : can run and do local check before making a PR and can customize with CI tool with shell, Codeclimate has a CLI
  • integrate with GitHub : yes, see details

so any idea about this tool? or is there more suitable tools can be shared?

[discuss] Add metrics for Edgex to enhance the system monitor

vinoyang
 

Copy from Github issue #263 :

Metrics and Log are two basic parts of a stable system. For enhancing the EdgeX's stability we should consider to monitor it in the future. The monitor would need to be provided metrics. So we should introduce the metrics infrastructure for it.

There are a few scenario where we could add metrics:

  • device data import count / rate;
  • support notification count / schedule count (trigger count)
  • export data count / rate;
  • ....

Although, metrics and log sometimes can match the same goal(provide information to calculate/aggregate/analysis). But they are different, here is a good article described the difference between them.

I find a popular go metrics package : go-metrics, which can write metrics into InfluxDB. And using this (we can also implement a logger to write log into InfluxDB), we could integrate the metrics with InfluxDB's full technology stack (monitor, dashboard, storage, computing).

----------------------------------------------------------------------------------------------------

drasko's comment : 

@yanghua good point here.

 

In Mainfllux we use metrics (and logs of course) for every service, something like this: https://github.com/mainflux/mainflux/blob/master/http/api/metrics.go, and those are then exported to Prometheus. We are using go-kit metrics because we are using go-kit in Mainfux, but go-metricslooks like a nice package and a good candidate for EdgeX.

Hope for more comment!

 

Re: [discuss] Introduce Codeclimate to improve code quality

Trevor.Conn@...
 

Dell Customer Communication

We might want to look at alternatives to CodeClimate given that it isn’t free. The code quality metrics and integration with GitHub are nice, remind me of SonarQube. It’s probably worth having a discussion about the topic of integrating a code quality pipeline via DevOps channels during the next Core WG meeting on Thursday. We could potentially also utilize Jenkins to trigger checks.

 

I thought this article provided a good overview of the kind of metrics a quality pipeline should capture.

https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567

 

Setting this up correctly (and monitoring it going forward) will require ownership from DevOps as well as each working group contributing to the codebase, although one working group might be able to chart a course for the others to follow.

 

Trevor

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Friday, June 8, 2018 9:03 PM
To: EdgeX-GoLang@...
Subject: [Edgex-golang] [discuss] Introduce Codeclimate to improve code quality

 

Copy from Github issue #286

based on the discussion in issue #280 , I find a tool name Codeclimate which may be suitable for us.

It contains many engines(plugins), one of the plugins is golint .

I have two basic requirement :

  • a CLI : can run and do local check before making a PR and can customize with CI tool with shell, Codeclimate has a CLI
  • integrate with GitHub : yes, see details

so any idea about this tool? or is there more suitable tools can be shared?

Re: [discuss] Add metrics for Edgex to enhance the system monitor

Trevor.Conn@...
 

Dell Customer Communication

Looping in Systems Mgmt group.

 

I’m going to initiate a discussion about a standardized structured logging format on the Core WG call on Thursday.

 

Trevor

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Friday, June 8, 2018 9:07 PM
To: EdgeX-GoLang@...
Subject: [Edgex-golang] [discuss] Add metrics for Edgex to enhance the system monitor

 

Copy from Github issue #263 :

Metrics and Log are two basic parts of a stable system. For enhancing the EdgeX's stability we should consider to monitor it in the future. The monitor would need to be provided metrics. So we should introduce the metrics infrastructure for it.

There are a few scenario where we could add metrics:

  • device data import count / rate;
  • support notification count / schedule count (trigger count)
  • export data count / rate;
  • ....

Although, metrics and log sometimes can match the same goal(provide information to calculate/aggregate/analysis). But they are different, here is a good article described the difference between them.

I find a popular go metrics package : go-metrics, which can write metrics into InfluxDB. And using this (we can also implement a logger to write log into InfluxDB), we could integrate the metrics with InfluxDB's full technology stack (monitor, dashboard, storage, computing).

----------------------------------------------------------------------------------------------------

drasko's comment : 

@yanghua good point here.

 

In Mainfllux we use metrics (and logs of course) for every service, something like this: https://github.com/mainflux/mainflux/blob/master/http/api/metrics.go, and those are then exported to Prometheus. We are using go-kit metrics because we are using go-kit in Mainfux, but go-metricslooks like a nice package and a good candidate for EdgeX.

Hope for more comment!

 

Re: [discuss] Introduce Codeclimate to improve code quality

vinoyang
 

Hi Trevor,

It seams the CodeClimate is free for open source forever : https://codeclimate.com/oss/ , the link shows there are many popular project use it, like Bootstrap, Angular, Node.js, jQuery, Rails and so on.

The article you mentioned is good.

No matter what tools would been chose. I think there is one import thing : contributor could run local check in their local machine. So if the tool has a CLI that would be better.

Vino yang.

Re: [discuss] Add metrics for Edgex to enhance the system monitor

vinoyang
 

Trevor,

I am glad to hear this! A standardized structured logging format is good.

If you can discuss metrics and introduce it to EdgeX at the meeting. It would be better.

Vino yang.

Re: [discuss] Introduce Codeclimate to improve code quality

Jeremy Phelps
 

Another option is to use SonarCube, we have several other projects that use this; ONAP for example has one here sonar.onap.org.
I did notice that there is a plugin supported by SonarSource as well https://github.com/SonarSource/sonar-go.  In addition ot being open source, it
also offers a CLI and a Jenkins plugin.

On Mon, Jun 11, 2018 at 10:08 AM, vinoyang <yanghua1127@...> wrote:
Hi Trevor,

It seams the CodeClimate is free for open source forever : https://codeclimate.com/oss/ , the link shows there are many popular project use it, like Bootstrap, Angular, Node.js, jQuery, Rails and so on.

The article you mentioned is good.

No matter what tools would been chose. I think there is one import thing : contributor could run local check in their local machine. So if the tool has a CLI that would be better.

Vino yang.


Re: [Edgex-tsc-core] [Edgex-golang] [discuss] Introduce Codeclimate to improve code quality

Keith Steele <keith@...>
 

Hi Trevor

Just a quick note back to say we don't necessarily have to use free stuff.
I have prepared the EdgeX board for possible purchases for the development environment... so if we feel there is something we really need but it would mean buying licenses then we can ask the board for money to do that.
It's also quite often the case that companies will give special deals to open source projects.
Let me know if it transpires we need to buy tools.

Regards
Keith

On 10 June 2018 at 20:36, <Trevor.Conn@...> wrote:

Dell Customer Communication

We might want to look at alternatives to CodeClimate given that it isn’t free. The code quality metrics and integration with GitHub are nice, remind me of SonarQube. It’s probably worth having a discussion about the topic of integrating a code quality pipeline via DevOps channels during the next Core WG meeting on Thursday. We could potentially also utilize Jenkins to trigger checks.

 

I thought this article provided a good overview of the kind of metrics a quality pipeline should capture.

https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567

 

Setting this up correctly (and monitoring it going forward) will require ownership from DevOps as well as each working group contributing to the codebase, although one working group might be able to chart a course for the others to follow.

 

Trevor

 

From: EdgeX-GoLang@lists.edgexfoundry.org [mailto:EdgeX-GoLang@lists.edgexfoundry.org] On Behalf Of vinoyang
Sent: Friday, June 8, 2018 9:03 PM
To: EdgeX-GoLang@lists.edgexfoundry.org
Subject: [Edgex-golang] [discuss] Introduce Codeclimate to improve code quality

 

Copy from Github issue #286

based on the discussion in issue #280 , I find a tool name Codeclimate which may be suitable for us.

It contains many engines(plugins), one of the plugins is golint .

I have two basic requirement :

  • a CLI : can run and do local check before making a PR and can customize with CI tool with shell, Codeclimate has a CLI
  • integrate with GitHub : yes, see details

so any idea about this tool? or is there more suitable tools can be shared?


Re: [Edgex-tsc-core] [Edgex-golang] [discuss] Introduce Codeclimate to improve code quality

Ian Johnson <ian.johnson@...>
 

At Canonical, we use CodeCov (which is free for open source projects + integrates with GitHub pull requests) and it works well for our large open source projects:

See for example snapd + snapcraft :

All you need to upload to codecov is a single curl command with the test results and there's various coverage parsing tools available. For go it's all built into the `go test` command when you use the -coverprofile option. For example this is all it takes for uploading to codecov from a travis build job : https://github.com/anonymouse64/asm2go/blob/master/.travis.yml#L20-L24
(I'm sure it would be simple to adapt this to a jenkins pipeline, etc.)

Ian


On Mon, Jun 11, 2018 at 8:09 AM Keith Steele <keith@...> wrote:
Hi Trevor

Just a quick note back to say we don't necessarily have to use free stuff.
I have prepared the EdgeX board for possible purchases for the development environment... so if we feel there is something we really need but it would mean buying licenses then we can ask the board for money to do that.
It's also quite often the case that companies will give special deals to open source projects.
Let me know if it transpires we need to buy tools.

Regards
Keith

On 10 June 2018 at 20:36, <Trevor.Conn@...> wrote:

Dell Customer Communication

We might want to look at alternatives to CodeClimate given that it isn’t free. The code quality metrics and integration with GitHub are nice, remind me of SonarQube. It’s probably worth having a discussion about the topic of integrating a code quality pipeline via DevOps channels during the next Core WG meeting on Thursday. We could potentially also utilize Jenkins to trigger checks.

 

I thought this article provided a good overview of the kind of metrics a quality pipeline should capture.

https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567

 

Setting this up correctly (and monitoring it going forward) will require ownership from DevOps as well as each working group contributing to the codebase, although one working group might be able to chart a course for the others to follow.

 

Trevor

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Friday, June 8, 2018 9:03 PM
To: EdgeX-GoLang@...
Subject: [Edgex-golang] [discuss] Introduce Codeclimate to improve code quality

 

Copy from Github issue #286

based on the discussion in issue #280 , I find a tool name Codeclimate which may be suitable for us.

It contains many engines(plugins), one of the plugins is golint .

I have two basic requirement :

  • a CLI : can run and do local check before making a PR and can customize with CI tool with shell, Codeclimate has a CLI
  • integrate with GitHub : yes, see details

so any idea about this tool? or is there more suitable tools can be shared?


Re: [discuss] Introduce Codeclimate to improve code quality

vinoyang
 

Hi Jeremy,

Obviously, SonarCube is a good and popular tool. But t I worry about one point : Golang is not supported natively by official. See here : https://www.sonarqube.org/features/multi-languages/

And the plugin you mentioned : https://github.com/SonarSource/sonar-go is not popular and also has 20+ issues about some rules, it may course some problem which can not be fixed as soon as possible.

However, Codeclimate support Golang natively : https://docs.codeclimate.com/docs/supported-languages-for-maintainability .

I just worry about the maintainability of two tools.

Gofmt git commit hook

Trevor.Conn@...
 

https://tip.golang.org/misc/git/pre-commit

 

Per our discussion in the Core WG call yesterday, I’m working today on looking for code quality tools that we can easily integrate and that will do as many of the required checks prior to merge as possible. I found the above git pre-commit script which can be used to run gofmt prior to commit on the client side. Sample output for poorly formatted code when I try to commit looks like this:

 

Go files must be formatted with gofmt. Please run:

  gofmt -w /Users/tconn/go/src/github.com/edgexfoundry/edgex-go/cmd/core-data/main.go

 

I can't commit until I fix the problems. The only downside, as I understand it, is that it relies on the developer to create the scripts and give it execute permissions. Plus I don’t understand how this can be source controlled. It looks like there are server-side git hooks as well.

 

https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

 

Would this be more appropriate as a server-side hook that runs during a push? If so, what’s the effort in setting that up in all Go repos? How about when we create new Go repos?

 

The script is linked from a larger blog post:

https://blog.golang.org/go-fmt-your-code


Using this strategy we could also wire in golint and other tools. Thoughts?


Trevor Conn
Senior Principal Software Engineer
Dell Technologies | IoT DellTech
Trevor_Conn@...
Round Rock, TX  USA

Re: [Edgex-tsc-devops] Gofmt git commit hook

Jeremy Phelps
 

Hi Trevor,
I am in favor of something like this generally.  I think because we use GitHub we will have to have client side hooks ( I will look into it further though ).  There are tools available to help share commit hook scripts such as https://github.com/git-hooks/git-hooks/wiki/Get-Started.  Lets put this on the table for discussion, we can ramp up the DevOps WG meetings again if we need extra time.
Jeremy

On Fri, Jun 15, 2018 at 12:12 PM, <Trevor.Conn@...> wrote:

https://tip.golang.org/misc/git/pre-commit

 

Per our discussion in the Core WG call yesterday, I’m working today on looking for code quality tools that we can easily integrate and that will do as many of the required checks prior to merge as possible. I found the above git pre-commit script which can be used to run gofmt prior to commit on the client side. Sample output for poorly formatted code when I try to commit looks like this:

 

Go files must be formatted with gofmt. Please run:

  gofmt -w /Users/tconn/go/src/github.com/edgexfoundry/edgex-go/cmd/core-data/main.go

 

I can't commit until I fix the problems. The only downside, as I understand it, is that it relies on the developer to create the scripts and give it execute permissions. Plus I don’t understand how this can be source controlled. It looks like there are server-side git hooks as well.

 

https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

 

Would this be more appropriate as a server-side hook that runs during a push? If so, what’s the effort in setting that up in all Go repos? How about when we create new Go repos?

 

The script is linked from a larger blog post:

https://blog.golang.org/go-fmt-your-code


Using this strategy we could also wire in golint and other tools. Thoughts?


Trevor Conn
Senior Principal Software Engineer
Dell Technologies | IoT DellTech
Trevor_Conn@...
Round Rock, TX  USA


Re: [Edgex-tsc-devops] Gofmt git commit hook

Trevor.Conn@...
 

Dell - Internal Use - Confidential

Thanks, Jeremy. It looks like the git-hooks utility you linked to below would allow us to create a git-hooks directory in our repo and then put the appropriate scripts in there. This would allow us to source control the scripts.

 

It also looks like their “community hooks” would allow us to create a repo specifically for githook scripts. Then all we’d have to include in our projects is a githooks.json file like this one. Execution requires internet connectivity though, obviously.

 

We’d need to require all developers to install the utility if we go down this route.

 

Trevor

 

From: EdgeX-TSC-DevOps@... [mailto:EdgeX-TSC-DevOps@...] On Behalf Of Jeremy Phelps
Sent: Friday, June 15, 2018 1:10 PM
To: Conn, Trevor
Cc: EdgeX-TSC-DevOps@...; EdgeX-GoLang@...
Subject: Re: [Edgex-tsc-devops] Gofmt git commit hook

 

Hi Trevor,

I am in favor of something like this generally.  I think because we use GitHub we will have to have client side hooks ( I will look into it further though ).  There are tools available to help share commit hook scripts such as https://github.com/git-hooks/git-hooks/wiki/Get-Started.  Lets put this on the table for discussion, we can ramp up the DevOps WG meetings again if we need extra time.

Jeremy

 

On Fri, Jun 15, 2018 at 12:12 PM, <Trevor.Conn@...> wrote:

https://tip.golang.org/misc/git/pre-commit

 

Per our discussion in the Core WG call yesterday, I’m working today on looking for code quality tools that we can easily integrate and that will do as many of the required checks prior to merge as possible. I found the above git pre-commit script which can be used to run gofmt prior to commit on the client side. Sample output for poorly formatted code when I try to commit looks like this:

 

Go files must be formatted with gofmt. Please run:

  gofmt -w /Users/tconn/go/src/github.com/edgexfoundry/edgex-go/cmd/core-data/main.go

 

I can't commit until I fix the problems. The only downside, as I understand it, is that it relies on the developer to create the scripts and give it execute permissions. Plus I don’t understand how this can be source controlled. It looks like there are server-side git hooks as well.

 

https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

 

Would this be more appropriate as a server-side hook that runs during a push? If so, what’s the effort in setting that up in all Go repos? How about when we create new Go repos?

 

The script is linked from a larger blog post:

https://blog.golang.org/go-fmt-your-code

 

Using this strategy we could also wire in golint and other tools. Thoughts?

 

Trevor Conn
Senior Principal Software Engineer
Dell Technologies | IoT DellTech
Trevor_Conn@...
Round Rock, TX  USA

 

Re: [Edgex-tsc-devops] Gofmt git commit hook

Jeremy Phelps
 

Yeah we will explore this further for sure.  I did confirm that we will not be able to do git hooks on GitHubs servers; it's an obvious security concern for them.  With some work we can make something with webhooks though.
Jeremy

On Fri, Jun 15, 2018 at 3:34 PM, <Trevor.Conn@...> wrote:

Dell - Internal Use - Confidential

Thanks, Jeremy. It looks like the git-hooks utility you linked to below would allow us to create a git-hooks directory in our repo and then put the appropriate scripts in there. This would allow us to source control the scripts.

 

It also looks like their “community hooks” would allow us to create a repo specifically for githook scripts. Then all we’d have to include in our projects is a githooks.json file like this one. Execution requires internet connectivity though, obviously.

 

We’d need to require all developers to install the utility if we go down this route.

 

Trevor

 

From: EdgeX-TSC-DevOps@lists.edgexfoundry.org [mailto:EdgeX-TSC-DevOps@lists.edgexfoundry.org] On Behalf Of Jeremy Phelps
Sent: Friday, June 15, 2018 1:10 PM
To: Conn, Trevor
Cc: EdgeX-TSC-DevOps@lists.edgexfoundry.org; EdgeX-GoLang@lists.edgexfoundry.org
Subject: Re: [Edgex-tsc-devops] Gofmt git commit hook

 

Hi Trevor,

I am in favor of something like this generally.  I think because we use GitHub we will have to have client side hooks ( I will look into it further though ).  There are tools available to help share commit hook scripts such as https://github.com/git-hooks/git-hooks/wiki/Get-Started.  Lets put this on the table for discussion, we can ramp up the DevOps WG meetings again if we need extra time.

Jeremy

 

On Fri, Jun 15, 2018 at 12:12 PM, <Trevor.Conn@...> wrote:

https://tip.golang.org/misc/git/pre-commit

 

Per our discussion in the Core WG call yesterday, I’m working today on looking for code quality tools that we can easily integrate and that will do as many of the required checks prior to merge as possible. I found the above git pre-commit script which can be used to run gofmt prior to commit on the client side. Sample output for poorly formatted code when I try to commit looks like this:

 

Go files must be formatted with gofmt. Please run:

  gofmt -w /Users/tconn/go/src/github.com/edgexfoundry/edgex-go/cmd/core-data/main.go

 

I can't commit until I fix the problems. The only downside, as I understand it, is that it relies on the developer to create the scripts and give it execute permissions. Plus I don’t understand how this can be source controlled. It looks like there are server-side git hooks as well.

 

https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

 

Would this be more appropriate as a server-side hook that runs during a push? If so, what’s the effort in setting that up in all Go repos? How about when we create new Go repos?

 

The script is linked from a larger blog post:

https://blog.golang.org/go-fmt-your-code

 

Using this strategy we could also wire in golint and other tools. Thoughts?

 

Trevor Conn
Senior Principal Software Engineer
Dell Technologies | IoT DellTech
Trevor_Conn@...
Round Rock, TX  USA

 


Re: [Edgex-tsc-devops] Gofmt git commit hook

Fede Claramonte
 

I think it is easier just to add that test to the test rule in the makefile. It will not be run automatically in the developers machine, but anyone should run make test before doing a PR. And it does not need any change in the server or CI.

Fede


On 06/15/2018 10:51 PM, Jeremy Phelps wrote:


Yeah we will explore this further for sure.  I did confirm that we will not be able to do git hooks on GitHubs servers; it's an obvious security concern for them.  With some work we can make something with webhooks though.
Jeremy

On Fri, Jun 15, 2018 at 3:34 PM, <Trevor.Conn@...> wrote:

Dell - Internal Use - Confidential

Thanks, Jeremy. It looks like the git-hooks utility you linked to below would allow us to create a git-hooks directory in our repo and then put the appropriate scripts in there. This would allow us to source control the scripts.

 

It also looks like their “community hooks” would allow us to create a repo specifically for githook scripts. Then all we’d have to include in our projects is a githooks.json file like this one. Execution requires internet connectivity though, obviously.

 

We’d need to require all developers to install the utility if we go down this route.

 

Trevor

 

From: EdgeX-TSC-DevOps@lists.edgexfoundry.org [mailto:EdgeX-TSC-DevOps@lists.edgexfoundry.org] On Behalf Of Jeremy Phelps
Sent: Friday, June 15, 2018 1:10 PM
To: Conn, Trevor
Cc: EdgeX-TSC-DevOps@lists.edgexfoundry.org; EdgeX-GoLang@lists.edgexfoundry.org
Subject: Re: [Edgex-tsc-devops] Gofmt git commit hook

 

Hi Trevor,

I am in favor of something like this generally.  I think because we use GitHub we will have to have client side hooks ( I will look into it further though ).  There are tools available to help share commit hook scripts such as https://github.com/git-hooks/git-hooks/wiki/Get-Started.  Lets put this on the table for discussion, we can ramp up the DevOps WG meetings again if we need extra time.

Jeremy

 

On Fri, Jun 15, 2018 at 12:12 PM, <Trevor.Conn@...> wrote:

https://tip.golang.org/misc/git/pre-commit

 

Per our discussion in the Core WG call yesterday, I’m working today on looking for code quality tools that we can easily integrate and that will do as many of the required checks prior to merge as possible. I found the above git pre-commit script which can be used to run gofmt prior to commit on the client side. Sample output for poorly formatted code when I try to commit looks like this:

 

Go files must be formatted with gofmt. Please run:

  gofmt -w /Users/tconn/go/src/github.com/edgexfoundry/edgex-go/cmd/core-data/main.go

 

I can't commit until I fix the problems. The only downside, as I understand it, is that it relies on the developer to create the scripts and give it execute permissions. Plus I don’t understand how this can be source controlled. It looks like there are server-side git hooks as well.

 

https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

 

Would this be more appropriate as a server-side hook that runs during a push? If so, what’s the effort in setting that up in all Go repos? How about when we create new Go repos?

 

The script is linked from a larger blog post:

https://blog.golang.org/go-fmt-your-code

 

Using this strategy we could also wire in golint and other tools. Thoughts?

 

Trevor Conn
Senior Principal Software Engineer
Dell Technologies | IoT DellTech
Trevor_Conn@...
Round Rock, TX  USA

 



Delhi -- Cleaner Separation of Internal/External Packages

Trevor.Conn@...
 

Dell Customer Communication

Hi all – I wanted to send out an email as a heads-up to one of the decisions that came from our last F2F in Palo Alto. This was discussed during the Device Services working group and I wanted to give everyone visibility and an opportunity to comment before changes get made.

 

The Problem

Currently the EdgeX GoLang implementation has no separation between its internal domain model and the models used to pass data out of the system. They are one and the same. Likewise, there is no visibility as to which service client is used by an external application versus solely used internal to the EdgeX platform. This means we are at risk of making changes to either our data models or our service client code that break any applications which have imported those items. We have no way to insulate our consumers from changes we make to the application based on an internal need, nor do we have the ability to hide internal/private details of our data model since our data model is also part of our client contract.

 

The Recommendation

In order to facilitate the separation of internal/external domain models and logic, we realized that we would need to restructure the application packages. At a high level, the changes we decided to make are as follows:

 

/edgex-go/core

·         will be re-purposed to contain those models and libraries which are usable by external applications

·         will include the following:

o   clients package – service clients usable by external applications

o   models package – data models usable by external applications

§  NOTE: These will be identical to the current property bags found today in /edgex-go/core/domain/models

/edgex-go/internal

·         This package will be extended to include all code that we do not want external applications explicitly relying on as recommended by the golang-standards project layout

·         That means the following packages will be moved into here in some fashion

o   /edgex-go/core/command

o   /edgex-go/core/core-data

o   /edgex-go/core/core-metadata

o   /edgex-go/export

o   /edgex-go/support

§  Consul-client, domain, l ogging

§  Service client packages will most likely go to core above

·         Domain models included herein may start off as identical to those found today in /edgex-go/core/domain/models but have the flexibility to change and move around over time according to the internal needs of the platform.

                         

We are looking at making this change early in the Delhi development cycle, so please provide feedback in a timely fashion if you have it. Also please note that any external applications which are directly linking to the current package structure within the master branch of EdgeX will break when we do this. That is unavoidable. We are doing this now so that the pill isn’t too bitter to swallow later.

 

I look forward to your comments.

 

Trevor Conn

Senior Principal Software Engineer

Dell Technologies | IoT DellTech

Trevor.Conn@...

Round Rock, TX USA

 

Re: Delhi -- Cleaner Separation of Internal/External Packages

vinoyang
 

Hi Trevor:

I totally agree the decisions. But two questions :

  1. What's the layout of internal modules' models package : one model package shared by all modules OR many model packages (and split some models which should not been shared and package into the specific modules)
  2. Can we refactor this in a separated branch? Not in master branch, so that we can develop features and fix issues normally.

Thanks
Vino

Re: Delhi -- Cleaner Separation of Internal/External Packages

Trevor.Conn@...
 

Dell Customer Communication

Vino – To answer your questions:

 

1.)    Initially, just to get the restructuring done, picture the contents of core/command, data and metadata being moved there as close to as-is as possible. There will also be a new models package for internal use. This will position us for a wider refactoring hopefully using DDD principles as a guide. An example of what this might look like (albeit without the package re-org being discussed in this thread) can be found in the holding area. This refactoring will then obviously change the structure of the packages in /internal, but who cares because applications won’t directly reference them.

2.)    I don’t think this will be necessary. I think the restructuring can be done fairly quickly and as a single check-in. We want it in the master branch quickly to stop the bad practice of importing EdgeX’s internals by outside applications and to position us for future efforts.

 

Trevor

 

From: EdgeX-GoLang@... [mailto:EdgeX-GoLang@...] On Behalf Of vinoyang
Sent: Friday, June 29, 2018 10:28 PM
To: EdgeX-GoLang@...
Subject: Re: [Edgex-golang] Delhi -- Cleaner Separation of Internal/External Packages

 

Hi Trevor:

I totally agree the decisions. But two questions :

  1. What's the layout of internal modules' models package : one model package shared by all modules OR many model packages (and split some models which should not been shared and package into the specific modules)
  2. Can we refactor this in a separated branch? Not in master branch, so that we can develop features and fix issues normally.


Thanks
Vino