Date   
Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 07/11/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 11 July 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

IT-16630 Request the instruction for replicating TIG Server of EdgeX Foundry

Cloud Tsai
 

Hi James and DevOps team,

Thanks for the kindly support.  Please see the response from Linux Foundation about the AWS server configuration.
Let's discuss when you have idea about the next steps.


---------- Forwarded message ---------
From: Project Services <notifications@...>
Date: Thu, 4 Jul 2019 at 01:22
Subject: IT-16630 Request the instruction for replicating TIG Server of EdgeX Foundry
To: <cherry@...>


Jordan Conway commented:

Hi Cherry,

I've attached the elastic beanstalk config folder as well as the terraform config we use to setup this instance.

It likely won't work "out of the box" because the terraform config relies on other multi-project/account modules across our infrastructure that are not easily shared but it will give you an idea of how the configuration is done.
edgex-tigstack-eb.tar.gz tigstack-eb.tf


System Administrator changed the status to Waiting for customer.


Jordan Conway changed the status to Resolved.


Jordan Conway resolved this as Done.


How was our service for this request?

Very poor

Poor

Neither good nor poor

Good

Very good



View request · Turn off this request's notifications

This is shared with Cherry Wang.



--
Best Regards,
Cloud Tsai

GitHub Audit: Empty repos

Gregg, James R
 

TSC Members,

An audit of the EdgeX Foundry GitHub repos was completed last week and briefly discussed in the DevOps WG.    Below are the result of the audit which identifies empty repositories.

 

We would to disposition all of the empty repos which do not have an owner or a plan for further development.  There should not be any duplicate repos where code could slip into the main organization without proper peer review by a WG lead.

 

Please review this below list of repos and report back to me any actions which are recommended (deletion, archive, plans for code drop etc..)

Label

Repository

Files

Last Commit

Notes

(EMPTY)

edgexfoundry/device-opcua-c

11/29/2018 20:58

Updated 8 months ago by JPWKU

(< 5 files)

edgexfoundry/jenkins_pipeline_presentation

Jenkinsfile, README.md

2/27/2018 21:58

 

 

Label

Repository

Files

Last Commit

Notes

Plan

(< 5 files)

edgexfoundry-holding/device-opcua

README.md

10/3/2018 1:28

Updated 9 months ago by jwagantall

[EMPTY]

edgexfoundry-holding/test

12/3/2018 4:44

(< 5 files)

edgexfoundry-holding/regression-test

README.md

7/10/2019 19:36

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/protocol-ezmq-python

README.md

10/30/2018 0:34

Updated 9 months ago by jwagantall

(< 5 files)

edgexfoundry-holding/edgex-taf

README.md

6/20/2019 14:28

Under development with plans for Fuji release

[EMPTY]

edgexfoundry-holding/device-bacnet-c

6/13/2019 7:06

(EMPTY)

edgexfoundry-holding/edgex-appl-services

12/4/2018 20:19

Updated 7 months ago by JPWKU

(< 5 files)

edgexfoundry-holding/ci-snaps-common

README.md

4/18/2019 17:41

Updated 3 months ago by MilJ19LF

[EMPTY]

edgexfoundry-holding/device-opcua-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/device-bluetooth-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/management-executor

5/28/2018 6:36

(< 5 files)

edgexfoundry-holding/edgex-mongo-init

README.md

7/10/2019 21:44

Open PR with current dialog

(< 5 files)

edgexfoundry-holding/edgex-taf-common

README.md

7/10/2019 19:26

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/device-snmp-go

README.md

5/7/2019 18:02

Updated 2 months ago by MilJ19LF

Already moved out and included in Edinburgh

(< 5 files)

edgexfoundry-holding/app-service-configurable

README.md

6/18/2019 21:14

Updated 24 days ago by MilJ19LF

Under development with plans for Fuji release

 

Thank you

 

James Gregg

IOTG RBHE DevOps | EdgeX Foundry DevOps

Email: james.r.gregg@...

Tel: (480) 552-7965

 

 

Re: GitHub Audit: Empty repos

Ian Johnson
 

Hi James,

You can delete the ci-snaps-common repo, it ended up not being needed since we just put the files for snap jobs in ci-management.

Thanks,
Ian

On Mon, Jul 15, 2019, 21:23 Gregg, James R <james.r.gregg@...> wrote:

TSC Members,

An audit of the EdgeX Foundry GitHub repos was completed last week and briefly discussed in the DevOps WG.    Below are the result of the audit which identifies empty repositories.

 

We would to disposition all of the empty repos which do not have an owner or a plan for further development.  There should not be any duplicate repos where code could slip into the main organization without proper peer review by a WG lead.

 

Please review this below list of repos and report back to me any actions which are recommended (deletion, archive, plans for code drop etc..)

Label

Repository

Files

Last Commit

Notes

(EMPTY)

edgexfoundry/device-opcua-c

11/29/2018 20:58

Updated 8 months ago by JPWKU

(< 5 files)

edgexfoundry/jenkins_pipeline_presentation

Jenkinsfile, README.md

2/27/2018 21:58

 

 

Label

Repository

Files

Last Commit

Notes

Plan

(< 5 files)

edgexfoundry-holding/device-opcua

README.md

10/3/2018 1:28

Updated 9 months ago by jwagantall

[EMPTY]

edgexfoundry-holding/test

12/3/2018 4:44

(< 5 files)

edgexfoundry-holding/regression-test

README.md

7/10/2019 19:36

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/protocol-ezmq-python

README.md

10/30/2018 0:34

Updated 9 months ago by jwagantall

(< 5 files)

edgexfoundry-holding/edgex-taf

README.md

6/20/2019 14:28

Under development with plans for Fuji release

[EMPTY]

edgexfoundry-holding/device-bacnet-c

6/13/2019 7:06

(EMPTY)

edgexfoundry-holding/edgex-appl-services

12/4/2018 20:19

Updated 7 months ago by JPWKU

(< 5 files)

edgexfoundry-holding/ci-snaps-common

README.md

4/18/2019 17:41

Updated 3 months ago by MilJ19LF

[EMPTY]

edgexfoundry-holding/device-opcua-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/device-bluetooth-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/management-executor

5/28/2018 6:36

(< 5 files)

edgexfoundry-holding/edgex-mongo-init

README.md

7/10/2019 21:44

Open PR with current dialog

(< 5 files)

edgexfoundry-holding/edgex-taf-common

README.md

7/10/2019 19:26

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/device-snmp-go

README.md

5/7/2019 18:02

Updated 2 months ago by MilJ19LF

Already moved out and included in Edinburgh

(< 5 files)

edgexfoundry-holding/app-service-configurable

README.md

6/18/2019 21:14

Updated 24 days ago by MilJ19LF

Under development with plans for Fuji release

 

Thank you

 

James Gregg

IOTG RBHE DevOps | EdgeX Foundry DevOps

Email: james.r.gregg@...

Tel: (480) 552-7965

 

 

Re: [Edgex-tsc-device-services] GitHub Audit: Empty repos

James.White2@...
 

James,

The following are still under active development/work and so I would request that the following be kept

 

edgexfoundry-holding/edgex-mongo-init

edgexfoundry-holding/edgex-taf-common

edgexfoundry-holding/app-service-configurable

Otherwise - +1 to drop or archive the others (drop empty repos and archive those with code).

Jim

From: EdgeX-TSC-Device-Services@... <EdgeX-TSC-Device-Services@...> On Behalf Of Gregg, James R
Sent: Monday, July 15, 2019 9:24 PM
To: EdgeX-TSC@...; edgex-tsc-device-services@...
Cc: EdgeX-TSC-DevOps@...
Subject: [Edgex-tsc-device-services] GitHub Audit: Empty repos

 

[EXTERNAL EMAIL]

TSC Members,

An audit of the EdgeX Foundry GitHub repos was completed last week and briefly discussed in the DevOps WG.    Below are the result of the audit which identifies empty repositories.

 

We would to disposition all of the empty repos which do not have an owner or a plan for further development.  There should not be any duplicate repos where code could slip into the main organization without proper peer review by a WG lead.

 

Please review this below list of repos and report back to me any actions which are recommended (deletion, archive, plans for code drop etc..)

Label

Repository

Files

Last Commit

Notes

(EMPTY)

edgexfoundry/device-opcua-c

11/29/2018 20:58

Updated 8 months ago by JPWKU

(< 5 files)

edgexfoundry/jenkins_pipeline_presentation

Jenkinsfile, README.md

2/27/2018 21:58

 

 

Label

Repository

Files

Last Commit

Notes

Plan

(< 5 files)

edgexfoundry-holding/device-opcua

README.md

10/3/2018 1:28

Updated 9 months ago by jwagantall

[EMPTY]

edgexfoundry-holding/test

12/3/2018 4:44

(< 5 files)

edgexfoundry-holding/regression-test

README.md

7/10/2019 19:36

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/protocol-ezmq-python

README.md

10/30/2018 0:34

Updated 9 months ago by jwagantall

(< 5 files)

edgexfoundry-holding/edgex-taf

README.md

6/20/2019 14:28

Under development with plans for Fuji release

[EMPTY]

edgexfoundry-holding/device-bacnet-c

6/13/2019 7:06

(EMPTY)

edgexfoundry-holding/edgex-appl-services

12/4/2018 20:19

Updated 7 months ago by JPWKU

(< 5 files)

edgexfoundry-holding/ci-snaps-common

README.md

4/18/2019 17:41

Updated 3 months ago by MilJ19LF

[EMPTY]

edgexfoundry-holding/device-opcua-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/device-bluetooth-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/management-executor

5/28/2018 6:36

(< 5 files)

edgexfoundry-holding/edgex-mongo-init

README.md

7/10/2019 21:44

Open PR with current dialog

(< 5 files)

edgexfoundry-holding/edgex-taf-common

README.md

7/10/2019 19:26

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/device-snmp-go

README.md

5/7/2019 18:02

Updated 2 months ago by MilJ19LF

Already moved out and included in Edinburgh

(< 5 files)

edgexfoundry-holding/app-service-configurable

README.md

6/18/2019 21:14

Updated 24 days ago by MilJ19LF

Under development with plans for Fuji release

 

Thank you

 

James Gregg

IOTG RBHE DevOps | EdgeX Foundry DevOps

Email: james.r.gregg@...

Tel: (480) 552-7965

 

 

Dell Corporation Limited is registered in England and Wales. Company Registration Number: 2081369
Registered address: Dell House, The Boulevard, Cain Road, Bracknell,  Berkshire, RG12 1LF, UK.
Company details for other Dell UK entities can be found on  www.dell.co.uk.

Re: GitHub Audit: Empty repos

Moonki Hong
 

Hi, Gregg,

 

We have a code drop plan for "edgexfoundry-holding/protocol-ezmq-python" this week. When we tried to do that, it is already deleted (or archived) from the given GitHub Org. Would you like to check it out and let us know, so that we will transfer the code into that repository.

 

Thank you in advance,

Peter Moonki Hong, Ph.D

Samsung Research Korea, Samsung Electronics Co., Ltd.

 

 

--------- Original Message ---------

Sender : Gregg, James R <james.r.gregg@...>

Date : 2019-07-16 11:23 (GMT+9)

Title : [Edgex-tsc-devops] GitHub Audit: Empty repos

 

TSC Members,

An audit of the EdgeX Foundry GitHub repos was completed last week and briefly discussed in the DevOps WG.    Below are the result of the audit which identifies empty repositories.

 

We would to disposition all of the empty repos which do not have an owner or a plan for further development.  There should not be any duplicate repos where code could slip into the main organization without proper peer review by a WG lead.

 

Please review this below list of repos and report back to me any actions which are recommended (deletion, archive, plans for code drop etc..)

Label

Repository

Files

Last Commit

Notes

(EMPTY)

edgexfoundry/device-opcua-c

11/29/2018 20:58

Updated 8 months ago by JPWKU

(< 5 files)

edgexfoundry/jenkins_pipeline_presentation

Jenkinsfile, README.md

2/27/2018 21:58

 

 

Label

Repository

Files

Last Commit

Notes

Plan

(< 5 files)

edgexfoundry-holding/device-opcua

README.md

10/3/2018 1:28

Updated 9 months ago by jwagantall

[EMPTY]

edgexfoundry-holding/test

12/3/2018 4:44

(< 5 files)

edgexfoundry-holding/regression-test

README.md

7/10/2019 19:36

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/protocol-ezmq-python

README.md

10/30/2018 0:34

Updated 9 months ago by jwagantall

(< 5 files)

edgexfoundry-holding/edgex-taf

README.md

6/20/2019 14:28

Under development with plans for Fuji release

[EMPTY]

edgexfoundry-holding/device-bacnet-c

6/13/2019 7:06

(EMPTY)

edgexfoundry-holding/edgex-appl-services

12/4/2018 20:19

Updated 7 months ago by JPWKU

(< 5 files)

edgexfoundry-holding/ci-snaps-common

README.md

4/18/2019 17:41

Updated 3 months ago by MilJ19LF

[EMPTY]

edgexfoundry-holding/device-opcua-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/device-bluetooth-c

3/29/2019 13:52

[EMPTY]

edgexfoundry-holding/management-executor

5/28/2018 6:36

(< 5 files)

edgexfoundry-holding/edgex-mongo-init

README.md

7/10/2019 21:44

Open PR with current dialog

(< 5 files)

edgexfoundry-holding/edgex-taf-common

README.md

7/10/2019 19:26

Under development with plans for Fuji release

(< 5 files)

edgexfoundry-holding/device-snmp-go

README.md

5/7/2019 18:02

Updated 2 months ago by MilJ19LF

Already moved out and included in Edinburgh

(< 5 files)

edgexfoundry-holding/app-service-configurable

README.md

6/18/2019 21:14

Updated 24 days ago by MilJ19LF

Under development with plans for Fuji release

 

Thank you

 

James Gregg

IOTG RBHE DevOps | EdgeX Foundry DevOps

Email: james.r.gregg@...

Tel: (480) 552-7965

 

 

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 07/18/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 18 July 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 07/25/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 25 July 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

2019-07-25 WG follow-up: injecting versioning information at build time

Jacob Blain Christen (Intel)
 

As discussed in this morning’s call, there are extant examples of injecting versioning information into your binaries at build time, most of which leverage `git describe`. I mentioned Consul and jq and have since poked around for some pointers to share.

 

Consul is built with gox which is a go toolchain wrapper that Mitchell Hashimoto wrote to deal with various shortcomings of early 1.x go. Don’t let that get in the way of understanding that gox does to pass through many command-line switches to `go build` and specifically to our interest -ldflags. All of this is to say, focus on the ldflags callout and realize that the Consul GNUMakefile is eventually invoking the go toolchain.

 

This link shows how Consul is invoking `git describe`:

https://github.com/hashicorp/consul/blob/421ecd32fc8fe9933cf50005eafc14705ffd45b1/GNUmakefile#L29

 

and this link shows how the Consul build script makes use of the exported GOLDFLAGS envvar:

https://github.com/hashicorp/consul/blob/63508c601538bfa11f1d0a644a4adb3ee01e489a/build-support/functions/20-build.sh#L306

 

 

Another good example is jq which is written in C. This link shows a utility script that invokes `git describe`:

https://github.com/stedolan/jq/blob/jq-1.6/scripts/version

 

and this link shows how that information is used to generate a version.h header:

https://github.com/stedolan/jq/blob/jq-1.6/Makefile.am#L96

 

 

These represent common patterns in my experience. We would probably want to lift from the Consul project how it invokes `git describe`, e.g. `git describe --tags --always --match "v*"` because this makes it more explicit that we only care about our semver information embedded in tags. The `--always` tells git describe to fallback to a 7 character sha prefix (instead of exiting with an error) if there are no matching tags which enough information for automation to assume and inject a default initial value for the version (if so desired).

 

I am happy to discuss this further and/or develop a plan forward via working session(s) but I hope this is at least a decent primer on what other projects our in the world are doing to inject version at build time.

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 08/01/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 1 August 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

Questions re: LF Edge Shared Infrastructure Proposal

Gregg, James R
 

Andy Grimberg made an initial proposal in the DevOps WG meeting last week , introducing an opportunity for using a shared infrastructure for all projects falling under the LF Edge umbrella.  Based on that initial discussion, I have the following list of questions which would be helpful to gain clarification about the project in terms of scope, schedule, budget.

 

Here are the additional “sandblast” of questions we did not specifically get a chance to answer in that meeting.

 

Scope

===========

Scope of the work

               - What work will the LF own within the proposed scope of work to transition to a common shared infrastructure?

               - What help is needed from the DevOps Open Source community?

Are there any other LF projects sharing a common build infrastructure?

 

What will the new GitHub repo structure look like?

What is the LF Release Engineer's role going to be in terms of ensuring that name collisions do not occur?

If it’s a shared infrastructure, do we have an opportunity to leverage common base build images for builds which are building Docker images?

 

It seems like one of the reasons for this proposal was to address a desire to reduce the size of duplicated infrastructure and thereby reduce technical debt when it comes to maintenance and ongoing support of the infrastructure.  Is there a plan to share a common DevOps WG under the LF Edge project so that communications are centralized?

 

Does the Linux Foundation have a plan to release roadmaps for when technical debt will be addressed? 

 

What other alternatives has the Linux Foundation considered to address the problem?

Are there any plans to look at leveraging Kubernetes for hosting build automation and leveraging more of a Container as a Service build automation model?

Has the Linux Foundation looked at Rancher Labs Rancher 2.0 for hosting a more modern CaaS platform?

What changes will be necessary to support all of the packer build images?

 

What optimizations of the shared common infrastructure will improve the overall build automation performance?

               - We have noticed service degradation when pulling images from upstream repos (docker hub or other repos)

               - We have seen what appears to be network degradation at times when pulling build dependencies

               - What other technical debt would be addressed within the proposed scope of work?

               - How will ARM builds be optimized in the new proposed shared / common infrastructure?

                              Note: ARM builds slower than Non-ARM

                              If there's more ARM builds happening (due to the shared infrastructure), how would the builds not all take longer to complete?

Schedule

===========

What's the timing for the proposal?

               - Need proposed start - end dates that do not conflict with release dates and/or current development

Are the resources committed to do the actual work within the timeframes?

How will the work be coordinated so as to not disrupt current development?

 

 

Budget

===========

What's the data that supports the claim that supports the proposal for a shared common infrastructure of all LF Edge projects?

Has the cost analysis been completed?

- please share the cost analysis of the proposed savings

               - number of builds for all projects (considered small if under < 500 total builds) - What happens if the next project to join LF Edge bumps the number of builds >1000?

               - shared resource model = shared support resources between LF Edge umbrella projects

Does the shared common build infrastructure also mean that the LF head count to support that build infrastructure is shared across all of the same LF Edge projects?

What's the actual savings?

 

 

Thank you Andy for coming into the DevOps WG meeting last week.  Hopefully we can get answers and clarify further so we can make some decisions and perhaps plan accordingly.

 

I appreciate your responses in advance and please let me know if any of my questions are not clear.

 

James Gregg

IOTG RBHE DevOps | EdgeX Foundry DevOps

Email: james.r.gregg@...

Tel: (480) 552-7965

 

Re: Questions re: LF Edge Shared Infrastructure Proposal

Trevor.Conn@...
 

Tracking additional questions I asked, see below.


1.) How will LF mitigate build worker starvation -- which we already see just in our own project, nevermind a shared environment?

2.) Will we maintain the same level of customizability for our build environment, for example the new auto-tagging Jenkins pipeline jobs?

3.) It would be nice to have the ability to cancel jobs. For example, a PR is created which kicks off verify jobs. A couple minutes later the dev resubmits a change to the same PR (like a missing rebase or something). We have to wait for those initial jobs to finish before the second jobs run.

    ** It sounded from the discussion like this was more related to the GitHub plugin, but it's still on my want list **

4.) What sort of dashboard will we have? Will we be able to see the overall shared stats in order to identify delays or quantify usage of the infra by project?


Trevor Conn
Technical Staff Engineer
Core Working Group Chair of EdgeX Foundry
Dell Technologies | IoT DellTech
Trevor_Conn@...
Round Rock, TX  USA


From: EdgeX-TSC-DevOps@... <EdgeX-TSC-DevOps@...> on behalf of Gregg, James R <james.r.gregg@...>
Sent: Monday, August 5, 2019 6:57 PM
To: edgex-tsc-devops@...
Subject: [Edgex-tsc-devops] Questions re: LF Edge Shared Infrastructure Proposal
 

[EXTERNAL EMAIL]

Andy Grimberg made an initial proposal in the DevOps WG meeting last week , introducing an opportunity for using a shared infrastructure for all projects falling under the LF Edge umbrella.  Based on that initial discussion, I have the following list of questions which would be helpful to gain clarification about the project in terms of scope, schedule, budget.

 

Here are the additional “sandblast” of questions we did not specifically get a chance to answer in that meeting.

 

Scope

===========

Scope of the work

               - What work will the LF own within the proposed scope of work to transition to a common shared infrastructure?

               - What help is needed from the DevOps Open Source community?

Are there any other LF projects sharing a common build infrastructure?

 

What will the new GitHub repo structure look like?

What is the LF Release Engineer's role going to be in terms of ensuring that name collisions do not occur?

If it’s a shared infrastructure, do we have an opportunity to leverage common base build images for builds which are building Docker images?

 

It seems like one of the reasons for this proposal was to address a desire to reduce the size of duplicated infrastructure and thereby reduce technical debt when it comes to maintenance and ongoing support of the infrastructure.  Is there a plan to share a common DevOps WG under the LF Edge project so that communications are centralized?

 

Does the Linux Foundation have a plan to release roadmaps for when technical debt will be addressed? 

 

What other alternatives has the Linux Foundation considered to address the problem?

Are there any plans to look at leveraging Kubernetes for hosting build automation and leveraging more of a Container as a Service build automation model?

Has the Linux Foundation looked at Rancher Labs Rancher 2.0 for hosting a more modern CaaS platform?

What changes will be necessary to support all of the packer build images?

 

What optimizations of the shared common infrastructure will improve the overall build automation performance?

               - We have noticed service degradation when pulling images from upstream repos (docker hub or other repos)

               - We have seen what appears to be network degradation at times when pulling build dependencies

               - What other technical debt would be addressed within the proposed scope of work?

               - How will ARM builds be optimized in the new proposed shared / common infrastructure?

                              Note: ARM builds slower than Non-ARM

                              If there's more ARM builds happening (due to the shared infrastructure), how would the builds not all take longer to complete?

Schedule

===========

What's the timing for the proposal?

               - Need proposed start - end dates that do not conflict with release dates and/or current development

Are the resources committed to do the actual work within the timeframes?

How will the work be coordinated so as to not disrupt current development?

 

 

Budget

===========

What's the data that supports the claim that supports the proposal for a shared common infrastructure of all LF Edge projects?

Has the cost analysis been completed?

- please share the cost analysis of the proposed savings

               - number of builds for all projects (considered small if under < 500 total builds) - What happens if the next project to join LF Edge bumps the number of builds >1000?

               - shared resource model = shared support resources between LF Edge umbrella projects

Does the shared common build infrastructure also mean that the LF head count to support that build infrastructure is shared across all of the same LF Edge projects?

What's the actual savings?

 

 

Thank you Andy for coming into the DevOps WG meeting last week.  Hopefully we can get answers and clarify further so we can make some decisions and perhaps plan accordingly.

 

I appreciate your responses in advance and please let me know if any of my questions are not clear.

 

James Gregg

IOTG RBHE DevOps | EdgeX Foundry DevOps

Email: james.r.gregg@...

Tel: (480) 552-7965

 

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 08/08/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 8 August 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 08/15/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 15 August 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 08/22/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 22 August 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

Scheduled maintenance: jenkins.edgexfoundry.org 2019-08-27 00:00-01:00 UTC

Eric Ball
 

What:
Software update to Jenkins.

When:
2019-08-27 00:00-01:00 UTC

Impact:
jenkins.edgexfoundry.org may be unavailable during this period.

Why:

Jenkins needs an upgrade to be compatible with the latest versions of several plugins, and to address some minor security issues.

Re: [Edgex-tsc] Scheduled maintenance: jenkins.edgexfoundry.org 2019-08-27 00:00-01:00 UTC

Eric Ball
 

Sorry, the correct day is 00:00 2019-08-29 (just under 24 hours from now). Apologies for the confusion.

On Tue, Aug 27, 2019 at 4:48 PM Eric Ball via Lists.Edgexfoundry.Org <eball=linuxfoundation.org@...> wrote:
What:
Software update to Jenkins.

When:
2019-08-27 00:00-01:00 UTC

Impact:
jenkins.edgexfoundry.org may be unavailable during this period.

Why:

Jenkins needs an upgrade to be compatible with the latest versions of several plugins, and to address some minor security issues.

POSTPONED: Scheduled maintenance: jenkins.edgexfoundry.org 2019-08-27 00:00-01:00 UTC

Eric Ball
 

Due to issues with our cloud provider's storage cluster, we are going to postpone this maintenance by 24 hours. It will now be scheduled for 00:00-01:00 UTC 2019-08-30.


On Tue, Aug 27, 2019 at 5:05 PM Eric Ball <eball@...> wrote:
Sorry, the correct day is 00:00 2019-08-29 (just under 24 hours from now). Apologies for the confusion.

On Tue, Aug 27, 2019 at 4:48 PM Eric Ball via Lists.Edgexfoundry.Org <eball=linuxfoundation.org@...> wrote:
What:
Software update to Jenkins.

When:
2019-08-27 00:00-01:00 UTC

Impact:
jenkins.edgexfoundry.org may be unavailable during this period.

Why:

Jenkins needs an upgrade to be compatible with the latest versions of several plugins, and to address some minor security issues.

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 08/29/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 29 August 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb

Upcoming Event: EdgeX TSC DevOps WG Meeting (Weekly) - Thu, 09/05/2019 7:00am-8:00am, Please RSVP #cal-reminder

EdgeX-TSC-DevOps@lists.edgexfoundry.org Calendar <EdgeX-TSC-DevOps@...>
 

Reminder: EdgeX TSC DevOps WG Meeting (Weekly)

When: Thursday, 5 September 2019, 7:00am to 8:00am, (GMT-07:00) America/Los Angeles

Where:https://zoom.us/j/967227826

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-DevOps@...

Description: EdgeX TSC DevOps and QA/Test WG Meeting. Meeting content posted to DevOps Wiki.
Meeting Lead: James Gregg, EdgeX DevOps WG Chair, james.r.gregg@...
-----
Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/967227826

Or iPhone one-tap :
    US: +16465588656,,967227826# or +16699006833,,967227826# 
Or Telephone:
    Dial(for higher quality, dial a number based on your current location): 
        US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)
    Meeting ID: 967 227 826
    International numbers available: https://zoom.us/u/bmaYSxofb