Date   

EdgeX QA/Test WG: Validation/Testing - Tue, 06/16/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 16 June 2020
4:00pm to 5:00pm
(GMT+00:00) UTC

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 06/16/2020 4:00pm-5:00pm, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 16 June 2020, 4:00pm to 5:00pm, (GMT+00:00) UTC

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 06/09/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 9 June 2020
4:00pm to 5:00pm
(GMT+00:00) UTC

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 06/09/2020 4:00pm-5:00pm, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 9 June 2020, 4:00pm to 5:00pm, (GMT+00:00) UTC

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX CLI survey

Jim White
 

EdgeX users and developers,
I am making one last pitch for your participation in our CLI survey.  The command line interface dev team is looking for critical feedback on how you use or would use the EdgeX CLI.  If you have yet filled out the survey, please do so.  You can find it at https://www.surveymonkey.com/r/QWKHZZZ or https://bitly.com/EdgeXCLI

thanks,
Jim White
CTO, IOTech
EdgeX Foundry co-founder & TSC Vice-chairman
On EdgeX Slack @ jpwhite
612-916-6693


EdgeX QA/Test WG: Validation/Testing - Tue, 06/02/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 2 June 2020
4:00pm to 5:00pm
(GMT+00:00) UTC

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 06/02/2020 4:00pm-5:00pm, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 2 June 2020, 4:00pm to 5:00pm, (GMT+00:00) UTC

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 05/26/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 26 May 2020
4:00pm to 5:00pm
(GMT+00:00) UTC

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 05/26/2020 4:00pm-5:00pm, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 26 May 2020, 4:00pm to 5:00pm, (GMT+00:00) UTC

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 05/19/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 19 May 2020
4:00pm to 5:00pm
(GMT+00:00) UTC

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 05/19/2020 4:00pm-5:00pm, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 19 May 2020, 4:00pm to 5:00pm, (GMT+00:00) UTC

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 05/12/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 12 May 2020
4:00pm to 5:00pm
(GMT+00:00) UTC

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 05/12/2020 4:00pm-5:00pm, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 12 May 2020, 4:00pm to 5:00pm, (GMT+00:00) UTC

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Re: [Edgex-tsc-devops] blackbox test release tag and schedule change

Ranganathan, Ramya
 

Lisa,

Yes this is a NEW requirement – not a NEW issue 😊

Let the community weigh in on it through the VOTE process.

-Ramya

 

From: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Sent: Thursday, May 7, 2020 8:51 AM
To: Cloud Tsai <cloud@...>; Ranganathan, Ramya <ramya.ranganathan@...>
Cc: Jim White <jim@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

The official release images should already be tested by the tests running against the master branch. At the time of release there is no difference between master images and the official release images.

 

I realize change is hard but I do recommend not creating extra work for yourself. If you want to create an extra branch and update the scripts that is fine. It is not a release artifact.

 

Blackbox testing has never been a release artifact. You are asking to ADD it to the artifacts. That is the core issue. That is why the community should weigh in on it. The process for everything in the past is to have the community weigh in on what we are supporting.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 7:10 PM
To: "Ranganathan, Ramya" <ramya.ranganathan@...>
Cc: "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>, Jim White <jim@...>, "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, James, and Ramya,

 

Thanks for taking care of this.

Yes, I agree with the proposal Lisa provided.

 

I don't see anything about tests in the release automation ADR, so I just thought blackbox test will not be one of the release artifacts.  My expectation was to create a normal branch like before.  The following steps are necessary anyway, or we won't have any final regression for the released images:

1.       Change the sync and deployment script to test against the official release images.

2.       Confirm all the test cases are passed.

Those work will be covered by QA working group.

I will join today's DevOps meeting to see what we should do.

 

 

On Thu, 7 May 2020 at 07:36, Ranganathan, Ramya <ramya.ranganathan@...> wrote:

Sounds like a plan Lisa!  Hope cloud is ok with this too..

 

 

From: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Sent: Wednesday, May 6, 2020 3:54 PM
To: Ranganathan, Ramya <ramya.ranganathan@...>; Jim White <jim@...>
Cc: Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

My proposed solution

 

  1. For Geneva we release a tag to create the desired snapshot
  2. For Hanoi and beyond we have a proposal that is brought up to the TSC and/or Architect’s meeting. Previous changes to our official release verbiage have required TSC vote. When I think back to Snaps and Swaggerhub these things were weighed by the community before being adopted. Essentially this is what is being asked, adding a new type of release artifact to the release.

 

From a DevOps perspective we have already provided the automation to release a git tag so that is covered. Of course any enhancements to the release automation can be scoped but we would need a proposal and some requirements gathering first.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 3:33 PM
To: Jim White <jim@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: Cloud Tsai <cloud@...>, "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

It would be appropriate for Test QA should own the test release strategy.

Actual tag creation should be part of the release automation process and potentially fall into Dev ops.

 

My 2 cents 😊

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Jim White
Sent: Wednesday, May 6, 2020 3:29 PM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: Ranganathan, Ramya <ramya.ranganathan@...>; Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

OK - which work group wants to host the discussion on this?  TestQA or DevOps??  Let's see if we can get consensus on a short term plan for Geneva, and then an owner to create a long term design/solution that we can review in a few weeks time and put in place for Hanoi.

 

 

Jim White

CTO, IOTech

EdgeX Foundry co-founder & TSC Vice-chairman

On EdgeX Slack @ jpwhite

612-916-6693

 

 

On Wed, 6 May 2020 at 15:19, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

I understand the need. I think the community should think about this issue a little more though.

In fact I see overlap in this request with what the direction the certification group has taken.

 

Like other the other repositories we can create a tag to give a downloadable snapshot for the repository at the time of release.

 

However, I think we should be careful. In general, we don’t usually offer much support once the release tags are cut.

I can see the scope of QA/Test blowing up if we have an official released test suite. With what resources would we support those tests? It doesn’t seem manageable long term.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 2:54 PM
To: Cloud Tsai <cloud@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Agree with Cloud. We need a way to snapshot the test version for major releases.

CD can still use master latest for nightly regressions.

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Wednesday, May 6, 2020 9:12 AM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, 

 

Thanks for the quick response.

We won't make any PR in cd-management.

 

However, we do need a branch or tag for blackbox test.

We need to take a snapshot which can pass all the Geneva test cases for users.

Users might want deploy the Geneva code, and clone a set of blackbox test to run on their own environment.

In this case, a clean test repository for Geneva only is important to them.

The blackbox test won't be only for our own CI/CD build, but also the spec and reference of how to interact with each service for real users..

Could you please advise whether you have any alternative solution for this requirement?  Thanks.

 

On Wed, 6 May 2020 at 23:01, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

No none of this is required!!!

 

The testing repositories should always pull things that are tagged master. There are no needed changes to the testing repositories for the release.

 

Please do not make any PRs in cd-management to release things. I’m managing that as the release tsar.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 6:40 AM
To: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>
Cc: "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi DevOps team,

 

I would like to give you guys a head up about what we plan to do for the release in blackbox test repositories and ask for the opinions.

 

I thought the concept should be the same as the modules, because we only need a tag.

After the Geneva images are released and we create the Geneva docker-compose file in developer-scripts, QA team will do the following actions:

  1. Add the Jenkinsfile into the blackbox test repositories: https://github.com/edgexfoundry/blackbox-testing and https://github.com/edgexfoundry/edgex-taf
  2. Change the sync and deployment script to test against the official release images.
  3. Confirm all the test cases are passed.
  4. Open a PR in cd-management to tag the release.
  5. Change  the sync and deployment script back to test against the master staged images.

Also, since the staged images build will be automatically triggered when each PR is merged, the schedule of blackbox daily run would not need to follow the image build time now.  I would like to modify the  blackbox daily run schedule to our morning (around 9 AM Taipei time, and it's 12 PM now).

We just need your help for PR review.  Please let me know if you have any questions or suggestions.  Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


Re: [Edgex-tsc-devops] blackbox test release tag and schedule change

Rashidi-ranjbar, Lisa A
 

The official release images should already be tested by the tests running against the master branch. At the time of release there is no difference between master images and the official release images.

 

I realize change is hard but I do recommend not creating extra work for yourself. If you want to create an extra branch and update the scripts that is fine. It is not a release artifact.

 

Blackbox testing has never been a release artifact. You are asking to ADD it to the artifacts. That is the core issue. That is why the community should weigh in on it. The process for everything in the past is to have the community weigh in on what we are supporting.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 7:10 PM
To: "Ranganathan, Ramya" <ramya.ranganathan@...>
Cc: "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>, Jim White <jim@...>, "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, James, and Ramya,

 

Thanks for taking care of this.

Yes, I agree with the proposal Lisa provided.

 

I don't see anything about tests in the release automation ADR, so I just thought blackbox test will not be one of the release artifacts.  My expectation was to create a normal branch like before.  The following steps are necessary anyway, or we won't have any final regression for the released images:

1.       Change the sync and deployment script to test against the official release images.

2.       Confirm all the test cases are passed.

Those work will be covered by QA working group.

I will join today's DevOps meeting to see what we should do.

 

 

On Thu, 7 May 2020 at 07:36, Ranganathan, Ramya <ramya.ranganathan@...> wrote:

Sounds like a plan Lisa!  Hope cloud is ok with this too..

 

 

From: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Sent: Wednesday, May 6, 2020 3:54 PM
To: Ranganathan, Ramya <ramya.ranganathan@...>; Jim White <jim@...>
Cc: Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

My proposed solution

 

  1. For Geneva we release a tag to create the desired snapshot
  2. For Hanoi and beyond we have a proposal that is brought up to the TSC and/or Architect’s meeting. Previous changes to our official release verbiage have required TSC vote. When I think back to Snaps and Swaggerhub these things were weighed by the community before being adopted. Essentially this is what is being asked, adding a new type of release artifact to the release.

 

From a DevOps perspective we have already provided the automation to release a git tag so that is covered. Of course any enhancements to the release automation can be scoped but we would need a proposal and some requirements gathering first.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 3:33 PM
To: Jim White <jim@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: Cloud Tsai <cloud@...>, "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

It would be appropriate for Test QA should own the test release strategy.

Actual tag creation should be part of the release automation process and potentially fall into Dev ops.

 

My 2 cents 😊

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Jim White
Sent: Wednesday, May 6, 2020 3:29 PM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: Ranganathan, Ramya <ramya.ranganathan@...>; Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

OK - which work group wants to host the discussion on this?  TestQA or DevOps??  Let's see if we can get consensus on a short term plan for Geneva, and then an owner to create a long term design/solution that we can review in a few weeks time and put in place for Hanoi.

 

 

Jim White

CTO, IOTech

EdgeX Foundry co-founder & TSC Vice-chairman

On EdgeX Slack @ jpwhite

612-916-6693

 

 

On Wed, 6 May 2020 at 15:19, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

I understand the need. I think the community should think about this issue a little more though.

In fact I see overlap in this request with what the direction the certification group has taken.

 

Like other the other repositories we can create a tag to give a downloadable snapshot for the repository at the time of release.

 

However, I think we should be careful. In general, we don’t usually offer much support once the release tags are cut.

I can see the scope of QA/Test blowing up if we have an official released test suite. With what resources would we support those tests? It doesn’t seem manageable long term.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 2:54 PM
To: Cloud Tsai <cloud@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Agree with Cloud. We need a way to snapshot the test version for major releases.

CD can still use master latest for nightly regressions.

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Wednesday, May 6, 2020 9:12 AM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, 

 

Thanks for the quick response.

We won't make any PR in cd-management.

 

However, we do need a branch or tag for blackbox test.

We need to take a snapshot which can pass all the Geneva test cases for users.

Users might want deploy the Geneva code, and clone a set of blackbox test to run on their own environment.

In this case, a clean test repository for Geneva only is important to them.

The blackbox test won't be only for our own CI/CD build, but also the spec and reference of how to interact with each service for real users..

Could you please advise whether you have any alternative solution for this requirement?  Thanks.

 

On Wed, 6 May 2020 at 23:01, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

No none of this is required!!!

 

The testing repositories should always pull things that are tagged master. There are no needed changes to the testing repositories for the release.

 

Please do not make any PRs in cd-management to release things. I’m managing that as the release tsar.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 6:40 AM
To: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>
Cc: "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi DevOps team,

 

I would like to give you guys a head up about what we plan to do for the release in blackbox test repositories and ask for the opinions.

 

I thought the concept should be the same as the modules, because we only need a tag.

After the Geneva images are released and we create the Geneva docker-compose file in developer-scripts, QA team will do the following actions:

  1. Add the Jenkinsfile into the blackbox test repositories: https://github.com/edgexfoundry/blackbox-testing and https://github.com/edgexfoundry/edgex-taf
  2. Change the sync and deployment script to test against the official release images.
  3. Confirm all the test cases are passed.
  4. Open a PR in cd-management to tag the release.
  5. Change  the sync and deployment script back to test against the master staged images.

Also, since the staged images build will be automatically triggered when each PR is merged, the schedule of blackbox daily run would not need to follow the image build time now.  I would like to modify the  blackbox daily run schedule to our morning (around 9 AM Taipei time, and it's 12 PM now).

We just need your help for PR review.  Please let me know if you have any questions or suggestions.  Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


Re: [Edgex-tsc-devops] blackbox test release tag and schedule change

Cloud Tsai
 

Hi Lisa, James, and Ramya,

Thanks for taking care of this.
Yes, I agree with the proposal Lisa provided.

I don't see anything about tests in the release automation ADR, so I just thought blackbox test will not be one of the release artifacts.  My expectation was to create a normal branch like before.  The following steps are necessary anyway, or we won't have any final regression for the released images:
  1. Change the sync and deployment script to test against the official release images.
  2. Confirm all the test cases are passed.
Those work will be covered by QA working group.
I will join today's DevOps meeting to see what we should do.


On Thu, 7 May 2020 at 07:36, Ranganathan, Ramya <ramya.ranganathan@...> wrote:

Sounds like a plan Lisa!  Hope cloud is ok with this too..

 

 

From: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Sent: Wednesday, May 6, 2020 3:54 PM
To: Ranganathan, Ramya <ramya.ranganathan@...>; Jim White <jim@...>
Cc: Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

My proposed solution

 

  1. For Geneva we release a tag to create the desired snapshot
  2. For Hanoi and beyond we have a proposal that is brought up to the TSC and/or Architect’s meeting. Previous changes to our official release verbiage have required TSC vote. When I think back to Snaps and Swaggerhub these things were weighed by the community before being adopted. Essentially this is what is being asked, adding a new type of release artifact to the release.

 

From a DevOps perspective we have already provided the automation to release a git tag so that is covered. Of course any enhancements to the release automation can be scoped but we would need a proposal and some requirements gathering first.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 3:33 PM
To: Jim White <jim@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: Cloud Tsai <cloud@...>, "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

It would be appropriate for Test QA should own the test release strategy.

Actual tag creation should be part of the release automation process and potentially fall into Dev ops.

 

My 2 cents 😊

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Jim White
Sent: Wednesday, May 6, 2020 3:29 PM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: Ranganathan, Ramya <ramya.ranganathan@...>; Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

OK - which work group wants to host the discussion on this?  TestQA or DevOps??  Let's see if we can get consensus on a short term plan for Geneva, and then an owner to create a long term design/solution that we can review in a few weeks time and put in place for Hanoi.

 

 

Jim White

CTO, IOTech

EdgeX Foundry co-founder & TSC Vice-chairman

On EdgeX Slack @ jpwhite

612-916-6693

 

 

On Wed, 6 May 2020 at 15:19, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

I understand the need. I think the community should think about this issue a little more though.

In fact I see overlap in this request with what the direction the certification group has taken.

 

Like other the other repositories we can create a tag to give a downloadable snapshot for the repository at the time of release.

 

However, I think we should be careful. In general, we don’t usually offer much support once the release tags are cut.

I can see the scope of QA/Test blowing up if we have an official released test suite. With what resources would we support those tests? It doesn’t seem manageable long term.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 2:54 PM
To: Cloud Tsai <cloud@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Agree with Cloud. We need a way to snapshot the test version for major releases.

CD can still use master latest for nightly regressions.

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Wednesday, May 6, 2020 9:12 AM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, 

 

Thanks for the quick response.

We won't make any PR in cd-management.

 

However, we do need a branch or tag for blackbox test.

We need to take a snapshot which can pass all the Geneva test cases for users.

Users might want deploy the Geneva code, and clone a set of blackbox test to run on their own environment.

In this case, a clean test repository for Geneva only is important to them.

The blackbox test won't be only for our own CI/CD build, but also the spec and reference of how to interact with each service for real users..

Could you please advise whether you have any alternative solution for this requirement?  Thanks.

 

On Wed, 6 May 2020 at 23:01, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

No none of this is required!!!

 

The testing repositories should always pull things that are tagged master. There are no needed changes to the testing repositories for the release.

 

Please do not make any PRs in cd-management to release things. I’m managing that as the release tsar.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 6:40 AM
To: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>
Cc: "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi DevOps team,

 

I would like to give you guys a head up about what we plan to do for the release in blackbox test repositories and ask for the opinions.

 

I thought the concept should be the same as the modules, because we only need a tag.

After the Geneva images are released and we create the Geneva docker-compose file in developer-scripts, QA team will do the following actions:

  1. Add the Jenkinsfile into the blackbox test repositories: https://github.com/edgexfoundry/blackbox-testing and https://github.com/edgexfoundry/edgex-taf
  2. Change the sync and deployment script to test against the official release images.
  3. Confirm all the test cases are passed.
  4. Open a PR in cd-management to tag the release.
  5. Change  the sync and deployment script back to test against the master staged images.

Also, since the staged images build will be automatically triggered when each PR is merged, the schedule of blackbox daily run would not need to follow the image build time now.  I would like to modify the  blackbox daily run schedule to our morning (around 9 AM Taipei time, and it's 12 PM now).

We just need your help for PR review.  Please let me know if you have any questions or suggestions.  Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai



--
Best Regards,
Cloud Tsai


Re: [Edgex-tsc-devops] blackbox test release tag and schedule change

Ranganathan, Ramya
 

Sounds like a plan Lisa!  Hope cloud is ok with this too..

 

 

From: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Sent: Wednesday, May 6, 2020 3:54 PM
To: Ranganathan, Ramya <ramya.ranganathan@...>; Jim White <jim@...>
Cc: Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

My proposed solution

 

  1. For Geneva we release a tag to create the desired snapshot
  2. For Hanoi and beyond we have a proposal that is brought up to the TSC and/or Architect’s meeting. Previous changes to our official release verbiage have required TSC vote. When I think back to Snaps and Swaggerhub these things were weighed by the community before being adopted. Essentially this is what is being asked, adding a new type of release artifact to the release.

 

From a DevOps perspective we have already provided the automation to release a git tag so that is covered. Of course any enhancements to the release automation can be scoped but we would need a proposal and some requirements gathering first.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 3:33 PM
To: Jim White <jim@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: Cloud Tsai <cloud@...>, "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

It would be appropriate for Test QA should own the test release strategy.

Actual tag creation should be part of the release automation process and potentially fall into Dev ops.

 

My 2 cents 😊

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Jim White
Sent: Wednesday, May 6, 2020 3:29 PM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: Ranganathan, Ramya <ramya.ranganathan@...>; Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

OK - which work group wants to host the discussion on this?  TestQA or DevOps??  Let's see if we can get consensus on a short term plan for Geneva, and then an owner to create a long term design/solution that we can review in a few weeks time and put in place for Hanoi.

 

 

Jim White

CTO, IOTech

EdgeX Foundry co-founder & TSC Vice-chairman

On EdgeX Slack @ jpwhite

612-916-6693

 

 

On Wed, 6 May 2020 at 15:19, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

I understand the need. I think the community should think about this issue a little more though.

In fact I see overlap in this request with what the direction the certification group has taken.

 

Like other the other repositories we can create a tag to give a downloadable snapshot for the repository at the time of release.

 

However, I think we should be careful. In general, we don’t usually offer much support once the release tags are cut.

I can see the scope of QA/Test blowing up if we have an official released test suite. With what resources would we support those tests? It doesn’t seem manageable long term.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 2:54 PM
To: Cloud Tsai <cloud@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Agree with Cloud. We need a way to snapshot the test version for major releases.

CD can still use master latest for nightly regressions.

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Wednesday, May 6, 2020 9:12 AM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, 

 

Thanks for the quick response.

We won't make any PR in cd-management.

 

However, we do need a branch or tag for blackbox test.

We need to take a snapshot which can pass all the Geneva test cases for users.

Users might want deploy the Geneva code, and clone a set of blackbox test to run on their own environment.

In this case, a clean test repository for Geneva only is important to them.

The blackbox test won't be only for our own CI/CD build, but also the spec and reference of how to interact with each service for real users..

Could you please advise whether you have any alternative solution for this requirement?  Thanks.

 

On Wed, 6 May 2020 at 23:01, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

No none of this is required!!!

 

The testing repositories should always pull things that are tagged master. There are no needed changes to the testing repositories for the release.

 

Please do not make any PRs in cd-management to release things. I’m managing that as the release tsar.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 6:40 AM
To: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>
Cc: "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi DevOps team,

 

I would like to give you guys a head up about what we plan to do for the release in blackbox test repositories and ask for the opinions.

 

I thought the concept should be the same as the modules, because we only need a tag.

After the Geneva images are released and we create the Geneva docker-compose file in developer-scripts, QA team will do the following actions:

  1. Add the Jenkinsfile into the blackbox test repositories: https://github.com/edgexfoundry/blackbox-testing and https://github.com/edgexfoundry/edgex-taf
  2. Change the sync and deployment script to test against the official release images.
  3. Confirm all the test cases are passed.
  4. Open a PR in cd-management to tag the release.
  5. Change  the sync and deployment script back to test against the master staged images.

Also, since the staged images build will be automatically triggered when each PR is merged, the schedule of blackbox daily run would not need to follow the image build time now.  I would like to modify the  blackbox daily run schedule to our morning (around 9 AM Taipei time, and it's 12 PM now).

We just need your help for PR review.  Please let me know if you have any questions or suggestions.  Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


Re: [Edgex-tsc-devops] blackbox test release tag and schedule change

Gregg, James R
 

Jim,

I think Lisa has done a good job of outlining what should happen for this.

If we need further discussion, we can bring it into the DevOps WG tomorrow morning.

 

~ James Gregg

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Jim White
Sent: Wednesday, May 6, 2020 3:29 PM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: Ranganathan, Ramya <ramya.ranganathan@...>; Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

OK - which work group wants to host the discussion on this?  TestQA or DevOps??  Let's see if we can get consensus on a short term plan for Geneva, and then an owner to create a long term design/solution that we can review in a few weeks time and put in place for Hanoi.

 

 

Jim White

CTO, IOTech

EdgeX Foundry co-founder & TSC Vice-chairman

On EdgeX Slack @ jpwhite

612-916-6693

 

 

On Wed, 6 May 2020 at 15:19, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

I understand the need. I think the community should think about this issue a little more though.

In fact I see overlap in this request with what the direction the certification group has taken.

 

Like other the other repositories we can create a tag to give a downloadable snapshot for the repository at the time of release.

 

However, I think we should be careful. In general, we don’t usually offer much support once the release tags are cut.

I can see the scope of QA/Test blowing up if we have an official released test suite. With what resources would we support those tests? It doesn’t seem manageable long term.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 2:54 PM
To: Cloud Tsai <cloud@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Agree with Cloud. We need a way to snapshot the test version for major releases.

CD can still use master latest for nightly regressions.

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Wednesday, May 6, 2020 9:12 AM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, 

 

Thanks for the quick response.

We won't make any PR in cd-management.

 

However, we do need a branch or tag for blackbox test.

We need to take a snapshot which can pass all the Geneva test cases for users.

Users might want deploy the Geneva code, and clone a set of blackbox test to run on their own environment.

In this case, a clean test repository for Geneva only is important to them.

The blackbox test won't be only for our own CI/CD build, but also the spec and reference of how to interact with each service for real users..

Could you please advise whether you have any alternative solution for this requirement?  Thanks.

 

On Wed, 6 May 2020 at 23:01, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

No none of this is required!!!

 

The testing repositories should always pull things that are tagged master. There are no needed changes to the testing repositories for the release.

 

Please do not make any PRs in cd-management to release things. I’m managing that as the release tsar.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 6:40 AM
To: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>
Cc: "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi DevOps team,

 

I would like to give you guys a head up about what we plan to do for the release in blackbox test repositories and ask for the opinions.

 

I thought the concept should be the same as the modules, because we only need a tag.

After the Geneva images are released and we create the Geneva docker-compose file in developer-scripts, QA team will do the following actions:

  1. Add the Jenkinsfile into the blackbox test repositories: https://github.com/edgexfoundry/blackbox-testing and https://github.com/edgexfoundry/edgex-taf
  2. Change the sync and deployment script to test against the official release images.
  3. Confirm all the test cases are passed.
  4. Open a PR in cd-management to tag the release.
  5. Change  the sync and deployment script back to test against the master staged images.

Also, since the staged images build will be automatically triggered when each PR is merged, the schedule of blackbox daily run would not need to follow the image build time now.  I would like to modify the  blackbox daily run schedule to our morning (around 9 AM Taipei time, and it's 12 PM now).

We just need your help for PR review.  Please let me know if you have any questions or suggestions.  Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


Re: [Edgex-tsc-devops] blackbox test release tag and schedule change

Rashidi-ranjbar, Lisa A
 

My proposed solution

 

  1. For Geneva we release a tag to create the desired snapshot
  2. For Hanoi and beyond we have a proposal that is brought up to the TSC and/or Architect’s meeting. Previous changes to our official release verbiage have required TSC vote. When I think back to Snaps and Swaggerhub these things were weighed by the community before being adopted. Essentially this is what is being asked, adding a new type of release artifact to the release.

 

From a DevOps perspective we have already provided the automation to release a git tag so that is covered. Of course any enhancements to the release automation can be scoped but we would need a proposal and some requirements gathering first.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 3:33 PM
To: Jim White <jim@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: Cloud Tsai <cloud@...>, "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

It would be appropriate for Test QA should own the test release strategy.

Actual tag creation should be part of the release automation process and potentially fall into Dev ops.

 

My 2 cents 😊

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Jim White
Sent: Wednesday, May 6, 2020 3:29 PM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: Ranganathan, Ramya <ramya.ranganathan@...>; Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

OK - which work group wants to host the discussion on this?  TestQA or DevOps??  Let's see if we can get consensus on a short term plan for Geneva, and then an owner to create a long term design/solution that we can review in a few weeks time and put in place for Hanoi.

 

 

Jim White

CTO, IOTech

EdgeX Foundry co-founder & TSC Vice-chairman

On EdgeX Slack @ jpwhite

612-916-6693

 

 

On Wed, 6 May 2020 at 15:19, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

I understand the need. I think the community should think about this issue a little more though.

In fact I see overlap in this request with what the direction the certification group has taken.

 

Like other the other repositories we can create a tag to give a downloadable snapshot for the repository at the time of release.

 

However, I think we should be careful. In general, we don’t usually offer much support once the release tags are cut.

I can see the scope of QA/Test blowing up if we have an official released test suite. With what resources would we support those tests? It doesn’t seem manageable long term.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 2:54 PM
To: Cloud Tsai <cloud@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Agree with Cloud. We need a way to snapshot the test version for major releases.

CD can still use master latest for nightly regressions.

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Wednesday, May 6, 2020 9:12 AM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, 

 

Thanks for the quick response.

We won't make any PR in cd-management.

 

However, we do need a branch or tag for blackbox test.

We need to take a snapshot which can pass all the Geneva test cases for users.

Users might want deploy the Geneva code, and clone a set of blackbox test to run on their own environment.

In this case, a clean test repository for Geneva only is important to them.

The blackbox test won't be only for our own CI/CD build, but also the spec and reference of how to interact with each service for real users..

Could you please advise whether you have any alternative solution for this requirement?  Thanks.

 

On Wed, 6 May 2020 at 23:01, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

No none of this is required!!!

 

The testing repositories should always pull things that are tagged master. There are no needed changes to the testing repositories for the release.

 

Please do not make any PRs in cd-management to release things. I’m managing that as the release tsar.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 6:40 AM
To: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>
Cc: "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi DevOps team,

 

I would like to give you guys a head up about what we plan to do for the release in blackbox test repositories and ask for the opinions.

 

I thought the concept should be the same as the modules, because we only need a tag.

After the Geneva images are released and we create the Geneva docker-compose file in developer-scripts, QA team will do the following actions:

  1. Add the Jenkinsfile into the blackbox test repositories: https://github.com/edgexfoundry/blackbox-testing and https://github.com/edgexfoundry/edgex-taf
  2. Change the sync and deployment script to test against the official release images.
  3. Confirm all the test cases are passed.
  4. Open a PR in cd-management to tag the release.
  5. Change  the sync and deployment script back to test against the master staged images.

Also, since the staged images build will be automatically triggered when each PR is merged, the schedule of blackbox daily run would not need to follow the image build time now.  I would like to modify the  blackbox daily run schedule to our morning (around 9 AM Taipei time, and it's 12 PM now).

We just need your help for PR review.  Please let me know if you have any questions or suggestions.  Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


Re: [Edgex-tsc-devops] blackbox test release tag and schedule change

Ranganathan, Ramya
 

It would be appropriate for Test QA should own the test release strategy.

Actual tag creation should be part of the release automation process and potentially fall into Dev ops.

 

My 2 cents 😊

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Jim White
Sent: Wednesday, May 6, 2020 3:29 PM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: Ranganathan, Ramya <ramya.ranganathan@...>; Cloud Tsai <cloud@...>; EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

OK - which work group wants to host the discussion on this?  TestQA or DevOps??  Let's see if we can get consensus on a short term plan for Geneva, and then an owner to create a long term design/solution that we can review in a few weeks time and put in place for Hanoi.

 

 

Jim White

CTO, IOTech

EdgeX Foundry co-founder & TSC Vice-chairman

On EdgeX Slack @ jpwhite

612-916-6693

 

 

On Wed, 6 May 2020 at 15:19, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

I understand the need. I think the community should think about this issue a little more though.

In fact I see overlap in this request with what the direction the certification group has taken.

 

Like other the other repositories we can create a tag to give a downloadable snapshot for the repository at the time of release.

 

However, I think we should be careful. In general, we don’t usually offer much support once the release tags are cut.

I can see the scope of QA/Test blowing up if we have an official released test suite. With what resources would we support those tests? It doesn’t seem manageable long term.

 

Lisa

 

From: "Ranganathan, Ramya" <ramya.ranganathan@...>
Date: Wednesday, May 6, 2020 at 2:54 PM
To: Cloud Tsai <cloud@...>, "Rashidi-ranjbar, Lisa A" <lisa.a.rashidi-ranjbar@...>
Cc: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>, "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: RE: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Agree with Cloud. We need a way to snapshot the test version for major releases.

CD can still use master latest for nightly regressions.

 

-Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Wednesday, May 6, 2020 9:12 AM
To: Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...>
Cc: EdgeX-TSC-DevOps@...; edgex-tsc-qa-test@...; Nevis, Bryon <bryon.nevis@...>; tw-engineers <tw-engineers@...>
Subject: Re: [Edgex-tsc-QA-Test] [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi Lisa, 

 

Thanks for the quick response.

We won't make any PR in cd-management.

 

However, we do need a branch or tag for blackbox test.

We need to take a snapshot which can pass all the Geneva test cases for users.

Users might want deploy the Geneva code, and clone a set of blackbox test to run on their own environment.

In this case, a clean test repository for Geneva only is important to them.

The blackbox test won't be only for our own CI/CD build, but also the spec and reference of how to interact with each service for real users..

Could you please advise whether you have any alternative solution for this requirement?  Thanks.

 

On Wed, 6 May 2020 at 23:01, Rashidi-ranjbar, Lisa A <lisa.a.rashidi-ranjbar@...> wrote:

No none of this is required!!!

 

The testing repositories should always pull things that are tagged master. There are no needed changes to the testing repositories for the release.

 

Please do not make any PRs in cd-management to release things. I’m managing that as the release tsar.

 

Lisa

 

From: <EdgeX-TSC-DevOps@...> on behalf of Cloud Tsai <cloud@...>
Date: Wednesday, May 6, 2020 at 6:40 AM
To: "EdgeX-TSC-DevOps@..." <EdgeX-TSC-DevOps@...>
Cc: "edgex-tsc-qa-test@..." <edgex-tsc-qa-test@...>, "Nevis, Bryon" <bryon.nevis@...>, tw-engineers <tw-engineers@...>
Subject: [Edgex-tsc-devops] blackbox test release tag and schedule change

 

Hi DevOps team,

 

I would like to give you guys a head up about what we plan to do for the release in blackbox test repositories and ask for the opinions.

 

I thought the concept should be the same as the modules, because we only need a tag.

After the Geneva images are released and we create the Geneva docker-compose file in developer-scripts, QA team will do the following actions:

  1. Add the Jenkinsfile into the blackbox test repositories: https://github.com/edgexfoundry/blackbox-testing and https://github.com/edgexfoundry/edgex-taf
  2. Change the sync and deployment script to test against the official release images.
  3. Confirm all the test cases are passed.
  4. Open a PR in cd-management to tag the release.
  5. Change  the sync and deployment script back to test against the master staged images.

Also, since the staged images build will be automatically triggered when each PR is merged, the schedule of blackbox daily run would not need to follow the image build time now.  I would like to modify the  blackbox daily run schedule to our morning (around 9 AM Taipei time, and it's 12 PM now).

We just need your help for PR review.  Please let me know if you have any questions or suggestions.  Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai