Date   

Re: Blackbox Tests are not Running

Gregg, James R
 

DevOps hasn’t owned any of this blackbox testing work. 

 

~ James Gregg

 

From: EdgeX-TSC-DevOps@... <EdgeX-TSC-DevOps@...> On Behalf Of Trevor.Conn@...
Sent: Wednesday, March 11, 2020 8:34 AM
To: edgex-tsc-devops@...; EdgeX-TSC-QA-Test@...
Subject: [Edgex-tsc-devops] Blackbox Tests are not Running
Importance: High

 

Hi all – I’m raising the flag, following on Bryon Nevis’ post in Slack yesterday, that Blackbox tests are broken. The run itself is failing and we’ve lost all history in Jenkins. As you can see when looking at the master build definition, there are now only two entries in the job history. Not sure what happened here, but please give this priority.

 

The error from the build console of job #2 is as follows:

 

10:37:01  > git --version # timeout=10

10:37:01 using GIT_SSH to set credentials SSH Credentials for GitHub

10:37:01  > git fetch --tags --progress git@...:edgexfoundry/blackbox-testing +refs/heads/*:refs/remotes/origin/* # timeout=10

10:37:01 ERROR: Error cloning remote repo 'origin'

10:37:01 hudson.plugins.git.GitException: Command "git fetch --tags --progress git@...:edgexfoundry/blackbox-testing +refs/heads/*:refs/remotes/origin/*" returned status code 128:

10:37:01 stdout:

10:37:01 stderr: fatal: Not a git repository (or any of the parent directories): .git

 

In addition, unsure if it’s related, I see an error in the job configuration’s “Build Environment” tab.

 

 

Trevor Conn

Director of Software Engineering (DMTS)

Core Working Group Chair of EdgeX Foundry

Dell Technologies | Edge / IoT DellTech

Trevor_Conn@...

Round Rock, TX USA

 


Blackbox Tests are not Running

Trevor.Conn@...
 

Hi all – I’m raising the flag, following on Bryon Nevis’ post in Slack yesterday, that Blackbox tests are broken. The run itself is failing and we’ve lost all history in Jenkins. As you can see when looking at the master build definition, there are now only two entries in the job history. Not sure what happened here, but please give this priority.

 

The error from the build console of job #2 is as follows:

 

10:37:01  > git --version # timeout=10

10:37:01 using GIT_SSH to set credentials SSH Credentials for GitHub

10:37:01  > git fetch --tags --progress git@...:edgexfoundry/blackbox-testing +refs/heads/*:refs/remotes/origin/* # timeout=10

10:37:01 ERROR: Error cloning remote repo 'origin'

10:37:01 hudson.plugins.git.GitException: Command "git fetch --tags --progress git@...:edgexfoundry/blackbox-testing +refs/heads/*:refs/remotes/origin/*" returned status code 128:

10:37:01 stdout:

10:37:01 stderr: fatal: Not a git repository (or any of the parent directories): .git

 

In addition, unsure if it’s related, I see an error in the job configuration’s “Build Environment” tab.

 

 

Trevor Conn

Director of Software Engineering (DMTS)

Core Working Group Chair of EdgeX Foundry

Dell Technologies | Edge / IoT DellTech

Trevor_Conn@...

Round Rock, TX USA

 


Cancelled Event: EdgeX QA/Test WG: Validation/Testing - Tuesday, 10 March 2020 #cal-cancelled

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Cancelled: EdgeX QA/Test WG: Validation/Testing

This event has been cancelled.

When:
Tuesday, 10 March 2020 11:00pm to
Wednesday, 11 March 2020 12:00am
(UTC+00:00) Europe/London

Where:
https://zoom.us/j/635232828

Organizer: EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Meeting tonight has been cancelled

Robin Chatterjee
 

Good morning,

The meeting that is scheduled for tonight (11 pm British Time) has been canceled.

Nonetheless, please email me any issues of concern that you believe need to be addressed by email and I will append it to the agenda so that we can discuss it on next Tuesday's meeting.

I look forward to speaking with everyone concerned next week (17th March 2020).

Best regards,

Robin CHATTERJEE
Technical Lead (IOTech Systems Limited) 
EdgeX QA/Test WG Chair ( https://www.edgexfoundry.org )
Email     : robin@...
Mobile   : +44 (0)7476525353



EdgeX QA/Test WG: Validation/Testing - Tue, 03/03/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 3 March 2020 11:00pm to
Wednesday, 4 March 2020 12:00am
(GMT+00:00) Europe/London

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 03/03/2020 11:00pm to Wed, 03/04/2020 12:00am, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 3 March 2020, 11:00pm to Wednesday, 4 March 2020, 12:00am (GMT+00:00) Europe/London

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 02/25/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 25 February 2020 11:00pm to
Wednesday, 26 February 2020 12:00am
(GMT+00:00) Europe/London

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 02/25/2020 11:00pm to Wed, 02/26/2020 12:00am, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 25 February 2020, 11:00pm to Wednesday, 26 February 2020, 12:00am (GMT+00:00) Europe/London

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 02/18/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 18 February 2020 11:00pm to
Wednesday, 19 February 2020 12:00am
(GMT+00:00) Europe/London

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 02/18/2020 11:00pm to Wed, 02/19/2020 12:00am, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 18 February 2020, 11:00pm to Wednesday, 19 February 2020, 12:00am (GMT+00:00) Europe/London

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 02/11/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 11 February 2020 11:00pm to
Wednesday, 12 February 2020 12:00am
(GMT+00:00) Europe/London

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 02/11/2020 11:00pm to Wed, 02/12/2020 12:00am, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 11 February 2020, 11:00pm to Wednesday, 12 February 2020, 12:00am (GMT+00:00) Europe/London

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


EdgeX QA/Test WG: Validation/Testing - Tue, 02/04/2020 #cal-notice

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <noreply@...>
 

EdgeX QA/Test WG: Validation/Testing

When:
Tuesday, 4 February 2020 11:00pm to
Wednesday, 5 February 2020 12:00am
(GMT+00:00) Europe/London

Where:
https://zoom.us/j/635232828

Organizer:
EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Upcoming Event: EdgeX QA/Test WG: Validation/Testing - Tue, 02/04/2020 11:00pm to Wed, 02/05/2020 12:00am, Please RSVP #cal-reminder

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Reminder: EdgeX QA/Test WG: Validation/Testing

When: Tuesday, 4 February 2020, 11:00pm to Wednesday, 5 February 2020, 12:00am (GMT+00:00) Europe/London

Where:https://zoom.us/j/635232828

An RSVP is requested. Click here to RSVP

Organizer: EdgeX-TSC-QA-Test@...

Description: EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Re: APIv2 Version Property and Backward/Forward Compatibility (Follow Up)

Michael Estrin <m.estrin@...>
 

​For historical purposes, quoting Rodney's Slack response to this message below:


Rodney  12 hours ago
I agree with 1 in your list above:  Changes to an endpoint’s DTOs across minor releases must be additive.

Rodney  12 hours ago
I also agree with 2, 3, 4, and 6.

Rodney  12 hours ago
With respect to “optional” fields within a DTO, I would ask that we also define how the API behaves if such a field is absent or otherwise blank.  For example, should an implementation assume a default value and what would that default value be?  This is important for making a distinction between an unspecified value or the value 0.  For example, a count of 0 could be interpreted as a literal truth (e.g. there are 0 devices connected to this service) whereas the absence of the value may be the count in unknown and should not be assumed to be zero.  Strings are another area of concern because an empty string may be sufficiently ambiguous as to create confusion in the users.

Rodney  12 hours ago
I am not onboard with how you wrote 5.  My thoughts are still working out whether or not it is possible to be “additive” with the interpretation of a property.

Rodney  12 hours ago
With respect to 6, I would reword it as some form of guidance.  One could argue that allowing 6 serves as a form of a “relief valve” by deterring the need for a major release bump.  It’s frequency of use needs to be monitored in that abuse of 6 is a signal that it is time to refactor/rebalance the APIs into a new major release.

Rodney  12 hours ago
With 6, some guidance should be written up.

Rodney  12 hours ago
Furthermore, with respect to the interpretation of optional fields should be part of what you refer to as contracts and therefore validated by QA/Test

Cross-posted to QA/Test, Core, and AppFunc.


Hope to see this as a discussion topic in next week's Certification working group meeting.


Respectfully suggest a follow-on ADR on eventually agreed-upon approach to backward-compatibility be written for the historical record.  See this solidly falling under Certification's mandate.



From: Estrin, Michael
Sent: Wednesday, January 22, 2020 11:09 AM
To: EdgeX-TSC-Certification@...; edgex-tsc-qa-test@...
Cc: Rodney Hess; Conn, Trevor; Jim White
Subject: APIv2 Version Property and Backward/Forward Compatibility (Follow Up)
 

Follow up to my message of earlier this week (https://lists.edgexfoundry.org/g/EdgeX-TSC-Certification/message/94​).  Aforementioned message was discussed in this week's Certification working group meeting.


I'm backing off my request to version HTTP API URLs and the batch envelope DTO's version field to include minor release (e.g. /api/v2.0/...).  After an implementation spike and some additional research, I realized the concern I was attempting to address with my earlier proposal was a result of the lack of automated testing to enforce backward-compatibility across minor releases.


I now believe we should version the HTTP API URLs and the batch envelope DTO's version field to major only (e.g. /api/v2/...).


I further believe we should codify the principles developers should follow to ensure backward-compatibility across minor releases.  I offer the following "starter list" for consideration:

  1. Changes to an endpoint's DTOs across minor releases must be additive.  Re-purposing an existing property is a backward-breaking change.
  2. New properties added to an existing endpoint's DTOs may only be "optional."  Adding a new "required" field to an existing DTO is a backward-breaking change.
  3. Changes to the "required"/"optional" designation of an individual endpoint's DTO's property are not allowed.  Varying whether or not a property is required across minor releases is a backward-breaking change.
  4. Changes to the structural schema (i.e. {"foo":{"bar":"baz"} => {"foo.bar": "baz"}) across minor releases is not allowed; this is a backward-breaking change.
  5. Changes to the fundamental interpretation of a property's content is not allowed.  Varying how a property's value is interpreted is a backward-breaking change.
  6. Any change required to add functionality in a minor release that cannot be made to an existing endpoint (because doing so would break backward compatibility) could be accomplished through the creation of a new endpoint and corresponding DTOs.
Finally, starting with API v2.0 I believe we should have automated contract testing across minor releases.

The APIv2 skeletal implementation (https://github.com/edgexfoundry/edgex-go/pull/2285) includes support for Golang-based acceptance testing.  I've structured the initial set of example tests to support backward-compatibility testing.  That is, each endpoint will be exercised using each minor release's DTOs to ensure later minor releases do not introduce behavior or schema changes that break the contract established by an earlier minor release.  I can expound further if desired and on request.


Cross-posted this to the QA/Test Working Group mailing list.  I believe QA/Test needs to address how their TAF implementation will enforce backward-compatibility via contract testing across minor releases.  I believe this is required functionality for certification.


I look forward to subsequent discussion, whether here on the mailing list or within the Certification or QA/Test working group meetings.


Cancelled Event: EdgeX QA/Test WG: Validation/Testing - Tuesday, 28 January 2020 #cal-cancelled

EdgeX-TSC-QA-Test@lists.edgexfoundry.org Calendar <EdgeX-TSC-QA-Test@...>
 

Cancelled: EdgeX QA/Test WG: Validation/Testing

This event has been cancelled.

When:
Tuesday, 28 January 2020 11:00pm to
Wednesday, 29 January 2020 12:00am
(UTC+00:00) Europe/London

Where:
https://zoom.us/j/635232828

Organizer: EdgeX-TSC-QA-Test@...

Description:
EdgeX QA/Test WG: Validation/Testing meeting. Meeting content posted to QA/Test Wiki.
Meeting Lead: Robin Chatterjee, EdgeX QA/Test WG Chair, robin@...
-----Join Zoom Meeting
https://zoom.us/j/635232828

One tap mobile
+16699006833,,635232828# US (San Jose)
+16465588656,,635232828# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 646 558 8656 US (New York)
        +1 855 880 1246 US Toll-free
        +1 877 369 0926 US Toll-free
Meeting ID: 635 232 828
Find your local number: https://zoom.us/u/abzs1KyOkW


Re: Meeting tonight has been cancelled

Michael Estrin <m.estrin@...>
 

​I'm requesting 15 minutes in an upcoming QA/Test Working Group meeting to discuss the need for (and TAF's approach to) contract testing to ensure backward compatibility across minor releases.  This will be required for certification (@Rodney).  This follows my posting to the working group's mailing list (https://lists.edgexfoundry.org/g/EdgeX-TSC-QA-Test/message/244).  Please advise if/when this request can be accommodated.​



From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> on behalf of Robin Chatterjee <robin@...>
Sent: Tuesday, January 28, 2020 3:20 AM
To: edgex-tsc-qa-test@...
Subject: [Edgex-tsc-QA-Test] Meeting tonight has been cancelled
 

[EXTERNAL EMAIL]

Good morning,

The meeting that is scheduled for tonight (11 pm British Time) has been cancelled.

Nonetheless, please email me any issues of concern that you believe need to be addressed by email and I will append it to the agenda so that we can discuss it on next Tuesday's meeting.

I look forward to speaking with everyone concerned next week (4th Feb 2020).

Best regards,

Robin CHATTERJEE
Technical Lead (IOTech Systems Limited) 
EdgeX QA/Test WG Chair ( https://www.edgexfoundry.org )
Email     : robin@...
Mobile   : +44 (0)7476525353


Meeting tonight has been cancelled

Robin Chatterjee
 

Good morning,

The meeting that is scheduled for tonight (11 pm British Time) has been cancelled.

Nonetheless, please email me any issues of concern that you believe need to be addressed by email and I will append it to the agenda so that we can discuss it on next Tuesday's meeting.

I look forward to speaking with everyone concerned next week (4th Feb 2020).

Best regards,

Robin CHATTERJEE
Technical Lead (IOTech Systems Limited) 
EdgeX QA/Test WG Chair ( https://www.edgexfoundry.org )
Email     : robin@...
Mobile   : +44 (0)7476525353


Re: edgex-taf file structure for review

Ranganathan, Ramya
 

>> The is to apply UC_ as naming convention, but we would like to categorize the in service first.  The UC_{xxx} folder will be under the service folder, such as functionalTest/core-data/UC_xxx.  Is it ok?

Yes  , this is perfect!

 

Thank you Cloud!

 

-Ramya

 

From: Cloud Tsai <cloud@...>
Sent: Thursday, January 23, 2020 8:30 AM
To: Ranganathan, Ramya <ramya.ranganathan@...>
Cc: edgex-tsc-qa-test@...
Subject: Re: [Edgex-tsc-QA-Test] edgex-taf file structure for review

 

Hi Ramya,

 

Thanks for the review and suggestions.

Here is the draft PR

I agree with your recommendations.

We will rename testModules as opposed to testCaseModules.

The is to apply UC_ as naming convention, but we would like to categorize the in service first.  The UC_{xxx} folder will be under the service folder, such as functionalTest/core-data/UC_xxx.  Is it ok?

 

On Thu, 23 Jan 2020 at 01:56, Ranganathan, Ramya <ramya.ranganathan@...> wrote:

Hi Cloud,

Thanks for the attachment.

Robin was mentioning in the WG that a PR has been opened for this revised TAF directory structure.

Could you please send me the link for that.

 

Couple of feedback:

  1. I may sound too picky, but according to me, testModule is generic and incomplete in the sense that it may or may not imply all aspects of test case definition.

I would propose testCaseModules as opposed to testModules. In software engineering, a test case is a specification of the inputs, execution conditions, testing procedure, and expected results that define a single test to be executed to achieve a particular software testing objective, such as to exercise a particular program path or to verify compliance with a specific requirement.[

In TAF, each test case is crafted as a module fulfilling all the above conditions which then forms the building block to execute it under various test scenarios.

 

  1. I see Cloud has termed the UC_ only for integration and E2E type of scenarios. I see the point here but just for uniform naming reasons, I am debating what if all the test scenarios can be named UC_.

I will think more about it. Will discuss with cloud on this and get his thoughts.

 

Thanks,

Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Tuesday, January 14, 2020 7:55 AM
To: edgex-tsc-qa-test@...
Subject: [Edgex-tsc-QA-Test] edgex-taf file structure for review

 

Hi all,

 

As the discussion, we will make all kinds of the new tests in edgex-taf repository, including functional test, integration test, performance test, and so on.

The attachment is the proposal of new file structure.  Please help take a look and let us know your thoughts.

Thanks.

 

--

Best Regards,

Cloud Tsai


 

--

Best Regards,

Cloud Tsai


Re: edgex-taf file structure for review

Cloud Tsai
 

Hi Ramya,

Thanks for the review and suggestions.
Here is the draft PR
I agree with your recommendations.
We will rename testModules as opposed to testCaseModules.
The is to apply UC_ as naming convention, but we would like to categorize the in service first.  The UC_{xxx} folder will be under the service folder, such as functionalTest/core-data/UC_xxx.  Is it ok?

On Thu, 23 Jan 2020 at 01:56, Ranganathan, Ramya <ramya.ranganathan@...> wrote:

Hi Cloud,

Thanks for the attachment.

Robin was mentioning in the WG that a PR has been opened for this revised TAF directory structure.

Could you please send me the link for that.

 

Couple of feedback:

  1. I may sound too picky, but according to me, testModule is generic and incomplete in the sense that it may or may not imply all aspects of test case definition.

I would propose testCaseModules as opposed to testModules. In software engineering, a test case is a specification of the inputs, execution conditions, testing procedure, and expected results that define a single test to be executed to achieve a particular software testing objective, such as to exercise a particular program path or to verify compliance with a specific requirement.[

In TAF, each test case is crafted as a module fulfilling all the above conditions which then forms the building block to execute it under various test scenarios.

 

  1. I see Cloud has termed the UC_ only for integration and E2E type of scenarios. I see the point here but just for uniform naming reasons, I am debating what if all the test scenarios can be named UC_.

I will think more about it. Will discuss with cloud on this and get his thoughts.

 

Thanks,

Ramya

 

From: EdgeX-TSC-QA-Test@... <EdgeX-TSC-QA-Test@...> On Behalf Of Cloud Tsai
Sent: Tuesday, January 14, 2020 7:55 AM
To: edgex-tsc-qa-test@...
Subject: [Edgex-tsc-QA-Test] edgex-taf file structure for review

 

Hi all,

 

As the discussion, we will make all kinds of the new tests in edgex-taf repository, including functional test, integration test, performance test, and so on.

The attachment is the proposal of new file structure.  Please help take a look and let us know your thoughts.

Thanks.

 

--

Best Regards,

Cloud Tsai



--
Best Regards,
Cloud Tsai