Test Cycle Test Outcomes Consolidated Triage Report
This report will help you with insights about the issues created/linked during the executions of the tests that are part of the Test Cycle. This report will also give you the information about how the 'capacity' is being utilized for the tests which are auto re-executed using the Worksoft AI/ML feature.
This is a very useful report which gives you information about the "Annotations" created for the issues along with the issue detail.
This report contains the 5 sheets which are detailed below.
- Auto_Triage_summary: This sheet details the summary about the number of tests executed within the specific test cycles, assigned labels information along with the links for redirecting to the other sheets available within the report.
- Test_Cycle_Level_Net_New_Issues: This sheet gives you the information about the new issues that are created for the Test Runs tagged to that specific Test Cycle.
For example. you have executed a batch of tests that are part of the Test Cycle and have logged the new issues for the failed tests. The issues which were created for the tests that are part of the current Test Cycle will be shown in this sheet.
Note: If you generate the Test Cycle Test Outcomes Report before triaging the failures, you see this sheet as empty. You need to regenerate the Test Cycle Test Outcomes Reports after creating the issues for the failed test runs under the current test cycle. - Test_Cycle_Level_Unique_Issues: This sheet details about the local issues that are either created or linked to the failed tests of the current Test Cycle.
For example, you have observed the failed tests in the current Test Cycle and the failures occurred are similar to the existing issues and hence, you have linked the failed test runs to those issues. In such a scenario, you can view all the linked issues information which were part of the different Test Cycles captured in this sheet. - TC_Level_Failed_Runs_Gap_Issues: This sheet details you about the failed runs and their execution details under that specific test cycle along with the match accuracy % or the RCF and Issues assignment made by Worksoft AI/ML.
- Flaky_Test_Auto_Reruns_Detail: This sheet gives you the complete information about the flaky tests which are auto-triggered using the Worksoft AI/ML.
To know more about the auto re-execution of the Test Run Failures, click here.
You can view the below-mentioned parameters information in the Issues related sheets (1st and 2nd sheet):
- Environment: Under which, the tests are executed.
- Issue Creation Info (Originating Test Cycle, Key(s) and Key Dates): Under this section, you can view the information about the Test Cycle under which the issue is created and the Issue key details.
- Issue Summary & Issue Detail: This column details the Summary & the description of the issues under this column.
- Annotation(s): The annotations added to the issues will be shown under this column.
- Total # of Impacted Test Runs: Under this column, the count of the number of tests that were linked to the respective issue part of the specific test cycle
- Root Cause for Failure: Labels tagged to the failed tests will be shown under this column
- Issue Traceability to Manual Test Case(s) & Application Modules: These two columns indicate the number of manual test cases and application modules labels are assigned in total to the issue tagged to the multiple runs.
- Impacted Testing Platforms: The OS on which the tests got executed are shown under this column.
- Test Runs: The Test Run Ids viewed under this column which upon clicking can be redirected to the respective Test Run detail in the Worksoft SaaS application.
- Unique Run Definitions Impacted: Run Definition names would be shown under this column.
- Worksoft AI/ML Match Accuracy (%) for Issue Linkage to the Run:
- Root Cause of Failure assigned to Test Runs: The RCF labels assigned to the tests can be viewed under this column.
- Worksoft AI/ML Match Accuracy (%) for Root Cause of Failure assignment to the Run: Under this column, you can view the match accuracy % of the RCF labels assigned by Worksoft AI/ML.
- Test Run Traceability to Manual Test Case(s) & Application Module(s): These columns show the name of the labels which are created as manual test cases and application module labels.
This report will help you discover how well your tests cover all the above-mentioned parameters.