Quality Reports :: Test Cycle Outcomes & Triage Comparison Report
Steps to generate a Comparison Report?
- Click on the 'Customize Report(s)' link under the report category captioned "Test Cycle Outcomes and Triage Comparison Reports'.
- On the pop-up screen, select or enter the identifier for the 'baseline' test cycle and at least one to up to 4 'target' test cycles that you want to compare to the 'baseline' test cycle. The baseline test cycle is the test cycle that the 'target' test cycles you choose will be compared to. Please note that each 'target' test cycle that you select will be compared to the 'baselines' and not amongst themselves.
- If your project lead, enabled you to inherit the project level settings for the test cycle outcomes and triage comparison reports, you can optionally click on the 'Advanced' link, and then override/deviate from the project level settings. Please click here to learn more about the various options that are available as a project level setting that will determine the following:
- Which worksheets are included in the generated report? Some of the sheets (like 'Flakiness Summary') can be optionally included. For each comparison of a target test cycle to a baseline a separate worksheet can be generated or comparison detail for all test cycles to the baseline can be included within a single worksheet.
- Which columns are included in the worksheets? User Notes, Traceability to the Application Modules / Manual Test Cases / Issues within your Atlassian JIRA account, etc, can be optionally printed as columns within the detail worksheets.
- What attributes are used to compare each of the target test cycle to the baseline test cycle
- Click 'Generate Report' button. That will trigger the generation of the report. You will see a node captioned "Request Date: [mon] [dd] [yyyy] [hh]:[mm]:[ss]" node within the "Test Cycle Outcomes and Triage Comparison Reports" tree, where:
- [mon] is the month
- [dd] is the day of the month
- [yyyy] is the 4 digit year
- [hh] is the hour part of the time of report generation request
- [mm] the minutes part of the time of report generation request, and
- [ss] the seconds part of the time of report generation request.
Until the report is generated you will see a message "Please wait.... Your custom reports are being prepared..."
If you mouse over on the "Request Date: [mon] [dd] [yyyy] [hh]:[mm]:[ss]" node within the "Test Cycle Outcomes and Triage Comparison Reports" tree, you will see details on the baseline test cycle, the target test cycles, who requested the report to be generated, when was the request made, when did the processing complete, etc.,
If you expand the node, you will also be able to see the name of the baseline test cycle and each target test cycle within the hierarchy. The mouse over on each of those test cycles will show additional detail about the test cycle like the start and end time for the test cycles, the execution window, etc.,
- Depending on the number of target test cycles that you have included in the comparison against your baseline test cycle, the number of test runs in each cycle and the configurations for the comparison reports, it may take anywhere from a minute to five minutes for the comparison report to be generated.
Email notification will go out to you when the report generation starts and when it completes. When you receive the email that the generation completed, you can go back to the Analytics (tree view) and look for the report. Alternatively, if you are already doing something else in your domain, you can go back to Analytics (tree view) and check to see if the report generation completed.
- Once the report generation completes, you will see an excel icon to the right of the data time when you initiated the report generation.If you click on the excel icon, the report will download to your computer as an Excel Spreadsheet. The filename for the report downloaded will be of the format:
"TestCycleComparisonReport_Generated_On_[mon]_[dd]_[yyyy]_[hh]_[mm]_[ss]"
An example is "TestCycleComparisonReport_Generated_On_Jan_15_2021_09_34_25".
What is in the Comparison Report?
- A 'Summary' worksheet: The Summary sheet provides high level information about the test cycle and various aggregated counts grouped into various categories.
- One or more 'Outcomes & Triage Detail' worksheets, depending on the project configuration for this report. See a sample below.
- Optionally, a 'Flakiness Summary' worksheet, depending on the project configuration for this report.
