With this feature, Worksoft SaaS Machine Learning will "implicitly" check if there are other failed test runs in the same test cycle that have the same or closely similar patterns of failure and automatically "recommend" test runs that guide you in choosing other failed test runs in the same test cycle that could benefit from a bulk assignment of triage information (root cause(s) of failure and/or issue linkages) from a source test run.

The recommendation engine will take a little bit of time to come back with recommendations. Each recommendation is color coded. The stronger the recommendation, the darker the green color in a new icon that appears right below the actions menu in the 'Actions' column of the Test Runs grid listing.

How it Works:

1. Before you can use this feature you need to triage a run (henceforth referred as source run) and assign RCF & Issue
2. You can initiate the request either from Test Runs home page or Execution results page of the source run
3. In the Test Runs Home page, you click on the three dots (in actions menu) and then <> to start the process. You also can do this from Execution Results page by clicking on three dots in the <> and then select <> to start the process.
4. In the overlay that shows up you will see either one or two tabs based on whether RCF or Issue or both are assigned to the source run. You will see two tabs if both RCF & Issue are assigned else one of the tab based on which one is assigned





5. Following are the inputs you can choose. Please note at present, even though you have individual tabs for RCF & Issue, system will use the values selected in RCF tab only

• Include Test Runs based on 'Assignment Status' of Root Causes of Failure and the 'User' that assigned the Root Causes:

Scope Selection(1): In this option you select scope of failed test runs that should be considered by the recommendation engine. You are presented with 5 options w/ default being “Test runs for which no Root Cause of Failure/Issue were ever assigned including currently”. This option is the most probable option that you would go with if you have started triage process once the test cycle execution is completed and you have new failures.

• Include Test Runs based on the 'Testing Context' associated with the Test Run

Scope Selection(2): This is part 2 of the selections that determines scope of failed tests to consider by the recommendation engine. There are two options offered and you need to select one before you can proceed. In your project if you use the same RD to execute multiple test cases, first option would be the one you likely go with and if the test cycle is a cross browser/compatibility test cycle then you would choose option 2.

       The next two selections influence the output of the recommendation engine.

• Exclude Test Runs based on the Match Accuracy of the Recommendation. Specify the minimum match accuracy % for a Test Run to be shown within the Worksoft AI/ML Recommendations

When the recommendation engine executes it will either come up w/ a set of runs with match accuracy percentage OR with no runs, in case it can’t find any pattern match. When the engine finds set of runs, the percentage you specify here will be used to filter out runs that have match accuracy less that this percentage and add remaining into “My Selections” tab for you to continue w/ further steps

• Limit of test runs to be recommended by Worksoft AI/ML

Based on the number of runs that are part of the scope and error pattern, there can be lot of runs that can show up. To ensure you get a chance to look at these runs and review those, number of runs that get loaded to “My Selections”, after applying threshold, are limited to the number you specify here. By default, system limits this to 250 runs

Once you submit the request based on the number of failures in the test cycle and the scope options you selected, the process can take a minute or more for the process to complete. If the process is taking more than a minute, you are given an option to wait or cancel the process. 

You can select these options on the popover shown. In addition to the options, you also get to see an Id (Process Id) that you want to save. This would help in case you select to leave the process and return later to check on the recommendations. You can use the Process Id captured in below accordion of Test Runs Search Criteria.

You can select these options on the popover shown. In addition to the options, you also get to see an Id (Process Id) that you want to save. This would help in case you select to leave the process and return later to check on the recommendations. You can use the Process Id captured in below accordion of Test Runs Search Criteria.

If you cancel the process, you can bulk assign RCF/Issue by going the manual route (where you select Runs manually) or can reinitiate the process.

If the request results in no runs found to have the same pattern with match accuracy greater than the threshold specified, you will be shown the same message and will land on the “My Selections” with no runs added to the tab.

If the request returns set of runs, then you land on “My Selections” tab with the runs added. In addition to the regular details (see in “Runs List” tab) you also get to see following additional details

• Match Accuracy. In the first column along w/ remove selection option you also see another icon w/ accuracy inscribed on top of it. This helps you find which ones are closer to the source run in terms of the failure and then take a decision to proceed w/ assigning RCF/Issue or remove them from selection
• In Failure Triage column, you see three icons. They represent whether the values match the source run or not. This is additional info that would help you decide on the reason for failure

With this feature you will get to see additional information in the Test Runs Home page. If you or someone else from your team used this feature and didn’t follow up w/ assigning RCF/issue to the runs, you will see a new icon in the first column.

This icon indicates that the specific run is part of the recommendation engine results. The number on the icon indicates whether the run is match with one source run or more and the colour indicates if the range of match accuracy.

• The higher the shade of green the greater is the match accuracy. If accuracy is < 50% then the colour is lighter green and if the number is > 90% then it tend to be on the darker side.
• If the count is 1 or greater you can click on the icon to see the details. Overlay that shows up has details of the source run along w/ RCF & Issue assigned to it. 

Error Condition

For any reason there is an error while the process is running, you will see following popup

If you go with the first option and that as-well result in similar outcome, you can raise support ticket with Worksoft SaaS team. While raising the ticket provide following details (but not limited to) 

• Source Test RunId
• User who attempted the process
• Date and time when the process is triggered