Analytics: Administration Level
Before you read this you may want to see:
Administration level reports helps the Worksoft SaaS Administrators to understand the application behavior and Quality at a high level based on the Test Executions.There are two categories
- Quality
- Subscription Usage
Quality:
The Test Execution Outcomes reports will provide rich analytics about the Quality Assurance initiatives up and down the chain of command.
Using this report you can gain a greater understanding of how quality assurance efforts are operating and to what extent they are actually succeeding.
Specifically, this report will help you visualize/understand how the test outcomes vary by application release/iterations across numerous dimensions like project environment, browser/operating system platform, testing purpose, etc.,
This rich analytics can be accessed in various ways across numerous dimensions:
Worksoft SaaS Subscriptions are based on Capacity and Concurrency. These reports will help you understand better on how the current subscription is helping you with your test runs.
This report will help you with insights into how you are utilizing the 'capacity' from your Worksoft SaaS Subscription across projects and users within your company.
This rich analytics can be accessed in various ways across numerous dimensions:
- At Company (Account-level) and Project levels
- For a specific Calendar Time Period
- For a specific Testing Purpose (Smoke, Regression, etc)
- By Automation State
- By Entity Version - Active/WIP
- By Browser and Operating System Platforms
- By Duration of Test Execution
- By Percentage Of Passed Test Instructions
- By Count of Test Run(s)
Subscription Usage:
Worksoft SaaS Subscriptions are based on Capacity and Concurrency. These reports will help you understand better on how the current subscription is helping you with your test runs.
Capacity Utilization
This report will help you with insights into how you are utilizing the 'capacity' from your Worksoft SaaS Subscription across projects and users within your company.
'Capacity', as you may know, means the “Maximum” number of “Minutes” of automated test executions/day ['Run Definitions' as well as 'Test Scenarios'] you can perform within your domain.
This report will help you gain answers to the following types of questions:
For example, maybe you can schedule tests for one browser/operating system at a specific time of the day or specific day of the week or specific day of the month and other browser/operating system at some other time window.
This report will help you gain answers to the following types of questions:
- Are you executing more tests on specific time slots of a calendar day, or specific days of the week or specific days of the month than others causing your test cycle to fit into long time windows?
- Is it better for you to break your test cycles differently so that you are optimally utilizing the full value of your subscription's capacity?
For example, maybe you can schedule tests for one browser/operating system at a specific time of the day or specific day of the week or specific day of the month and other browser/operating system at some other time window.
Are your tests experiencing better outcomes if the tests are running in a time window than you are currently running them? You may see impacts of flakiness vary because of some jobs running on your servers where you AUT is hosted (say a backup job) or may be because of the number of manual users of the AUT accessing the app at the same time automation is accessing, etc.
Concurrency Usage
This report will help you with insights into how you are utilizing the 'concurrency' from your Worksoft SaaS Subscription, over a time period, across projects and users within your company.
'Concurrency', as you know, allows you to execute tests in your parallel to reduce the overall time it takes to finish all your tests.
This report will help you gain answers to the following types of questions:
- Are you scheduling your automated tests all at once and causing many of them to get queued up because of non-availabilty of adequate concurrency?
- Is it better for you to more evenly distribute the test execution schedules?
- If you are already optimally using your current concurrency, is it time for you to make changes to your Worksoft SaaS subscription to get higher levels of concurrency?
Subscription Allocation to Projects & Users
This is an excel report that provides details about allocation of capacity & concurrency for different projects & users in your domain. You can use the report to review & plan how to allocate available capacity & concurrency along with execution priority to the users.