Testing Contexts - Basic Options
Before you read this article see :
Main Article
The Basic Options let you specify the following:
- One of the environments where your web/mobile application is deployed at or is accessible from.
- The platform or device on using which you want to run the test. This will include cloud platform you want to use to run your test, the browser/version, operating system/version, device/version, screen orientation [Landscape/Portrait], screen resolution, etc
- The level of visual and/or textual logs (Screenshots, Videos, Selenium/WebDriver/appium/Selendroid Logs, Cloud Platform Logs, etc) you want Worksoft SaaS to generate when your Test Runs are executed.
Environment:
One of the environments where your web/mobile application is deployed at or accessible from that you created in 'Environments' screen accessible from 'More' >> 'Environments' menu. This drop-down lists out all the available Environments that you have configured in Worksoft SaaS.
Device Settings:
This section allows you to specify:
- Cloud Platform: The cloud on which you want your test to get executed. Default platform is 'Worksoft SaaS Cloud'. If you want to use your existing account with one of the external (3rd party) cloud platforms (like Sauce Labs), and have registered such an account within you Worksoft SaaS domain, you can also choose a cloud platform different than 'Worksoft SaaS Cloud' using the drop-down.
- Device Type: You can choose 'Desktop' or 'Mobile Tablet' or 'Mobile Phone'. Based on the Device type that you select, further options change. When you choose Device Type 'Mobile Tablet' or 'Mobile Phone', you need to specify:
- Device Orientation: Landscape or Portrait
- Device: You can pick the actual device you want to run your tests on (example: Apple-iPad-Pro)
- Operating System: The options in the drop-down will be auto-filtered based on the 'Device Type' you choose.
- Browser: The options in the drop-down will be auto-filtered based on the 'OS' you choose.
- Screen Resolution: This is only applicable for Device Type 'Desktop'.
Execution Logs Settings:
The Execution Log Settings consist of Screen capture, Video Capture, and Selenium/WebDriver Log Capture common across all the Cloud Platforms.
- Screen Capture: Image screenshots will be captured if you choose one of the many options that are available which can be viewed and downloaded.
- First you have to decide if you want screenshots taken for "All Instructions" within your test whether they pass or fail OR only for those Instructions in your test that fail ("On Error"). Screenshots on Error Only consume less minutes (obviously) from the "Daily Capacity" available within your Worksoft SaaS domain.
- Second, you have to decide when you want screenshots available to you. You can choose from 'Real-Time' or 'Post-Run' or 'Time-Lagged'.
- 'Real-Time' means screenshot is available immediately when an Instruction completes execution (consumes most minutes from your 'Daily Capacity').
- 'Post-Run' means screenshots are available ONLY after the entire test completes execution (consumes least minutes from your 'Daily Capacity').
- 'Time-Lagged' means screenshots are available for a set of instructions that completed execution with a time lag (Minutes consumed from your 'Daily Capacity' is usually between the minutes consumed for the other 2 options). This is enabled only for Cloud Platform 'BrowserStack'.
- Video Capture: If you enable this option, Video will be captured of your test in execution and made available to you a few minutes after the test execution completes. If you choose this option, your test execution will take more time and hence more minutes are consumed from your 'Daily Capacity' that comes with your Worksoft SaaS subscription level. This option is only available when you choose Cloud Platform to be either 'Worksoft SaaS Cloud' or 'Sauce Labs'.
- Selenium/WebDriver Log Capture: If you are familiar with raw Selenium (or Selendroid or Appium) runtime standard output logs, you can turn this option on, and once the test completes the log will be made available to you.
However,you may see the additional drop-down(s) that which would help with more information. These more logs get enabled based on the Cloud Platform that you select.
When you choose "Cloud Platform" as "Worksoft SaaS Cloud"(Default Cloud Platform) or "BrowserStack", You can view Network Log and Console Logs that are captured during the run.
- Network Log Capture: This log provides the response of the network pane in '.har' format from the Developer Tools (F12) in the browser when the application test is being run. This log provides details of all the calls that happen and their status codes along with the response time graphs of each event of every page load during application Test Run.
- Console Log Capture: These are .json logs that are generated in the console pane of the Developer Tools Pane(F12) of the browser during the Test Run of the Application Under Test(AUT). These contain warning and error logs that appear in the console for every URL visited along with the time stamp.
When you choose "Cloud Platform" as "Sauce Labs", you have a log which is exclusively accessible for Sauce Labs i.e, Cloud Logs Capture. Also, you can also select Network Log and Console Log Capture for this cloud provider.
- Cloud Log Capture: This log is only available for those of you that use your own account at external (3rd party) cloud platform account (like for example at Sauce Labs) to execute your test. Worksoft SaaS can automatically download the Sauce Labs log from your account there and make it available within your Worksoft SaaS domain.
For Performance Testing- the cloud platform is Worksoft SaaS Cloud where the Test Runs are executed on the headless browser. A drop-down to select "VNC Viewer" is visible along with the Video Screen Resolution option.
When you have more than one secure tunnel to access your application, Worksoft SaaS also allows you to specify which of those tunnel you want Worksoft SaaS to use to access your application when executing your tests. Please refer to the article "Secure Cloud Tunnels" for more details.
Worksoft SaaS QaCONNECT REST API also allows you to deviate from/override the Secure Cloud Tunnel that is within the Testing Context when scheduling/triggering test executions remotely. This is a powerful feature that will let you configure ‘Tunnel A’ as the ‘default’ within Worksoft SaaS (after you create other tunnels for the same application/environment combination) but when executing/triggering tests remotely using QaCONNECT REST API, you can indicate to Worksoft SaaS for ‘Tunnel B’ to be used instead (all without you editing the Testing Context within Worksoft SaaS). Please refer to the article on 'QaCONNECT'.
Time is Precious – Tips for Efficient Use:
To help you efficiently use the minutes you have within your domain's "Daily Capacity" each day, here are a few tips:
- Avoid “Real Time” screenshots for “All Instructions" unless you really need it.This significantly increases your overall execution time.
- The “Video Capture” feature adds approximately 30% more time to your execution. So, use it sparingly and when needed.
- We strongly recommend “Post Run Completion” option over “Real Time” option because of the execution time savings.
- For example, if you schedule your run definitions in the overnight hours in an automated fashion, driven from your Continuous Integration (CI) tools, it may not make a big difference to you, if your screenshots are available real-time or a few minutes after the test execution is completed BUT will save quite a few minutes from your available capacity for a particular day.
- As a rule of thumb, we recommend the use of “All Instructions – Real Time” option only for Scenario Validations or Run Definitions that you initiate for “debugging” purposes.