Skip to content

Data Definition: Test Cycle Runner Inputs

This data definition specifies the structure for the data file(s) that will contain the inputs to the "Worksoft SaaS Test Cycle Runner" component that is responsible to create test cycle(s) and schedule one or more "Application Test Drivers", which in turn will schedule the application tests that will get run within the newly created test cycle(s).

Attribute Purpose / Description Example Input
execution
_controls
Optional.

If no value passed, default is "all"

This attribute controls the "level" at which information will be logged into the log file that will automatically get created by Worksoft SaaS and saved within the "Files" module of your project when the Test Cycle Runner gets executed. This file will be available for you to download for a maximum of 7 days.

The options for this parameter are "error" and "all". Application test drivers that experience errors will be logged if you set the value of this parameter to "error". Both Application Test Drivers that experience errors and successful executions will logged if you choose a value of "all". If you don't pass a value to this attribute, the default value of "all" will be applied automatically.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "logLevel": ["all" | "error"]
}
execution
_indicator
Mandatory

This attribute controls whether or not a particular row gets picked up from a data file for scheduling by the Test Cycle Runner.

The options for this parameter are "Yes" and "No". Rows that have a value of "No" will not get picked by the Test Cycle Runner. This is mandatory parameter that requires you to populate a value ("Yes" or "No") for every row in the data file.

test_cycle
_info
Mandatory

This attribute can be used the specify the information that will be used by the Test Cycle Runner to create the test cycle.

This is mandatory parameter that requires you to populate a JSON value for every row in the data file.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "testCycleIdentifier": "",
  "testCycleDescription": "",
  "plannedWindowStartTime": "dd-mon-yyyy hh:mi",
  "plannedWindowEndTime": "dd-mon-yyyy hh:mi",
  "doNotAbortEntireScheduleIfValidationErrorsImpactSomeRuns": "true"
}

Please note:

  • The parameter "testCycleDescription" is optional.

  • You must populate a value for the "plannedWindowStartTime' and "plannedWindowEndTime" in the format "dd-mon-yyyy hh:mi". Both are required. The end time has to be later than the start time. Your profile timezone will be considered while schedulding the tests

  • The parameter "doNotAbortEntireScheduleIfValidationErrorsImpactSomeRuns", if set to "true", can be used to inform Worksoft SaaS to not abort the test cycle runner in case some of the application test drivers scheduled by the test cycle runner experience validation/other failures during scheduling. optional. If you feed a value of "false", if any of the application test drivers scheduled experience validation/other failures, the entire batch of application test drivers will be skipped from scheduling. This is optional parameter and default value is "true"
test_cycle
_asset
_capture
_overrides
Optional

This attribute allows you to OPTIONALLY pass 'override' values for "Execution Asset Log Capture modes" for all the Application Test Runs that get executed by the Test Cycle Runner.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "screenshotLogCaptureMode": ["01"|"11"|"12"|"21"|"22"],
  "videoLogCaptureMode": ["00"|"01"],
  "webDriverLogCaptureMode": ["00"|"01"],
  "cloudLogCaptureMode": ["00"|"01"],
  "networkLogCaptureMode": ["00"|"01"],
  "consoleLogCaptureMode": ["00"|"11"|"12"|"13"|"14"]
}

The valid options for the "screenshotLogCaptureMode" are:

  • 00 - Off (None)
  • 11 - All Instructions - Real Time
  • 12 - All Instructions - Post Run
  • 21 - On Error Only - Real Time
  • 22 - On Error Only - Post Run

The valid options for the "videoLogCaptureMode" are:

  • 00 - Off
  • 01 - On

The valid options for the "webDriverLogCaptureMode" are:

  • 00 - Off
  • 01 - On

The valid options for the "cloudLogCaptureMode" are:

  • 00 - Off
  • 01 - On

The valid options for the "networkLogCaptureMode" are:

  • 00 - Off
  • 01 - On

The valid options for the "consoleLogCaptureMode" are:

  • 00 - Disable
  • 11 - Errors
  • 12 - Warnings
  • 13 - Info
  • 14 - Verbose
test_cycle
_label
_overrides
Optional

This attribute allows you to OPTIONALLY pass 'override' values for Labels for all the Application Test Runs that get executed by the Test Cycle Runner.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "PARENT_LABEL_NAME": [
    "labelNameToBeUsedForOverridingAnyPreviousLabelsAssigned"
  ]
}

An example value if you want to override the currently assigned label (value) within the parent label "Test Case" with a new value of label "User Places an order greater than $1000" is as follows:

{
  "Test Case": [
    "User places an order greater than $1000"
  ]
}

More than one label can be passed as the value for the override within each parent label. In other words, you can override with labels A, B within a parent label "x" and with labels C, D, E within another parent label "y".

test_cycle
_failed_test
_triage
_overrides
Optional

This attribute can be used the specify if and how automatic triage is performed by Worksoft SaaS for all the Application Test Runs that get executed by the Test Cycle Runner if its outcome of the test runs is 'Completed w/ Failures' or 'System Aborted' or 'Aborted'. Automatic triage involves the assignment of a root cause of failure label(s) and the marking of the test run to not show on reports generated for the test cycle within which the test executes.

This is an optional parameter that does NOT require you to populate a JSON value for every row in the data file. if you don't feed in a value for a specific row, then the Application Test Runs scheduled for that row will NOT be auto-triaged by Worksoft SaaS.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "autoTriageFailedRunsIndicator": ["true"|"false"],
  "maxTimesToRerunAllFlakyTests": ["0"|"1"|"2"|"3"],
  "holdFlakyTestAutoRerunUntillAllOriginalTestsCompletes": ["true"|"false"],
  "considerAnyFailedTestWithNoRootCauseForFailureAsFlakyIndicator": ["true"|"false"],
  "considerAnyFailedTestWithNonFlakinessRootCauseForFailureAsFlakyIndicator": ["true"|"false"],
  "includeUserDefinedVariablesForWorksoftAIMLOverrides": ["true"|"false"]
}

Please note:

  • The parameter "autoTriageFailedRunsIndicator" controls whether or not auto triage must be done by Worksoft SaaS for runs with failed outcomes. It is optional (equivalent to you passing a value of "false"). Note that the project lead has to enable the passage of a value of 'false' for this parameter at the project/user level on "My Project Settings" screen.

  • The parameter "maxTimesToRerunAllFlakyTests" controls the number of times automatic re-execution is performed for a test run if its outcome is 'Completed w/ Failure' or 'System Aborted' or 'Aborted'. The acceptable values for this parameter are: 0,1,2,3. Defaults to "2" if you do not pass a value into this parameter.

  • The parameter "holdFlakyTestAutoRerunUntillAllOriginalTestsCompletes" controls whether or not the automatic re-execution of flaky tests, if any, by Worksoft AI/ML should be delayed (held) until all the original tests scheduled as part of the test cycle complete their execution. This parameter accepts 'true' or 'false' as values with the former being the default. Stated differently, by default, all automatic re-executions of flaky tests, if any, by Worksoft AI/ML will held up until after all original tests scheduled as part of the test cycle have their execution completed.

  • The parameter "considerAnyFailedTestWithNoRootCauseForFailureAsFlakyIndicator" when set will expand the scope of automatic re-executions to any latest execution of a test that have ended in 'Completed w/ failures', 'System Aborted' and 'Aborted' that have the following conditions: (a) Root Cause for Failure IS one of the flakiness labels (Application Flakiness, Worksoft Flakiness) (currently this is implicitly implemented for the 2 parameters listed above) (b) No Root Cause of Failure IS assigned by Worksoft AI/ML or humans so far. This parameter accepts 'true' or 'false' as values. Defaults to 'false' if you do not pass a value into this parameter.

  • The parameter "considerAnyFailedTestWithNonFlakinessRootCauseForFailureAsFlakyIndicator" when set will expand the scope of automatic re-executions to any latest execution of a test that have ended in 'Completed w/ failures', 'System Aborted' and 'Aborted' that have the following conditions: (a) Root Cause for Failure is NOT one of the flakiness labels (Application Flakiness, Worksoft SaaS Flakiness). This parameter accepts 'true' or 'false' as values. Defaults to 'false' if you do not pass a value into this parameter.

  • The parameter "includeUserDefinedVariablesForWorksoftAIMLOverrides" allows you to include User-Defined Variables in the automatic triage of failed test runs performed by Worksoft AI/ML. Only the User-Defined Variables that are either used for Dynamic Binding of Data Files to Run Definitions or those for which values are overridden via QaCONNECT REST API will be considered. It defaults to 'False' if you do not pass a value into this parameter.
test_cycle
_timings
Required for the first schedule.

It should not be passed for subsequent schedules.

This attribute can be used the specify the information that will be used by the Test Cycle Runner to create and schedule application test runs for more than one test cycle in one shot. For example, you may want to schedule the once a week full regression test cycle for the next 4 weeks or your daily smoke test for the next 7 days, etc.,

If a value is passed into this column, the identifiers for each of the test cycles created will be dynamically derived using the value of the parameter "testCycleIdentifier" in value for the attribute "test_cycle_info" and appending a date and the frequency keyword string of "Daily" or "Weekly" or "Monthly". This is done so that the requirement in Worksoft SaaS that each Test Cycle Identifier must be unique within a project.

This is an optional but conditionally required parameter that does NOT require you to populate a JSON value for every row in the data file. It is ONLY required in case you want to create test cycles for the next N days or weeks or months in one shot. If you are using the scheduler to trigger re-execution of tests within a previously created test cycle(s), you must NOT pass a value to this attribute.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "executionFrequencyKeywordToBeAppendedToTestCycleIdentifier": [
    "Hourly"|"Daily"|"Weekly"|"Monthly"],
  "numberOfTestCyclesToBeAutoCreatedBasedOnExecutionFrequencyKeyword": "AN_INTEGER_COUNT"
}

Please note:

  • You must populate a value for both the parameters "executionFrequencyKeywordToBeAppendedToTestCycleIdentifier" and "numberOfTestCyclesToBeAutoCreatedBasedOnExecutionFrequencyKeyword" if you pass any value for this attribute.

  • The value you pass for the parameter "numberOfTestCyclesToBeAutoCreatedBasedOnExecutionFrequencyKeyword' has to be an integer greater than 1 but less than 30.
execution
_schedule
_params
Optional.

For the first request within a cycle, do not pass a value into the "startTimeFor
Schedule
Execution" parameter.

Worksoft SaaS will use the value of the "plannedWindow
StartTime" as the schedule start time.

For schedules beyond the first one, you are expected to pass a value for the parameter "startTimeFor
Schedule
Execution".

Otherwise, the current (system) time will be used as the start time for the new schedule.

This attribute can be used the specify the information that will be used by the Test Cycle Runner to (a) get the application test runs scheduled by the Test Cycle Runner to be executed under another User that is specified by the "userAlias" and (b) know the start date and time for the new schedule of tests within a previously created test cycle.

This is an optional but conditionally required parameter that does NOT require you to populate a JSON value for every row in the data file. For the first request within a test cycle, do not pass a value into the "startTimeForScheduleExecution" parameter. Worksoft SaaS will use the value of the "plannedWindowStartTime" as the schedule start time. For any schedule beyond the first one, you are expected to pass a value for the parameter "startTimeForScheduleExecution". Otherwise, the current (system) time will be used as the start time for the new schedule.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "startTimeForScheduleExecution": "dd-mon-yyyy hh:mi",
  "userAliasToScheduleAllRunsInTheSchedule": "USER_ALIAS"
}

Please note:

  • The value you pass into the parameter "startTimeForScheduleExecution" must be in the format "dd-mon-yyyy hh:mi"
test_cycle
_notification
_request
Optional

This attribute can be used optionally, if the user would to receive notifications upon completion of execution of test runs in the test cycle. Worksoft SaaS supports two kinds of notifications i.e. email & slack. By using the parameters in the JSON, user can provide list of email addresses the notification is to be sent to.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "notificationRecipientsForEmail": [
    "user1@yourEmailDomain.com",
    "user2@yourEmailDomain.com"
  ],
  "executionStatusNotificationMethods": ["01"|"02"]
}

Please note:

  • The parameter "executionStatusNotificationMethods" can be used to specify how the notifications will be sent out by Worksoft SaaS in the context of the tests scheduled using the Test Cycle Runner. Currently Email and Slack options are supported for this parameter. Valid options are '01' for 'Email' and '02' for 'Slack'. No notifications are sent if the parameter is not specified.

  • The parameter "notificationRecipientsForEmail" can be used to specify the list of email recipients to whom the mail notifications will be sent by Worksoft SaaS in the context of the tests scheduled using the Test Cycle Runner. If this field is left blank, email will be sent to the email address that corresponds to the user account that is used to make a request to the QaCONNECT REST service.

  • When '01' is specified in "executionStatusNotificationMethods" it is mandatory to pass "notificationRecipientsForEmail"
datafile_with
_runs_to_be
_scheduled
Mandatory

This attribute can be used the feed the names of the Data Files that should be used by the Test Cycle Runner to get the list of application test drivers to be executed that in turn will schedule the application test runs.

This is a mandatory parameter that you are required to populate a JSON value for every row in the data file. At least one data file muse be passed as value.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

[
  {
    "dataFileName": "NAME_OF_YOUR_DATA_FILE",
    "dataFileStatus": "STATUS_OF_YOUR_DATA_FILE",
    "dataFilterName": "NAME_OF_YOUR_DATAFILTER_TO_BE_APPLIED_TO_THE_DATA_FILE",
    "userDefinedVariableValueOverrides": [
      {
        "name": "NAME_OF_THE_USER_DEFINED_VARIABLE",
        "value": "VALUE_OF_THE_USER_DEFINED_VARIABLE"
      }
    ]
  }
]

Please note:

  • The status of the Data File can be "Active" or "WIP". If you dont' pass a value for this parameter, a default value of 'Active' is used.

  • The optional parameter "dataFilterName" can be used by you to specify a data filter, if any, that you would like Worksoft SaaS to use to filter a subset of records from the Data File.

  • Optionally, you can pass the override values for any user-defined variables used, if any, within the data filters along with the data filter name.
report
_generation
Optional

This attribute can be used the specify information that will indicate to Worksoft SaaS if clipboard reports and the test cycle hierarchy reports must be generated automatically once all the application test runs scheduled within the Test Cycle(s) complete their execution.

This is an optional parameter that you are NOT required to populate a JSON value for every row in the data file.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

{
  "clipboardReportGeneration": ["true"|"false"],
  "testcycleReportGeneration": ["true"|"false"]
}

Please note:

  • The default value for both parameters is "false" if you don't pass a value.
inputs_for
_test_cycle
_clipboard
_generation
Optional

This attribute can be used the configure how Worksoft SaaS should consolidate data from clipboards (that are populated by automated test runs with data from run-level datasets using the command 'ddPersistDatasetToTestCycleClipboard') which are scoped at the level of a test run id (within a test cycle) to 'output' that is at the level of "test cycle".

The output is created as an analytics report that appears as an Excel file within the 'Quality' pane of Analytics module within your Worksoft SaaS app OR as a data file (that appears within the 'Data Files' module). The naming convention for the consolidated output file(s) is as follows:

"testCycleIdentifier_nameOfDataDefinitionOfDataset_dateTimestamp", where "dataTimestamp" is a unique identifier that is of the format "ddmmyyhhmmssSSS" [example: "31072018061327021" that represents 31st day of 7th month of year 2018 at 06 hrs 13 minutes 27 seconds and 21 milliseconds].

This is an optional parameter that you are NOT required to populate a JSON value for every row in the data file.

Use the following JSON format when you populate data into the column of values for this attribute within the data files:

Please include here the JSON request for the QaCONNECT REST API service: 

https://www.web.worksoft.cloud/rest/v1/:domainname/executions/create-consolidated-output-from-clipboards

Example Input JSON:

{
  "consolidatedOutputType": "R",
  "dataDefinitions": [
    {
      "name": "YOUR_DATADEFINITION_NAME",
      "status": "YOUR_DATADEFINITION_STATUS"
    },
    {
      "name": "YOUR_DATADEFINITION_NAME1",
      "status": "YOUR_DATADEFINITION_STATUS1"
    }
  ]
}

Please note:

  • The parameter "consolidatedOutputType"must be passed a value of 'R' if you want the consolidated output to appear as a 'report' within the 'Quality' pane of the 'Analytics' module of the Worksoft SaaS application. Instead, if you want the consolidated output to appear as a 'data file' within the 'Data Files' module of the Worksoft SaaS application, pass a value of 'D'. Please note that if you pass a value of 'D', the consolidated output will be truncated to the top 5000 rows if the consolidation across clipboards results in rows that exceed 5000, the max number of data rows currently allowed in a data file. When you pass an 'R', no such truncation will occur.

  • The parameter "dataDefinitions" can be used to specify the list of Data Definitions along with their status that you want to limit the aggregation of data from the clip boards populated by the application test runs within the test cycle. The status of the Data Definition passed as input to the above mentioned parameter accepts as values 'WIP' and 'Active'. The value defaults to 'WIP' if you pass a value into this parameter.
developer
_notes
Optional This attribute can be optionally used by you to add a mnemonic kind of free form text that will help you easily remember the purpose behind each row in the data file that feeds into the Test Cycle Runner.

Feedback and Knowledge Base