Skip to content

Commands for logging and reporting on 'Application Performance'

You can use this set of commands to log the timing metrics about the performance of your application under test, from within your automated tests during their run-time execution. 

Using these commands your tests can capture/log the time taken for key events (like page load, DOM load, etc) or the specific actions to be performed within your test.

At the time of development of your automated tests (specifically the test scripts), you can build logic within your scripts to read from a user-defined variable (or from a data file) the log level that determined how much information you want to be captured/logged from within your automated test during its run-time execution. At the time of execution of the automated tests, you dynamically pass the value to the user-defined variable (or feed value from within a data file bound to a test) that defines the actual amount of timing metrics captured/logged within your test. The log levels supported are DEBUG, INFO, WARN, FATAL and ERROR. 

The commands in this category also allow your tests to persist the timing metrics captured/logged (in a Data Set) to a Data File within your project.

Sometimes you may want to log the cumulative execution time of a block of test instructions within your test as a key event's timing metric you want to report on. Some other times, you may want to get key application performance related timing metric (like page load time, DOM load time, etc., ) using JavaScript functions (using libraries like Resource Timing API that many browsers natively support, that you can get using runScript command within Worksoft SaaS) or using the Worksoft SaaS supported functions by tapping into the 'runEventTimingFunction' command and log the response from those functions as your key event timing metric that you want to report on. In the same test, Worksoft SaaS allows you to log some key event timings using the former approach described above and some other key event timing metrics using the latter approach.

At a high level, here is a flow that you will have to implement to reap the value from the commands in this category.

First, define your dataset using the command 'createDataset'                 

Then,
a) For each block of test instructions that you want the 'cumulative' execution time captured as a key event timing, define a block like the one portrayed below:
# Command Target Value
Timer for instructions block starts
1 timerForTestInstructionsBlockStart
LogLevel Minutes/Seconds/MilliSeconds/Hours
To set an identifier

2

setIdentifierForKeyEvent
IdentifierName (Optional)  
A sequence of instructions to perform various actions
3 Instruction-1

 
4 Instruction-2

 
5 Instruction-n

 
To set the instruction execution timings into a DataSet
6 ddAppendInstructionBlockTimingToDataSet
datasetName attributeName1:${contextString},attributeName2:${contextString},...
Timer for instructions block ends
7 timerForTestInstructionsBlockEnd

 
b) For each key event timing, you want to capture using some open source/commercial 3rd party JavaScript library (like Resource Timing API) or a custom JavaScript function your developed OR using one of the Worksoft SaaS supported JavaScript resource timing functions, define a block like the one portrayed below:
# Command Target Value
Timer for key events block starts
1 timerForKeyEventBlockStart
LogLevel
To set a context for a key event
2 setContextForKeyEvent
IdentifierName  
To capture the event metrics

3
runEventTimingFunction
(OR)
runScript(Some custom javascript function)


functionName||M/S/MS/H


PageLoad,DomLoad,TFB
If "runEventTimingFunction" is used, then the below command is not required

4
setEventTiming
(Default time is in MilliSeconds)
responseTimeInMilliseconds
 
To set the key event timings into a DataSet
5 ddAppendKeyEventTimingToDataSet datasetName
 attributeName1:${value1},attributeName2:${value2},..
Timer for key events block ends
6 timerForKeyEventBlockEnd

Lastly, the key events' timing metrics that are captured using either or both of the approaches described above, can be persisted into 'Data Files' within your project, from within your automated tests in execution. Optionally, if you want, you can additionally log the key event timing values into analytics reports for performance testing using  'echoForReporting' command.

The table(s) below lists the commands in the ascending alphabetic order within each category along with a brief description of the purpose behind the command. By clicking on the hyperlinked command name in the table(s) below, you can review more detailed information about a specific command that includes the syntax for you to follow when using the commands. Please note that these commands (constructs) are supported within Worksoft SaaS application (Edit Test Script screen) only and NOT supported in QaSCRIBE test authoring tool.  
Command Purpose
ddAppendInstructionBlockTimingToDataSet This command appends to a Data Set (previously defined within the automated test) the cumulative execution time of test instructions within the timerForTestInstructionsBlockStart and timerForTestInstructionsBlockEnd block.

In the Data Definition that defines the 'Data Set', it is mandatory to have an 'Attribute' with the name "ResponseTime" so that the cumulative execution time of a block of instructions can get persisted into the column in the 'Data Set' that corresponds to attribute 'ResponseTime' in the data definition.
ddAppendKeyEventTimingToDataSet This command appends to a Data Set (previously defined within the automated test) the key event timing captured by use of either the command 'runScript' followed by the command 'setEventTiming' OR the command 'runEventTimingFunction' within the timerForKeyEventBlockStart and timerForKeyEventBlockEnd block.
runEventTimingFunction This command is used to get the performance timing metric for key events like DOMLoad, PageLoad, etc. using pre-built (Worksoft SaaS natively supported) javascript functions based on the Resource Timing API that many browsers support. If you want to use your own JavaScript functions instead use the command 'runScript' to execute your custom JavaScript function and then log the return value from those functions by using the 'setEventTiming' command.
setContextForKeyEvent This command is used within the timerForKeyEventBlockStart and timerForKeyEventBlockEnd to set the context of the key event.

The context is defined using a name you assign to the key event along with the 'column' in the data set (actually the 'attribute' name within the data definition that defines the data set) you want the timing metric persisted into.
setEventTiming This command is used within the timerForKeyEventBlockStart and timerForKeyEventBlockEnd to log the return value from a javascript function you execute using the 'runScript' command.

The value of the key event time passed into this command must be in Milliseconds.
setIdentifierForKeyEvent This command is used within the timerForTestInstructionsBlockStart and timerForTestInstructionsBlockEnd block to assign an identifier for the key event.

This command is mandatory. We use this command to identify the row in the dataset for which we want to context information captured within this block.
storeElapsedTime This command is used to calculate the elapsed time without using the JavaScript function.
timerForKeyEventBlockStart This command is used to indicate the start of the block.
Please note it is mandatory to have a User Defined Variable with the name "LogLevel".
timerForKeyEventBlockEnd This command is to indicate the end of the block.
timerForTestInstructionsBlockStart This command is mandatory to start the block of commands for calculating the time it took to execute the commands until the timerForTestInstructionsBlockEnd is encountered.
timerForTestInstructionsBlockEnd This command is to indicate the end of the block.

Feedback and Knowledge Base