Understanding Loadtest Results
Last updated
Was this helpful?
Last updated
Was this helpful?
The Results view is accessible for completed jobs and provides access to all information collected during a test run. It also allows you to compare different instances with each other:
The Result Details tab contains a number of charts which provide metrics related to the performance of the Load Test. It displays the metrics for a specific Test Result within the Test Instance you are viewing. It contains a header which displays the Concurrent Users which were specified when the test was configured. It also displays the start and end times of the Test. Select the Test Result to view by selecting a number of users from the “Users” dropdown; the metrics displayed on the charts are specific to the test run which was selected. The charts can be manipulated to show or hide additional data, printed, or expanded to provide a better view of the chart; see Understanding Load Test Result Charts for more details.
The following screenshot shows metrics for a Test Result which ran a .class file with 2 users (the .class file name has been obfuscated). Response times can be compared to the number of Active Users - virtual users who have started but not yet finished the Test.
The Pages & URLs tab shows detailed data about URL and page response times, and provides access to response time breakdowns.
This page contains detailed results for each individual URL call in the test.
The Instance section shows basic information about the test.
Item
Description
Concurrent Users
Test Sequence Times
Start and end for the test sequence run.
Create Quick PDF Report
In the Results section you can select which run to display in the charts and tables. Each run in the instance is indicated by a tab showing the number of users.
By default, information about all runs is displayed:
To select a specific run, click the tab with the desired number of users:
The table corresponding to the run is shown in the Pages & Urls section.
Errors show detailed information about any possible errors if any have occurred.
The Instance section shows basic information about the test.
Item
Description
Concurrent Users
Test Sequence Times
Start and end for the test sequence run.
Create Quick PDF Report
The Error Overview table shows the number of errors per page and test.
Each page is a row in the table, with a column for each test.
This view allows you to see which tests contain errors and select them for display in the details chart and table.
The Errors Over Time chart shows error occurrences during the test for the selected location.
The chart shows the number of errors during the test for the selected test.
Select Test
To display the chart, you need to select the test in the dropdown menu.
The Live Errors Feed table shows information about the last recorded errors chart.
Column
Description
Page
Page order number and name in scenario.
URL
URL Index
Error number relative to the URL.
Time Offset
Time when the error occured, relative to the start of test.
Error Type
Actions
Buttons for possible actions to perform on the error.
Actions
Column
Action
Description
Details
The Error Details dialog provides access to a number of tabs containing detailed information relating to the error.
The Summary section shows the received content of the response.
Row
Description
Page
Page order number and name in scenario.
Error Data
Time when error occured: Timestamp, time after start (number of seconds, and converted time).
URL
URL Exec Step
Execution step when error occured.
Exec Agent Socket Create Time
Creation time for the execution agent connection.
TCP Connection Reuse Count
Number of reuses of the TCP connection.
TCP Local Address
TCP local IP address.
TCP Local Port
Local port for TCP connection.
Remote TCP Address
TCP remote address.
TCP Remote Port
Remote port for TCP connection.
SSL Session Id
Error Log
Scrollable log view. Clicking the header opens the full log in a separate browser window.
The log in the Summary section can be viewed in a separate browser tab.
In the Request Headers tab, information about the request preceding the error is displayed.
The table displays the HTTP Methods and URL associated with the error at the top. Following that is a table containing all headers sent with the request.
The Request Content tab shows the sent content of the request if any exists.
The Response Headers tab shows the received response headers.
The table displays the HTTP Status Codes and text error at the top.
Following that is a table containing all headers provided in the response.
The Response Content tab, shows the received content of the response, if any.
To view the content in a browser, click “Preview Response Content”. Your default browser opens to display the content.
Depending on which integrations you have enabled, various integration tabs may be available as part of the results view.
The Transactions tab presents information about User Defined Transactions.
The Transactions Summary table shows an overview of the transactions for a selected result.
With the All Transaction Times Percentile chart, you get a statistical view of how the transaction times are distributed.
The All Transaction Times chart shows the transaction times for the selected transaction.
In the Full Passed Transaction Times Percentile chart, you get a statistical view of how the transaction times are distributed for the passed transactions.
The Settings Used tab shows a summary of all settings used for a load test.
The Instance section shows basic information about the test.
Preview unavailable
Concurrent Users
Test Sequence Times
Start and end for the test sequence run.
Create Quick PDF Report
In the Results section, you can select which run to display.
To select a specific result set:
Click the tab with the desired number of users
The Settings Used tab shows a summary of all settings used for a load test.
Item
Description
Scenario
Scenario
Scenario used in the job.
Loadtest Options
Number of Concurrent Users
Loadtest Duration
Total duration of the test.
Rampup Time
Run Multiple Sequential Tests
Location
Cluster
Scenario Options
User input Text Files
User Defined Vars
Variables defined in the scenario.
Advanced Options
Max Loops Per User
Request Timeout (sec)
Time to wait for responses.
Additional Options
Distribute Load on All Available Datacenters
Client Options
User Agent
Network Bandwidth Downlink (Mbit/s):
Desktop
Limit on downloaded desktop traffic.
Mobile
Limit on downloaded mobile traffic.
Network Bandwidth Uplink (Mbit/s):
Desktop
Limit on uploaded desktop traffic.
Mobile
Limit on uploaded mobile traffic.
Client Side Monitoring
DNS
DNS Hosts file
DNS Server
Resolve DNS for Each Executed Loop
DNS Translation file
Reporting
Email Preliminary Report To
Address for test reports.
Reporting History
Limit on number of tests to include in test history.
Execution
Schedule
Type of scheduling for the job.
Schedule Date
Date for schedule.
Test Information
Attached to Project
Project the test belongs to.
Job Comment
Comments added to the job settings.
Tags
Scenario Tags applied to the scenario.
If the job uses Client Side Monitoring, this tab shows real browser rendering times corresponding to the behavior of a real user accessing the application during the load test.
You will be able to see the total Response Time measured from a real browser for each separate run on this page.
The Instance section shows basic information about the test:
Item
Description
Concurrent Users
Test Sequence Times
Start and end for the test sequence run.
Create Quick PDF Report
In the Results section, you can select which run to display in the charts and tables.
To select a specific result set:
Click the tab with the desired number of users
For each location, a Location Overview chart displays a graph containing the page response times and the number of URL errors for all runs.
Select the desired location, e.g. Los Angeles, from the dropdown menu to pick which location to show in the chart.
The chart shows the page response times and the number of URL errors overall runs for the selected location.
The Run Details shows information about the run and provides access to related screenshots.
The run details contain a timestamp for when the test started (with offset showing time elapsed from the start of test in parentheses).
If screenshots are available, a link to open them for viewing is also shown.
To see the screenshots:
Click the View Screenshot link.
The screenshots are displayed as an overlay on your browser:
The snapshots provide a browser view of the page at the time the URL occurred.
Navigate
When you hover the mouse over the left or right side of the snapshot, navigation buttons are shown, allowing you to browse through multiple snapshots, if available.
The Select Run dropdown menu allows you to pick which test run to show in the run Run Details and Pages sections
The URL Errors table shows any errors that have occurred for the page during the test.
Each URL is displayed as a row in the table.
Column
Description
#
ID number.
Code
Error code, if any.
URL
URL associated with the error.
Time
Time elapsed from the start of the test.
Error
Error message.
The Pages table(s) shows information about the page(s) in the test and the URLs called for each page during the test run.
Each page is displayed in a table, with URLs displayed in rows:
Column
Description
Comment
URL
Called URL.
Time
Time until received response.
Size
Response size.
More Information
Show more information about the URL.
0s, 1s, 2s….
Graphical representation of the response times.
More information about the response is available for each URL. To see more information about a particular URL, click the “See More” link. A popup containing details about the URL is shown:
Item
Description
Mime Type
HTTP Response
Error Message
Returned error message (if the URL is associated with an error).
To see the timing details about a particular URL:
Click the representation for the desired URL
A popup containing details about the URL is shown:
Information about the HTTP call is shown at the top.
Item
Description
HTTP Response
Error Message
Returned error message (if the URL is associated with an error).
The Response Time Details section shows a graphical representation of the flow and a legend with times.
The Total Response Time is displayed as a summary of the detailed view.
Number of .
Button to form the results.
Number of .
Button to form the results.
and URL associated with the error.
and message for the error.
Open the dialog.
and URL associated with the error.
Session ID for connection.
Number of .
Button to the form results.
Number of in the test.
Time to .
Used .
Location used to generate load.
Connected .
Maximum number of per .
Additional provided as part of the job.
Enabled
Client .
Enabled .
Custom .
Custom used for lookups.
Whether lookup is performed for each .
Custom translation file used.
Number of .
Button to the form results.
Click to see about the response.
Click to see .
for the response message.
Response .
Response .
