Understanding Loadtest Results

The Results view is accessible for completed jobs and provides access to all information collected during a test run. It also allows you to compare different instances with each other:

Result Details

The Result Details tab contains a number of charts which provide metrics related to the performance of the Load Test. It displays the metrics for a specific Test Result within the Test Instance you are viewing. It contains a header which displays the Concurrent Users which were specified when the test was configured. It also displays the start and end times of the Test. Select the Test Result to view by selecting a number of users from the “Users” dropdown; the metrics displayed on the charts are specific to the test run which was selected. The charts can be manipulated to show or hide additional data, printed, or expanded to provide a better view of the chart; see Understanding Load Test Result Charts for more details.

The following screenshot shows metrics for a Test Result which ran a .class file with 2 users (the .class file name has been obfuscated). Response times can be compared to the number of Active Users - virtual users who have started but not yet finished the Test.

Pages & URLs

The Pages & URLs tab shows detailed data about URL and page response times, and provides access to response time breakdowns.

This page contains detailed results for each individual URL call in the test.

Instance

The Instance section shows basic information about the test.

Item	Description
Concurrent Users	Number of Users.
Test Sequence Times	Start and end for the test sequence run.
Create Quick PDF Report	Button to Quick Report form the results.

Results

In the Results section you can select which run to display in the charts and tables. Each run in the instance is indicated by a tab showing the number of users.

By default, information about all runs is displayed:

To select a specific run, click the tab with the desired number of users:

The table corresponding to the run is shown in the Pages & Urls section.

Errors

Errors show detailed information about any possible errors if any have occurred.

Instance

The Instance section shows basic information about the test.

Item	Description
Concurrent Users	Number of users.
Test Sequence Times	Start and end for the test sequence run.
Create Quick PDF Report	Button to create a PDF report form the results.

Error Overview

The Error Overview table shows the number of errors per page and test.

Each page is a row in the table, with a column for each test.

This view allows you to see which tests contain errors and select them for display in the details chart and table.

Chart

The Errors Over Time chart shows error occurrences during the test for the selected location.

The chart shows the number of errors during the test for the selected test.

Select Test

To display the chart, you need to select the test in the dropdown menu.

Live Error Feed

The Live Errors Feed table shows information about the last recorded errors chart.

Column	Description
Page	Page order number and name in scenario.
URL	HTTP Methods and URL associated with the error.
URL Index	Error number relative to the URL.
Time Offset	Time when the error occured, relative to the start of test.
Error Type	HTTP Status Codes and message for the error.
Actions	Buttons for possible actions to perform on the error.

Actions

Column	Action	Description
	Details	Open the Error Details dialog.

Error Details

The Error Details dialog provides access to a number of tabs containing detailed information relating to the error.

Summary

The Summary section shows the received content of the response.

Row	Description
Page	Page order number and name in scenario.
Error Data	Time when error occured: Timestamp, time after start (number of seconds, and converted time).
URL	HTTP Method and URL associated with the error.
URL Exec Step	Execution step when error occured.
Exec Agent Socket Create Time	Creation time for the execution agent connection.
TCP Connection Reuse Count	Number of reuses of the TCP connection.
TCP Local Address	TCP local IP address.
TCP Local Port	Local port for TCP connection.
Remote TCP Address	TCP remote address.
TCP Remote Port	Remote port for TCP connection.
SSL Session Id	Session ID for SSL connection.
Error Log	Scrollable log view. Clicking the header opens the full log in a separate browser window.

Log

The log in the Summary section can be viewed in a separate browser tab.

Request

In the Request Headers tab, information about the request preceding the error is displayed.

The table displays the HTTP Methods and URL associated with the error at the top. Following that is a table containing all headers sent with the request.

Request Content

The Request Content tab shows the sent content of the request if any exists.

Response Headers

The Response Headers tab shows the received response headers.

The table displays the HTTP Status Codes and text error at the top.

Following that is a table containing all headers provided in the response.

Response Content

The Response Content tab, shows the received content of the response, if any.

To view the content in a browser, click “Preview Response Content”. Your default browser opens to display the content.

Integrations

Depending on which integrations you have enabled, various integration tabs may be available as part of the results view.

Transactions

The Transactions tab presents information about User Defined Transactions.

Summary

The Transactions Summary table shows an overview of the transactions for a selected result.

With the All Transaction Times Percentile chart, you get a statistical view of how the transaction times are distributed.

The All Transaction Times chart shows the transaction times for the selected transaction.

In the Full Passed Transaction Times Percentile chart, you get a statistical view of how the transaction times are distributed for the passed transactions.

Settings Used

The Settings Used tab shows a summary of all settings used for a load test.

Instance

The Instance section shows basic information about the test.

Preview unavailable

Concurrent Users	Number of Users.
Test Sequence Times	Start and end for the test sequence run.
Create Quick PDF Report	Button to the Quick Report form results.

Results

In the Results section, you can select which run to display.

To select a specific result set:

• Click the tab with the desired number of users

The Settings Used tab shows a summary of all settings used for a load test.

Item	Description
Scenario
Scenario	Scenario used in the job.
Loadtest Options
Number of Concurrent Users	Number of virtual users in the test.
Loadtest Duration	Total duration of the test.
Rampup Time	Time to ramp-up.
Run Multiple Sequential Tests	Used execution mode.
Location
Cluster	Location cluster(s) used to generate load.
Scenario Options
User input Text Files	Connected input file(s).
User Defined Vars	Variables defined in the scenario.
Advanced Options
Max Loops Per User	Maximum number of loops per users.
Request Timeout (sec)	Time to wait for responses.
Additional Options	Additional command line options provided as part of the job.
Distribute Load on All Available Datacenters	Enabled load distribution
Client Options
User Agent	Client User-Agent.
Network Bandwidth Downlink (Mbit/s):
Desktop	Limit on downloaded desktop traffic.
Mobile	Limit on downloaded mobile traffic.
Network Bandwidth Uplink (Mbit/s):
Desktop	Limit on uploaded desktop traffic.
Mobile	Limit on uploaded mobile traffic.
Client Side Monitoring	Enabled Client Side Monitoring.
DNS
DNS Hosts file	Custom hosts file.
DNS Server	Custom DNS server used for lookups.
Resolve DNS for Each Executed Loop	Whether DNS lookup is performed for each loop.
DNS Translation file	Custom DNS server translation file used.
Reporting
Email Preliminary Report To	Address for test reports.
Reporting History	Limit on number of tests to include in test history.
Execution
Schedule	Type of scheduling for the job.
Schedule Date	Date for schedule.
Test Information
Attached to Project	Project the test belongs to.
Job Comment	Comments added to the job settings.
Tags	Scenario Tags applied to the scenario.

Client Side Monitoring

If the job uses Client Side Monitoring, this tab shows real browser rendering times corresponding to the behavior of a real user accessing the application during the load test.

You will be able to see the total Response Time measured from a real browser for each separate run on this page.

Instance Results

The Instance section shows basic information about the test:

Item	Description
Concurrent Users	Number of Users.
Test Sequence Times	Start and end for the test sequence run.
Create Quick PDF Report	Button to the Quick Report form results.

Results

In the Results section, you can select which run to display in the charts and tables.

To select a specific result set:

• Click the tab with the desired number of users

Location Overview Chart

For each location, a Location Overview chart displays a graph containing the page response times and the number of URL errors for all runs.

Select the desired location, e.g. Los Angeles, from the dropdown menu to pick which location to show in the chart.

The chart shows the page response times and the number of URL errors overall runs for the selected location.

Run Details

The Run Details shows information about the run and provides access to related screenshots.

The run details contain a timestamp for when the test started (with offset showing time elapsed from the start of test in parentheses).

If screenshots are available, a link to open them for viewing is also shown.

Snapshot

To see the screenshots:

• Click the View Screenshot link.

The screenshots are displayed as an overlay on your browser:

The snapshots provide a browser view of the page at the time the URL occurred.

Navigate

When you hover the mouse over the left or right side of the snapshot, navigation buttons are shown, allowing you to browse through multiple snapshots, if available.

Select Run

The Select Run dropdown menu allows you to pick which test run to show in the run Run Details and Pages sections

URL Errors

The URL Errors table shows any errors that have occurred for the page during the test.

Each URL is displayed as a row in the table.

Column	Description
#	ID number.
Code	Error code, if any.
URL	URL associated with the error.
Time	Time elapsed from the start of the test.
Error	Error message.

Pages

The Pages table(s) shows information about the page(s) in the test and the URLs called for each page during the test run.

Each page is displayed in a table, with URLs displayed in rows:

Column	Description	Comment
URL	Called URL.
Time	Time until received response.
Size	Response size.
More Information	Show more information about the URL.	Click to see more information about the response.
0s, 1s, 2s….	Graphical representation of the response times.	Click to see timing details.

The “More Information” Column

More information about the response is available for each URL. To see more information about a particular URL, click the “See More” link. A popup containing details about the URL is shown:

Item	Description
Mime Type	MIME Type for the response message.
HTTP Response	Response HTTP Status Codes.
Error Message	Returned error message (if the URL is associated with an error).

View Details

To see the timing details about a particular URL:

• Click the representation for the desired URL

A popup containing details about the URL is shown:

Information about the HTTP call is shown at the top.

Item	Description
HTTP Response	Response HTTP Status Codes.
Error Message	Returned error message (if the URL is associated with an error).

The Response Time Details section shows a graphical representation of the flow and a legend with times.

The Total Response Time is displayed as a summary of the detailed view.