Test Reports

HILSTER Testing Framework includes test reports for HTML, XML and JSON.

HTML Test Report

The HTML test report is a monolithic Javascript application that does not need to load any external data.

_images/htmltestreport.png

Overview

When you load the test report a progress bar appears and the percentage of loaded tests is shown. The HTML test report loads very fast.

On the left side you find an overview about the test run. It includes the total number of tests, the number of successful tests, the number of failed tests, the number of buggy tests and the number of skipped tests.

At the bottom of the left area you can find information about the host and the platform the test was run on and read the licensing information, too.

This area can be hidden and shown by clicking the burger icon next to the logo.

In the top right corner you can see the test title and the test date.

Test Results

On the right side you can find the main content. The main content contains the test results. Clicking on a result toggles its visibility. By default all test results are collapsed.

When opening a test result the Javascript application will scroll to the ideal position.

Every test result contains different entries like the result itself, a traceback, metadata, documentation, tags, attachments, captured stdout and captured stderr.

Long captured output (stdout and stderr) has to be loaded by clicking the show more link. This helps to improve loading speed.

By clicking on an item in the overview the main content is filtered accordingly.

File Attachments

The HTML test report supports file attachments that you can attach to a test using htf.TestCase.attach_file() (or htf.TestCase.attachFile()).

File attachments are not supported by all browsers. For more information see Browser Limitations.

URL Attachments

You can also attach URLs by using htf.TestCase.attach_url() (or htf.TestCase.attachUrl()) in your test.

Searching

When clicking on the magnifier on the top a search box appears and an x to hide it again.

By default the search is a fulltext search for the titles of the tests only.

To search all text use the prefix global:. For example global:foo searches for foo globally.

Advanced Searching

The search phrase can also be a regular expression that can also be prefixed.

For example to search for tests that include foo or bar globally you would use global:foo|bar.

To search in specific result fields, e.g. Tags in all results containing it you can use the name of the field as a prefix. The prefix for Tags is tags:, the prefix for Documentation is documentatation: and so on.

Hotkeys

The HTML-Testreport supports the following hotkeys:

Key Description
h Shows help (any key but h in help hides it)
esc Reset view: hide help, hide search box, show all results
a Shows all results
s Shows successes only
f Shows failures
e Shows errors only
S Shows skipped only
F Shows the search box and sets the focus in it
a Collapses all uncollapsed test results
b Toggles the sitebar

The only hotkey you have to remember is h.

Browser Limitations

File attachments are realized using data-urls.

Some Microsoft-browsers do not support them at all. Chrome and Chromium have security features but support an acceptable workaround.

The following browser limitations are known.

Browser Limitations
Internet Explorer 11
  • Does not load the test report when files are attached.
Edge
  • Attachments cannot be viewed or downloaded. Edge does not support data-urls to be used for downloading or viewing. There is no known workaround. Everything else works well.
Chrome/Chromium
  • Attachments cannot be viewed or downloaded with a single click on it. As a workaround you can use the context menu and use safe as or open in another tab.

Your browser must have Javascript enabled to view the test report. There are some plugins like NoScript that can cause problemes. Also old versions of Internet Explorer do not load Javascript by default.