Analyze run results

When you run LeanFT tests created from a LeanFT project template, an HTML report is automatically generated. The report includes summary information about the run as well as detailed information about the captured steps.

The Basics

  • For Java and C# tests, you can use LeanFT > View Last Run Results to open the most recent LeanFT HTML Report that was generated from the test that ran inside the IDE during the current IDE session.

    Note: By default, the report is generated to a folder called RunResults in the project's output path. For example: \bin\Debug\RunResults\runresults.html

    JavaScript tests: Open the report from this location.

  • Use the Search bar and Previous/Next Error buttons inside the HTML page to navigate your report.
  • By default, the report includes all steps that involve LeanFT SDK test objects and result in an interaction with objects in your application.
  • Use Reporter steps to include custom details in the report about events that occur during your test.
  • By default, the report does not include screen snapshots, but you can modify the test settings to instruct LeanFT to capture snapshots for steps with errors or for every reported step.

Sample report for a simple Web test:

Back to top

Add more information to your test report

You can use the Reporter.ReportEvent method to add custom steps to your test report. Depending on the override you use, you can do one or more of the following:

  • Add a node to the test flow tree with step details.
  • Set the status for the step. This status impacts the overall status of the test.
  • Provide an image to include in the step details.  This could be an image from a file location or an image retrieved in an earlier step.
  • Pass an exception for inclusion in the step details.

You can also use the Reporter object to:

  • Add custom information to the top section of the report.

  • Start and end a reporting context, which adds another labeled level to the hierarchy of the test flow tree.

  • Initiate report-related activities in a custom framework that are handled automatically in the LeanFT testing frameworks.

  • Create custom verifications and display them in the test report.

For syntax details for the Reporter object, see:

Back to top

Customize report behavior

You can customize report behavior and preferences such as:

  • The title and description of the HTML report.

  • Where the report gets generated, and the folder and file name for the report.

    You can also control whether the report folder gets overwritten every time you run a particular test or if a new folder is created each time.

  • Which steps are included in the report.

    For example, all object steps, only steps with warnings or errors, or only steps with errors. You can also use this option to completely turn off the HTML report generation.

  • Whether and when snapshots are captured with steps in the report.

There are several to customize report behavior:

Note: This section is relevant for Java and C#. To set up the report for JavaScript tests, see Setting up and working with the JavaScript SDK.

  1. Edit the report options as needed in each project's test settings file.

  2. Override the ReportConfiguration with new settings.

    For example, to create a new report file for every run in Eclipse (Java):

    @Override
    public ModifiableReportConfiguration getReportConfiguration(){
        ModifiableReportConfiguration config = new ModifiableReportConfiguration();
        config.setOverridingExisting(true);
        return config;
    }
    
  3. Provide the modified settings from a central configuration project to multiple tests.

    C#:

    Use the GetReportConfiguration method in your project's UnitTestBase to override the default report with a provided assembly.

    For syntax details, see .NET SDK Reference.

    Java:

    Pass a leanft.properties file to the Java test project you are running using the lftConfigFile system variable.

    For example:

    java "C:\MyTests\MyTest1.jar" –DlftConfigFile="c:\tests\myConfig.properties"

Back to top

Share the report

In many cases, the generated HTML file is a standalone file.

If you selected to capture snapshots of all or failed steps, or your test includes steps that return images or other files, you may want to move the report to another location or share the report with others. In this case, make sure to keep the generated Resources folder together with the HTML file.

Back to top

See also: