ant gui
|
Executing Tests |
| Previous | Next | Contents |
The Jakarta EE 9 Platform TCK uses the JavaTest harness to execute the tests in the test suite. For detailed instructions that explain how to run and use JavaTest, see the JavaTest User’s Guide and Reference.
This chapter includes the following topics:
|
Note
|
The instructions in this chapter assume that you have installed and configured your test environment as described inChapter 4, "Installation," and Chapter 5, "Setup and Configuration," respectively. |
The following are assumed in this chapter:
Jakarta EE 9 CI is installed and configured as described in this guide.
Detailed configuration will vary from product to product. In this guide, we provide details for configuring the Jakarta EE CI, Eclipse GlassFish 6.0. If you are using another CI, refer to that product’s setup and configuration documentation.
Java SE 8 software is correctly installed and configured on the host machine.
Jakarta EE 9 Platform TCK is installed and configured as described in this guide.
Implementations of the technologies to be tested are properly installed and configured.
There are two general ways to run Jakarta EE 9 Platform TCK using the JavaTest harness software:
Through the JavaTest GUI
From the command line in your shell environment
Running the JavaTest harness from JavaTest GUI is recommended for initial configuration procedures, for validating your configuration, for selecting tests to run, and for general ease-of-use when running tests and viewing test reports.
Running the JavaTest harness from the command line is useful in headless server configurations, and for running tests in batch mode.
|
Note
|
The |
Set the TS_HOME environment variable to the directory in which the
Jakarta EE 9 Platform TCK is installed.
Change to the <TS_HOME>/bin directory.
Ensure that the ts.jte file contains information relevant to your
setup.
Refer to Chapter 5, "Setup and Configuration,"
for detailed configuration instructions.
Execute the ant gui target to start the JavaTest GUI:
ant gui
Using the JavaTest GUI to run TCK tests is described later in this
guide. For detailed information about using the JavaTest interface, see
the JavaTest User’s Guide.
|
Note
|
The forward and reverse keywords are available to filter the interop and/or rebuildable tests during a selected test run when running tests in the following directory only:
Forward tests are interop tests that run from the Vendor Implementation
to the Compatible Implementation, as well as rebuildable tests that run
only against the Vendor Implementation. Reverse tests (with test names
ending in To set one of these keywords in the Javatest GUI, select the Configure menu item, then select Change Configuration, then select Keywords, and set the appropriate keyword. When one of these keywords has been set, executing tests in the directories above causes only those tests that match the keyword to be run. This can be useful when trying to debug failures with a particular test configuration. Note, however, for certification all tests in both directions must pass. |
Set the TS_HOME environment variable to the directory in which
Jakarta EE 9 Platform TCK was installed.
Change to any subdirectory under <TS_HOME>/src/com/sun/ts/tests.
Ensure that the ts.jte file contains information relevant to your
setup.
Refer to Chapter 5, "Setup and Configuration,"
for detailed configuration instructions.
Execute the runclient Ant target to start the JavaTest:
ant runclient
This runs all tests in the current directory and any subdirectories.
To run the Jakarta EE 9 Platform TCK signature tests, enter the following commands:
cd <TS_HOME>/src/com/sun/ts/tests/signaturetest/javaee
ant runclient
To run a single test directory in the forward direction, enter the
following commands:
cd <TS_HOME>/src/com/sun/ts/tests/jaxws/api/jakarta_xml_ws/Dispatch
ant -Dkeywords=forward runclient
Start the JavaTest GUI and step through the basic configuration steps, if required, as described in Section 5.5.2, "The Configuration Interview."
In the JavaTest GUI tree view, expand the following directories:
com, sun, ts, tests, samples.
Highlight the samples directory, right-click, and choose Execute
These Tests.
If a work directory has not been specified, you are prompted to specify
or create a new one.
From the JavaTest main menu, select File, then select Create Work Directory. The Create Work Directory dialog is displayed.
Locate or enter the name of the directory to which the test harness
will write temporary files (for example, /tmp/JTWork), and click
Create.
From the JavaTest main menu, select Run Tests, then select Start to
run the default tests.
If your configuration information is incomplete, you are prompted to
supply the missing parameters.
The JavaTest status bar grows while JavaTest tracks statistics relative
to the files done, tests found, and tests done.
Check the results.
Test progress and results are displayed by the JavaTest harness.
Go to the <TS_HOME>/src/com/sun/ts/tests/samples directory.
Start the the test run by executing the following command:
ant runclient
All sample tests will be run, and should pass.
Generate test reports by executing the following commands:
Change to the <TS_HOME>/bin directory:
cd <TS_HOME>/bin
Run the report Ant target:
ant report
Reports are written to the report directory you specified in
<TS_HOME>/bin/ts.jte. If no report directory is specified, reports are
written to the /tmp/JTreport directory (Solaris/Linux) or
C:\temp\JTreport (Windows).
From the JavaTest main menu, select Configure, then select Edit Configuration.
In the Configuration Editor, select Specify Tests to Run? from the
option list on the left.
You are asked whether you want to run all or a subset of the test suite.
Click Yes, and then Next to run a subset of tests.
Select the tests you want to run from the displayed test tree, and
then click Done.
You can select entire branches of the test tree, or use Ctrl+Click or
Shift+Click to select multiple tests or ranges of tests, respectively.
After clicking Done, you are returned to the JavaTest main window.
Select Run Tests, then select Start to run the tests you selected.
Change to the directory containing the tests you want to run.
For example, <TS_HOME>/src/com/sun/ts/tests/samples.
Start the test run by executing the following command:
ant runclient
The tests in <TS_HOME>/src/com/sun/ts/tests/samples and its
subdirectories are run.
7.4.3 To Run a Subset of Tests in Batch Mode Based on Prior Result Status ^^^^^^^^^^^^^^^^^^^^^^^^^
You can run certain tests in batch mode based on the test’s prior run
status by specifying the priorStatus system property when invoking
Ant.
Invoke ant with the priorStatus property.
The accepted values for the priorStatus property are any combination
of the following:
fail
pass
error
notRun
For example, you could run all Jakarta EE 9 tests with a status of failed and error by invoking the following commands:
cd <TS_HOME>/src/com/sun/ts/tests/ejb
ant -DpriorStatus="fail,error" runclient
Note that multiple priorStatus values must be separated by commas.
The Jakarta EE TCK includes some tests that may be optional depending on
your implementation. For example, certain technologies are now optional
for implementations of the full Jakarta EE Platform. There are other
technologies which are optional for Web Profile implementations, but may
be implemented. If implemented, optional tests must be run and pass.
There are two mechanisms in place in the TCK which control whether or
not a given set of tests is run - the javaee.level property in the
ts.jte file (see Section 7.5.1, "Setting the
javaee.level Property") and keywords (see Section 7.5.2,
"Using Keywords to Create Groups and Subsets of Tests").
The ts.jte file includes the javaee.level property. This property
serves two purposes. First, it is used to determine whether the
implementation under test is a Jakarta EE Full profile (full) or Jakarta EE
Web profile (web). Either "full" or "web" must be specified in the list
values. A setting of "full" instructs the test harness to deploy EAR
files. A setting of "web" instructs the test harness to deploy WAR
files. The javaee.level property is also used to help determine which
APIs in the signature tests are to be tested. The comments that precede
the property setting in the ts.jte file provide additional information
about setting this property.
The default setting is as follows:
javaee.level=full
Each test in TCK has keywords associated with it. The keywords are used to create groups and subsets of tests. At test execution time, a user can tell the test harness to only run tests with or without certain keywords. This mechanism is used to select or omit testing on selected optional technologies. The "keywords" property can be set to a set of available keywords joined by "&" and/or "|".
To set the keywords system property at runtime, you must either pass it
on the command line via -Dkeywords="" or in the JavaTest GUI, by
opening the test suite and performing the following steps:
Select View, then select Filters, then select CurrentConfiguration.
Select Configure, then select ChangeConfiguration, then select Keywords.
In the Keywords dialog, select the Select Tests that Match check
box, specify the desired keyword in the field, then click Done.
Only tests that have been tagged with that keyword will be enabled in
the test tree.
The examples in the sections that follow show how to use keywords to run required technologies in both the Full and Web profile, run/omit running optional sets of tests in TCK, and run the Interoperability and Rebuildable tests in forward and reverse directions.
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords=javaee runclient
Only tests that are required by the Full Profile will be run.
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords=javaee_web_profile runclient
Only tests that are required by the Web Profile will be run.
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords="javaee & !connector" runclient
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords=ejb runclient
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords=ejb32 runclient
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords=ejb_web_profile runclient
Keywords can be used to run subsets of tests from areas that are not required by the Jakarta EE 9 platform specification. Table 7-1 lists optional subsets of tests that can be run for the Full Profile and provides the technology-to-keyword mappings for each of the optional areas.
Subsets
| Technology | Keyword |
|---|---|
EJB 1.x, CMP, BMP, entity beans |
|
EJB 2.x, CMP, BMP, entity beans |
|
EJBQL |
|
JAXR |
|
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords=javaee_optional runclient
Keywords can be used to run subsets of tests from additional areas that
are not required by the Jakarta EE 9 Web Profile specification. For
example, if your server implements the Jakarta EE 9 Web Profile and the
Jakarta Connector Architecture 1.7 technology, set the keywords to
javaee_web_profile|connector_web_profile to enable running tests for
both areas. The command below shows how to specify these keywords to run
the tests in both areas.
ant -Dkeywords="(javaee_web_profile|connector_web_profile) runclient
Table 7-2 lists optional subsets of tests that can be run for the Web Profile and provides the technology-to-keyword mappings for each of the optional areas.
Subsets
| Technology | Keyword |
|---|---|
Jakarta Connectors |
|
Jakarta Authorization (formerly JACC) |
|
Jakarta Authentication (formerly JASPIC) |
|
Jakarta Mail (formerly JavaMail) |
|
Jakarta Registries (formerly JAXR) |
|
Jakarta Messaging(formerly JMS) |
|
XA |
|
To add tests for other technologies, select the appropriate keyword from Table 7-2. This table provides a mapping of keywords to optional technologies (test directories) in the test suite and indicates optional test areas for the Jakarta EE 9 Web Profile.
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords=javaee_web_profile_optional runclient
cd <TS_HOME>/src/com/sun/ts/tests
ant -Dkeywords="javaee_web_profile | jacc_web_profile | jaspic_web_profile" runclient
Table 1-1 shows the TCK keywords you can use to test optional Jakarta Enterprise Beans (formerly EJB) Lite components. Components denoted with an asterisk (*) are pruned components; components without an asterisk are not required by EJB Lite.
| Component | TCK Keyword |
|---|---|
Message-Driven Beans |
|
1x CMP/BMP Entity Beans * |
ejb_1x_optional |
2x CMP/BMP Entity Beans, Remote/Home Component, Local/Home Component * |
|
3x Remote |
|
EJB QL * |
|
Persistent Timer Service |
|
Remote asyncrhonous session bean |
|
EJB Embeddable Container |
|
Support for the following features has been made optional in this release:
EJB 2.1 and earlier Entity Bean Component Contract for Container-Managed Persistence and Bean-Managed Persistence
Client View of an EJB 2.1 and earlier Entity Bean
EJB QL: Query Language for Container-Managed Persistence Query Methods
The following vehicle keywords can be used to select or exclude the vehicles in which tests are run:
connectorservlet_vehicle
ejblitesecuredjsp_vehicle
ejbliteservlet_vehicle
ejbliteservlet2_vehicle
jaspicservlet_vehicle
pmservlet_vehicle
puservlet_vehicle
wsservlet_vehicle
servlet_vehicle
jsp_vehicle
web_vehicle
appclient_vehicle
wsappclient_vehicle
ejb_vehicle
wsejb_vehicle
These vehicles are defined in the
<TS_HOME>/src/com/sun/ts/tests/common/vehicle subdirectory structures.
ant -Dkeywords="ejb_vehicle" runclient
ant -Dkeywords="!ejb_vehicle" runclient
The forward and reverse keywords can be used to filter the interop
and/or rebuildable tests during a selected test run when running tests
in one of the following directories only:
<TS_HOME>/src/com/sun/ts/tests/jaxws
<TS_HOME>/src/com/sun/ts/tests/jws
<TS_HOME>/src/com/sun/ts/tests/interop
Forward tests are interop tests that run from the Vendor
Implementation to the Compatible Implementation, as well as rebuildable
tests that run only against the Vendor Implementation. Reverse tests
(with test names ending in _reverse) are interop tests that run from the
Compatible Implementation to the Vendor Implementation, as well as
rebuildable tests that run only against the Compatible Implementation.
To set one of these keywords when running in command-line mode, set the appropriate keyword using the keyword system property.
ant -Dkeywords=forward runclient
ant -Dkeywords=reverse runclient
To set one of these keywords in the Javatest GUI, select the Configure menu item, then select Change Configuration, then select Keywords, and set the appropriate keyword.
When one of these keywords has been set, executing tests in the directories above causes only those tests that match the keyword to be run. This can be useful when trying to debug failures with a particular test configuration. Note, however, for certification all tests in both directions must pass.
If you are running Interop or XML Web Service reverse tests, which run
against the Jakarta EE 9 CI, you must start the standalone deployment
server in a separate shell on the same host as the TCK harness. The
default deployment porting implementation goes through a standalone
deployment server with a dedicated classpath. To start the standalone
deployment server, change to the <TS_HOME>/bin directory and execute
the start.auto.deployment.server Ant task.
The following directories require rebuilding, which is done by running
the configure.datasource.tests Ant target:
com/sun/ts/tests/ejb30/lite/packaging/war/datasource
com/sun/ts/tests/ejb30/assembly/appres
com/sun/ts/tests/ejb30/misc/datasource
When the configure.datasource.tests Ant target is run from any
directory, it rebuilds these directories and any required
subdirectories.
The com/sun/ts/tests/jms/ee20/resourcedefs directory must also be
rebuilt. Run the build.special.webservices.clients Ant target to
rebuild the tests in this directory.
The database properties in the TCK bundle are set to Derby database. If
any other database is used, the update.metadata.token.values ant
target needs to be executed for metadata-complete tests.
The following directories require rebuilding:
src\com\sun\ts\tests\appclient\deploy\metadatacomplete\testapp.
This can be done by running the update.metadata.token.values Ant
target.
A set of report files is created for every test run. These report files can be found in the report directory you specify. After a test run is completed, the JavaTest harness writes HTML reports for the test run. You can view these files in the JavaTest ReportBrowser when running in GUI mode, or in the web browser of your choice outside the JavaTest interface.
To see all of the HTML report files, enter the URL of the report.html
file. This file is the root file that links to all of the other HTML
reports.
The JavaTest harness also creates a summary.txt file in the report
directory that you can open in any text editor. The summary.txt file
contains a list of all tests that were run, their test results, and
their status messages.
Although you can run the Ant report target from any test directory, its
support is not guaranteed in the lower level directories. It is
recommended that you always run the report target from <TS_HOME>/bin,
from which reports are generated containing information about which
tests were or were not run.
From the JavaTest main menu, select Report, then select Create
Report.
You are prompted to specify a directory to use for your test reports.
Specify the directory you want to use for your reports, and then
click OK.
Use the Filter list to specify whether you want to generate reports for
the current configuration, all tests, or a custom set of tests.
You are asked whether you want to view report now.
Click Yes to display the new report in the JavaTest ReportBrowser.
Specify where you want to create the test report.
To specify the report directory from the command line at runtime, use:
ant report -Dreport.dir="report_dir"
Reports are written for the last test run to the directory you specify.
To specify the default report directory, set the report.dir
property in <TS_HOME>/bin/ts.jte.
For example, report.dir="/home/josephine/reports".
To disable reporting, set the report.dir property to "none",
either on the command line or in ts.jte.
For example:
ant -Dreport.dir="none"
Troubleshooting
Although you can run the report Ant target from any test directory,
its support is not guaranteed in the lower level directories. It is
recommended that you always run the report target from
<TS_HOME)/bin, from which reports are generated containing information
about which tests were or were not run.cc
From the JavaTest main menu, select Report, then select Open Report.
You are prompted to specify the directory containing the report you want
to open.
Select the report directory you want to open, and then click Open.
The selected report set is opened in the JavaTest Report Browser.
Use the Web browser of your choice to view the report.html file in the
report directory you specified from the command line or in ts.jte.
The current report directory is displayed when you run the report
target.
| Previous | Next | Contents |
Copyright © 2018, 2020 Oracle and/or its affiliates. All rights reserved.