|
|
@ -3,12 +3,18 @@
|
|
|
|
ctest(1)
|
|
|
|
ctest(1)
|
|
|
|
********
|
|
|
|
********
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. contents::
|
|
|
|
|
|
|
|
|
|
|
|
Synopsis
|
|
|
|
Synopsis
|
|
|
|
========
|
|
|
|
========
|
|
|
|
|
|
|
|
|
|
|
|
.. parsed-literal::
|
|
|
|
.. parsed-literal::
|
|
|
|
|
|
|
|
|
|
|
|
ctest [<options>]
|
|
|
|
ctest [<options>]
|
|
|
|
|
|
|
|
ctest <path-to-source> <path-to-build> --build-generator <generator>
|
|
|
|
|
|
|
|
[<options>...] [-- <build-options>...] [--test-command <test>]
|
|
|
|
|
|
|
|
ctest (-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>)
|
|
|
|
|
|
|
|
[-- <dashboard-options>...]
|
|
|
|
|
|
|
|
|
|
|
|
Description
|
|
|
|
Description
|
|
|
|
===========
|
|
|
|
===========
|
|
|
@ -55,17 +61,21 @@ Options
|
|
|
|
``-F``
|
|
|
|
``-F``
|
|
|
|
Enable failover.
|
|
|
|
Enable failover.
|
|
|
|
|
|
|
|
|
|
|
|
This option allows ctest to resume a test set execution that was
|
|
|
|
This option allows CTest to resume a test set execution that was
|
|
|
|
previously interrupted. If no interruption occurred, the ``-F`` option
|
|
|
|
previously interrupted. If no interruption occurred, the ``-F`` option
|
|
|
|
will have no effect.
|
|
|
|
will have no effect.
|
|
|
|
|
|
|
|
|
|
|
|
``-j <jobs>, --parallel <jobs>``
|
|
|
|
``-j <jobs>, --parallel <jobs>``
|
|
|
|
Run the tests in parallel using the given number of jobs.
|
|
|
|
Run the tests in parallel using the given number of jobs.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to run the tests in parallel using given
|
|
|
|
This option tells CTest to run the tests in parallel using given
|
|
|
|
number of jobs. This option can also be set by setting the
|
|
|
|
number of jobs. This option can also be set by setting the
|
|
|
|
environment variable ``CTEST_PARALLEL_LEVEL``.
|
|
|
|
environment variable ``CTEST_PARALLEL_LEVEL``.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This option can be used with the :prop_test:`PROCESSORS` test property.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See `Label and Subproject Summary`_.
|
|
|
|
|
|
|
|
|
|
|
|
``--test-load <level>``
|
|
|
|
``--test-load <level>``
|
|
|
|
While running tests in parallel (e.g. with ``-j``), try not to start
|
|
|
|
While running tests in parallel (e.g. with ``-j``), try not to start
|
|
|
|
tests when they may cause the CPU load to pass above a given threshold.
|
|
|
|
tests when they may cause the CPU load to pass above a given threshold.
|
|
|
@ -74,7 +84,7 @@ Options
|
|
|
|
``TestLoad`` option of the `CTest Test Step`_.
|
|
|
|
``TestLoad`` option of the `CTest Test Step`_.
|
|
|
|
|
|
|
|
|
|
|
|
``-Q,--quiet``
|
|
|
|
``-Q,--quiet``
|
|
|
|
Make ctest quiet.
|
|
|
|
Make CTest quiet.
|
|
|
|
|
|
|
|
|
|
|
|
This option will suppress all the output. The output log file will
|
|
|
|
This option will suppress all the output. The output log file will
|
|
|
|
still be generated if the ``--output-log`` is specified. Options such
|
|
|
|
still be generated if the ``--output-log`` is specified. Options such
|
|
|
@ -84,37 +94,37 @@ Options
|
|
|
|
``-O <file>, --output-log <file>``
|
|
|
|
``-O <file>, --output-log <file>``
|
|
|
|
Output to log file.
|
|
|
|
Output to log file.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to write all its output to a log file.
|
|
|
|
This option tells CTest to write all its output to a log file.
|
|
|
|
|
|
|
|
|
|
|
|
``-N,--show-only``
|
|
|
|
``-N,--show-only``
|
|
|
|
Disable actual execution of tests.
|
|
|
|
Disable actual execution of tests.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to list the tests that would be run but not
|
|
|
|
This option tells CTest to list the tests that would be run but not
|
|
|
|
actually run them. Useful in conjunction with the ``-R`` and ``-E``
|
|
|
|
actually run them. Useful in conjunction with the ``-R`` and ``-E``
|
|
|
|
options.
|
|
|
|
options.
|
|
|
|
|
|
|
|
|
|
|
|
``-L <regex>, --label-regex <regex>``
|
|
|
|
``-L <regex>, --label-regex <regex>``
|
|
|
|
Run tests with labels matching regular expression.
|
|
|
|
Run tests with labels matching regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to run only the tests whose labels match the
|
|
|
|
This option tells CTest to run only the tests whose labels match the
|
|
|
|
given regular expression.
|
|
|
|
given regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
``-R <regex>, --tests-regex <regex>``
|
|
|
|
``-R <regex>, --tests-regex <regex>``
|
|
|
|
Run tests matching regular expression.
|
|
|
|
Run tests matching regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to run only the tests whose names match the
|
|
|
|
This option tells CTest to run only the tests whose names match the
|
|
|
|
given regular expression.
|
|
|
|
given regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
``-E <regex>, --exclude-regex <regex>``
|
|
|
|
``-E <regex>, --exclude-regex <regex>``
|
|
|
|
Exclude tests matching regular expression.
|
|
|
|
Exclude tests matching regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to NOT run the tests whose names match the
|
|
|
|
This option tells CTest to NOT run the tests whose names match the
|
|
|
|
given regular expression.
|
|
|
|
given regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
``-LE <regex>, --label-exclude <regex>``
|
|
|
|
``-LE <regex>, --label-exclude <regex>``
|
|
|
|
Exclude tests with labels matching regular expression.
|
|
|
|
Exclude tests with labels matching regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to NOT run the tests whose labels match the
|
|
|
|
This option tells CTest to NOT run the tests whose labels match the
|
|
|
|
given regular expression.
|
|
|
|
given regular expression.
|
|
|
|
|
|
|
|
|
|
|
|
``-FA <regex>, --fixture-exclude-any <regex>``
|
|
|
|
``-FA <regex>, --fixture-exclude-any <regex>``
|
|
|
@ -137,11 +147,13 @@ Options
|
|
|
|
``-D <dashboard>, --dashboard <dashboard>``
|
|
|
|
``-D <dashboard>, --dashboard <dashboard>``
|
|
|
|
Execute dashboard test.
|
|
|
|
Execute dashboard test.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to act as a CDash client and perform a
|
|
|
|
This option tells CTest to act as a CDash client and perform a
|
|
|
|
dashboard test. All tests are <Mode><Test>, where Mode can be
|
|
|
|
dashboard test. All tests are <Mode><Test>, where Mode can be
|
|
|
|
Experimental, Nightly, and Continuous, and Test can be Start,
|
|
|
|
Experimental, Nightly, and Continuous, and Test can be Start,
|
|
|
|
Update, Configure, Build, Test, Coverage, and Submit.
|
|
|
|
Update, Configure, Build, Test, Coverage, and Submit.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See `Dashboard Client`_.
|
|
|
|
|
|
|
|
|
|
|
|
``-D <var>:<type>=<value>``
|
|
|
|
``-D <var>:<type>=<value>``
|
|
|
|
Define a variable for script mode.
|
|
|
|
Define a variable for script mode.
|
|
|
|
|
|
|
|
|
|
|
@ -153,35 +165,33 @@ Options
|
|
|
|
``-M <model>, --test-model <model>``
|
|
|
|
``-M <model>, --test-model <model>``
|
|
|
|
Sets the model for a dashboard.
|
|
|
|
Sets the model for a dashboard.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to act as a CDash client where the ``<model>``
|
|
|
|
This option tells CTest to act as a CDash client where the ``<model>``
|
|
|
|
can be ``Experimental``, ``Nightly``, and ``Continuous``.
|
|
|
|
can be ``Experimental``, ``Nightly``, and ``Continuous``.
|
|
|
|
Combining ``-M`` and ``-T`` is similar to ``-D``.
|
|
|
|
Combining ``-M`` and ``-T`` is similar to ``-D``.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See `Dashboard Client`_.
|
|
|
|
|
|
|
|
|
|
|
|
``-T <action>, --test-action <action>``
|
|
|
|
``-T <action>, --test-action <action>``
|
|
|
|
Sets the dashboard action to perform.
|
|
|
|
Sets the dashboard action to perform.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to act as a CDash client and perform some
|
|
|
|
This option tells CTest to act as a CDash client and perform some
|
|
|
|
action such as ``start``, ``build``, ``test`` etc. See
|
|
|
|
action such as ``start``, ``build``, ``test`` etc. See
|
|
|
|
`Dashboard Client Steps`_ for the full list of actions.
|
|
|
|
`Dashboard Client Steps`_ for the full list of actions.
|
|
|
|
Combining ``-M`` and ``-T`` is similar to ``-D``.
|
|
|
|
Combining ``-M`` and ``-T`` is similar to ``-D``.
|
|
|
|
|
|
|
|
|
|
|
|
``--track <track>``
|
|
|
|
See `Dashboard Client`_.
|
|
|
|
Specify the track to submit dashboard to
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Submit dashboard to specified track instead of default one. By
|
|
|
|
|
|
|
|
default, the dashboard is submitted to Nightly, Experimental, or
|
|
|
|
|
|
|
|
Continuous track, but by specifying this option, the track can be
|
|
|
|
|
|
|
|
arbitrary.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``-S <script>, --script <script>``
|
|
|
|
``-S <script>, --script <script>``
|
|
|
|
Execute a dashboard for a configuration.
|
|
|
|
Execute a dashboard for a configuration.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to load in a configuration script which sets
|
|
|
|
This option tells CTest to load in a configuration script which sets
|
|
|
|
a number of parameters such as the binary and source directories.
|
|
|
|
a number of parameters such as the binary and source directories.
|
|
|
|
Then ctest will do what is required to create and run a dashboard.
|
|
|
|
Then CTest will do what is required to create and run a dashboard.
|
|
|
|
This option basically sets up a dashboard and then runs ``ctest -D``
|
|
|
|
This option basically sets up a dashboard and then runs ``ctest -D``
|
|
|
|
with the appropriate options.
|
|
|
|
with the appropriate options.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See `Dashboard Client`_.
|
|
|
|
|
|
|
|
|
|
|
|
``-SP <script>, --script-new-process <script>``
|
|
|
|
``-SP <script>, --script-new-process <script>``
|
|
|
|
Execute a dashboard for a configuration.
|
|
|
|
Execute a dashboard for a configuration.
|
|
|
|
|
|
|
|
|
|
|
@ -190,16 +200,12 @@ Options
|
|
|
|
script may modify the environment and you do not want the modified
|
|
|
|
script may modify the environment and you do not want the modified
|
|
|
|
environment to impact other ``-S`` scripts.
|
|
|
|
environment to impact other ``-S`` scripts.
|
|
|
|
|
|
|
|
|
|
|
|
``-A <file>, --add-notes <file>``
|
|
|
|
See `Dashboard Client`_.
|
|
|
|
Add a notes file with submission.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to include a notes file when submitting
|
|
|
|
|
|
|
|
dashboard.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``-I [Start,End,Stride,test#,test#|Test file], --tests-information``
|
|
|
|
``-I [Start,End,Stride,test#,test#|Test file], --tests-information``
|
|
|
|
Run a specific number of tests by number.
|
|
|
|
Run a specific number of tests by number.
|
|
|
|
|
|
|
|
|
|
|
|
This option causes ctest to run tests starting at number Start,
|
|
|
|
This option causes CTest to run tests starting at number Start,
|
|
|
|
ending at number End, and incrementing by Stride. Any additional
|
|
|
|
ending at number End, and incrementing by Stride. Any additional
|
|
|
|
numbers after Stride are considered individual test numbers. Start,
|
|
|
|
numbers after Stride are considered individual test numbers. Start,
|
|
|
|
End,or stride can be empty. Optionally a file can be given that
|
|
|
|
End,or stride can be empty. Optionally a file can be given that
|
|
|
@ -214,11 +220,11 @@ Options
|
|
|
|
``--rerun-failed``
|
|
|
|
``--rerun-failed``
|
|
|
|
Run only the tests that failed previously.
|
|
|
|
Run only the tests that failed previously.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to perform only the tests that failed during
|
|
|
|
This option tells CTest to perform only the tests that failed during
|
|
|
|
its previous run. When this option is specified, ctest ignores all
|
|
|
|
its previous run. When this option is specified, CTest ignores all
|
|
|
|
other options intended to modify the list of tests to run (``-L``, ``-R``,
|
|
|
|
other options intended to modify the list of tests to run (``-L``, ``-R``,
|
|
|
|
``-E``, ``-LE``, ``-I``, etc). In the event that CTest runs and no tests
|
|
|
|
``-E``, ``-LE``, ``-I``, etc). In the event that CTest runs and no tests
|
|
|
|
fail, subsequent calls to ctest with the ``--rerun-failed`` option will run
|
|
|
|
fail, subsequent calls to CTest with the ``--rerun-failed`` option will run
|
|
|
|
the set of tests that most recently failed (if any).
|
|
|
|
the set of tests that most recently failed (if any).
|
|
|
|
|
|
|
|
|
|
|
|
``--repeat-until-fail <n>``
|
|
|
|
``--repeat-until-fail <n>``
|
|
|
@ -236,7 +242,7 @@ Options
|
|
|
|
``--interactive-debug-mode [0|1]``
|
|
|
|
``--interactive-debug-mode [0|1]``
|
|
|
|
Set the interactive mode to 0 or 1.
|
|
|
|
Set the interactive mode to 0 or 1.
|
|
|
|
|
|
|
|
|
|
|
|
This option causes ctest to run tests in either an interactive mode
|
|
|
|
This option causes CTest to run tests in either an interactive mode
|
|
|
|
or a non-interactive mode. On Windows this means that in
|
|
|
|
or a non-interactive mode. On Windows this means that in
|
|
|
|
non-interactive mode, all system debug pop up windows are blocked.
|
|
|
|
non-interactive mode, all system debug pop up windows are blocked.
|
|
|
|
In dashboard mode (Experimental, Nightly, Continuous), the default
|
|
|
|
In dashboard mode (Experimental, Nightly, Continuous), the default
|
|
|
@ -246,36 +252,139 @@ Options
|
|
|
|
``--no-label-summary``
|
|
|
|
``--no-label-summary``
|
|
|
|
Disable timing summary information for labels.
|
|
|
|
Disable timing summary information for labels.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest not to print summary information for each
|
|
|
|
This option tells CTest not to print summary information for each
|
|
|
|
label associated with the tests run. If there are no labels on the
|
|
|
|
label associated with the tests run. If there are no labels on the
|
|
|
|
tests, nothing extra is printed.
|
|
|
|
tests, nothing extra is printed.
|
|
|
|
|
|
|
|
|
|
|
|
``--no-subproject-summary``
|
|
|
|
See `Label and Subproject Summary`_.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--no-subproject-summary``
|
|
|
|
Disable timing summary information for subprojects.
|
|
|
|
Disable timing summary information for subprojects.
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest not to print summary information for each
|
|
|
|
This option tells CTest not to print summary information for each
|
|
|
|
subproject associated with the tests run. If there are no subprojects on the
|
|
|
|
subproject associated with the tests run. If there are no subprojects on the
|
|
|
|
tests, nothing extra is printed.
|
|
|
|
tests, nothing extra is printed.
|
|
|
|
|
|
|
|
|
|
|
|
``--build-and-test <path-to-source> <path-to-build>``
|
|
|
|
See `Label and Subproject Summary`_.
|
|
|
|
Configure, build and run a test.
|
|
|
|
|
|
|
|
|
|
|
|
``--build-and-test``
|
|
|
|
|
|
|
|
See `Build and Test Mode`_.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--test-output-size-passed <size>``
|
|
|
|
|
|
|
|
Limit the output for passed tests to ``<size>`` bytes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--test-output-size-failed <size>``
|
|
|
|
|
|
|
|
Limit the output for failed tests to ``<size>`` bytes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--overwrite``
|
|
|
|
|
|
|
|
Overwrite CTest configuration option.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
By default CTest uses configuration options from configuration file.
|
|
|
|
|
|
|
|
This option will overwrite the configuration option.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--force-new-ctest-process``
|
|
|
|
|
|
|
|
Run child CTest instances as new processes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
By default CTest will run child CTest instances within the same
|
|
|
|
|
|
|
|
process. If this behavior is not desired, this argument will
|
|
|
|
|
|
|
|
enforce new processes for child CTest processes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--schedule-random``
|
|
|
|
|
|
|
|
Use a random order for scheduling tests.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This option will run the tests in a random order. It is commonly
|
|
|
|
|
|
|
|
used to detect implicit dependencies in a test suite.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--submit-index``
|
|
|
|
|
|
|
|
Legacy option for old Dart2 dashboard server feature.
|
|
|
|
|
|
|
|
Do not use.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--timeout <seconds>``
|
|
|
|
|
|
|
|
Set a global timeout on all tests.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This option will set a global timeout on all tests that do not
|
|
|
|
|
|
|
|
already have a timeout set on them.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--stop-time <time>``
|
|
|
|
|
|
|
|
Set a time at which all tests should stop running.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Set a real time of day at which all tests should timeout. Example:
|
|
|
|
|
|
|
|
``7:00:00 -0400``. Any time format understood by the curl date parser
|
|
|
|
|
|
|
|
is accepted. Local time is assumed if no timezone is specified.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--print-labels``
|
|
|
|
|
|
|
|
Print all available test labels.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This option will not run any tests, it will simply print the list of
|
|
|
|
|
|
|
|
all labels associated with the test set.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. include:: OPTIONS_HELP.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. _`Label and Subproject Summary`:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Label and Subproject Summary
|
|
|
|
|
|
|
|
============================
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CTest prints timing summary information for each label and subproject
|
|
|
|
|
|
|
|
associated with the tests run. The label time summary will not include labels
|
|
|
|
|
|
|
|
that are mapped to subprojects.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When the :prop_test:`PROCESSORS` test property is set, CTest will display a
|
|
|
|
|
|
|
|
weighted test timing result in label and subproject summaries. The time is
|
|
|
|
|
|
|
|
reported with `sec*proc` instead of just `sec`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The weighted time summary reported for each label or subproject j is computed
|
|
|
|
|
|
|
|
as::
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Weighted Time Summary for Label/Subproject j =
|
|
|
|
|
|
|
|
sum(raw_test_time[j,i] * num_processors[j,i], i=1...num_tests[j])
|
|
|
|
|
|
|
|
|
|
|
|
This option tells ctest to configure (i.e. run cmake on), build,
|
|
|
|
for labels/subprojects j=1...total
|
|
|
|
and or execute a test. The configure and test steps are optional.
|
|
|
|
|
|
|
|
The arguments to this command line are the source and binary
|
|
|
|
where:
|
|
|
|
directories.
|
|
|
|
|
|
|
|
The ``--build-generator`` option *must* be provided to use
|
|
|
|
* raw_test_time[j,i]: Wall-clock time for the ith test for the jth label or
|
|
|
|
``--build-and-test``. If ``--test-command`` is specified then that will be
|
|
|
|
subproject
|
|
|
|
run after the build is complete. Other options that affect this
|
|
|
|
* num_processors[j,i]: Value of the CTest PROCESSORS property for the ith test
|
|
|
|
mode are ``--build-target``, ``--build-nocmake``, ``--build-run-dir``,
|
|
|
|
for the jth label or subproject
|
|
|
|
``--build-two-config``, ``--build-exe-dir``,
|
|
|
|
* num_tests[j]: Number of tests associated with the jth label or subproject
|
|
|
|
``--build-project``, ``--build-noclean`` and ``--build-options``.
|
|
|
|
* total: Total number of labels or subprojects that have at least one test run
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Therefore, the weighted time summary for each label or subproject represents
|
|
|
|
|
|
|
|
the amount of time that CTest gave to run the tests for each label or
|
|
|
|
|
|
|
|
subproject and gives a good representation of the total expense of the tests
|
|
|
|
|
|
|
|
for each label or subproject when compared to other labels or subprojects.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For example, if "SubprojectA" showed "100 sec*proc" and "SubprojectB" showed
|
|
|
|
|
|
|
|
"10 sec*proc", then CTest allocated approximately 10 times the CPU/core time
|
|
|
|
|
|
|
|
to run the tests for "SubprojectA" than for "SubprojectB" (e.g. so if effort
|
|
|
|
|
|
|
|
is going to be expended to reduce the cost of the test suite for the whole
|
|
|
|
|
|
|
|
project, then reducing the cost of the test suite for "SubprojectA" would
|
|
|
|
|
|
|
|
likely have a larger impact than effort to reduce the cost of the test suite
|
|
|
|
|
|
|
|
for "SubprojectB").
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. _`Build and Test Mode`:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Build and Test Mode
|
|
|
|
|
|
|
|
===================
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CTest provides a command-line signature to to configure (i.e. run cmake on),
|
|
|
|
|
|
|
|
build, and or execute a test::
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ctest --build-and-test <path-to-source> <path-to-build>
|
|
|
|
|
|
|
|
--build-generator <generator> [<options>...] [-- <build-options>...]
|
|
|
|
|
|
|
|
[--test-command <test>]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The configure and test steps are optional. The arguments to this command line
|
|
|
|
|
|
|
|
are the source and binary directories. The ``--build-generator`` option *must*
|
|
|
|
|
|
|
|
be provided to use ``--build-and-test``. If ``--test-command`` is specified
|
|
|
|
|
|
|
|
then that will be run after the build is complete. Other options that affect
|
|
|
|
|
|
|
|
this mode include:
|
|
|
|
|
|
|
|
|
|
|
|
``--build-target``
|
|
|
|
``--build-target``
|
|
|
|
Specify a specific target to build.
|
|
|
|
Specify a specific target to build.
|
|
|
|
|
|
|
|
|
|
|
|
This option goes with the ``--build-and-test`` option, if left out the
|
|
|
|
If left out the ``all`` target is built.
|
|
|
|
``all`` target is built.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--build-nocmake``
|
|
|
|
``--build-nocmake``
|
|
|
|
Run the build without running cmake first.
|
|
|
|
Run the build without running cmake first.
|
|
|
@ -324,67 +433,47 @@ Options
|
|
|
|
``--test-command``
|
|
|
|
``--test-command``
|
|
|
|
The test to run with the ``--build-and-test`` option.
|
|
|
|
The test to run with the ``--build-and-test`` option.
|
|
|
|
|
|
|
|
|
|
|
|
``--test-output-size-passed <size>``
|
|
|
|
|
|
|
|
Limit the output for passed tests to ``<size>`` bytes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--test-output-size-failed <size>``
|
|
|
|
|
|
|
|
Limit the output for failed tests to ``<size>`` bytes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--test-timeout``
|
|
|
|
``--test-timeout``
|
|
|
|
The time limit in seconds, internal use only.
|
|
|
|
The time limit in seconds
|
|
|
|
|
|
|
|
|
|
|
|
``--tomorrow-tag``
|
|
|
|
|
|
|
|
Nightly or experimental starts with next day tag.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This is useful if the build will not finish in one day.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--ctest-config``
|
|
|
|
|
|
|
|
The configuration file used to initialize CTest state when submitting dashboards.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This option tells CTest to use different initialization file instead
|
|
|
|
.. _`Dashboard Client`:
|
|
|
|
of CTestConfiguration.tcl. This way multiple initialization files
|
|
|
|
|
|
|
|
can be used for example to submit to multiple dashboards.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--overwrite``
|
|
|
|
|
|
|
|
Overwrite CTest configuration option.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
By default ctest uses configuration options from configuration file.
|
|
|
|
Dashboard Client
|
|
|
|
This option will overwrite the configuration option.
|
|
|
|
================
|
|
|
|
|
|
|
|
|
|
|
|
``--extra-submit <file>[;<file>]``
|
|
|
|
CTest can operate as a client for the `CDash`_ software quality dashboard
|
|
|
|
Submit extra files to the dashboard.
|
|
|
|
application. As a dashboard client, CTest performs a sequence of steps
|
|
|
|
|
|
|
|
to configure, build, and test software, and then submits the results to
|
|
|
|
|
|
|
|
a `CDash`_ server. The command-line signature used to submit to `CDash`_ is::
|
|
|
|
|
|
|
|
|
|
|
|
This option will submit extra files to the dashboard.
|
|
|
|
ctest (-D <dashboard> | -M <model> -T <action> | -S <script> | -SP <script>)
|
|
|
|
|
|
|
|
[-- <dashboard-options>...]
|
|
|
|
|
|
|
|
|
|
|
|
``--force-new-ctest-process``
|
|
|
|
Options for Dashboard Client include:
|
|
|
|
Run child CTest instances as new processes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
By default CTest will run child CTest instances within the same
|
|
|
|
``--track <track>``
|
|
|
|
process. If this behavior is not desired, this argument will
|
|
|
|
Specify the track to submit dashboard to
|
|
|
|
enforce new processes for child CTest processes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--schedule-random``
|
|
|
|
Submit dashboard to specified track instead of default one. By
|
|
|
|
Use a random order for scheduling tests.
|
|
|
|
default, the dashboard is submitted to Nightly, Experimental, or
|
|
|
|
|
|
|
|
Continuous track, but by specifying this option, the track can be
|
|
|
|
|
|
|
|
arbitrary.
|
|
|
|
|
|
|
|
|
|
|
|
This option will run the tests in a random order. It is commonly
|
|
|
|
``-A <file>, --add-notes <file>``
|
|
|
|
used to detect implicit dependencies in a test suite.
|
|
|
|
Add a notes file with submission.
|
|
|
|
|
|
|
|
|
|
|
|
``--submit-index``
|
|
|
|
This option tells CTest to include a notes file when submitting
|
|
|
|
Legacy option for old Dart2 dashboard server feature.
|
|
|
|
dashboard.
|
|
|
|
Do not use.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--timeout <seconds>``
|
|
|
|
``--tomorrow-tag``
|
|
|
|
Set a global timeout on all tests.
|
|
|
|
Nightly or experimental starts with next day tag.
|
|
|
|
|
|
|
|
|
|
|
|
This option will set a global timeout on all tests that do not
|
|
|
|
This is useful if the build will not finish in one day.
|
|
|
|
already have a timeout set on them.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--stop-time <time>``
|
|
|
|
``--extra-submit <file>[;<file>]``
|
|
|
|
Set a time at which all tests should stop running.
|
|
|
|
Submit extra files to the dashboard.
|
|
|
|
|
|
|
|
|
|
|
|
Set a real time of day at which all tests should timeout. Example:
|
|
|
|
This option will submit extra files to the dashboard.
|
|
|
|
``7:00:00 -0400``. Any time format understood by the curl date parser
|
|
|
|
|
|
|
|
is accepted. Local time is assumed if no timezone is specified.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
``--http1.0``
|
|
|
|
``--http1.0``
|
|
|
|
Submit using HTTP 1.0.
|
|
|
|
Submit using HTTP 1.0.
|
|
|
@ -399,26 +488,6 @@ Options
|
|
|
|
this to maintain compatibility with an older version of CDash which
|
|
|
|
this to maintain compatibility with an older version of CDash which
|
|
|
|
doesn't support compressed test output.
|
|
|
|
doesn't support compressed test output.
|
|
|
|
|
|
|
|
|
|
|
|
``--print-labels``
|
|
|
|
|
|
|
|
Print all available test labels.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This option will not run any tests, it will simply print the list of
|
|
|
|
|
|
|
|
all labels associated with the test set.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. include:: OPTIONS_HELP.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. _`Dashboard Client`:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Dashboard Client
|
|
|
|
|
|
|
|
================
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CTest can operate as a client for the `CDash`_ software quality dashboard
|
|
|
|
|
|
|
|
application. As a dashboard client, CTest performs a sequence of steps
|
|
|
|
|
|
|
|
to configure, build, and test software, and then submits the results to
|
|
|
|
|
|
|
|
a `CDash`_ server.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. _`CDash`: http://cdash.org/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Dashboard Client Steps
|
|
|
|
Dashboard Client Steps
|
|
|
|
----------------------
|
|
|
|
----------------------
|
|
|
|
|
|
|
|
|
|
|
@ -731,7 +800,6 @@ Configuration settings to specify the version control tool include:
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_UPDATE_VERSION_ONLY`
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_UPDATE_VERSION_ONLY`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Additional configuration settings include:
|
|
|
|
Additional configuration settings include:
|
|
|
|
|
|
|
|
|
|
|
|
``NightlyStartTime``
|
|
|
|
``NightlyStartTime``
|
|
|
@ -773,6 +841,7 @@ Configuration settings include:
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
|
|
|
|
* :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
|
|
|
|
* :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See `Label and Subproject Summary`_.
|
|
|
|
|
|
|
|
|
|
|
|
.. _`CTest Build Step`:
|
|
|
|
.. _`CTest Build Step`:
|
|
|
|
|
|
|
|
|
|
|
@ -804,6 +873,8 @@ Configuration settings include:
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
|
|
|
|
* :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
|
|
|
|
* :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See `Label and Subproject Summary`_.
|
|
|
|
|
|
|
|
|
|
|
|
``MakeCommand``
|
|
|
|
``MakeCommand``
|
|
|
|
Command-line to launch the software build process.
|
|
|
|
Command-line to launch the software build process.
|
|
|
|
It will be executed in the location specified by the
|
|
|
|
It will be executed in the location specified by the
|
|
|
@ -847,6 +918,7 @@ Configuration settings include:
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
|
|
|
|
* `CTest Script`_ variable: :variable:`CTEST_LABELS_FOR_SUBPROJECTS`
|
|
|
|
* :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
|
|
|
|
* :module:`CTest` module variable: ``CTEST_LABELS_FOR_SUBPROJECTS``
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
See `Label and Subproject Summary`_.
|
|
|
|
|
|
|
|
|
|
|
|
``TestLoad``
|
|
|
|
``TestLoad``
|
|
|
|
While running tests in parallel (e.g. with ``-j``), try not to start
|
|
|
|
While running tests in parallel (e.g. with ``-j``), try not to start
|
|
|
@ -1082,3 +1154,5 @@ See Also
|
|
|
|
========
|
|
|
|
========
|
|
|
|
|
|
|
|
|
|
|
|
.. include:: LINKS.txt
|
|
|
|
.. include:: LINKS.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. _`CDash`: http://cdash.org/
|
|
|
|