257 lines
12 KiB
Plaintext
257 lines
12 KiB
Plaintext
RunCMake.CPack is a test module that is intended for testing of package
|
|
generators that can be validated from command line.
|
|
|
|
-------------
|
|
Adding a test
|
|
-------------
|
|
|
|
CPack test root directory: 'Tests/RunCMake/CPack/tests'.
|
|
|
|
All phases are executed separately for each generator that is bound to a test.
|
|
Tests for each generator are subtests of test 'RunCMake.CPack_<generator_name>'.
|
|
|
|
Each test must also be added to 'RunCMakeTest.cmake' script located in CPack
|
|
test root directory.
|
|
|
|
Line that adds a test is:
|
|
run_cpack_test(<test_name> "<generator_name_list>" <compile_stage>
|
|
"<packaging_type_list>")
|
|
|
|
<generator_name_list> may be one generator e.g. "RPM" or multiple e.g. "RPM;DEB"
|
|
and will be run for all listed generators. In test and verification scripts
|
|
current generator can be accessed through GENERATOR_TYPE variable.
|
|
|
|
<compile_stage> is a boolean variable that enables or disables compile stage -
|
|
most tests don't require compilation as a non binary file can be used for
|
|
package content but sometimes an executable or a library has to be created
|
|
before the packaging stage.
|
|
|
|
<packaging_type_list> can be a list of one or more packaging types: MONOLITHIC,
|
|
COMPONENT or GROUP - each type sets per generator default variables which can
|
|
be overwritten in the test if needed
|
|
(see <generator_type>/packaging_<packaging_type>_default.cmake for the variables
|
|
that are set by default for each packaging type).
|
|
Alternatively CUSTOM value can be set which means that default values will not
|
|
be set and that values will be set manually in the test itself.
|
|
|
|
Alternatively a test with subtests can be added:
|
|
run_cpack_test_subtests(<test_name> "<subtests_list>" "<generator_name_list>"
|
|
<compile_stage> "<packaging_type_list>")
|
|
|
|
<subtests_list> is the only new parameter and it is a list of names that will
|
|
be used for subtests. In test and verification scripts subtest name can be
|
|
accessed through RunCMake_SUBTEST_SUFFIX variable.
|
|
|
|
Also source package tests can be added:
|
|
run_cpack_source_test(<test_name> "<generator_name_list>" true)
|
|
|
|
Test consists of
|
|
- test prerequirements phase (optional)
|
|
- CMake execution phase
|
|
- build phase (optional and not available for source package tests)
|
|
- CPack execution phase
|
|
- verification of generated files
|
|
|
|
The phases are executed once per specified generator, packaging type and subtest
|
|
combinatiion.
|
|
|
|
test prerequirements phase (optional):
|
|
--------------------------------------
|
|
|
|
In some cases individual tests for certain generator need additional
|
|
prerequirements met.
|
|
|
|
In such cases '<test_name>/<generator_name>-Prerequirements.cmake' file
|
|
containing 'function(get_test_prerequirements found_var config_file)' should be
|
|
created. Function should return true in found_var if all prerequirements are
|
|
met. config_file variable contains the location of config file generated by the
|
|
generator so that this function can check if prerequired variable is set in the
|
|
file.
|
|
NOTE: All required programs should be searched for in generator prerequirements
|
|
function and only checked for the variable in configure file in per test
|
|
function.
|
|
|
|
If prerequirements are not met test will be skipped outputting
|
|
'<test_name> - SKIPPED' string. Note that this doesn't fail the entire test
|
|
group.
|
|
|
|
TODO: skipped tests should provide expected error string so test should fail
|
|
if error string is not found in the output of run test (this would add
|
|
'EXPECTED FAIL' string on success and 'ERROR' on failure).
|
|
|
|
CMake execution phase:
|
|
----------------------
|
|
|
|
To add a new CPack test we first create a '<test_name>/test.cmake' script that
|
|
contains CMake commands that should be used as a preparation script for
|
|
generation of different types of packages. This script is placed into CPack
|
|
test root directory.
|
|
|
|
If test will be used for multiple generators but some of them require some
|
|
generator specific commands then those commands should be added to 'test.cmake'
|
|
script wrapped with 'if(GENERATOR_TYPE STREQUAL <name_of_the_generator>)'.
|
|
|
|
NOTE: In some cases (for example when testing CPackComponent.cmake functions)
|
|
the test has to run some functions after CPack.cmake is included. In such cases
|
|
a function run_after_include_cpack can be declared in test.cmake file and that
|
|
function will run after the inclusion of CPack.cmake.
|
|
|
|
NOTE: During CMake configure stage developer warnings may be expected. In such
|
|
cases an expected output regular expression can be provided by creating
|
|
'<test_name>/configure-stdout.txt' and/or '<test_name>/configure-stderr.txt'
|
|
file. There are also more specialized versions of the file available:
|
|
- configure-${PACKAGING_TYPE}-${SUBTEST_SUFFIX}-std${o}.txt
|
|
- configure-${SUBTEST_SUFFIX}-std${o}.txt
|
|
- configure-${PACKAGING_TYPE}-std${o}.txt
|
|
|
|
build phase (optional and not available for source package tests)
|
|
-----------------------------------------------------------------
|
|
|
|
This phase only runs make command.
|
|
|
|
NOTE: By default all tests have CMAKE_BUILD_TYPE variable set to Debug.
|
|
|
|
CPack execution phase:
|
|
----------------------
|
|
|
|
Only executes CPack for content that was generated during CMake execution
|
|
phase.
|
|
|
|
NOTE: By default CPACK_PACKAGE_NAME variable is set to lower case test name.
|
|
|
|
Verification of generated files:
|
|
--------------------------------
|
|
|
|
Verification of generated files consists of two phases
|
|
- mandatory verification phase
|
|
- optional verification phase
|
|
|
|
Mandatory verification phase checks that expected files were generated and
|
|
contain expected files.
|
|
Mandatory verification phase also checks that no other unexpected package files
|
|
were generated (this is executed even if EXPECTED_FILES_COUNT contains 0 in
|
|
order to verify that no files were generated).
|
|
CMake script '<test_name>/ExpectedFiles.cmake' is required by
|
|
this step and must contain
|
|
- EXPECTED_FILES_COUNT variable that contains the number of expected files that
|
|
will be generated (0 or more)
|
|
|
|
- EXPECTED_FILE_<file_number_starting_with_1> that contains globing expression
|
|
that uniquely defines expected file name (will be used to find expected file)
|
|
and should be present once for each expected file.
|
|
NOTE: This variable should be used only as last resort as it sets generator
|
|
specific globbing expression. Each generator can recreate file name from
|
|
parts that are already populated by default but can always be
|
|
overwritten if needed:
|
|
- EXPECTED_FILE_<file_number_starting_with_1>_NAME is the name component of
|
|
the file (by default it is set to package name that is same as test name
|
|
in lowercase)
|
|
- EXPECTED_FILE_<file_number_starting_with_1>_VERSION is the version of the
|
|
package (by default it is set to '0.1.1')
|
|
- EXPECTED_FILE_<file_number_starting_with_1>_REVISION is the revision of the
|
|
package (by default it is set to '1')
|
|
|
|
- EXPECTED_FILE_CONTENT_<file_number_starting_with_1> that contains regular
|
|
expression of files that should be present in generated file and should be
|
|
present once for each expected file
|
|
NOTE: This variable should be used only as last resort as it sets generator
|
|
specific regular expression.
|
|
EXPECTED_FILE_CONTENT_<file_number_starting_with_1>_LIST should be
|
|
preferred as it requires a list of expected files and directories that
|
|
is later changed automatically depending on the generator so expected
|
|
package content can be written only once per test for all generators.
|
|
|
|
- EXPECTED_FILE_PACKAGING_PREFIX and
|
|
EXPECTED_FILE_<file_number_starting_with_1>_PACKAGING_PREFIX variables can be
|
|
set to explicitly specified CPACK_PACKAGING_PREFIX value. By default this
|
|
variable does not need to be set as it is implicitly set to package generator
|
|
specific prefix.
|
|
|
|
Optional verification phase is generator specific and is optionally executed.
|
|
This phase is executed if '<test_name>/VerifyResult.cmake' script exists.
|
|
|
|
VerifyResult.cmake script also automatically prints out standard output and
|
|
standard error from CPack execution phase that is compared with
|
|
'<test_name>/<generator_name>-stdout.txt' regular expression and
|
|
'<test_name>/<generator_name>-stderr.txt' regular expresson respectively.
|
|
NOTE: For subtests generator name can also be suffixed with subtest name and/or
|
|
packaging type (MONOLITHIC, COMPONENT, GROUP) and in such cases the
|
|
preferences of which file will be used are as follows:
|
|
- generator name + packaging type + subtest name
|
|
- generator name + packaging type
|
|
- generator name + subtest name
|
|
- generator name
|
|
- subtest name
|
|
- default generator
|
|
File name format: '<generator_name>-<packaging_type>-<subtest_name>-std<type>.txt'
|
|
where <type> can either be 'out' or 'err'.
|
|
File name format: '<generator_name>-<packaging_type>-std<type>.txt'
|
|
where <type> can either be 'out' or 'err'.
|
|
File name format: '<generator_name>-<subtest_name>-std<type>.txt' where
|
|
<type> can either be 'out' or 'err'.
|
|
NOTE: If none of the comparison files are present then the default generator
|
|
file is used if present.
|
|
|
|
----------------------
|
|
Adding a new generator
|
|
----------------------
|
|
|
|
To add a new generator we must
|
|
- add new generator directory (e.g. RPM for RPM generator) to CPack test root
|
|
directory that contains 'Helpers.cmake' script.
|
|
- In this script some functions must exist:
|
|
- getPackageContent: This function should list files that are contained in
|
|
a package.
|
|
Function parameters:
|
|
+ FILE variable that will contain path to file for which the content
|
|
should be listed
|
|
+ RESULT_VAR that will tell the function which variable in parent scope
|
|
should contain the result (list of files inside package file)
|
|
- getPackageNameGlobexpr: This function should generate package name
|
|
globbing expression.
|
|
Function parameters:
|
|
+ NAME that will contain the expected package name
|
|
+ COMPONENT that will contain the expected package component
|
|
+ VERSION that will contain the expected package version
|
|
+ REVISION that will contain the expected package revision number
|
|
+ FILE_NO that will contain the file number for which the globbing
|
|
expression is generated
|
|
+ RESULT_VAR that will tell the function which variable in parent scope
|
|
should contain the result (file name globbing expression)
|
|
- getPackageContentList: This function should return a list of files and
|
|
directories that are in the provided package.
|
|
Function parameters:
|
|
+ FILE that will contain the package file for which the package content
|
|
should be returned.
|
|
+ RESULT_VAR that will tell the function which variable in parent scope
|
|
should contain the result (list of package content)
|
|
- toExpectedContentList: This function should convert an expected package
|
|
content list into one that is expected for the
|
|
generator (e.g. rpm packages have install/relocate
|
|
path prefixes which aren't part of the package so
|
|
those paths have to be removed from the expected
|
|
content list).
|
|
Function parameters:
|
|
+ FILE_NO that will contain the file number for which the conversion
|
|
should be performed
|
|
+ CONTENT_VAR that will contain the input list and is also the variable
|
|
in parent scope which should contain the result (converted content list)
|
|
- add 'Prerequirements.cmake' script to generator directory. In this script a
|
|
function named 'get_test_prerequirements' must exist. This function should
|
|
set a variable in parent scope (name of the variable is the first parameter)
|
|
that tells if prerequirements for test execution are met (certain programs,
|
|
OS specifics, ...) and create a config file (name of the variable which
|
|
contains file name and path is provided with the second parameter) that
|
|
should contain 'set' commands for variables that will later be used in tests
|
|
(e.g. location of dpkg program for DEB packages)
|
|
- add tests the same way as described above
|
|
- add generator to 'add_RunCMake_test_group' function call that is located in
|
|
RunCMake CMakeLists.txt file
|
|
- if needed add 'packaging_<packaging_type>_default.cmake' scripts that define
|
|
default variables that will be set for each packaging type (MONOLITHIC,
|
|
COMPONENT and GROUP)
|
|
- if needed add 'default_expected_std<type>.txt' files where <type> is either
|
|
'out' or 'err' which will contain default expected output of package
|
|
generation regular expression.
|
|
- add generator to list of other CPack generators in RunCMake/CMakeLists.txt
|