2009 -1098 P Trilinos Usage of CMake CTest CDash Roscoe

2009 -1098 P Trilinos Usage of CMake/CTest/CDash Roscoe A. Bartlett http: //www. cs. sandia. gov/~rabartl/ Department of Optimization & Uncertainty Estimation Sandia National Laboratories Kitware/SNL CMake/CTest/CDash Workshop May 7, 2009 Page Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC 04 -94 AL 85000.

Primary Motivations for Switching Trilinos • Native support for MS Windows (MS Visual Studio projects and selfextracting installers) and Mac (XCode project files and binary installers) • Improve robustness of our development and testing environments – Improve the stability of important Trilinos code – Support stronger collaborations with important customer codes (i. e. Almost Continuous Integration) – Better support R&D of deep stacks of vertically integrated algorithms CMake Transition • Decrease the maintenance costs of the framework and testing infrastructure Autotools Total Effort – Reduce maintenance cost by 1/2 to 1/3! – Allow Jim Willenbring and Brent Perschbacher to do more important things CMake Time Page

Gains & (Initial) Looses Switching to CMake for Builds • What we gain: – Full dependency tracking of every kind possible on all platforms (i. e. header to object, object to library, library to executable, and build system files to all built files) – Support for shared libraries on a variety of platforms – Support for MS Windows (i. e. Visual Studio projects, Windows installers, etc. ) – Simplified build system and easier maintenance (extremely easy to add new packages and maintain existing packages) – Improved mechanism for extending capabilities (as compared to M 4 in autotools) – Ability to affect the development of the build tools with good existing collaborations – Significant ``in house'' knowledge-base – One hundred percent automated intra-package dependency tracking and handling (built into the prototype Trilinos/CMake build system) • What we lose: – CMake requires that all uses have 'cmake' installed on their machine when building from source and users will need to have at a very recent version of cmake. (However, cmake is very easy to build from source) Page

Gains (no losses) Switching to CTest/CDash for Testing • What we gain: – Test time-outs (this is a major maintenance issue for the current Perl-based test harness) – Memory testing with Valgrind and purify that is backed up by Kitware and a larger development community – Line coverage testing that is backed up by Kitware and a large development community – Support for selecting and excluding subsets of tests based on regular expressions or named keywords – Better integration with the build system (e. g. easier to support more advanced features like PBS batch systems and flexible testing control) – Better tracking of specific tests (i. e. each and every test can have a unique name that is easy to find) • What we lose: – Nothing! Page

Design Principles for Trilinos CMake Build System: #1 • Make it exceedingly easy to define CMake files for new packages and to define libraries, tests, and examples in those packages. • Create a design for building individual package CMake files that automatically results in uniformity of how things are done. – Use standard macros to define every package's main features – Simplify maintenance. • Allow changes to logic and functionality that apply to all Trilinos packages without having to touch each individual Trilinos package's CMake files. • Provide 100% automatic intra-package dependencies handling – Avoid mistakes – Avoid duplication – Robustifies a number of important features. • Provide built-in automated support for as many critical software engineering practices a possible – Proper pre-checkin testing when continuous integration is being performed Page

Design Principles for Trilinos CMake Build System: #2 • Avoid duplication of all kinds as much as possible. – Fundamental software maintenance issue. • The build system should be able to reproduce 100% update-to-date output by simply typing ‘make’. – E. g. Coping test input files to binary directory • Aggregate as much common functionality as possible to the top-level CMake files but allow individual CMake packages to refine the logic if they really need to. • Accept extra complexity at the global framework level to reduce it at the package level – Apply solid software engineering design principles to manage the complexity – Remove duplication Page

Outline of CMake Package. Arch Build System Trilinos/ CMake. Lists. txt # Top-level build file cmake/ # Cmake support Trilinos. Packages. cmake # List of Trilinos packages Trilinos. TPLs. cmake # Lists of Trilinos TPLs. . . package_arch # Generic package architecture macros Package. Arch. Global. Macros. cmake Package. Library. Macros. cmake Package. Add. Executable. cmake. . . packages/. . . epetraext/ CMake. Lists. txt # Top-level package build file cmake/ Dependencies. cmake # Defines intra-package dependency and TPL lists Epetra. Ext_config. h. in # Copied from. . /src/ and hand modified src/ CMake. Lists. txt # Defines library sources and library(s) test/ CMake. Lists. txt Matrix/ CMake. Lists. txt # Define actual test executables and test runs. . . Page

Adding Packages in Trilinos. Packages. cmake SET( Trilinos_PACKAGES_AND_DIRS_AND_CATEGORIES Teuchos teuchos PS RTOp rtop PS Epetra epetra PS Triutils triutils PS Epetra. Ext epetraext PS Thyra thyra PS RBGen rbgen SS For. Trilinos EX Py. Trilinos SS. . . ) • Adding a new Trilinos Package is a 1 -line addition at the Framework Level! • NOTE: The packages must be listed in a order of strictly increasing dependences! • NOTE: If you get the ordering wrong, the automated dependency handling CMake scripts will automatically detect this and issue a very good error messages before the build is performed! • PS = Primary Stable Code, SS = Secondary Stable Code, EX = Experimental Code Page

Defining a Package’s CMake Build Files Trilinos/packges/ epetraext/CMake. Lists. txt INCLUDE(Package. Macros) INCLUDE(Add. Option. And. Define) # # A) Define the package # PACKAGE(Epetra. Ext) # # B) Set up package-specific options # ADD_OPTION_AND_DEFINE(${PACKAGE_NAME}_BUILD_TRANSFORM HAVE_TRANSFORM "Enable transform functionality. " ON ). . . # # C) Add the libraries, tests, and examples # ADD_SUBDIRECTORY(src) PACKAGE_ADD_TEST_DIRECTORIES(test) # # D) Do standard postprocessing # PACKAGE_POSTPROCESS() Page • Utility macros provide framework hooks into package functionality – Defines common behavior across all packages – Avoids duplication – Facilitates maintenance epetraext/cmake/Depencencies. cmake SET(LIB_REQUIRED_DEP_PACKAGES Epetra Teuchos) SET(LIB_OPTIONAL_DEP_PACKAGES Triutils) SET(TEST_REQUIRED_DEP_PACKAGES) SET(TEST_OPTIONAL_DEP_PACKAGES) SET(LIB_REQUIRED_DEP_TPLS) SET(LIB_OPTIONAL_DEP_TPLS) SET(TEST_REQUIRED_DEP_TPLS) SET(TEST_OPTIONAL_DEP_TPLS) • Intra-package dependencies ! – Used in all intra-package dependency handing – All header-file paths and link libraries and directories are handled automatically – These dependencies can not be wrong! (i. e. the libs and execs would not build and link otherwise) – Optional dependencies create variables _ENABLE_ and HAVE__ !

Automatic Intra-Package Dependency Handling $. /do-configure -D Trilinos_DUMP_PACKAGE_DEPENDENCIES: BOOL=ON Configuring Trilinos build directory Printing package dependencies. . . -- Teuchos_FORWARD_LIB_REQUIRED_DEP_PACKAGES='RTOp; Epetra. Ext; Isorropia; Thyra; Galeri; Amesos; Ifpack; Belos; RBGen' -- Teuchos_FORWARD_LIB_OPTIONAL_DEP_PACKAGES='Aztec. OO; ML' -- Epetra_FORWARD_LIB_REQUIRED_DEP_PACKAGES='Triutils; Epetra. Ext; Isorropia; Aztec. OO; Galeri; Amesos; Ifpack; Belos' -- Epetra_FORWARD_LIB_OPTIONAL_DEP_PACKAGES='Thyra; ML; RBGen' -- Zoltan_FORWARD_LIB_REQUIRED_DEP_PACKAGES='Isorropia'. . . ----- Epetra. Ext_LIB_REQUIRED_DEP_PACKAGES='Epetra; Teuchos' Epetra. Ext_LIB_OPTIONAL_DEP_PACKAGES='Triutils' Epetra. Ext_FORWARD_LIB_OPTIONAL_DEP_PACKAGES='Isorropia; Thyra; Galeri; Amesos; ML' Epetra. Ext_FORWARD_TEST_OPTIONAL_DEP_PACKAGES='Belos' -- Isorropia_LIB_REQUIRED_DEP_PACKAGES='Teuchos; Epetra; Zoltan' -- Isorropia_LIB_OPTIONAL_DEP_PACKAGES='Epetra. Ext' -- Isorropia_FORWARD_LIB_OPTIONAL_DEP_PACKAGES='ML' -- Thyra_LIB_REQUIRED_DEP_PACKAGES='RTOp; Teuchos' -- Thyra_LIB_OPTIONAL_DEP_PACKAGES='Epetra. Ext; Epetra' -- Thyra_FORWARD_LIB_REQUIRED_DEP_PACKAGES='Stratimikos'. . . -- Stratimikos_LIB_REQUIRED_DEP_PACKAGES='Thyra' -- Stratimikos_LIB_OPTIONAL_DEP_PACKAGES='Amesos; Aztec. OO; Belos; Ifpack; ML' -- Stratimikos_TEST_OPTIONAL_DEP_PACKAGES='Triutils' Page

Adding a Library in PACKAGE/src/CMake. Lists. txt INCLUDE(Package. Library. Macros) # A) Package-specific configuration options • Dependent package header directories, libraries, and library link directories handled automatically! PACKAGE_CONFIGURE_FILE(${PROJECT_NAME}_config. h) # B) Define the header and source files (and directories) INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR}) SET(HEADERS Epetra. Ext_Config. Defs. h. . . ) SET(SOURCES Epetra. Ext_Product. Operator. cpp. . . ). . . # C) Define the targets for package's library(s) PACKAGE_ADD_LIBRARY( epetraext HEADERS ${HEADERS} SOURCES ${SOURCES} ) Page • Macros provide uniform behavior across all libraries across all packages! • Test-only libraries are supported with ‘TESTONLY’ flag (see NOX for an example)

Adding a Test in PACKAGE/test/CMake. Lists. txt epetraext/test/CMake. Lists. txt # Compile against epetra_test_err. h in all tests? INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR}) • Add common header include dirs . . . ADD_SUBDIRECTORY(Matrix) • Just include the subdirectories epetraext/test/Matrix/CMake. Lists. txt INCLUDE(Package. Add. Executable. And. Test) INCLUDE(Package. Copy. Files. To. Binary. Dir) PACKAGE_ADD_EXECUTABLE_AND_TEST( Matrix_test SOURCES cxx_main. cpp COMM serial mpi ) • All header paths, link libraries etc are handled automatically! • Define executable and test in one shot! • 100% correct dependency tracking! PACKAGE_COPY_FILES_TO_BINARY_DIR( Epetra. Ext. Matrix. Copy. Files DEST_FILES infile. AB infile. ATBT infile. AB 2 infiles infile. ABT infile. AB 3 infile. ATB 2 C. mtx C 4 x 4. mtx C 4 x 12 x 4. mtx C 4 x 6. mtx C 6 x 4. mtx C 6 x 6. mtx mat 6 x 4. mtx mat 6 x 6. mtx mat 12 x 4. mtx mat 4 x 12. mtx mat 4 x 4. mtx mat 4 x 6. mtx YTC. mtx Y_transp. mtx roman 2 romancase 2 cdt_case cdt_d. mtx cdt_m. mtx cdt_tce. mtx SOURCE_DIR ${${PACKAGE_NAME}_SOURCE_DIR}/test/Matrix SOURCE_PREFIX "src_“ EXEDEPS Matrix_test ) Page

Defining slightly more complex tests thyra/test/operator_solve/CMake. Lists. txt INCLUDE(Package. Add. Executable. And. Test) PACKAGE_ADD_EXECUTABLE( test_linear_op_with_solve SOURCES test_linear_op_with_solve. cpp COMM serial mpi ) PACKAGE_ADD_TEST( test_linear_op_with_solve NAME test_linear_op_with_solve_n 1_n 2 ARGS "--n=1" "--n=2" NUM_MPI_PROCS 1 COMM serial mpi ) PACKAGE_ADD_TEST( test_linear_op_with_solve NAME test_linear_op_with_solve_n 4 ARGS "--n=4" NUM_MPI_PROCS 1 COMM serial mpi XHOST s 858352 s 903186 ) Page • Define test cases separately from executable if needed!

Defining More Sophisticated Tests Current Limits of built-in CTest tests: • Runs a single executable (or script) with a set of arguments • Pass/fail determined by return value (0), or regex fail or regex any pass on STDOUT • Runs test in current directory What we would like: • Run multiple commands • Define pass/fail for each command in various ways (examine other output files, examine output from multiple commands, etc. ) • Run tests in subdirectories created for the test Solution: • Write user-defined CMake function PACKAGE_ADD_ADVANCED_TEST(. . . ) • Write ‘cmake –P’ scripts Page

Defining More Sophisticated Tests Define advanced test: PACKAGE_ADD_ADVANCED_TEST( TEST_0 (EXEC | CMND ). . . [TEST_1 [EXEC | CMND ]. . . [TEST_N [EXEC | CMND ]. . . ] [OVERALL_WORKING_DIRECTORY ] [FAIL_FAST] [KEYWORDS . . . ] [COMM [serial] [mpi]] [OVERALL_NUM_MPI_PROCS ] [HOST . . . ] [XHOST . . . ] [FINAL_PASS_REGULAR_EXPRESSION | FINAL_FAIL_REGULAR_EXPRESSION ] ) Where: TEST_ EXEC [NOEXEPREFIX] [NOEXESUFFIX] [ADD_DIR_TO_NAME] | CMND [ARGS . . . ] [MESSAGE ""] [WORKING_DIRECTORY ] [NUM_MPI_PROCS ] [OUTPUT_FILE ] [NO_ECHO_OUTPUT]] [PASS_ANY | PASS_REGULAR_EXPRESSION "" | PASS_REGULAR_EXPRESSION_ALL "" "". . . "" | FAIL_REGULAR_EXPRESSION "" | STANDARD_PASS_OUTPUT Page ]