Features/Meson: Difference between revisions

From QEMU
Line 99: Line 99:
;add_global_dependency or <tt><nowiki>dependency::get_compile_args</nowiki></tt>
;add_global_dependency or <tt><nowiki>dependency::get_compile_args</nowiki></tt>
:Needed to move glib detection to Meson.
:Needed to move glib detection to Meson.
;[https://github.com/mesonbuild/meson/issues/8333 Can't extract objects form custom_target()]
:extract_objects does not support generated file (not needed yet but could be surprising)
;[https://github.com/mesonbuild/meson/issues/7585 test doesn't take custom_target index objects]
:see tests/qapi-schema/meson.build
;[https://github.com/mesonbuild/meson/issues/8824 Add "c_winlibs" und "cpp_winlibs" to target functions]
;[https://github.com/mesonbuild/meson/issues/8824 Add "c_winlibs" und "cpp_winlibs" to target functions]
:Should be useful for qemu-ga and friends.
:Should be useful for qemu-ga and friends.
Line 113: Line 109:
;[https://github.com/mesonbuild/meson/pull/8151 transitively pass compile flags for internal dependency]
;[https://github.com/mesonbuild/meson/pull/8151 transitively pass compile flags for internal dependency]
:Enables a better fix for https://bugs.launchpad.net/qemu/+bug/1909256
:Enables a better fix for https://bugs.launchpad.net/qemu/+bug/1909256
;[https://github.com/mesonbuild/meson/pull/9400 allow custom targets in extract_objects and test]
:Useful for modinfo to receive object files instead of source files; segue to [https://github.com/mesonbuild/meson/pull/8900 this one that was merged in 0.58.2]


=== Merged for 0.57 ===
=== meson test ===
<!--
;[https://github.com/mesonbuild/meson/issues/7830 Improve `meson test`] (0.57)
;[https://github.com/mesonbuild/meson/pull/7876 Allow internal dependencies for "links" if they don't have sources]
:Probably useful in order to move glib detection to Meson.
;[https://github.com/mesonbuild/meson/pull/7900 Avoid build.ninja changes due to order of hash table iteration]
:Avoids rebuilding a lot of files after "touch ../meson.build && make"
;[https://github.com/mesonbuild/meson/pull/7902/ mtest: only build what is needed for the tests]
:Makes "meson test" match Makefile.mtest behavior with respect to rebuilds
;[https://github.com/mesonbuild/meson/pull/8231  Allow passing external program and dependency objects to summary()]
:Makes summary nicer (both code and looks) - broken in 0.58.1 and 0.59.1
-->
;[https://github.com/mesonbuild/meson/issues/7830 Improve `meson test`]
:Get rid of scripts/tap-driver.pl
:Get rid of scripts/tap-driver.pl
;[https://github.com/mesonbuild/meson/pull/8288 introduce add_test_setup(exclude suites: ...) keyword argument]
;[https://github.com/mesonbuild/meson/pull/8288 introduce add_test_setup(exclude suites: ...) keyword argument] (0.57)
:Can be useful to simplify invocation of "slow" vs. "quick" tests without a Makefile wrapper
:Can be useful to simplify invocation of "slow" vs. "quick" tests without a Makefile wrapper
;[https://github.com/mesonbuild/meson/pull/9410 mtest: limit "magic" CTRL+C behavior to process group leaders] (0.59.3)
:Needed to call "meson test" from make


=== Merged for 0.58.2 ===
=== extract_objects ===
'''[https://github.com/mesonbuild/meson/pull/8900 extract_objects: fixes, tests and documentation for using the result in a custom_target]'''
;'''[https://github.com/mesonbuild/meson/pull/8900 extract_objects: fixes, tests and documentation for using the result in a custom_target]''' (0.58.2)
:Useful to parse compile_commands.json in scripts/modinfo.sh
:Useful to parse compile_commands.json in scripts/modinfo.sh
=== Merged for 0.59 ===
;[https://github.com/mesonbuild/meson/pull/9400 allow custom targets in extract_objects and test] (0.61.0)
'''[https://github.com/mesonbuild/meson/pull/8512 Utility methods for feature objects]'''
:Useful for modinfo to receive object files instead of source files
 
=== Others ===
'''[https://github.com/mesonbuild/meson/pull/8512 Utility methods for feature objects]''' (0.59.0)
:Simplifies use of get_option()
:Simplifies use of get_option()
=== Merged for 0.59.3 ===
;[https://github.com/mesonbuild/meson/pull/9014 "links" test with file argument should use the appropriate compiler for the source file] (0.60.0)
;[https://github.com/mesonbuild/meson/pull/9395 introspect: include choices for array options]
;[https://github.com/mesonbuild/meson/pull/9410 mtest: limit "magic" CTRL+C behavior to process group leaders]
:Needed to call "meson test" from make
 
=== Merged for 0.60 ===
;[https://github.com/mesonbuild/meson/pull/9014 "links" test with file argument should use the appropriate compiler for the source file]
:Needed to move C++ compiler detection
:Needed to move C++ compiler detection
;Support for dependency('iconv')

Revision as of 15:29, 7 December 2021

For the rationale behind the conversion and the development notes, see Features/Meson/Design.

Summary of changes

New system requirements
  • Ninja 1.7
  • Meson 0.55.3 (in order to use the system-wide install).
Source code changes
  • You cannot #include "config-target.h" and "config-devices.h". Instead, you #include CONFIG_TARGET and CONFIG_DEVICES (i.e. filenames are specified via macros; meson.build takes care of defining the macros).
  • .c files (including .inc.c files) cannot be #included. They have been renamed to .c.inc, for consistency with existing .rst.inc files.
  • The name of files produced by decodetree is fixed (the C code for XYZ.decode ends up in decode-XYZ.c.inc).
  • Subdirectories with trace events require a forwarding header file; see hw/scsi/trace.h for an example.
Build layout changes
  • The Makefile is not recursive anymore. You don't make aarch64-softmmu/all or make arm-linux-user/all, you make qemu-system-aarch64 or make qemu-arm.
  • Emulator binaries have moved to the root of the build directory. For now, paths such as cris-softmmu/qemu-system-cris are preserved for backwards compatibility through symlinks (created by configure). They are just a user convenience however. You cannot "rm" them to force a rebuild, for example.
  • Other binaries have moved to the directory where they reside in the source tree. For example, "virtiofsd" has moved to "tools/virtiofsd/virtiofsd".
  • Most binaries are built by default by make all, including several in contrib/ that weren't built before. Currently, vhost-user-blk and rdmacm-mux are not because they fail on 32-bit and big-endian platforms respectively.
  • Sphinx manuals are only built if their constituent files are changed, therefore "make sphinxdocs" will be plenty fast when you've only modified a file or two.
  • Dependencies added to libqemuutil.a will propagate to all programs that link to it. If util/ code has dependencies, it should be conditional on have_system or have_block if applicable (this was a bug in the Makefiles).
  • For bisection: incremental builds across the conversion work fine in the forwards direction. They probably don't work at all backwards across the conversion, so you'll need to throw away your build tree when bisection moves you backwards across the conversion. This is mitigated by the fact that bisection usually starts at a release, and the patches have been applied on top of the 5.1.0 tag.
New conventions
  • New configure options should also be added to meson_options.txt so that dependencies are tested in meson.build rather than configure. How to do so is documented in docs/devel.
Impact to developers
  • If you were already using out-of-tree (VPATH) builds, nothing changes; an incremental pull of these changes should still build.
  • If you are used to in-tree builds, you'll want to do make distclean prior to merging in these changes (if you forget, git will remind you about various trace.h files that still exist as built files in your in-tree build vs. checked-in files post-patch). After that one-time clean, you can then proceed to do ./configure && make as before, but things will now automatically create a subdirectory build/ on your behalf where the actual build is performed.

CI holes

The following issues were _not_ found by GitLab CI:

  • SDL 2.0.8 requires -Wno-undef
  • s390x-softmmu is special for cross-compilation, because it is the only target to use the host C compiler.
  • There is special code in configure to handle a C++ compiler that does not play well with the C compiler.
  • Not all linux-user TCG tests are covered.
  • Big endian platforms are not covered.
  • ARM host platforms are not covered.
  • 32-bit POSIX platforms are not covered.
  • SystemTap is not covered.
  • daxctl is not covered.
  • --disable-tools --enable-system builds are not covered.
  • Peter uses Ubuntu+MXE instead of Fedora-mingw to build cross compilers. docker-test-mingw only works for Fedora and should be made more generic. Also container images for Fedora-mingw should be prepared in the same way as the existing container images for Debian+MXE.

Reported and unfixed bugs

  • meson.build files are missing copyright/license header comments

Known issues

  • Meson does not like outputs of custom_target (e.g. tracetool and qapi-gen.py) to be in multiple directories. Right now this is only a problem for the QAPI tests.

Next steps

Move pkg-config tests to meson.build
Replace declare_dependency statements with dependency statements. Note that Meson warns for dependencies available only as shared libraries, so if a dependency is only used by emulators or tools it should be guarded by "if have_system" or "if have_block".
Benefit: allows removing *_CFLAGS and *_LIBS variables from config-host.mak. Allows using dependency objects instead of CONFIG_* symbols when declaring sourcesets.
Change simple config-host.mak symbols to Meson options
Some symbols (e.g. CONFIG_PARALLELS) are not anymore used in the Makefiles, and are only placed in config-host.mak to be read from Meson. Instead pass them through -D... command-line options
Benefit: allows removing CONFIG_* variables from config-host.mak.
Move other compile tests from configure to meson.build
Write a linter for meson.build
Visit the output of meson introspect --ast meson.build (including subdirectories) and warn for:
  • dependencies searched with a method other than "pkg-config" or "system"
  • dependencies lacking "static: enable_static"
  • variables not defined on all paths (Meson accepts undefined variables on the RHS of short-circuiting boolean operators)

TBD

Create configure help and command line parsing code from meson_options.txt.
Use the output of "meson introspect --buildoptions" to initialize variables, parse command line options, print the help and create -D command line options for meson. PoC here (requires developers to install "jq").
Benefit: remove code duplication, centralize command line handling
Decide what to do about remaining build rules
Here you can find old patches to convert the build rules in roms/ and pc-bios/. However, they should probably use their own build system with full support for cross-compilation, a la tests/tcg.


Pending Meson changes

Issues

add_global_dependency or dependency::get_compile_args
Needed to move glib detection to Meson.
Add "c_winlibs" und "cpp_winlibs" to target functions
Should be useful for qemu-ga and friends.

Pull requests

python: add automatic dependency tracking for scripts
Avoids listing sources for qapi-gen.py and tracetool.py
add fs.expand method
Could be useful to compute TARGET_DIRS
transitively pass compile flags for internal dependency
Enables a better fix for https://bugs.launchpad.net/qemu/+bug/1909256

meson test

Improve `meson test` (0.57)
Get rid of scripts/tap-driver.pl
introduce add_test_setup(exclude suites: ...) keyword argument (0.57)
Can be useful to simplify invocation of "slow" vs. "quick" tests without a Makefile wrapper
mtest: limit "magic" CTRL+C behavior to process group leaders (0.59.3)
Needed to call "meson test" from make

extract_objects

extract_objects: fixes, tests and documentation for using the result in a custom_target (0.58.2)
Useful to parse compile_commands.json in scripts/modinfo.sh
allow custom targets in extract_objects and test (0.61.0)
Useful for modinfo to receive object files instead of source files

Others

Utility methods for feature objects (0.59.0)

Simplifies use of get_option()
"links" test with file argument should use the appropriate compiler for the source file (0.60.0)
Needed to move C++ compiler detection