Merges

How to merge pull requests into QEMU

This is a work in progress, and has some missing sections and marked TODO items!

Summary of requirements

Verify the GPG-signed tag
Merge the pullreq into a 'staging' branch
Push to the gitlab qemu-project/qemu.git staging branch
Check that gitlab CI passed
Optionally, do local builds
Eyeball the commit logs to check for weirdness
Push to qemu-project/qemu.git master
Reply to the pullreq submitter saying you've applied it (or not, if it failed)

I do most of this with the 'apply-pullreq' script, which you can find in this git repo, along with some other scripts mentioned in this document:

https://git.linaro.org/people/peter.maydell/misc-scripts.git/

(You won't necessarily need every script in there.)

This script assumes:

you run it in a local QEMU git repo that you're only using for pullreq handling
you use 'master' for the local branch that tracks upstream master and 'staging' for the local branch that tracks staging
you have a git remote named 'publish-gitlab' which can push to the project's gitlab repo
that you have in your gpg keyring the keys of the people you are pulling from

Setting up the git repo to use for pullreq handling

IME it is best to keep the local directory you use for pullreq handling entirely separate from whatever other git trees you are doing local development in. This avoids the potential for accidents where local changes end up being committed upstream by mistake.

To set this up, clone https://gitlab.com/qemu-project/qemu.git as normal. Add the remote for pushing changes upstream:

publish-gitlab  git@gitlab.com:qemu-project/qemu.git

Check you have been set up on gitlab to have permissions to push to the project's repo. At the moment this means your gitlab account needs to have at least the "Developer" role.

Create the 'staging' branch:

$ git checkout master && git pull && git checkout -b staging

Setting up local gpg keyring

TODO

Basic workflow

$ git checkout master && git pull && git checkout staging && git reset --hard master
# apply-pullreq wants the URL and tagname -- you can just copy and paste this
# directly from the pull request cover letter URL
$ apply-pullreq https://some/url/qemu.git tags/whatever
# Wait for gitlab CI and for any local builds
# Check the commit logs look OK
$ git log --stat master..staging
# If satisfied, push to master
$ git push publish-gitlab staging:master

If something goes wrong, you can throw away what you were trying to merge by repeating the initial

$ git checkout master && git pull && git checkout staging && git reset --hard master

By default, apply-pullreq will simply refuse to apply a pullreq with a submodule update in it. This is a guard against submaintainers who accidentally include submodule changes (usually accidental reverts of a recent submodule change); git makes this sadly easy to do by mistake. In this case, you need to check that the submodule change is intentional (ie that it seems to be associated with a commit that is deliberately updating that submodule), and if so retry, passing apply-pullreq the --submodule-ok option (which just tells it to not bail out when it sees the submodule change).

You can make apply-pullreq skip the GPG signature verification with the --no-verify-sig option, but you should only do this in very limited circumstances when you want to test a branch as if it were a pull request merge; don't do a merge this way for anything you are actually going to push upstream.

The apply-pullreq script will try to run a script named parallel-buildtest, which it expects to do whatever local builds you are running. My version of this is in the misc-scripts git repo, but this is where it gets rather ad-hoc, and you may not want to use my version. At a minimum, you need to have the script do a

git push --force publish-gitlab staging:staging

to trigger the gitlab CI pipeline run. My script also pushes to a personal git repo, and uses GNU parallel (NB: not the Debian 'moreutils' parallel!) to kick off builds on a bunch of different machines. The set of things I still do with this ad-hoc testing are:

ppc64 host
aarch32 build, via a chroot setup on an aarch64 machine
freebsd, netbsd, openbsd VMs on an x86-64 host (ie what you get from "make vm-build-netbsd", "make vm-build-openbsd" and "make vm-build-freebsd")
OSX
on an x86-64 host, some setups I think gitlab CI still hasn't covered:
- incremental build
- clang sanitizer build
- all-linux-static running the linux-user-test binaries as well as check-tcg (this might be covered by gitlab well enough these days; must check)

In all these cases by 'build' I mean the equivalent of "make && make check".

If you are in a position to do perhaps a subset of this ad-hoc testing that would be nice, but it isn't mandatory.

Finding pull requests, and replying to them

You can find pull requests by looking for emails to the qemu-devel@nongnu.org list which include the phrase "for you to fetch changes up to" and do not include the phrases "not for master" or "PULL SUBSYSTEM". You should set up an email filter to tag these so you reliably see them.

When you try to apply a pull request but it fails testing in some way, reply telling the submitter what went wrong.

When you apply a pull request and successfully merge it, reply to let the submitter know. You should use a template something like:

Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/7.0 for any user-visible changes.

(with the URL updated to match whatever the upcoming release is). The reference to the changelog is important to nudge submaintainers into doing updates of the changelog as they go along and not forget about it entirely.

Making release candidates and releases

For the moment I cover here only the technical side of "how to make the tag", not the "confirming dates, writing up the Planning wiki page, pushing back on pull requests which don't meet the freeze criteria" side of things.

TODO some advice on that stuff would be useful to cover eventually.

You can use the 'qemu-tag-release' script to make both the rcN and the final release tags. This script updates the VERSION file, makes a signed tag and pushes it to master. It assumes (but does not check!) that you are on the 'staging' branch and have already tested and pushed the last merge to master. You need to pass it both the number for VERSION and the tag name, which looks like this:

qemu-tag-release 6.1.90 v6.2.0-rc0
qemu-tag-release 6.1.91 v6.2.0-rc1
[etc]
qemu-tag-release 6.2.0 v6.2.0

This is because our VERSION file number has the convention that it is e.g. 6.1.0 for the v6.1 release, 6.1.50 while master is open for general development, 6.1.90 for rc0, 6.1.91 for rc1, and so on, and then finally 6.2.0 for the v6.2 release.

Run the script, use 'git show' to check you're happy with the commit it just made, and run the 'git push' command it tells you to push the tag to gitlab.

All further release and tag work is handled by Mike Roth; you just need to email him to let him know you've pushed the tag (either rcN or final), and he will create tarballs and send out announcement emails.

TODO we should get Mike to write up his process for his side of things...

Once the final release has been made, the first thing to be done is to run

qemu-open-devtree 6.3

which will update VERSION to '6.2.50' to indicate that development is open again. As with the tag script, you should check the commit and then push it to master. (The argument to qemu-open-devtree is used only in the commit message; the new number for VERSION is determined automatically from its previous contents.)