Guessing relevant test modules with Fault Line
Launchpad’s test suite takes a long time to run. Far too long to wait for when you’ve made a change. And the likelihood that you’ve broken any given test is pretty small. So you probably want to start by running the tests that were likely to break.
You can usually guess which tests to run; if you’ve changed “archive.py“, you should probably run “test_archive“. But some connections are easier to miss, so I’ve hacked up a Fault Line, a bzr plugin that uses past changes to guess which test files correlate to the files you’ve changed. You can run it like so:
bin/test -m $(bzr fault-line --module-regex)
This will look at all the files you’ve changed, look at their recent history, and see which files tended to change in the last 100 revisions where you changed the specified files. The --module-regex option causes it to output a regular expression, assuming that the test files are Python modules. Otherwise, it would just output a list of the test files it found.
Thanks to Jelmer Vernoij, this is even easier to achieve for testing bzr. You just need to run “bzr selftest --auto“.
To install Fault Line, just run:
bzr branch lp:fault-line ~/.bazaar/plugins/faultline
It currently requires bzr 2.3 or later (e.g. stock Natty bzr).
Fault line is pretty limited right now. For example, the way it guesses what’s a test file is hard-coded. But a few days ago, it wasn’t even a proof of concept. Who knows what the future holds?
