This adds a performance tracking framework that can run a set of tests over
specified git revisions. The ruby script for generating the reports comes from
similar performance tracking in GEGL. The framework permits evaluating new
tests against older version of clutter.
The tests themselves go through a few hoops for disabling framerate limiting in
both mesa and clutter.
When running make check the tests will be run and lines of the form:
@ test-state: 40.51 fps
will be left in the output, a script can scrape these lines out of a build log
on a buildbot to in other ways track performance.
When running tests under Valgrind it would be useful to pass a
suppression file for the known one-off allocations done by Clutter
and by its dependencies. This trims the output of Valgrind and
improves the ability to actually spot leaks.
framework
* configure.ac:
* tests/*:
The tests have been reorganised into different categories: conformance,
interactive and micro benchmarks.
- conformance tests can be run as part of automated tests
- interactive tests are basically all the existing tests
- micro benchmarks focus on a single performance metric
I converted the timeline tests to conformance tests and also added some
tests from Neil Roberts and Ebassi.
Note: currently only the conformance tests use the glib test APIs,
though the micro benchmarks should too.
The other change is to make the unit tests link into monolithic binaries
which makes the build time for unit tests considerably faster. To deal
with the extra complexity this adds to debugging individual tests I
have added some sugar to the makefiles so all the tests can be run
directly via a symlink and when an individual test is run this way,
then a note is printed to the terminal explaining exactly how that test
may be debugged using GDB.
There is a convenience make rule: 'make test-report', that will run all
the conformance tests and hopefully even open the results in your web
browser. It skips some of the slower timeline tests, but you can run
those using 'make full-report'
