[ << Regression tests ]	[Top][Contents][Index][ ? ]			[ Programming work >> ]
[ < Introduction to regression tests ]		[  Up : Regression tests ]	[ Compiling regression tests > ]

9.2 Precompiled regression tests

Regression test output

As part of the release process, the regression tests are run for every LilyPond release. Full regression test output is available for every stable version and the most recent development version.

Regression test output is available in HTML and PDF format. Links to the regression test output are available at the developer’s resources page for the version of interest.

The latest stable version of the regtests is found at:

http://lilypond.org/doc/stable/input/regression/collated-files.html

The latest development version of the regtests is found at:

http://lilypond.org/doc/latest/input/regression/collated-files.html

Regression test comparison

Each time a new version is released, the regtests are compiled and the output is automatically compared with the output of the previous release. The result of these comparisons is archived online:

http://lilypond.org/test/

Checking these pages is a very important task for the LilyPond project. You are invited to report anything that looks broken, or any case where the output quality is not on par with the previous release, as described in Bug reports.

What to look for

The test comparison shows all of the changes that occurred between the current release and the prior release. Each test that has a significant (noticeable) difference in output is displayed, with the old version on the left and the new version on the right.

Some of the small changes can be ignored (slightly different slur shapes, small variations in note spacing), but this is not always the case: sometimes even the smallest change means that something is wrong. To help in distinguishing these cases, we use bigger staff size when small differences matter.

Staff size 30 generally means "pay extra attention to details". Staff size 40 (two times bigger than default size) or more means that the regtest is about the details.

Staff size smaller than default doesn’t mean anything.

Regression tests whose output is the same for both versions are not shown in the test comparison.

• Images: green blurs in the new version show the approximate location of elements in the old version.

  There are often minor adjustments in spacing which do not indicate any problem.

• Log files: show the difference in command-line output.

  The main thing to examine are any changes in page counts – if a file used to fit on 1 page but now requires 4 or 5 pages, something is suspicious!

• Profile files: give information about TODO? I don’t know what they’re for. Apparently they give some information about CPU usage. If you got tons of changes in cell counts, this probably means that you compiled make test-baseline with a different amount of CPU threads than make check. Try redoing tests from scratch with the same number of threads each time – see Saving time with the ‘-j’ option.

---

[ << Regression tests ]	[Top][Contents][Index][ ? ]			[ Programming work >> ]
[ < Introduction to regression tests ]		[  Up : Regression tests ]	[ Compiling regression tests > ]

<< Back to Documentation Index

LilyPond — Contributor’s Guide v2.21.0 (development-branch).