14.6.4.4 GOP-PROP 4 - lessons from 2.14

History

A brief history of releases:

date (YYYY-MM-DD)versioncomment
2008-10-282.11.63nobody checking regtests
2008-11-172.11.64
2008-11-292.11.65
2008-12-232.12.0
2009-01-01somewhere around here, Graham becomes officially release manager, but Han-Wen still builds the actual releases
2009-01-012.12.1
2009-01-252.12.2
2009-02-282.13.0
2009-06-012.13.1note jump in time!
2009-06-272.13.2first Graham release?
2009-07-032.13.3
2009-09-09Graham arrives in Glasgow, gets a powerful desktop computer, and begins serious work on GUB (sending bug reports to Jan). It takes approximately 100 hours until GUB is stable enough to make regular releases.
2009-09-242.13.4
2009-10-022.13.5
2009-10-222.13.6
2009-11-052.13.7
...
2010-01-132.12.3
...
2010-03-192.13.16Bug squad starts doing a few regtest comparisons, but IIRC the effort dies out after a few weeks (BLUE)
...
2010-08-042.13.29Phil starts checking regtests (BLUE)
...
2011-01-122.13.46release candidate 1 (GREEN)
...
2011-05-302.13.63release candidate 7 (GREEN)
2011-06-062.14.0

Carl’s analysis of the bugs

A ‘csv’ spreadsheet is available.

 
https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html
lilypond-issues-analysis.csv
lilypond-issues-analysis-trim-duplicates.csv

There 148 issues marked with Priority=Critical in the tracker.

I’ve done an analysis, and it looks to me like there was initially a backlog of critical issues that weren’t fixed, and little work was being done to eliminate critical issues.

Somewhere about 2010-08-01, critical issues started to disappear, but occasional new ones appeared.

There were a couple of major changes that introduced unanticipated regressions (new spacing code, beam collision avoidance). These produced more than the expected number of regressions.

It appears to me that we didn’t really get serious about eliminating critical bugs until about 2010-06-15 or so. After that point, the number of critical bugs more-or-less steadily decreased until we got to a release candidate.

Of particular interest, the first release candidate of 2.14 was released on 2011-01-12. Over the next 10 days, about a dozen bugs were reported and fixed. Release candidate 2 came out on 2011-02-09. No surge of bugs occurred with this release. Candidate 3 came out on 2011-03-13; we got 2 bugs per week. Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came out on 2011-04-07. We got a couple of bugs per week.

Notes, commentary, and opinions

Han-Wen: Overall, I think this cycle took too long
Mike: I agree
Graham: +1

Discussions

 
https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html
https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html


LilyPond — Contributor’s Guide v2.19.14 (development-branch).