=head1 NAME

LilyGuts - doco to the internals of GNU LilyPond

=head1 DESCRIPTION

This page documents some aspects of the internals of GNU LilyPond. Some of
this stuff comes from e-mail I wrote, some from e-mail others wrote,
some are large comments taken away from the headers

You should use doc++ to take a peek at the sources.

This should become a Hacking-HOWTO. If you find any confusing stuff
here, let me know.  I am a hacker, and don't spend enough time doccing
what I write.

If you finally have mastered some internal piece of lily, your
explanation could be added here.

=head1 OVERVIEW

GNU LilyPond is a "multi-pass" system. The different passes have been
created so that they do not depend on each other. In a later stage
some parts may be moved into libraries, or seperate programs, or they
might be integrated in larger systems.

=over 4

=item Parsing:

No difficult algorithms. Associated datastructures have prefix Input
(eg Input_score, Input_command). The .ly file is read, and converted
to 

=item Creating elements

Requests are processed and used to create elements (like balls, stems,
slurs etc). This is done by a hierarchy of "brokers" (called
Translators: the ones for paper output are Engravers, for MIDI
Performers), which swallow requests, broadcast them and couple
different elements.

In this step data-structures for the next steps are created and filled
with data: PScore, PCol.

=item Preprocessing

Some dependencies are resolved, such as the direction of stems, beams,

=item Calculation:

This step uses structures which have names starting with 'P'.
linebreaks and horizontal positions of PCols are determined. 

Through some magical interactions with Line_of_score and Super_elem
(check out the source) the "lines" are produced. 

All other spanners can figure across which lines they are spread. If
applicable, they break themselves into pieces. After this, each piece
works (or, if there are no pieces) the spanner throws out any
dependencies which are in the wrong line.

=item Postprocesing:

Some items and all spanners need computation after the PCol positions
are determined.

=item Output paper

Very simple, just walk all Line_of_* and follow the links over there

=item Output midi

The columns of requests (from the Setting up step) are used to
determine what to output when.

=back

=head1 INTERNALS

This chapter deals with the internals of Mudela. 

=head2 Requests

As you can see, most information is stored in the form of a request. 
In music typesetting, the user might want to cram a lot more symbols
on the paper than actually fits. To reflect this idea (the user asks
more than we can do), the container for this data is called Request.

A request is done to the C<Staff> which contains the
C<Voice_element>. The staff decides whether to to honor the request,
ignore it, or merge it with other requests. Merging of requests is
preferably done with other requests done by members of the same
voicegroups (beams, brackets, stems)

The result of a request will be an C<Item> or a C<Spanner>, which
will be put on a C<PStaff>. Note that the C<PStaff> and the original
C<Staff> need not have anything in common. For example, the
``double'' piano Staff could interpret commands which juggle
melodies across the left and right hand, and may put the result in
two five-line PStaffs (maybe with extra PStaffs to carry the dynamic
signs and any lyric.

The class C<Staff> should be thought as a container for the
C<Voice>s, and an interpreter for C<Request>s.
Different staffs can produce different outputs; a melodious voice
which is put into a percussion-Staff, will be typeset as the rythm of
that voice.

After C<Staff> made up her mind, the resultant items and
spanners are put on the PScore.

Some of the important requests are:

=over 4

=item C<Barcheck_req>

Checks during music processing if start of this voice element
coincides with the start of a measure. Handy to check if you left out
some voice elts.

=item C<Note_req>

Staff has to decide if the ball should be hanging left or right. This
influences the horizontal dimensions of a column, and this  is why
request processing should be done before horizontal spacing.

Other voices' frivolities may cause the need for accidentals, so this
is also for the  C<Staff> to decide. The  C<Staff> can decide on positioning
based on ottava commands and the appropriate clef.

=item C<Rest_req>

Typeset a rest.

=item C<Span_req>

This type of request typically results in the creation of a C<Spanner>

=item C<Beam_req>

Start/stop a beam.

Staff has to combine this request with the stem_request, since the
number of flags that a stem wants to carry will determine the
number of beams.

=item  C<Dynamic>

Each dynamic is bound to one note (a crescendo spanning multiple
notes is thought to be made of two "dynamics": a start and a stop).
Dynamic changes can occur in a smaller time than the length of its
note, therefore fore each  C<Dynamic> request carries a time, measured
from the start of its note.

=head2 Voice groups

Voice group is a (time-dependent) collection of voices which share
some characteristics (slurs, stems) at some time.

=head1 Request_engraver

In the previous section the idea of Request has been explained, but
this only solves one half of the problem. The other half is
deciding which requests should be honored, which should merged with
other requests, and which should be ignored. Consider this (pseudo)input

	< % chord
		\music { [c() c] }
		\music { [e() e] }
	>

Both the c and e are part of a chord (they are in the same
Voice_group), so they should share the beams, and the two [ ] pairs
should be merged. The slurs OTOH are specific for each voice, so they
should not be shared.

The judge in this "allocation" problem is Staff (actually, it's child
C<Complex_staff>). It uses the C<Request_engraver> to do most of the
work.  For each request C<Complex_staff> queries so-called
C<Request_engraver>s if they want to accept a request eg, the
C<Notehead_engraver> will accept C<Note_req>s, and turn down
C<Slur_req>s. If C<Complex_staff> cannot find a engraver that wants
the request, it is junked (with a warning message).

After all requests have been either assigned, or junked, the Engraver
will process the requests (which usually means creating an C<Item> or
C<Spanner>). If a C<Request_engraver> creates something, it tells
C<Complex_staff>. If all requests have been processed, then each
Engraver is notified of any created Staff_element.

=head2 example:

	c4

produces:

	note_request (duration 1/4)
	stem_request (duration 1/4)

note_request will be taken by a C<Notehead_engraver>, stem_request
will be taken by a C<Stem_beam_engraver>. C<Notehead_engraver> creates
a C<Notehead>, C<Stem_beam_engraver> creates a C<Stem>. Both announce
this to the Staff. Staff will tell C<Stem_beam_engraver> about the
C<Notehead>, which will add the C<Notehead> to the C<Stem> it just
created.

To decide on merging, C<Complex_staff> has grouped several
engravers. There are a few groups:

=over 4

=item *

Staff wide, contains

	Local_key_engraver
	Bar_engraver
	Key_engraver
	Meter_engraver
	Clef_engraver

=item *

Voice group, contains

	Stem_beam_engraver
	Script_engraver
	Text_engraver

=item *

Voice, contains
	
	Slur_engraver
	Notehead_engraver

=back

=head1 ITEMS and SPANNERS

The symbols that are printed, are generated by items and spanners
(staff-elements). An item has one horizontal position, whereas a
spanner spans several columns.

=head1 DEPENDENCIES

In music symbols depend on each other: the stems of a beam should
point in the same direction as the beam itself, so the stems of a beam
depend on the beam. In the same way do scripts depend on the direction
of the stem. To reflect this, LilyPond has the notion of dependency. 
It works in the same fashion that C<make> uses to build programs: before
a stem is calculated, its dependencies (the beam) should be
calculated. Before a slur is calculated, its dependencies (stems, noteheads)
should be calculated.

=head1 BREAKING

So what's the deal with PREBREAK and POSTBREAK and all this
stuff?

Let's take text as an example. In German some compound
words change their spelling if they are broken: "backen" becomes
"bak-ken".  TeX has a mechanism to deal with this, you would define
the spelling of "backen" in TeX in this way

	\discretionary{bak-}{ken}{backen}

These 3 arguments are called "prebreak", "postbreak" and "nobreak"
text.

The same problem exists when typesetting music. If a line of music is
broken, the next line usually gets a clef. So in TeX terms, the clef
is a postbreak. The same thing happens with meter signs: Normally the
meter follows the bar. If a line is broken at that bar, the bar along
with the meter stays on the "last" line, but the next line also gets a
meter sign after the clef. Using the previous notation,

	\discretionary{bar meter}{clef meter}{ bar meter }

In GNU Lilypond, we have the same concepts (and the same
terminology). Each (nonrhythmic) symbol is typeset in  a nonrhythmic column
At a breakpoint, multiple symbols are printed; symbols to be printed
if the line is not broken, symbols to appear on the previous line, and
on the next line if it is broken.

=head1 SPACING

I think my method is the most elegant algorithm i've seen so far.
Some terminology: I call a vertical group of symbols (notes) which
start at the same time a "column".  Each line of a score has notes in
it, grouped in columns. The difference in starting time between those
columns makes it possible to determine ideal distances between those
columns.

Example:

		time ----->

		col1    col2    col3    col4


	voice1  1               1

	voice2  2       2       2       2


	(1 is a whole note, 2 a half note.)

	time_difference (col1 , col2) = 0.5 wholes,
	time_difference (col1 , col3) = 1 wholes,
	time_difference (col2 , col3) = 0.5 wholes,
	etc.

these differences are translated into ideal distances (these translations
have been the subject of discussion in this thread).

        distance (col1,col2) = 10 pt
        distance (col1,col3) = 14.1 pt
        distance (col2,col3) = 10 pt
        etc.

as you can see, these distance are conflicting. So instead of
satisfying all those ideals simultaneously, a compromise is sought.

This is Columbus' egg: GNU LilyPond attaches "springs" to each
column-pair.  each spring has an equilibrium-position which is equal to
the above mentioned distance, so

spring (col1, col2) and spring (col2,col3) try to push column 1
and 3 away (to a distance of 20pt) from each other, whereas the spring
between col 1 and col 3 tries to pull those two together (to a
distance of 14.1 pt). The net result of this pushing and pulling is an
equilibrium situation (the pushing cancels the pulling), which can be
calculated as the solution of Quadratic program: it is the solution
with minimum potential energy, for you physicists out there.

This algorithm for doing one line, gives a "badness" parameter for
each line (the potential energy). Now one can use TeX's algorithm for
making paragraphs (using this new version of "badness"): one should
try to minimise the overall badness of a paragraph. GNU LilyPond also
uses the concept of pre- and post-breaks.

(actually, it is a bit more complicated: each column also has a
minimum distance to other columns, to prevent symbols from running
into symbols of other columns.)

This of course does not solve the problem of generating the
springs. This is an area that needs a lot of work, and the optimal
solution to find is not of a mathematical nature.


