LilyPond — Contributor’s Guide

Appendices

1. Introduction to contributing

This chapter presents a quick overview of ways that people can help LilyPond.

1.1 Help us

We need you!

Thank you for your interest in helping us — we would love to see you get involved! Your contribution will help a large group of users make beautifully typeset music.

Even working on small tasks can have a big impact: taking care of them allows experienced developers work on advanced tasks, instead of spending time on those simple tasks.

For a multi-faceted project like LilyPond, sometimes it’s tough to know where to begin. In addition to the avenues proposed below, you can send an e-mail to the lilypond-devel@gnu.org mailing list, and we’ll help you to get started.

Simple tasks

No programming skills required!

• Mailing list support: answer questions from fellow users.
• Bug reporting: help users create proper Bug reports, and/or join the Bug Squad to organize Issues.
• Documentation: small changes can be proposed by following the guidelines for Documentation suggestions.
• LilyPond Snippet Repository (LSR): create and fix snippets following the guidelines in Adding and editing snippets.
• Discussions, reviews, and testing: the developers often ask for feedback about new documentation, potential syntax changes, and testing new features. Please contribute to these discussions!

Advanced tasks

These jobs generally require that you have the source code and can compile LilyPond.

Contributors using Linux or FreeBSD may also use Lilydev, but if they prefer their own development environment, they should read Working with source code, and Compiling.

Begin by reading Summary for experienced developers.

• Documentation: for large changes, see Documentation work.
• Website: the website is built from the normal documentation source. See the info about documentation, and also Website work.
• Translations: see Translating the documentation, and Translating the website.
• Bugfixes or new features: read Programming work.

1.2 Overview of work flow

Git is a version control system that tracks the history of a program’s source code. The LilyPond source code is maintained as a Git repository, which contains:

• all of the source files needed to build LilyPond, and
• a record of the entire history of every change made to every file since the program was born.

The ‘official’ LilyPond Git repository is hosted by the GNU Savannah software forge at http://git.sv.gnu.org.

Changes made within one contributor’s copy of the repository can be shared with other contributors using patches. A patch is a text file that indicates what changes have been made. If a contributor’s patch is approved for inclusion (usually through the mailing list), someone on the current development team will push the patch to the official repository.

The Savannah software forge provides two separate interfaces for viewing the LilyPond Git repository online: cgit and gitweb.

Git is a complex and powerful tool, but tends to be confusing at first, particularly for users not familiar with the command line and/or version control systems. We have created the lily-git graphical user interface to ease this difficulty.

Compiling (‘building’) LilyPond allows developers to see how changes to the source code affect the program itself. Compiling is also needed to package the program for specific operating systems or distributions. LilyPond can be compiled from a local Git repository (for developers), or from a downloaded tarball (for packagers). Compiling LilyPond is a rather involved process, and most contributor tasks do not require it.

Contributors can contact the developers through the ‘lilypond-devel’ mailing list. The mailing list archive is located at http://lists.gnu.org/archive/html/lilypond-devel/. If you have a question for the developers, search the archives first to see if the issue has already been discussed. Otherwise, send an email to lilypond-devel@gnu.org. You can subscribe to the developers’ mailing list here: http://lists.gnu.org/mailman/listinfo/lilypond-devel.

1.3 Summary for experienced developers

If you are already familiar with typical open-source tools, here’s what you need to know:

• source repository: hosted by GNU savannah.

  http://git.savannah.gnu.org/gitweb/?p=lilypond.git
  

• issue tracker: currently hosted by Sourceforge.

  https://sourceforge.net/p/testlilyissues/issues/
  

• patch review: Reitveld – the collaborative code review tool.

  https://codereview.appspot.com
  

• environment variables: many maintenance scripts, and many instructions in this guide rely on predefined Environment variables.
• mailing lists: given on Contact.
• Git branches:
  • master: always base your work from this branch, but never push directly to it. Patches are always pushed directly to the staging branch instead.
  • staging: always push to this branch after a successful patch review cycle (see below).
  • translation: Translators should base their work on this branch only and push any translation patches directly to it as well.
  • dev/foo: feel free to push any new branch name under dev/.
• regression tests: also known as “regtests”. A collection of more than a thousand .ly files that are used to track LilyPond’s engraving output between released stable and unstable versions as well as checked for all patches submitted for testing.

  If a patch introduces any unintentional changes to any of the regtests it is very likely it will be rejected (to be fixed) – always make sure that, if you expect any regression test changes, that they are explained clearly as part of the patch description when submitting for testing. For more information see Regression tests.

• reviews: after finishing work on a patch or branch:
  1. upload it with our custom git-cl ‘helper-script’; see git-cl. In addition to uploading patches to the Google’s Rietveld code review tool the script will also update the issue tracker (or add a new issue as appropriate) so that any reference to the patch is not lost. The current “status” of any patch submitted is always managed on the issue tracker; also see Issues.

     Once submitted the patch will be given a status of Patch-new and will enter the “Patch Countdown”. More information on this can be found in the section Uploading a patch for review.

  2. Patches are generally tested within 24 hours of submission. Once it has passed the basic tests – make, make doc and a make test-baseline/check –, the tracker will be updated and the patch’s status will change to Patch-review for other developers to examine.
  3. Every third day, the “Patch Meister” will examine the issue tracker and the Rietveld code review tool for the submitted patch, looking for any comments by other developers. Depending on what has been posted, the patch will be either; “moved on” to the next patch status (Patch-countdown); set back to Patch-needs_work; or if more discussion is needed, left at Patch-review. In all cases the issue tracker (not the Rietveld code review tool) will be updated by the Patch Meister accordingly.
  4. Once another three days have passed, any patch that has been given Patch-countdown status will be changed to Patch-push, the issue tracker is updated, and the developer can now push it directly to the staging branch (or email the patch – created with git format-patch command – to one of the other developers who can push it for you).
  5. Automatic scripts run every few hours to merge the staging branch with master.

  Advanced note: This process does means that most patches will take about a week before finally being merged into master. With the limited resources for reviewing patches available and a history of unintended breakages in the master branch (from patches that have not had time to be reviewed properly), this is the best compromise we have found.

1.4 Mentors

We have a semi-formal system of mentorship, similar to the medieval “journeyman/master” training system. New contributors will have a dedicated mentor to help them “learn the ropes”.

Contributor responsibilities

1. Ask your mentor which sections of the CG you should read.
2. If you get stuck for longer than 10 minutes, ask your mentor. They might not be able to help you with all problems, but we find that new contributors often get stuck with something that could be solved/explained with 2 or 3 sentences from a mentor.
3. If you have been working on a task much longer than was originally estimated, stop and ask your mentor. There may have been a miscommunication, or there may be some time-saving tips that could vastly simply your task.
4. Send patches to your mentor for initial comments.
5. Inform your mentor if you’re going to be away for a month, or if you leave entirely. Contributing to lilypond isn’t for everybody; just let your mentor know so that we can reassign that work to somebody else.
6. Inform your mentor if you’re willing to do more work – we always have way more work than we have helpers available. We try to avoid overwhelming new contributors, so you’ll be given less work than we think you can handle.

Mentor responsibilities

1. Respond to questions from your contributor(s) promptly, even if the response is just “sorry, I don’t know” or “sorry, I’m very busy for the next 3 days; I’ll get back to you then”. Make sure they feel valued.
2. Inform your contributor(s) about the expected turnaround for your emails – do you work on lilypond every day, or every weekend, or what? Also, if you’ll be unavailable for longer than usual (say, if you normally reply within 24 hours, but you’ll be at a conference for a week), let your contributors know. Again, make sure they feel valued, and that your silence (if they ask a question during that period) isn’t their fault.
3. Inform your contributor(s) if they need to do anything unusual for the builds, such as doing a “make clean / doc-clean” or switching git branches (not expected, but just in case...)
4. You don’t need to be able to completely approve patches. Make sure the patch meets whatever you know of the guidelines (for doc style, code indentation, whatever), and then send it on to -devel for more comments. If you feel confident about the patch, you can push it directly (this is mainly intended for docs and translations; code patches should almost always go to -devel before being pushed).
5. Keep track of patches from your contributor. Either upload them to Rietveld yourself, or help+encourage them to upload the patches themselves. When a patch is on Rietveld, it’s your responbility to get comments for it, and to add a link to the patch to the google tracker. (tag it “patch-new”, or “patch-review” if you feel very confident in it)
6. Encourage your contributor to review patches, particularly your own! It doesn’t matter if they’re not familiar with C++ / scheme / build system / doc stuff – simply going through the process is valuable. Besides, anybody can find a typo!
7. Contact your contributor at least once a week. The goal is just to get a conversation started – there’s nothing wrong with simply copy&pasting this into an email:

   Hey there,
   
   How are things going?  If you sent a patch and got a review, do
   you know what you need to fix?  If you sent a patch but have no
   reviews yet, do you know when you will get reviews?  If you are
   working on a patch, what step(s) are you working on?
   

2. Quick start

Want to submit a patch for LilyPond? Great! Never created a patch before? Never compiled software before? No problem! This chapter is for you and will help you do this as quickly and easily as possible.

2.1 LilyDev

There is a ‘remix’ of Debian GNU/Linux – known as “LilyDev” for short – which includes all the necessary software and tools to compile LilyPond, the documentation and the website (also see Website work).

While compiling LilyPond on Mac OS and Windows is possible, both environments are complex to set up. LilyDev can be easily installed and run inside a ‘virtual machine’ on either of these operating systems relatively easily using readily available virtualization software. We recommend using VirtualBox as it is available for all major operating systems and is very easy to install & configure.

The LilyDev disk image can also be written to a USB device or ‘burnt’ to a DVD – it is approximately 900 MB in size – and installed just like any standard GNU/Linux distribution.

The current image is based on a 32-bit version of Debian 8 (‘Jessie’) and the disk image was generated using Debian live-build 4.

Download the LilyDev disk image file (a .iso file) from here:

https://github.com/fedelibre/LilyDev/releases/latest

If you are not familiar with GNU/Linux, it may be beneficial to read a few “introduction to Linux” type web pages.

Installing LilyDev in VirtualBox

This section discusses how to install and use LilyDev with VirtualBox.

1. Download VirtualBox from here:

   http://www.virtualbox.org/wiki/Downloads
   

   Note: In virtualization terminology, the operating system where VirtualBox is installed is known as the host. LilyDev will be installed ‘inside’ VirtualBox as a guest.

2. Start the VirtualBox software and click ‘New’ to create a new “virtual machine”.

   The ‘New Virtual Machine Wizard’ will walk you through setting up your guest virtual machine. Choose an appropriate name for your LilyDev installation and select the ‘Linux’ operating system. When selecting the ‘version’ choose ‘Debian (32 bit)’ (don’t use the ‘64 bit’ option). If you do not have that specific option choose ‘Linux 2.6’ (again do not choose any option that has 64 bit next to it).

3. Select the amount of RAM you will allow the LilyDev guest to use from your host operating system when it is running. If possible, use at least 700 MB of RAM; the more RAM you can spare from your host the better, although LilyDev will currently use no more than 4 GB (4096 MB) even if you are able to assign more.
4. For your ‘Virtual Hard Disk’, leave the ‘Create new hard disk’ option checked, use the default ‘VDI’ and “Dynamically allocated” options for the virtual hard drive. A complete compile of everything (code, docs, regression tests) can reach 10 GB so size your virtual disk and its location accordingly.
5. Verify the summary details and click ‘Create’, when you are satisfied. Your new guest will be displayed in the VirtualBox window.

   Note: The image contains a ‘686-pae’ kernel, so you must enable PAE within the virtual machine’s settings – click on System → Processor and select ‘Extended features: Enable PAE/NX’.

6. Click the ‘Start’ button and the ‘First Run Wizard’ will prompt you for the installation media. Click the browse icon, locate the LilyDev disk image file that you downloaded (the .iso file) and click through the wizard to begin the installation process.
7. When the LilyDev disk image boots for the first time, choose either the ‘Install’ or the ‘Graphical install’ menu item. The installer will then walk you through the complete installation process.
8. At the “Partition disks” stage, do not be afraid to select “Guided - use entire disk”, since this refers to your virtual disk, not your computer’s own hard disk.
9. Continue to click through the rest of the wizard, filling in any appropriate details when asked, and wait for the install to complete. This will take about 10 minutes or so on a reasonably modern computer.
10. When the installation is completed, just click on ‘Continue’ (you do not have to remove any media since you installed LilyDev from a Disk image, which is just a file on your computer). The installer will reboot the virtual machine.

LilyDev should now be installed and running!

Configuring LilyDev in VirtualBox

VirtualBox has extra ‘guest additions’ which although are not necessary to use LilyDev or compile LilyPond, do provide some additional features to your Virtual Machine to make it easier to work with. Such as being able to dynamically resize the LilyDev window, allow seamless interaction with your mouse pointer on both the host and guest and let you copy/paste between your host and guest if needed.

1. Select the ‘Devices’ menu from the virtual machine window and choose ‘Install Guest Additions...’. This will automount a CD which will prompt you to autorun it. Click OK and follow the instructions. It is recommended to reboot the guest when the installation is complete.

   Other virtualization software will also have their own ‘guest’ additions, follow the normal procedures for your virtualization software with LilyDev as the client.

2. Restart LilyDev to complete the installation of the guest additions.

   Advanced note: If you do any kernel upgrades, you may need to reinstall the additional software. Just follow the step above again and reboot when the reinstallation is complete.

Other items that may be helpful:

• In the settings for the virtual machine, set the network to Bridged mode to allow you to access shared folders when using Windows hosts.
• Set up any additional features, such as ‘Shared Folders’ between your main operating system and LilyDev. This is distinct from the networked share folders in Windows. Consult the external documentation for this.

  Some longtime contributors have reported that ‘shared folders’ are rarely useful and not worth the fuss, particularly since files can be shared over a network instead.

• Pasting into a terminal is done with Ctrl+Shift+v.
• Right-click allows you to edit a file with the text editor (default is Leafpad).

Known issues and warnings

Not all hardware is supported in all virtualization tools. In particular, some contributors have reported problems with USB network adapters. If you have problems with network connection (for example Internet connection in the host system is lost when you launch virtual system), try installing and running LilyDev with your computer’s built-in network adapter used to connect to the network. Refer to the help documentation that comes with your virtualization software.

2.2 lily-git

The ‘LilyPond Contributor’s Git Interface’ (otherwise known as lily-git.tcl) is a simple-to-use GUI to help you download and update the LilyPond source code as well as an aid to making software patches.

Where to get lily-git

Depending on your development environment, lily-git may already be installed on your computer.

• If you are using LilyDev (see LilyDev) then lily-git should already be installed and ready to run. If this is not the case you can easily turn it on by adding the following line in ~/.bashrc:

  # add lily-git to the PATH
  PATH=$LILYPOND_GIT/scripts/auxiliar:"${PATH}"
  

• For those not using LilyDev, lily-git can be obtained by downloading the software directly. See Manually installing lily-git.tcl.
• lily-git is part of the LilyPond source code and is located in ‘$LILYPOND_GIT/scripts/auxiliar/lily-git.tcl’.

Using lily-git to download the source code

1. Type the following command into a Terminal:

   lily-git.tcl
   

   You will be prompted to enter a name and email address into the lily-git UI. This information is used to label any patches you create (using the lily-git UI or git via the command line) and can be changed later if required. See Configuring Git.

2. Click on the Submit button to update lily-git with the information.
3. Click on the “Get source” button.

   A directory called ‘lilypond-git’ is created within your home directory and the entire source code will start to be downloaded into it.

   Note: Be patient! There is no progress bar in the lily-git UI but the complete source is around 180 MB.

   When the source code has been downloaded, the “command output” window in the lily-git UI will update and display “Done” on the very last line and the button label will change to say “Update source”.

   Note: Some contributors have reported that occasionally nothing happens at this step at all. If this occurs, then try again in a few minutes – it could be an intermittant network problem. If the problem persists, please ask for help.

4. Close the lily-git GUI and navigate to the ‘lilypond-git’ directory to view and edit the source files.

If this is the first time you will be attempting to compile LilyPond, please see the section Compiling with LilyDev before continuing.

How to use lily-git

Here is a brief description of what each button does in the lily-git UI.

1. Update source

Click the “Update source” button to get any recent changes to the source code that have been added by other contributors since your last session.

2a. New local commit

A single commit typically represents one logical set of related changes (such as a bug-fix), and may incorporate changes to multiple files at the same time.

When you’re finished making the changes for a commit, click the “New local commit” button. This will open the “Git Commit Message” window. The message header is required, and the message body is optional.

After entering a commit message, click “OK” to finalize the commit.

2b. Amend previous commit

You can go back and make changes to the most recent commit with the “Amend previous commit” button. This is useful if a mistake is found after you have clicked the “New local commit” button.

To amend the most recent commit, re-edit the source files as needed and then click the “Amend previous commit” button. The earlier version of the commit is not saved, but is replaced by the new one.

3. Make patch set

Before making a patch set from any commits, you should click the “Update source” button to make sure the commits are based on the most recent remote snapshot.

When you click the “Make patch set” button, lily-git.tcl will produce patch files for any new commits, saving them to the current directory. The command output will display the name of the new patch files near the end of the output:

0001-CG-add-lily-git-instructions.patch
Done.

Send patch files to the appropriate place:

• If you have a mentor, send it to them via email.
• Translators should send patches to translations@lilynet.net.
• More experienced contributors should upload the patch for web-based review. This requires additional software and use of the command-line; see Uploading a patch for review.
• If you have trouble uploading the patch for review, ask for help on lilypond-devel@gnu.org.

The “Abort changes – Reset to origin” button

The button labeled “Abort changes – Reset to origin” will copy all changed files to a subdirectory of ‘$LILYPOND_GIT’ named ‘aborted_edits/’, and will reset the repository to the current state of the remote repository (at git.sv.gnu.org).

2.3 git-cl

Git-cl is a ‘helper script’ that uploads patches to Google’s Rietveld Code Review Tool – used by the developers for patch review – and, at the same time, updates LilyPond’s issue tracker.

Installing git-cl

1. Download git-cl by running the command:

   git clone https://github.com/gperciva/git-cl.git
   

   or, if that command fails for any reason, try:

   git clone git://github.com/gperciva/git-cl.git
   

2. Add the ‘git-cl/’ directory to your PATH or create a symbolic link to the git-cl and upload.py scripts in one of your PATH directories (e.g. ‘$HOME/bin’).

   In GNU/Linux you can add directories to PATH by adding this line to your ‘.bashrc’ file located in your home directory:

   PATH=~/directory_containing_git-cl:"${PATH}"
   

Updating git-cl

LilyDev users should make sure that they always have the latest version of git-cl installed. It is possible that changes have been made to git-cl that are not (yet) included in the version of LilyDev that you are using.

Using a terminal run the following commands:

cd ~/git-cl/
git pull

This will download and update you to the lastest version of git-cl.

Configuring git-cl

Because git-cl updates two separate websites (Google’s Rietveld Code Review Tool and LilyPond’s issue tracker) you must have a valid user account (login and password) for both sites.

Set up a login account for Rietveld Code Review Tool

For the Rietveld Code Review Tool you will need a Google account but this does not require ‘Google’ email address; i.e. any email address for your Google account can be used. Just select the option “I prefer to use my current email address” when you sign up with Google.

Set up a login account for LilyPond’s Issue Tracker

Please register a user account at https://sourceforge.net/user/registration preferably using the same email address that you want to use LilyPond Developer mailing list login.

Once you have created this Sourceforge user account, send an email to the LilyPond Developer’s mailing list (lilypond-devel@gnu.org) asking for write access to the issue tracker along with your Sourceforce Username (not email address) and someone will then be able to set this up for you.

Authorizing git-cl for the LilyPond issue tracker

The git-cl command itself also needs to be ‘authorized’ so that it can access the LilyPond issue tracker.

1. Once you have been given a valid login for the LilyPond issue tracker, go to the ‘Account settings’ and select the ‘OAuth’ tab.
2. Locate the ‘Register New Application’ section and enter git-cl in the ‘Application Name:’ field.
3. Click on the ‘Register new application’ button. You should now see ‘git-cl’ listed under the ‘My Applications’ section.
4. Click on the ‘Generate Bearer Token’ button. You should now see ‘git-cl’ listed under the ‘Authorized Applications’ section along with a value for the ‘Bearer Token’ entry. This value is used, in the next steps, to allow git-cl to access and update the LilyPond issue tracker.

Installing ca-certificates

In order to have git-cl properly update issues on the SourceForge Allura issue tracker, you must have the package ca-certificates installed. You can check to see if the package is installed with

apt --installed list | grep ca-certificates

If ca-certificates is installed, you will get a result that shows the version that is installed. If it is not installed, there will be no version displayed.

Install ca-certificates with the following:

sudo apt-get install ca-certificates

Running git-cl for the first time

1. Using a terminal, move to the top level of the $LILYPOND_GIT directory and then run git-cl with the config option:

   cd $LILYPOND_GIT
   git-cl config
   

   You will see a series of prompts. For most of them you can simply accept the default value by responding with a newline (i.e. by pressing return or enter).

2. The prompt for the Rietveld server (the patch review tool), which defaults to codereview.appspot.com

   Rietveld server (host[:port]) [codereview.appspot.com]:
   

3. The prompt for the Allura server (the issue tracker), which defaults to https://sourceforge.net/p/testlilyissues/issues/

   Allura server [https://sourceforge.net/p/testlilyissues/issues/]:
   

4. When prompted for the Allura bearer token copy/paste the value generated in the previous steps for Authorising git-cl for the LilyPond issue tracker

   Allura bearer token (see https://sourceforge.net/auth/oauth/): fdbfca60801533465480

   Note: The above is a ‘fake’ bearer token used just for illustration. Do not use this value.

5. Finally, the prompt for the CC list, which defaults to lilypond-devel@gnu.org, the LilyPond Developer’s email list.

   CC list ("x" to clear) [lilypond-devel@gnu.org]:
   

The git-cl script should now be correctly configured for use.

2.4 Compiling with LilyDev

LilyDev is our ‘remix’ of Debian which contains all the necessary dependencies to do LilyPond development; for more information, see LilyDev.

Preparing the build

To prepare the build directory, enter (or copy&paste) the below text. This should take less than a minute.

cd $LILYPOND_GIT
sh autogen.sh --noconfigure
mkdir -p build/
cd build/
../configure

Building lilypond

Compiling LilyPond will take anywhere between 1 and 15 minutes on most ‘modern’ computers – depending on CPU and available RAM. We also recommend that you minimize the terminal window while it is building; this can help speed up on compilation times.

cd $LILYPOND_GIT/build/
make

It is possible to run make with the -j option to help speed up compilation times even more. See Compiling LilyPond

You may run the compiled lilypond with:

cd $LILYPOND_GIT/build/
out/bin/lilypond my-file.ly

Building the documentation

Compiling the documentation is a much more involved process, and will likely take 2 to 10 hours.

cd $LILYPOND_GIT/build/
make
make doc

The documentation is put in ‘out-www/offline-root/’. You may view the html files by entering the below text; we recommend that you bookmark the resulting page:

firefox $LILYPOND_GIT/build/out-www/offline-root/index.html

Installing

Don’t. There is no reason to install LilyPond within LilyDev. All development work can (and should) stay within the ‘$LILYPOND_GIT’ directory, and any personal composition or typesetting work should be done with an official GUB release.

Problems and other options

To select different build options, or isolate certain parts of the build, or to use multiple CPUs while building, read Compiling.

In particular, contributors working on the documentation should be aware of some bugs in the build system, and should read the workarounds in Generating documentation.

2.5 Now start work!

LilyDev users may now skip to the chapter which is aimed at their intended contributions:

• Documentation work
• Translating the documentation
• Website work
• Regression tests
• Programming work

These chapters are mainly intended for people not using LilyDev, but they contain extra information about the “behind-the-scenes” activities. We recommend that you read these at your leisure, a few weeks after beginning work with LilyDev.

• Working with source code
• Compiling

3. Working with source code

Advanced contributors will find this material quite useful, particularly if they are working on major new features.

3.1 Manually installing lily-git.tcl

We have created an easy-to-use GUI to simplify git for new contributors. If you are comfortable with the command-line, then skip ahead to Starting with Git.

1. If you haven’t already, download and install Git.
   • Windows users: download the .exe file labeled “Full installer for official Git” from:

     https://git-for-windows.github.io/
     

   • Other operating systems: either install git with your package manager, or download it from the “Binaries” section of:

     http://git-scm.com/download
     

2. Download the lily-git.tcl script from:

   http://git.sv.gnu.org/cgit/lilypond.git/plain/scripts/auxiliar/lily-git.tcl

3. To run the program from the command line, navigate to the directory containing lily-git.tcl and enter:

   wish lily-git.tcl
   

4. Click on the “Get source” button.

   This will create a directory called ‘lilypond-git/’ within your home directory, and will download the source code into that directory (around 150 Mb). When the process is finished, the “Command output” window will display “Done”, and the button label will change to say “Update source”.

5. Navigate to the ‘lilypond-git/’ directory to view the source files.

Further instructions are in How to use lily-git.

3.2 Starting with Git

Using the Git program directly (as opposed to using the lily-git.tcl GUI) allows you to have much greater control over the contributing process. You should consider using Git if you want to work on complex projects, or if you want to work on multiple projects concurrently.

3.2.1 Setting up

Installing Git

If you are using a Unix-based machine, the easiest way to download and install Git is through a package manager such as rpm or apt-get – the installation is generally automatic. The only required package is (usually) called git-core, although some of the auxiliary git* packages are also useful (such as gitk).

Alternatively, you can visit the Git website (http://git-scm.com/) for downloadable binaries and tarballs.

Initializing a repository

Once Git is installed, get a copy of the source code:

git clone git://git.sv.gnu.org/lilypond.git ~/lilypond-git

The above command will put the it in ‘~/lilypond-git’, where ~ represents your home directory.

Technical details

This creates (within the ‘$LILYPOND_GIT’ directory) a subdirectory called ‘.git/’, which Git uses to keep track of changes to the repository, among other things. Normally you don’t need to access it, but it’s good to know it’s there.

Configuring Git

Before working with the copy of the main LilyPond repository, you should configure some basic settings with the git config command. Git allows you to set both global and repository-specific options.

To configure settings that affect all repositories, use the ‘--global’ command line option. For example, the first two options that you should always set are your name and email, since Git needs these to keep track of commit authors:

git config --global user.name "John Smith"
git config --global user.email john@example.com

To configure Git to use colored output where possible, use:

git config --global color.ui auto

The text editor that opens when using git commit can also be changed. If none of your editor-related environment variables are set ($GIT_EDITOR, $VISUAL, or $EDITOR), the default editor is usually vi or vim. If you’re not familiar with either of these, you should probably change the default to an editor that you know how to use. For example, to change the default editor to nano, enter:

git config --global core.editor nano

Finally, and in some ways most importantly, let’s make sure that we can easily see the state of our working copy, without the need of typing git status repeatedly. If you’re not using LilyDev, add the following lines to your ‘~/.bashrc’:

export PS1="\u@\h \w\$(__git_ps1)$ "
export GIT_PS1_SHOWDIRTYSTATE=true
export GIT_PS1_SHOWUNTRACKEDFILES=true
export GIT_PS1_SHOWUPSTREAM=auto

The first line will show the branch we’re on. The other lines will use some symbols next to the branch name to indicate some kind of state. “*” means that there are unstaged changes, “+” indicates staged changes; if there are untracked files, a “%” will appear. Finally, we can also see if our HEAD is behind (“<”) or ahead (“>”) of its upstream, and if they have diverged (“<>”) or they are synced (“=”).

You may need to install the additional bash-completion package, but it is definitely worth it. After installation you must log out, and then log back in again to enable it.

Technical details

Git stores the information entered with git config --global in the file ‘.gitconfig’, located in your home directory. This file can also be modified directly, without using git config. The ‘.gitconfig’ file generated by the above commands would look like this:

[user]
        name = John Smith
        email = john@example.com
[color]
        ui = auto
[core]
        editor = nano

Using the git config command without the ‘--global’ option configures repository-specific settings, which are stored in the file ‘.git/config’. This file is created when a repository is initialized (using git init), and by default contains these lines:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true

However, since different repository-specific options are recommended for different development tasks, it is best to avoid setting any now. Specific recommendations will be mentioned later in this manual.

3.2.2 Git for the impatient

Ok, so you’ve been using lily-git.tcl for a while, but it’s time to take the next step. Since our review process delays patches by 60-120 hours, and you want to be able to work on other stuff while your previous work is getting reviewed, you’re going to use branches.

You can think of a branch as being a separate copy of the source code. But don’t worry about it.

Start work: make a new branch

Let’s pretend you want to add a section to the Contributor’s Guide about using branches.

Start by updating the repository, then making a new branch. Call the branch anything you want as long as the name starts with dev/. Branch names that don’t begin with dev/ are reserved for special things in lilypond.

git checkout master
git pull -r origin master
git branch dev/cg

Switch to that branch

Nothing has happened to the files yet. Let’s change into the new branch. You can think of this as “loading a file”, although in this case it’s really “loading a directory and subdirectories full of files”.

git checkout dev/cg

Your prompt now shows you that you’re on the other branch:

gperciva@LilyDev:~/lilypond-git (dev/cg)$

To be able to manage multiple lilypond issues at once, you’ll need to switch branches. You should have each lilypond issue on a separate branch. Switching branches is easy:

git checkout master
git checkout origin/staging
git checkout origin/release/unstable
git checkout dev/cg

Branches that begin with origin/ are part of the remote repository, rather than your local repository, so when you check them out you get a temporary local branch. You should never make changes directly on a branch beginning with origin/. You get changes into the remote repository by making them in local branches, and then pushing them to origin/staging as described below.

Make your changes

Edit files, then commit them.

git commit -a

Remember how I said that switching to a branch was like “loading a directory”? Well, you’ve just “saved a directory”, so that you can “load” it later.

When you create a new file, you need to add it to git, then commit it:

git add input/regression/avoid-crash-on-condition.ly
git commit -a

Edit more files. Commit them again. Edit yet more files, commit them again. Go eat dinner. Switch to master so you can play with the latest changes from other developers. Switch back to your branch and edit some more. Commit those changes.

At this stage, don’t worry about how many commits you have.

Save commits to external files

Branches are nerve-wracking until you get used to them. You can save your hard work as individual ‘.patch’ files. Be sure to commit your changes first.

git commit -a
git format-patch master

I personally have between 4 and 20 of those files saved in a special folder at any point in time. Git experts might laugh as that behavior, but I feel a lot better knowing that I’ve got those backups.

Prepare your branch for review

After committing, you can update your branch with the latest master:

git commit -a
git checkout master
git pull -r origin master
git checkout dev/cg
git rebase master

Due to the speed of lilypond development, sometimes master has changed so much that your branch can no longer be applied to it. In that happens, you will have a merge conflict. Stop for a moment to either cry or have a stiff drink, then proceed to Merge conflicts.

Upload your branch

Finally, you’re finished your changes. Time to upload for review. Make sure that you’re on your branch, then upload:

git checkout dev/cg
git-cl upload master

Wait for reviews

While you’re waiting for a countdown and reviews, go back to master, make a dev/doc-beams branch, and start adding doc suggestions from issue 12345 from the tracker. Or make a dev/page-breaks and fix bug in page breaking. Or whatever. Don’t worry, your dev/cg is safe.

Combining commits (optional unless you have broken commits)

Does the history of your branch look good?

gitk

If you have a lot of commits on your branch, you might want to combine some of them. Alternately, you may like your commits, but want to edit the commit messages.

git rebase -i master

Follow instructions on the screen.

Push to staging

When you’ve got the coveted Patch-push status, time to prepare your upload:

git fetch
git rebase origin/staging dev/cg~0
gitk HEAD

If everything looks good, push it:

git push origin HEAD:staging

Then change back to your working branch:

git checkout dev/cg

Delete your branch (safe)

After a few hours, if there’s nothing wrong with your branch, it should be automatically moved to origin/master. Update, then try removing your branch:

git checkout master
git pull -r origin master
git branch -d dev/cg

The last command will fail if the contents of dev/cg are not present in origin/master.

Delete your branch (UNSAFE)

Sometimes everything goes wrong. If you want to remove a branch even though it will cause your work to be lost (that is, if the contents of dev/cg are not present in master), follow the instructions in “Delete your branch (safe)”, but replace the -d on the final line with a -D.

3.2.3 Other repositories

We have a few other code repositories.

lilypond-extra

There is a separate repository for general administrative scripts, as well as pictures and media files for the website. People interested in working on the website should download this repository, and set their $LILYPOND_WEB_MEDIA_GIT environment variable to point to that repository.

https://github.com/gperciva/lilypond-extra

To configure an environment variable in bash (the default for most GNU/Linux distributions),

export LILYPOND_WEB_MEDIA_GIT=$HOME/dir/of/lilypond-extra/

Be aware that lilypond-extra is the definitive source for some binary files - in particular PDF versions of papers concerning LilyPond. To add further PDFs of this sort, all that is necessary is to add the PDF to lilypond-extra and then add a reference to it in the documentation. The file will then be copied to the website when make website is run.

However, pictures that are also used in the documentation build are mastered in the main git repository. If any of these is changed, it should be updated in git, and then the updates copied to lilypond-extra.

Grand Unified Builder (GUB)

Another item of interest might be the Grand Unified Builder, our cross-platform building tool. Since it is used by other projects as well, it is not stored in our gub repository. For more info, see http://lilypond.org/gub.

There are two locations for this repository: the version being used to build lilypond, which is at

http://github.com/gperciva/gub

and the original version by Jan Nieuwenhuizen, kept at

http://github.com/janneke/gub

LilyPad

Our binary releases on MacOS X and Windows contain a lightweight text editor.

To make any modifications the Windows editor, you will need to do the following:

1. Clone the git repository from https://github.com/gperciva/lilypad
2. Make changes to the source, and check it compiles. In a Windows environment MinGW provides both a Git installation and a gcc compiler. This can be obtained from http://www.mingw.org/
3. Update the version which is contained in the ‘rsrc.rc’. Check this compiles, too.
4. Commit the changes with an informative commit message.
5. Push the changes to github. You will need to use syntax similiar to this:

   git push https://UserName@github.com/gperciva/lilypad.git
   

   You will need to have push access to the git repository for this to be successful.

6. Make a tarball of the source code to be used by GUB by pulling the updated repository from GitHub. Ensure that the tarball has the correct Version number.
7. Copy the tarball to http://lilypond.org/downloads/gub-sources/lilypad/. You will need to have SSH access to lilypond.org. If you do not, contact the Release Manager via the lilypond-devel mailing list.
8. Update GUB to make it use the new tarball by editing ‘gub/specs/lilypad.py’ and changing the source = line to point to the new source.
9. Push this updated ‘lilypad.py’ version to the GUB repository on GitHub.
10. Test the changes with a new GUB compile.

yet more repositories

There are a few other repositories floating around, which will hopefully be documented in the near future.

3.2.4 Downloading remote branches

Organization of remote branches

The main LilyPond repository is organized into branches to facilitate development. These are often called remote branches to distinguish them from local branches you might create yourself (see Using local branches).

The master branch contains all the source files used to build LilyPond, which includes the program itself (both stable and development releases), the documentation (and its translations), and the website. Generally, the master branch is expected to compile successfully.

The translation branch is a side branch that allows translators to work without needing to worry about compilation problems. Periodically, the Translation Meister (after verifying that it doesn’t break compilation), will merge this branch into staging to incorporate recent translations. Similarly, the master branch is usually merged into the translation branch after significant changes to the English documentation. See Translating the documentation for details.

LilyPond repository sources

The recommended source for downloading a copy of the main repository is:

git://git.sv.gnu.org/lilypond.git

However, if your internet router filters out connections using the GIT protocol, or if you experience difficulty connecting via GIT, you can try these other sources:

ssh://git.sv.gnu.org/srv/git/lilypond.git
http://git.sv.gnu.org/r/lilypond.git

The SSH protocol can only be used if your system is properly set up to use it. Also, the HTTP protocol is slowest, so it should only be used as a last resort.

Downloading individual branches

Once you have initialized an empty Git repository on your system (see Initializing a repository), you can download a remote branch into it. Make sure you know which branch you want to start with.

To download the master branch, enter the following:

git remote add -ft master -m master \
  origin git://git.sv.gnu.org/lilypond.git/

To download the translation branch, enter:

git remote add -ft translation -m \
  translation origin git://git.sv.gnu.org/lilypond.git/

The git remote add process could take up to ten minutes, depending on the speed of your connection. The output will be something like this:

Updating origin
remote: Counting objects: 235967, done.
remote: Compressing objects: 100% (42721/42721), done.
remote: Total 235967 (delta 195098), reused 233311 (delta 192772)
Receiving objects: 100% (235967/235967), 68.37 MiB | 479 KiB/s, done.
Resolving deltas: 100% (195098/195098), done.
From git://git.sv.gnu.org/lilypond
 * [new branch]      master     -> origin/master
From git://git.sv.gnu.org/lilypond
 * [new tag]         flower/1.0.1 -> flower/1.0.1
 * [new tag]         flower/1.0.10 -> flower/1.0.10
⋮
 * [new tag]         release/2.9.6 -> release/2.9.6
 * [new tag]         release/2.9.7 -> release/2.9.7

When git remote add is finished, the remote branch should be downloaded into your repository—though not yet in a form that you can use. In order to browse the source code files, you need to create and checkout your own local branch. In this case, however, it is easier to have Git create the branch automatically by using the checkout command on a non-existent branch. Enter the following:

git checkout -b branch origin/branch

where branch is the name of your tracking branch, either master or translation.

Git will issue some warnings; this is normal:

warning: You appear to be on a branch yet to be born.
warning: Forcing checkout of origin/master.
Branch master set up to track remote branch master from origin.
Already on 'master'

By now the source files should be accessible—you should be able to edit any files in the ‘$LILYPOND_GIT’ directory using a text editor of your choice. But don’t start just yet! Before editing any source files, learn how to keep your changes organized and prevent problems later—read Basic Git procedures.

Technical Details

The git remote add command should add some lines to your local repository’s ‘.git/config’ file:

[remote "origin"]
        url = git://git.sv.gnu.org/lilypond.git/
        fetch = +refs/heads/master:refs/remotes/origin/master

Downloading all remote branches

To download all remote branches at once, you can clone the entire repository:

git clone git://git.sv.gnu.org/lilypond.git

Other branches

Most contributors will never need to touch the other branches. If you wish to do so, you will need more familiarity with Git; please see Other Git documentation.

• dev/XYZ: These branches are for individual developers. They store code which is not yet stable enough to be added to the master branch.
• stable/XYZ: The branches are kept for archival reasons.
• archive/XYZ: The branches are kept for archival reasons.

3.3 Basic Git procedures

3.3.1 The Git contributor’s cycle

Here is a simplified view of the contribution process on Git:

1. Update your local repository by pulling the most recent updates from the remote repository.
2. Edit source files within your local repository’s working directory.
3. Commit the changes you’ve made to a local branch.
4. Generate a patch to share your changes with the developers.

3.3.2 Pulling and rebasing

When developers push new patches to the git.sv.gnu.org repository, your local repository is not automatically updated. It is important to keep your repository up-to-date by periodically pulling the most recent commits from the remote branch. Developers expect patches to be as current as possible, since outdated patches require extra work before they can be used.

Occasionally you may need to rework some of your own modifications to match changes made to the remote branch (see Resolving conflicts), and it’s considerably easier to rework things incrementally. If you don’t update your repository along the way, you may have to spend a lot of time resolving branch conflicts and reconfiguring much of the work you’ve already done.

Fortunately, Git is able to resolve certain types of branch conflicts automatically with a process called rebasing. When rebasing, Git tries to modify your old commits so they appear as new commits (based on the latest updates). For a more involved explanation, see the git-rebase man page.

To pull without rebasing (recommended for translators), use the following command:

git pull    # recommended for translators

If you’re tracking the remote master branch, you should add the ‘-r’ option (short for ‘--rebase’) to keep commits on your local branch current:

git pull -r # use with caution when translating

If you don’t edit translated documentation and don’t want to type ‘-r’ every time, configure the master branch to rebase by default with this command:

git config branch.master.rebase true

If pull fails because of a message like

error: Your local changes to 'Documentation/learning/tutorial.itely'
would be overwritten by merge.  Aborting.

or

Documentation/learning/tutorial.itely: needs update
refusing to pull with rebase: your working tree is not up-to-date

it means that you have modified some files in you working tree without committing changes (see Commits); you can use the git stash command to work around this:

git stash      # save uncommitted changes
git pull -r    # pull using rebase (translators omit "-r")
git stash pop  # reapply previously saved changes

Note that git stash pop will try to apply a patch, and this may create a conflict. If this happens, see Resolving conflicts.

TODO: I think the next paragraph is confusing. Perhaps prepare the reader for new terms ‘committish’ and ‘head’? -mp

TODO: when committishes automatic conditional update have been tested and documented, append the following to the warning above: Note that using update-committishes make target generally touches committishes.

Technical details

The git config command mentioned above adds the line rebase = true to the master branch in your local repository’s ‘.git/config’ file:

[branch "master"]
        remote = origin
        merge = refs/heads/master
        rebase = true

3.3.3 Using local branches

Creating and removing branches

Local branches are useful when you’re working on several different projects concurrently. To create a new branch, enter:

git branch name

To delete a branch, enter:

git branch -d name

Git will ask you for confirmation if it sees that data would be lost by deleting the branch. Use ‘-D’ instead of ‘-d’ to bypass this. Note that you cannot delete a branch if it is currently checked out.

Listing branches and remotes

You can get the exact path or URL of all remote branches by running:

git remote -v

To list Git branches on your local repositories, run

git branch     # list local branches only
git branch -r  # list remote branches
git branch -a  # list all branches

Checking out branches

To know the currently checked out branch, i.e. the branch whose source files are present in your working tree, read the first line of the output of

git status

The currently checked out branch is also marked with an asterisk in the output of git branch.

You can check out another branch other_branch, i.e. check out other_branch to the working tree, by running

git checkout other_branch

Note that it is possible to check out another branch while having uncommitted changes, but it is not recommended unless you know what you are doing; it is recommended to run git status to check this kind of issue before checking out another branch.

Merging branches

To merge branch foo into branch bar, i.e. to “add” all changes made in branch foo to branch bar, run

git checkout bar
git merge foo

If any conflict happens, see Resolving conflicts.

There are common usage cases for merging: as a translator, you will often want the Translations meister to merge master into translation; on the other hand, the Translations meister wants to merge translation into staging whenever he has checked that translation builds successfully.

3.3.4 Commits

Understanding commits

Technically, a commit is a single point in the history of a branch, but most developers use the term to mean a commit object, which stores information about a particular revision. A single commit can record changes to multiple source files, and typically represents one logical set of related changes (such as a bug-fix). You can list the ten most recent commits in your current branch with this command:

git log -10 --oneline

If you’re using an older version of Git and get an ‘unrecognized argument’ error, use this instead:

git log -10 --pretty=oneline --abbrev-commit

More interactive lists of the commits on the remote master branch are available at http://git.sv.gnu.org/gitweb/?p=lilypond.git;a=shortlog and http://git.sv.gnu.org/cgit/lilypond.git/log/.

How to make a commit

Once you have modified some source files in your working directory, you can make a commit with the following procedure:

1. Make sure you’ve configured Git properly (see Configuring Git). Check that your changes meet the requirements described in Code style and/or Documentation policy. For advanced edits, you may also want to verify that the changes don’t break the compilation process.
2. Run the following command:

   git status
   

   to make sure you’re on the right branch, and to see which files have been modified, added or removed, etc. You may need to tell Git about any files you’ve added by running one of these:

   git add file  # add untracked file individually
   git add .     # add all untracked files in current directory
   

   After git add, run git status again to make sure you got everything. You may also need to modify ‘GNUmakefile’.

3. Preview the changes about to be committed (to make sure everything looks right) with:

   git diff HEAD
   

   The HEAD argument refers to the most recent commit on the currently checked-out branch.

4. Generate the commit with:

   git commit -a
   

   The ‘-a’ is short for ‘--all’ which includes modified and deleted files, but only those newly created files that have previously been added.

Commit messages

When you run the git commit -a command, Git automatically opens the default text editor so you can enter a commit message. If you find yourself in a foreign editing environment, you’re probably in vi or vim. If you want to switch to an editor you’re more familiar with, quit by typing :q! and pressing <Enter>. See Configuring Git for instructions on changing the default editor.

In any case, Git will open a text file for your commit message that looks like this:

# Please enter the commit message for your changes.  Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#	modified:   working.itexi
#

Your commit message should begin with a one-line summary describing the change (no more than 50 characters long), and if necessary a blank line followed by several lines giving the details:

Doc: add Baerenreiter and Henle solo cello suites

Added comparison of solo cello suite engravings to new essay with
high-res images, fixed cropping on Finale example.

Commit messages often start with a short prefix describing the general location of the changes.

• Doc: and Doc-**: If a commit affects the documentation in English (or in several languages simultaneously) the commit message should be prefixed with “Doc: ”. If the commit affects only one of the translations, the commit message should be prefixed with “Doc-**: ”, where ** is the two-letter language code.
• Web: and Web-**: Commits that affect the website should use “Web: ” for English, and “Web-**: ” for other languages.
• CSS: Commits that change CSS files should use “Web: CSS: ” or “Doc: CSS: ” depending on whether they affect the website or the documentation/manuals.
• Changes to a single file are often prefixed with the name of the file involved.

Visit the links listed in Understanding commits for examples.

3.3.5 Patches

How to make a patch

If you want to share your changes with other contributors and developers, you need to generate patches from your commits. We prefer it if you follow the instructions in Uploading a patch for review. However, we present an alternate method here.

You should always run git pull -r (translators should leave off the ‘-r’) before doing this to ensure that your patches are as current as possible.

Once you have made one or more commits in your local repository, and pulled the most recent commits from the remote branch, you can generate patches from your local commits with the command:

git format-patch origin

The origin argument refers to the remote tracking branch at git.sv.gnu.org. This command generates a separate patch for each commit that’s in the current branch but not in the remote branch. Patches are placed in the current working directory and will have names that look something like this:

0001-Doc-Fix-typos.patch
0002-Web-Remove-dead-links.patch
⋮

Send an email (must be less than 64 KB) to lilypond-devel@gnu.org briefly explaining your work, with the patch files attached. Translators should send patches to translations@lilynet.net. After your patches are reviewed, the developers may push one or more of them to the main repository or discuss them with you.

Emailing patches

The default x-diff MIME type associated with patch files (i.e., files whose name ends in .patch) means that the encoding of line endings may be changed from UNIX to DOS format when they are sent as attachments. Attempting to apply such an inadvertently altered patch will cause git to fail with a message about ‘whitespace errors’.

The solution to such problems is surprisingly simple—just change the default file extension of patches generated by git to end in .txt, for example:

git config format.suffix '.patch.txt'

This should cause email programs to apply the correct base64 encoding to attached patches.

If you receive a patch with DOS instead of UNIX line-endings, it can be converted back using the dos2unix utility.

Lots of useful information on email complications with patches is provided on the Wine wiki at http://wiki.winehq.org/GitWine.

3.3.6 Uploading a patch for review

Any non-trivial change should be uploaded to our “Rietveld” code review website:

http://codereview.appspot.com/

You can upload a patch for review by using our custom git-cl ‘helper-script’. This section assumes you have already installed, updated, and configured git-cl. See git-cl.

There are two methods, depending on your git setup.

• Master branch: (easy option)

  If you added your patch to master, then:

  git pull -r
  git-cl upload origin/master
  

  If you have git push ability, make sure that you remove your patch (with git rebase or git reset) before pushing other stuff.

  Notifications of patches are automatically added to our issue tracker to reduce the chance of patches getting lost. To suppress this (not recommended), add the -n / --no-code-issue option.

• Separate branch: (complicated option)

  Ensure your changes are committed in a separate branch, which should differ from the reference branch to be used (usually origin/master) by just the changes to be uploaded. Checkout the branch with the changes:

  git checkout some-branch-with-changes
  

  If the reference branch is to be origin/master, ensure that the branch containing the changes is up-to-date with it. Use git rebase or git pull -r to rebase the branch to the head of origin/master. For example:

  git pull -r origin master
  

  Finally, start the upload by entering:

  git-cl upload <reference SHA1 ID>
  

  where <reference SHA1 ID> is the SHA1 ID of the commit to be used as a reference source for the patch. Generally, this will be the SHA1 ID of origin/master, and in that case you can just use the command:

  git-cl upload origin/master
  

First you will see a terminal editor where you can edit the message that will accompany your patch. git-cl will respect the EDITOR environment variable if defined, otherwise it will use vi as the default editor.

After prompting for your Google email address and password, the patch set will be posted to Rietveld, and you will be given a URL for your patch.

Announcing your patch set

You should then announce the patch by logging into the code review issue webpage and using “Publish + Mail Comments” to add a (mostly bogus) comment to your issue. The text of your comment will be sent to our developer mailing list.

Revisions

As revisions are made in response to comments, successive patch sets for the same issue can be uploaded by reissuing the git-cl command with the modified branch checked out.

Sometimes in response to comments on revisions, the best way to work may require creation of a new branch in git. In order to associate the new branch with an existing Rietveld issue, the following command can be used:

git-cl issue issue-number

where issue-number is the number of the existing Rietveld issue.

Resetting git-cl

If git-cl becomes confused, you can “reset” it by running:

git-cl issue 0

3.3.7 The patch review cycle

Your patch will be available for reviews for the next few hours or days. Three times a week, patches with no known problems are gathered into a “patch countdown” and their status changed to patch-countdown. The countdown is a 48-hour waiting period in which any final reviews or complaints should be made.

During the countdown, your patch may be set to patch-needs_work, indicating that you should fix something (or at least discuss why the patch needs no modification). If no problems are found, the patch will be set to patch-push.

Once a patch has patch-push, it should be sent to your mentor for uploading. If you have git push ability, look at Pushing to staging.

• Patches get added to the tracker and to Rietveld by the “git-cl” tool, with a status of “patch-new”.
• The automated tester, Patchy, verifies that the patch can be applied to current master. By default, it checks that the patch allows make and make test to complete successfully. It can also be configured to check that make doc is successful. If it passes, Patchy changes the status to “patch-review” and emails the developer list. If the patch fails, Patchy sets it to “patch-needs_work” and notifies the developer list.
• The Patch Meister reviews the tracker periodically, to list patches which have been on review for at least 24 hours. The list is found at

  http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch%20patch=review&sort=modified+patch&colspec=ID%20Type%20Status%20Priority%20Owner%20Patch%20Summary%20Modified

• For each patch, the Handler reviews any discussion on the tracker and on Rietveld, to determine whether the patch can go forward. If there is any indication that a developer thinks the patch is not ready, the Handler marks it “patch-needs_work” and makes a comment regarding the reason, referring to the Rietveld item if needed.
• Patches with explicit approval, or at least no negative comment, can be updated to “patch-countdown”. When saving the tracker item, clear the “send email” box to prevent sending notification for each patch.
• The Patch Meister sends an email to the developer list, with a fixed subject line, to enable filtering by email clients:

  PATCH: Countdown to 20130113
  

  The text of the email sets the deadline for this countdown batch. At present, batches are done on Tuesday, Thursday and Sunday evenings.

  To create the countdown announcement, use the make-countdown-announcement.sh script, which takes the deadline date, and optionally your name. Follow the instructions provided:

  cd $LILYPOND_GIT
  scripts/auxiliar/make-countdown-announcement.sh "Jan 1, 2001" James
  

  The script produces an announcement that is easily readable in all email clients. Also, whenever a new contributor submits a patch, you will be prompted to add the new username and author name to the script itself, and then commit those changes to the main git repository.

• On the scheduled countdown day, the Patch Meister reviews the previous list of patches on countdown, with the same procedure and criteria as before. Patches with no controversy can be set to “patch-push” with a courtesy message added to the comment block.
• Roughly at six month intervals, the Patch Meister can list the patches which have been set to “patch-needs-work” and send the results to the developer list for review. In most cases, these patches should be marked “patch-abandoned” but this should come from the developer if possible.
• As in most organisations of unpaid volunteers, fixed procedures are useful in as much as they get the job done. In our community, there is room for senior developers to bypass normal patch handling flows, particularly now that the testing of patches is largely automated. Similarly, the minimum age of 24 hours can reasonably be waived if the patch is minor and from an experienced developer.

3.4 Advanced Git procedures

It is possible to work with several branches on the same local Git repository; this is especially useful for translators who may have to deal with both translation and a stable branch, e.g. stable/2.12.

Some Git commands are introduced first, then a workflow with several Git branches of LilyPond source code is presented.

3.4.1 Merge conflicts

To be filled in later, and/or moved to a different section. I just wanted to make sure that I had a stub ready somewhere.

3.4.2 Advanced Git concepts

A bit of Git vocabulary will be explained below. The following is only introductory; for a better understanding of Git concepts, you may wish to read Other Git documentation.

The git pull origin command above is just a shortcut for this command:

git pull git://git.sv.gnu.org/lilypond.git/ branch:origin/branch

where branch is typically master or translation; if you do not know or remember, see Downloading remote branches to remember which commands you issued or which source code you wanted to get.

A commit is a set of changes made to the sources; it also includes the committish of the parent commit, the name and e-mail of the author (the person who wrote the changes), the name and e-mail of the committer (the person who brings these changes into the Git repository), and a commit message.

A committish is the SHA1 checksum of a commit, a number made of 40 hexadecimal digits, which acts as the internal unique identifier for this commit. To refer to a particular revision, don’t use vague references like the (approximative) date, simply copy and paste the committish.

A branch is nothing more than a pointer to a particular commit, which is called the head of the branch; when referring to a branch, one often actually thinks about its head and the ancestor commits of the head.

Now we will explain the two last commands you used to get the source code from Git—see Downloading individual branches.

git remote add -ft branch -m branch \
  origin git://git.sv.gnu.org/lilypond.git/

git checkout -b branch origin/branch

The git remote has created a branch called origin/branch in your local Git repository. As this branch is a copy of the remote branch web from git.sv.gnu.org LilyPond repository, it is called a remote branch, and is meant to track the changes on the branch from git.sv.gnu.org: it will be updated every time you run git pull origin or git fetch origin.

The git checkout command has created a branch named branch. At the beginning, this branch is identical to origin/branch, but it will differ as soon as you make changes, e.g. adding newly translated pages or editing some documentation or code source file. Whenever you pull, you merge the changes from origin/branch and branch since the last pulling. If you do not have push (i.e. “write”) access on git.sv.gnu.org, your branch will always differ from origin/branch. In this case, remember that other people working like you with the remote branch branch of git://git.sv.gnu.org/lilypond.git/ (called origin/branch on your local repository) know nothing about your own branch: this means that whenever you use a committish or make a patch, others expect you to take the latest commit of origin/branch as a reference.

Finally, please remember to read the man page of every Git command you will find in this manual in case you want to discover alternate methods or just understand how it works.

3.4.3 Resolving conflicts

Occasionally an update may result in conflicts – this happens when you and somebody else have modified the same part of the same file and git cannot figure out how to merge the two versions together. When this happens, you must manually merge the two versions.

If you need some documentation to understand and resolve conflicts, see paragraphs How conflicts are presented and How to resolve conflicts in git merge man page.

If all else fails, you can follow the instructions in Reverting all local changes. Be aware that this eliminates any changes you have made!

3.4.4 Reverting all local changes

Sometimes git will become hopelessly confused, and you just want to get back to a known, stable state. This command destroys any local changes you have made in the currently checked-out branch, but at least you get back to the current online version:

git reset --hard origin/master

3.4.5 Working with remote branches

Fetching new branches from git.sv.gnu.org

To fetch and check out a new branch named branch on git.sv.gnu.org, run from top of the Git repository

git config --add remote.origin.fetch \
  +refs/heads/branch:refs/remotes/origin/branch

git checkout --track -b branch origin/branch

After this, you can pull branch from git.sv.gnu.org with:

git pull

Note that this command generally fetches all branches you added with git remote add (when you initialized the repository) or git config --add, i.e. it updates all remote branches from remote origin, then it merges the remote branch tracked by the current branch into the current branch. For example, if your current branch is master, origin/master will be merged into master.

Local clones, or having several working trees

If you play with several Git branches, e.g. master, translation, stable/2.12), you may want to have one source and build tree for each branch; this is possible with subdirectories of your local Git repository, used as local cloned subrepositories. To create a local clone for the branch named branch, run

git checkout branch
git clone -lsn . subdir
cd subdir
git reset --hard

Note that subdir must be a directory name which does not already exist. In subdir, you can use all Git commands to browse revisions history, commit and uncommit changes; to update the cloned subrepository with changes made on the main repository, cd into subdir and run git pull; to send changes made on the subrepository back to the main repository, run git push from subdir. Note that only one branch (the currently checked out branch) is created in the subrepository by default; it is possible to have several branches in a subrepository and do usual operations (checkout, merge, create, delete...) on these branches, but this possibility is not detailed here.

When you push branch from subdir to the main repository, and branch is checked out in the main repository, you must save uncommitted changes (see git stash) and do git reset --hard in the main repository in order to apply pushed changes in the working tree of the main repository.

3.4.6 Git log

The commands above don’t only bring you the latest version of the sources, but also the full history of revisions (revisions, also called commits, are changes made to the sources), stored in the ‘.git’ directory. You can browse this history with

git log     # only shows the logs (author, committish and commit message)
git log -p  # also shows diffs
gitk        # shows history graphically

3.4.7 Applying remote patches

TODO: Explain how to determine if a patch was created with git format-patch.

Well-formed git patches created with git format-patch should be committed with the following command:

git am patch

Patches created without git format-patch can be applied in two steps. The first step is to apply the patch to the working tree and the index:

git apply --index patch

The second step is to commit the changes and give credit to the author of the patch. This can be done with the following command:

git commit --author="John Smith <john@example.com>"

Please note that using the --index option for patching is quite important here and cannot reliably be replaced by using the -a option when committing: that would only commit files from the working tree that are already registered with git, so every file that the patch actually adds, like a regtest for a fixed bug, would get lost. For the same reason, you should not use the git-independent ‘patch’ program for applying patches.

3.4.8 Cleaning up multiple patches

If you have been developing on your own branch for a while, you may have more commmits than is really sensible. To revise your work and condense commits, use:

git rebase origin/master
git rebase -i origin/master

3.4.9 Commit access

Most contributors are not able to commit patches directly to the main repository—only members of the LilyPond development team have commit access. If you are a contributor and are interested in joining the development team, contact the Project Manager through the mailing list (lilypond-devel@gnu.org). Generally, only contributors who have already provided a number of patches which have been pushed to the main repository will be considered for membership.

If you have been approved by the Project Manager, use the following procedure to obtain commit access:

1. If you don’t already have one, set up a Savannah user account at https://savannah.gnu.org/account/register.php. If your web browser responds with an “untrusted connection” message when you visit the link, follow the steps for including the CAcert root certificate in your browser, given at http://savannah.gnu.org/tls/tutorial/.

   Note: Savannah will silently put your username in lower-case – do not try to use capital letters.

2. After registering, if you are not logged in automatically, login at https://savannah.gnu.org/account/login.php—this should take you to your “my” page (https://savannah.gnu.org/my/).
3. Click on the “My Groups” link to access the “My Group Membership” page. From there, find the “Request for Inclusion” box and search for “LilyPond”. Among the search results, check the box labeled “GNU LilyPond Music Typesetter” and write a brief (required) message for the Project Manager (“Hey it’s me!” should be fine).

   Note that you will not have commit access until the Project Manager activates your membership. Once your membership is activated, LilyPond should appear under the heading “Groups I’m Contributor of” on your “My Group Membership” page.

4. Generate an SSH ‘rsa’ key pair. Enter the following at the command prompt:

   ssh-keygen -t rsa
   

   When prompted for a location to save the key, press <ENTER> to accept the default location (‘~/.ssh/id_rsa’).

   Next you are asked to enter an optional passphrase. On most systems, if you use a passphrase, you will likely be prompted for it every time you use git push or git pull. You may prefer this since it can protect you from your own mistakes (like pushing when you mean to pull), though you may find it tedious to keep re-entering it.

   You can change/enable/disable your passphrase at any time with:

   ssh-keygen -f ~/.ssh/id_rsa -p
   

   Note that the GNOME desktop has a feature which stores your passphrase for you for an entire GNOME session. If you use a passphrase to “protect you from yourself”, you will want to disable this feature, since you’ll only be prompted once. Run the following command, then logout of GNOME and log back in:

   gconftool-2 --set -t bool \
     /apps/gnome-keyring/daemon-components/ssh false
   

   After setting up your passphrase, your private key is saved as ‘~/.ssh/id_rsa’ and your public key is saved as ‘~/.ssh/id_rsa.pub’.

5. Register your public SSH ‘rsa’ key with Savannah. From the “My Account Configuration” page, click on “Edit SSH Keys”, then paste the contents of your ‘~/.ssh/id_rsa.pub’ file into one of the “Authorized keys” text fields, and click “Update”.

   Savannah should respond with something like:

   Success: Key #1 seen Keys registered
   

6. Configure Git to use the SSH protocol (instead of the GIT protocol). From your local Git repository, enter:

   git config remote.origin.url \
     ssh://user@git.sv.gnu.org/srv/git/lilypond.git
   

   replacing user with your Savannah username.

7. After your membership has been activated and you’ve configured Git to use SSH, test the connection with:

   git pull --verbose
   

   SSH should issue the following warning:

   The authenticity of host 'git.sv.gnu.org (140.186.70.72)' can't
   be established.
   RSA key fingerprint is
   80:5a:b0:0c:ec:93:66:29:49:7e:04:2b:fd:ba:2c:d5.
   Are you sure you want to continue connecting (yes/no)?
   

   Make sure the RSA key fingerprint displayed matches the one above. If it doesn’t, respond “no” and check that you configured Git properly in the previous step. If it does match, respond “yes”. SSH should then issue another warning:

   Warning: Permanently added 'git.sv.gnu.org,140.186.70.72' (RSA) to
   the list of known hosts.
   

   The list of known hosts is stored in the file ‘~/.ssh/known_hosts’.

   At this point, you are prompted for your passphrase if you have one, then Git will attempt a pull.

   If git pull --verbose fails, you should see error messages like these:

   Permission denied (publickey).
   fatal: The remote end hung up unexpectedly
   

   If you get the above error, you may have made a mistake when registering your SSH key at Savannah. If the key is properly registered, you probably just need to wait for the Savannah server to activate it. It usually takes a few minutes for the key to be active after registering it, but if it still doesn’t work after an hour, ask for help on the mailing list.

   If git pull --verbose succeeds, the output will include a ‘From’ line that shows ‘ssh’ as the protocol:

   From ssh://git.sv.gnu.org/srv/git/lilypond
   

   If the protocol shown is not ‘ssh’, check that you configured Git properly in the previous step.

8. Test your commit access with a dry run:

   Note: Do not push directly to master; instead, push to staging. See Pushing to staging.

   git push --dry-run --verbose
   

   Note that recent versions of Git (Git 1.6.3 or later) will issue a big warning if the above command is used. The simplest solution is to tell Git to push all matching branches by default:

   git config push.default matching
   

   Then git push should work as before. For more details, consult the git push man page.

9. Repeat the steps from generating an RSA key through to testing your commit access, for each machine from which you will be making commits, or you may simply copy the files from your local ‘~/.ssh’ folder to the same folder on the other machine.

Technical details

• On Firefox, to view or remove the CAcert root certificate, go to: Edit > Preferences > Advanced > Encryption > View Certificates > Authorities > Certificate Name > Root CA > CA Cert Signing Authority.
• The git config commands above should modify your local repository’s ‘.git/config’ file. These lines:

  [remote "origin"]
          url = git://git.sv.gnu.org/lilypond.git/
  

  should now be changed to:

  [remote "origin"]
          url = ssh://user@git.sv.gnu.org/srv/git/lilypond.git
  

  where user is your login name on Savannah.

• Similarly, the git config push.default matching command should add these lines to ‘.git/config’:

  [push]
          default = matching
  

Known issues and warnings

Encryption protocols, including ssh, generally do not permit packet fragmentation to avoid introducing a point of insecurity. This means that the maximum packet size must not exceed the smallest MTU (Maximum Transmission Unit) set in the routers along the path. This smallest MTU is determined by a procedure during call set-up which relies on the transmission over the path of ICMP packets. If any of the routers in the path block ICMP packets this mechanism fails, resulting in the possibility of packets being transmitted which exceed the MTU of one of the routers. If this happens the packet is discarded, causing the ssh session to hang, timeout or terminate with the error message

ssh: connect to host <host ip addr> port 22: Bad file number
fatal: The remote end hung up unexpectedly

depending on precisely when in the proceedings the first large packet is transmitted. Most routers on the internet have MTU set to 1500, but routers installed in homes to connect via broadband may use a slightly smaller MTU for efficient transmission over ATM. If this problem is encountered a possible work-around is to set the MTU in the local router to 1500.

3.4.10 Pushing to staging

Do not push directly to the git master branch. Instead, push to staging.

You will not see your patch on origin/master until some automatic tests have been run. These tests are run every couple of hours; please wait at least 12 hours before wondering if your patch has been lost. Note that you can check the commits on origin/staging by looking at the git web interface on savannah.

It may happen occasionally that the staging branch breaks automated testing. In this case the automatic move of staging material to master gets halted in order to avoid broken material entering master. This is a safety net. Please do not try breaking out from it by adding fixes on top of staging: in that case the whole sequence will end up in master after all, defeating the purpose of the system. The proper fix usually involves rewriting the staging branch and is best left to core developers after discussion on the developer list.

Before pushing to staging it is a good practice to check whether staging is ahead of master, and if so, wait until master has caught up with staging before pushing. This simplifies things if changes to staging have to be backed out for some reason. To check whether master has caught up with staging you can look at the git web interface on savannah, or do:

git fetch
gitk

and check that origin/master is at the same commit as origin/staging. Another option is to see if any commits are listed when you do:

git fetch
git log origin/master..origin/staging

If your work is in a patch file

Assuming that your patch is in a file called ‘0001-my-patch.patch’ (see Patches), and you are currently on git master, do:

git checkout staging
git pull -r
git am 0001-my-patch.patch
gitk
git push origin staging
git checkout master

If your work is in a branch

If you are working on branches and your work is in my_branch_name, then do:

git checkout my_branch_name
git pull -r origin staging

This will rebase your branch on origin/staging. At this point git will let you know if there are any conflicts. If so, resolve them before continuing:

gitk
git push origin HEAD:staging

3.5 Git on Windows

TODO: Decide what to do with this... Pare it down? Move paragraphs next to analogous Unix instructions? -mp

3.5.1 Background to nomenclature

Git is a system for tracking the changes made to source files by a distributed set of editors. It is designed to work without a master repository, but we have chosen to have a master repository for LilyPond files. Editors hold a local copy of the master repository together with any changes they have made locally. Local changes are held in a local ‘branch’, of which there may be several, but these instructions assume you are using just one. The files visible in the local repository always correspond to those on the currently ‘checked out’ local branch.

Files are edited on a local branch, and in that state the changes are said to be ‘unstaged’. When editing is complete, the changes are moved to being ‘staged for commit’, and finally the changes are ‘committed’ to the local branch. Once committed, the changes (called a ‘commit’) are given a unique 40-digit hexadecimal reference number called the ‘Committish’ or ‘SHA1 ID’ which identifies the commit to Git. Such committed changes can be sent to the master repository by ‘pushing’ them (if you have write permission) or by sending them by email to someone who has, either as a complete file or as a ‘diff’ or ‘patch’ (which send just the differences from the master repository).

3.5.2 Installing git

Obtain Git from https://git-for-windows.github.io/.

Note that most users will not need to install SSH. That is not required until you have been granted direct push permissions to the master git repository.

Start Git by clicking on the desktop icon. This will bring up a command line bash shell. This may be unfamiliar to Windows users. If so, follow these instructions carefully. Commands are entered at a $ prompt and are terminated by keying a newline.

3.5.3 Initialising Git

Decide where you wish to place your local Git repository, creating the folders in Windows as necessary. Here we call the folder to contain the repository [path]/Git, but if you intend using Git for other projects a directory name like lilypond-git might be better. You will need to have space for around 100Mbytes.

Start the Git bash shell by clicking on the desk-top icon installed with Git and type

cd [path]/Git

to position the shell at your new Git repository.

Note: if [path] contains folders with names containing spaces use

cd "[path]/Git"

Then type

git init

to initialize your Git repository.

Then type (all on one line; the shell will wrap automatically)

git remote add -ft master origin git://git.sv.gnu.org/lilypond.git

to download the lilypond master files.

We now need to generate a local copy of the downloaded files in a new local branch. Your local branch needs to have a name. It is usual to call it ‘master’ and we shall do that here.

To do this, type

git checkout -b master origin/master

This creates a second branch called ‘master’. You will see two warnings (ignore these), and a message advising you that your local branch ‘master’ has been set up to track the remote branch. You now have two branches, a local branch called ‘master’, and a tracking branch called ‘origin/master’, which is a shortened form of ‘remotes/origin/master’.

Return to Windows Explorer and look in your Git repository. You should see lots of folders. For example, the LilyPond documentation can be found in [path]/Git/Documentation/.

The Git bash shell is terminated by typing exit or by clicking on the usual Windows close-window widget.

3.5.4 Git GUI

Almost all subsequent work will use the Git Graphical User Interface, which avoids having to type command line commands. To start Git GUI first start the Git bash shell by clicking on the desktop icon, and type

cd [path]/Git
git gui

The Git GUI will open in a new window. It contains four panels and 7 pull-down menus. At this stage do not use any of the commands under Branch, Commit, Merge or Remote. These will be explained later.

The top panel on the left contains the names of files which you are in the process of editing (Unstaged Changes), and the lower panel on the left contains the names of files you have finished editing and have staged ready for committing (Staged Changes). At present, these panels will be empty as you have not yet made any changes to any file. After a file has been edited and saved the top panel on the right will display the differences between the edited file selected in one of the panels on the left and the last version committed on the current branch.

The panel at bottom right is used to enter a descriptive message about the change before committing it.

The Git GUI is terminated by entering CNTL-Q while it is the active window or by clicking on the usual Windows close-window widget.

3.5.5 Personalising your local git repository

Open the Git GUI, click on

Edit -> Options

and enter your name and email address in the left-hand (Git Repository) panel. Leave everything else unchanged and save it.

Note that Windows users must leave the default setting for line endings unchanged. All files in a git repository must have lines terminated by just a LF, as this is required for Merge to work, but Windows files are terminated by CRLF by default. The git default setting causes the line endings of files in a Windows git repository to be flipped automatically between LF and CRLF as required. This enables files to be edited by any Windows editor without causing problems in the git repository.

3.5.6 Checking out a branch

At this stage you have two branches in your local repository, both identical. To see them click on

Branch -> Checkout

You should have one local branch called ‘master’ and one tracking branch called ‘origin/master’. The latter is your local copy of the ‘remotes/origin/master’ branch in the master LilyPond repository. The local ‘master’ branch is where you will make your local changes.

When a particular branch is selected, i.e., checked out, the files visible in your repository are changed to reflect the state of the files on that branch.

3.5.7 Updating files from ‘remote/origin/master’

Before starting the editing of a file, ensure your local repository contains the latest version of the files in the remote repository by first clicking

Remote -> Fetch from -> origin

in the Git GUI.

This will place the latest version of every file, including all the changes made by others, into the ‘origin/master’ branch of the tracking branches in your git repository. You can see these files by checking out this branch, but you must never edit any files while this branch is checked out. Check out your local ‘master’ branch again.

You then need to merge these fetched files into your local ‘master’ branch by clicking on

Merge -> Local Merge

and if necessary select the local ‘master’ branch.

Note that a merge cannot be completed if you have made any local changes which have not yet been committed.

This merge will update all the files in the ‘master’ branch to reflect the current state of the ‘origin/master’ branch. If any of the changes conflict with changes you have made yourself recently you will be notified of the conflict (see below).

3.5.8 Editing files

First ensure your ‘master’ branch is checked out, then simply edit the files in your local Git repository with your favourite editor and save them back there. If any file contains non-ASCII characters ensure you save it in UTF-8 format. Git will detect any changes whenever you restart Git GUI and the file names will then be listed in the Unstaged Changes panel. Or you can click the Rescan button to refresh the panel contents at any time. You may break off and resume editing any time.

The changes you have made may be displayed in diff form in the top right-hand panel of Git GUI by clicking on the file name shown in one of the left panels.

When your editing is complete, move the files from being Unstaged to Staged by clicking the document symbol to the left of each name. If you change your mind it can be moved back by clicking on the ticked box to the left of the name.

Finally the changes you have made may be committed to your ‘master’ branch by entering a brief message in the Commit Message box and clicking the Commit button.

If you wish to amend your changes after a commit has been made, the original version and the changes you made in that commit may be recovered by selecting

Commit -> Amend Last Commit

or by checking the Amend Last Commit radio button at bottom right. This will return the changes to the Staged state, so further editing made be carried out within that commit. This must only be done before the changes have been Pushed or sent to your mentor for Pushing - after that it is too late and corrections have to be made as a separate commit.

3.5.9 Sending changes to ‘remotes/origin/master’

If you do not have write access to ‘remotes/origin/master’ you will need to send your changes by email to someone who does.

First you need to create a diff or patch file containing your changes. To create this, the file must first be committed. Then terminate the Git GUI. In the git bash shell first cd to your Git repository with

cd [path]/Git

if necessary, then produce the patch with

git format-patch origin

This will create a patch file for all the locally committed files which differ from ‘origin/master’. The patch file can be found in [path]/Git and will have a name formed from the commit message.

3.5.10 Resolving merge conflicts

As soon as you have committed a changed file your local master branch has diverged from origin/master, and will remain diverged until your changes have been committed in remotes/origin/master and Fetched back into your origin/master branch. Similarly, if a new commit has been made to remotes/origin/master by someone else and Fetched, your local master branch is divergent. You can detect a divergent branch by clicking on

Repository -> Visualise all branch history

This opens up a very useful new window called ‘gitk’. Use this to browse all the commits made by yourself and others.

If the diagram at top left of the resulting window does not show your master tag on the same node as the remotes/origin/master tag your branch has diverged from origin/master. This is quite normal if files you have modified yourself have not yet been Pushed to remotes/origin/master and Fetched, or if files modified and committed by others have been Fetched since you last Merged origin/master into your local master branch.

If a file being merged from origin/master differs from one you have modified in a way that cannot be resolved automatically by git, Merge will report a Conflict which you must resolve by editing the file to create the version you wish to keep.

This could happen if the person updating remotes/origin/master for you has added some changes of his own before committing your changes to remotes/origin/master, or if someone else has changed the same file since you last fetched the file from remotes/origin/master.

Open the file in your editor and look for sections which are delimited with ...

[to be completed when I next have a merge conflict to be sure I give the right instructions -td]

3.5.11 Other actions

The instructions above describe the simplest way of using git on Windows. Other git facilities which may usefully supplement these include

• Using multiple local branches (Create, Rename, Delete)
• Resetting branches
• Cherry-picking commits
• Pushing commits to remote/origin/master
• Using gitk to review history

Once familiarity with using git on Windows has been gained the standard git manuals can be used to learn about these.

3.6 Repository directory structure

Prebuilt Documentation and packages are available from:

    http://www.lilypond.org

LilyPond development is hosted at:

    http://savannah.gnu.org/projects/lilypond

Here is a simple explanation of the directory layout for
LilyPond's source files.


.                        Toplevel READMEs, ChangeLog,
|                          build bootstrapping, patches
|                          for third party programs
|
|-- Documentation/       Top sources for most of the manuals
|   |
|   |
|   |   INDIVIDUAL CHAPTERS FOR EACH MANUAL:
|   |     Note: "Snippets" and "Internals Reference" are
|   |     auto-generated during the Documentation Build process.
|   |
|   |
|   |-- contributor/     Contributor's Guide
|   |-- essay/           Essay on automated music engraving
|   |-- extending/       Extending the functionality of LilyPond
|   |-- learning/        Learning Manual
|   |-- notation/        Notation Reference
|   |-- usage/           Runnning the programs that come with LilyPond
|   |-- web/             The website
|   |
|   |
|   |   TRANSLATED MANUALS:
|   |     Each language's directory can contain...
|   |       1) translated versions of:
|   |          * top sources for manuals
|   |          * individual chapters for each manual
|   |       2) a texidocs/ directory for snippet translations
|   |
|   |-- ca/              Catalan
|   |-- cs/              Czech
|   |-- de/              German
|   |-- es/              Spanish
|   |-- fr/              French
|   |-- hu/              Hungarian
|   |-- it/              Italian
|   |-- ja/              Japanese
|   |-- nl/              Dutch
|   |-- zh/              Chinese
|   |
|   |
|   |   MISCELLANEOUS DOC STUFF:
|   |
|   |-- css/             CSS files for HTML docs
|   |-- included/        .ly files used in the manuals
|   |-- logo/            Web logo and "note" icon
|   |-- ly-examples/     .ly files for the "Examples" webpage
|   |-- misc/            Old announcements, ChangeLogs and NEWS
|   |-- pictures/        Images used (eps/jpg/png/svg)
|   |   `-- pdf/         (pdf)
|   |-- po/              Translated build/maintenance scripts
|   |-- snippets/        Auto-generated from the LSR and from ./new/
|   |   `-- new/         Snippets too new for the LSR
|   `-- topdocs/         AUTHORS, INSTALL, README
|
|
|   C++ SOURCES:
|
|-- flower/              A simple C++ library
|-- lily/                C++ sources for the LilyPond binary
|
|
|   LIBRARIES:
|
|-- ly/                  .ly \include files
|-- mf/                  MetaFont sources for Emmentaler fonts
|-- ps/                  PostScript library files
|-- scm/                 Scheme sources for LilyPond and subroutine files
|-- tex/                 TeX and texinfo library files
|
|
|   SCRIPTS:
|
|-- config/              Autoconf helpers for configure script
|-- python/              Python modules, MIDI module
|   `-- auxiliar/        Python modules for build/maintenance
|-- scripts/             End-user scripts (--> lilypond/usr/bin/)
|   |-- auxiliar/        Maintenance and non-essential build scripts
|   `-- build/           Essential build scripts
|
|
|   BUILD PROCESS:
|   (also see SCRIPTS section above)
|
|-- make/                Specific make subroutine files
|-- stepmake/            Generic make subroutine files
|
|
|   REGRESSION TESTS:
|
|-- input/
|   `-- regression/      .ly regression tests
|       |-- abc2ly/      .abc regression tests
|       |-- lilypond-book/  lilypond-book regression tests
|       |-- midi/        midi2ly regression tests
|       `-- musicxml/    .xml and .itexi regression tests
|
|
|   MISCELLANEOUS:
|
|-- elisp/               Emacs LilyPond mode and syntax coloring
|-- vim/                 Vi(M) LilyPond mode and syntax coloring
`-- po/                  Translations for binaries and end-user scripts

3.7 Other Git documentation

• Official git man pages: http://www.kernel.org/pub/software/scm/git/docs/
• More in-depth tutorials: http://git-scm.com/documentation
• Book about git: Pro Git
• Github help: http://help.github.com/ (very highly recommended by Graham)

4. Compiling

This chapter describes the process of compiling the LilyPond program from source files.

4.1 Overview of compiling

Compiling LilyPond from source is an involved process, and is only recommended for developers and packagers. Typical program users are instead encouraged to obtain the program from a package manager (on Unix) or by downloading a precompiled binary configured for a specific operating system. Pre-compiled binaries are available on the Download page.

Compiling LilyPond from source is necessary if you want to build, install, or test your own version of the program.

A successful compile can also be used to generate and install the documentation, incorporating any changes you may have made. However, a successful compile is not a requirement for generating the documentation. The documentation can be built using a Git repository in conjunction with a locally installed copy of the program. For more information, see Building documentation without compiling.

Attempts to compile LilyPond natively on Windows have been unsuccessful, though a workaround is available (see LilyDev).

4.2 Requirements

4.2.1 Requirements for running LilyPond

This section contains the list of separate software packages that are required to run LilyPond.

• DejaVu fonts These are normally installed by default.
• FontConfig Use version 2.4.0 or newer.
• Freetype Use version 2.1.10 or newer.
• Ghostscript Use version 8.60 or newer.
• Guile Use version 1.8.8. Version 2.x of Guile is not currently supported.
• Pango User version 1.12 or newer.
• Python Use version 2.4 or newer.
• International fonts. For example:

  Fedora:

  fonts-arabic
  fonts-hebrew
  fonts-ja
  fonts-xorg-truetype
  taipeifonts
  ttfonts-ja
  ttfonts-zh_CN
  

  Debian based distributions:

  emacs-intl-fonts
  fonts-ipafont-gothic
  fonts-ipafont-mincho
  xfonts-bolkhov-75dpi
  xfonts-cronyx-75dpi
  xfonts-cronyx-100dpi
  xfonts-intl-.*
  

  These are normally installed by default and are required only to create music with international text or lyrics.

4.2.2 Requirements for compiling LilyPond

This section contains instructions on how to quickly and easily get all the software packages required to build LilyPond.

Most of the more popular Linux distributions only require a few simple commands to download all the software needed. For others, there is an explicit list of all the individual packages (as well as where to get them from) for those that are not already included in your distributions’ own repositories.

Fedora

The following instructions were tested on ‘Fedora’ versions 22 & 23 and will download all the software required to both compile LilyPond and build the documentation.

• Download and install all the LilyPond build-dependencies (approximately 700MB);

  sudo dnf builddep lilypond --nogpgcheck
  

• Download and install additional ‘build’ tools required for compiling;

  sudo dnf install autoconf gcc-c++
  

• Download texi2html 1.82 directly from: http://download.savannah.gnu.org/releases/texi2html/texi2html-1.82.tar.gz;

  texi2html is only required if you intend to compile LilyPond’s own documentation (e.g. to help with any document writing). The version available in the Fedora repositories is too new and will not work. Extract the files into an appropriate location and then run the commands;

  ./configure
  make
  sudo make install
  

  This should install texi2html 1.82 into /usr/local/bin, which will normally take priority over /usr/bin where the later, pre-installed versions gets put. Now verify that your operating system is able to see the correct version of texi2html.

  texi2html --version
  

• Although not ‘required’ to compile LilyPond, if you intend to contribute to LilyPond (codebase or help improve the documentation) then it is recommended that you also need to install git.

  sudo dnf install git
  

  Also see Starting with Git.

• To use the lily-git.tcl GUI;

  sudo dnf install tk
  

  See lily-git.

Linux Mint

The following instructions were tested on ‘Linux Mint 17.1’ and ‘LMDE - Betsy’ and will download all the software required to both compile LilyPond and build the documentation..

• Enable the sources repository;
  1. Using the Software Sources GUI (located under Administration).
  2. Select Official Repositories.
  3. Check the Enable source code repositories box under the Source Code section.
  4. Click the Update the cache button and when it has completed, close the Software Sources GUI.
• Download and install all the LilyPond build-dependencies (approximately 200MB);

  sudo apt-get build-dep lilypond
  

• Download and install additional ‘build’ tools required for compiling;

  sudo apt-get install autoconf fonts-texgyre texlive-lang-cyrillic
  

• Although not ‘required’ to compile LilyPond, if you intend to contribute to LilyPond (codebase or help improve the documentation) then it is recommended that you also need to install git.

  sudo apt-get install git
  

  Also see Starting with Git.

• To use the lily-git.tcl GUI;

  sudo apt-get install tk
  

  Also see lily-git.

OpenSUSE

The following instructions were tested on ‘OpenSUSE 13.2’ and will download all the software required to both compile LilyPond and build the documentation.

• Add the sources repository;

  sudo zypper addrepo -f \ "http://download.opensuse.org/source/distribution/13.2/repo/oss/" sources

• Download and install all the LilyPond build-dependencies (approximately 680MB);

  sudo zypper source-install lilypond
  

• Download and install additional ‘build’ tools required for compiling;

  sudo zypper install make
  

• Although not ‘required’ to compile LilyPond, if you intend to contribute to LilyPond (codebase or help improve the documentation) then it is recommended that you also need to install git.

  sudo zypper install git
  

  Also see Starting with Git.

• To use the lily-git.tcl GUI;

  sudo zypper install tk
  

  Also see lily-git.

Ubuntu

The following commands were tested on Ubuntu versions 14.04 LTS, 14.10 and 15.04 and will download all the software required to both compile LilyPond and build the documentation.

• Download and install all the LilyPond build-dependencies (approximately 200MB);

  sudo apt-get build-dep lilypond
  

• Download and install additional ‘build’ tools required for compiling;

  sudo apt-get install autoconf fonts-texgyre texlive-lang-cyrillic
  

• Although not ‘required’ to compile LilyPond, if you intend to contribute to LilyPond (codebase or help improve the documentation) then it is recommended that you also need to install git.

  sudo apt-get install git
  

  Also see Starting with Git.

• To use the lily-git.tcl GUI;

  sudo apt-get install tk
  

  Also see lily-git.

Other

The following individual software packages are required just to compile LilyPond.

• GNU Autoconf
• GNU Bison

  Use version 2.0 or newer.

• GNU Compiler Collection

  Use version 3.4 or newer (4.x recommended).

• Flex
• FontForge

  Use version 20060125 or newer (we recommend using at least 20100501); it must also be compiled with the ‘--enable-double’ switch, else this can lead to inaccurate intersection calculations which end up with poorly-rendered glyphs in the output.

• GNU gettext

  Use version 0.17 or newer.

• GNU Make

  Use version 3.78 or newer.

• MetaFont

  The mf-nowin, mf, mfw or mfont binaries are usually packaged along with TeX.

• MetaPost

  The mpost binary is also usually packaged with TeX.

• Perl
• Texinfo

  Use version 4.11 or newer.

• Type 1 utilities

  Use version 1.33 or newer.

• Cyrillic fonts

  Often packaged in repositories as texlive-lang-cyrillic.

• TeX Gyre ‘OTF’ font packages. As of LilyPond version 2.19.26, the previous default serif, san serif and monospace fonts now use Tex Gyre’s Schola, Heros and Cursor fonts respectively. Also See Fonts.

  Some distributions do not always provide ‘OTF’ font files in the Tex Gyre packages from their repositories. Use the command fc-list | grep texgyre to list the fonts available to your system and check that the appropriate *.otf files are reported. If they are not then download and manually extract the ‘OTF’ files to either your local ~/.fonts/ directory or use the configure command and the --with-texgyre-dir=/path_to_otf_files/ option.

  The following font families are required:

  Schola, Heros and Cursor.

4.2.3 Requirements for building documentation

The entire set of documentation for the most current build of LilyPond is available online at http://lilypond.org/doc/v2.19/Documentation/web/development, but you can also build them locally from the source code. This process requires some additional tools and packages.

• Everything listed in Requirements for compiling LilyPond
• ImageMagick
• Netpbm
• gzip
• rsync
• Texi2HTML

  Use version 1.82. Later versions will not work.

  Download texi2html 1.82 directly from: http://download.savannah.gnu.org/releases/texi2html/texi2html-1.82.tar.gz;

  Extract the files into an appropriate location and then run the commands;

  ./configure
  make
  sudo make install
  

  Now verify that your operating system is able to see the correct version of texi2html.

  texi2html --version
  

• Fonts required to build the documentation in addition to those required to run LilyPond:

  gsfonts
  fonts-linuxlibertine
  fonts-liberation
  fonts-dejavu
  fonts-freefont-otf
  ttf-bitstream-vera
  texlive-fonts-recommended
  ttf-xfree86-nonfree
  

4.3 Getting the source code

Downloading the Git repository

In general, developers compile LilyPond from within a local Git repository. Setting up a local Git repository is explained in Starting with Git.

Downloading a source tarball

Packagers are encouraged to use source tarballs for compiling.

The tarball for the latest stable release is available on the Source page.

The latest source code snapshot is also available as a tarball from the GNU Savannah Git server.

All tagged releases (including legacy stable versions and the most recent development release) are available here:

http://download.linuxaudio.org/lilypond/source/

Download the tarball to your ‘~/src/’ directory, or some other appropriate place.

Unpack the tarball with this command:

tar -xzf lilypond-x.y.z.tar.gz

This creates a subdirectory within the current directory called ‘lilypond-x.y.z/’. Once unpacked, the source files occupy about 40 MB of disk space.

Windows users wanting to look at the source code may have to download and install the free-software 7zip archiver to extract the tarball.

4.4 Configuring make

4.4.1 Running ./autogen.sh

After you unpack the tarball (or download the Git repository), the contents of your top source directory should be similar to the current source tree listed at http://git.sv.gnu.org/gitweb/?p=lilypond.git;a=tree.

Next, you need to create the generated files; enter the following command from your top source directory:

./autogen.sh --noconfigure

This will generate a number of files and directories to aid configuration, such as ‘configure’, ‘README.txt’, etc.

Next, create the build directory with:

mkdir build/
cd build/

We heavily recommend building lilypond inside a separate directory with this method.

4.4.2 Running ../configure

Configuration options

The ../configure command (generated by ./autogen.sh) provides many options for configuring make. To see them all, run:

../configure --help

Checking build dependencies

When ../configure is run without any arguments, it will check to make sure your system has everything required for compilation:

../configure

If any build dependency is missing, ../configure will return with:

ERROR: Please install required programs:  foo

The following message is issued if you are missing programs that are only needed for building the documentation:

WARNING: Please consider installing optional programs:  bar

If you intend to build the documentation locally, you will need to install or update these programs accordingly.

Configuring target directories

If you intend to use your local build to install a local copy of the program, you will probably want to configure the installation directory. Here are the relevant lines taken from the output of ../configure --help:

By default, ‘make install’ will install all the files in ‘/usr/local/bin’, ‘/usr/local/lib’ etc. You can specify an installation prefix other than ‘/usr/local’ using ‘‘--prefix’’, for instance ‘‘--prefix=$HOME’’.

A typical installation prefix is ‘$HOME/usr’:

../configure --prefix=$HOME/usr

Note that if you plan to install a local build on a system where you do not have root privileges, you will need to do something like this anyway—make install will only succeed if the installation prefix points to a directory where you have write permission (such as your home directory). The installation directory will be automatically created if necessary.

The location of the lilypond command installed by this process will be ‘prefix/bin/lilypond’; you may want to add ‘prefix/bin/’ to your $PATH if it is not already included.

It is also possible to specify separate installation directories for different types of program files. See the full output of ../configure --help for more information.

If you encounter any problems, please see Problems.

4.5 Compiling LilyPond

4.5.1 Using make

LilyPond is compiled with the make command. Assuming make is configured properly, you can simply run:

make

‘make’ is short for ‘make all’. To view a list of make targets, run:

make help

TODO: Describe what make actually does.

See also

Generating documentation provides more info on the make targets used to build the LilyPond documentation.

4.5.2 Saving time with the ‘-j’ option

If your system has multiple CPUs, you can speed up compilation by adding ‘-jX’ to the make command, where ‘X’ is one more than the number of cores you have. For example, a typical Core2Duo machine would use:

make -j3

If you get errors using the ‘-j’ option, and ‘make’ succeeds without it, try lowering the X value.

Because multiple jobs run in parallel when ‘-j’ is used, it can be difficult to determine the source of an error when one occurs. In that case, running ‘make’ without the ‘-j’ is advised.

4.5.3 Compiling for multiple platforms

If you want to build multiple versions of LilyPond with different configuration settings, you can use the ‘--enable-config=conf’ option of configure. You should use make conf=conf to generate the output in ‘out-conf’. For example, suppose you want to build with and without profiling, then use the following for the normal build

./configure --prefix=$HOME/usr/ --enable-checking
make

and for the profiling version, specify a different configuration

./configure --prefix=$HOME/usr/ --enable-profiling \
  --enable-config=prof --disable-checking
make conf=prof

If you wish to install a copy of the build with profiling, don’t forget to use conf=CONF when issuing make install:

make conf=prof install

See also

Installing LilyPond from a local build

4.5.4 Useful make variables

If a less verbose build output if desired, the variable QUIET_BUILD may be set to 1 on make command line, or in ‘local.make’ at top of the build tree.

4.6 Post-compilation options

4.6.1 Installing LilyPond from a local build

If you configured make to install your local build in a directory where you normally have write permission (such as your home directory), and you have compiled LilyPond by running make, you can install the program in your target directory by running:

make install

If instead, your installation directory is not one that you can normally write to (such as the default ‘/usr/local/’, which typically is only writeable by the superuser), you will need to temporarily become the superuser when running make install:

sudo make install

or…

su -c 'make install'

If you don’t have superuser privileges, then you need to configure the installation directory to one that you can write to, and then re-install. See Configuring target directories.

4.6.2 Generating documentation

Documentation editor’s edit/compile cycle

• Initial documentation build:

  make [-jX]
  make [-jX CPU_COUNT=X] doc          ## can take an hour or more
  make [-jX CPU_COUNT=X] doc-stage-1  ## to build only PDF documentation
  

• Edit/compile cycle:

  ## edit source files, then…
  
  make [-jX]                  ## needed if editing outside
                              ##   Documentation/, but useful anyway
                              ##   for finding Texinfo errors.
  make [-jX CPU_COUNT=X] doc  ## usually faster than initial build.
  

• Reset:

  It is generally possible to remove the compiled documentation from your system with ‘make doc-clean’, but this method is not 100% guaranteed. Instead, if you want to be sure you have a clean system, we recommend that you delete your ‘build/’ directory, and begin compiling from scratch. Since the documentation compile takes much longer than the non-documentation compile, this does not increase the overall time by a great deal.

Building documentation

After a successful compile (using make), the documentation can be built by issuing:

make doc

or, to build only the PDF documentation and not the HTML,

make doc-stage-1

After this initial build, make doc only makes changes to the documentation where needed, so it may only take a minute or two to test changes if the documentation is already built.

If make doc succeeds, the HTML documentation tree is available in ‘out-www/offline-root/’, and can be browsed locally. Various portions of the documentation can be found by looking in ‘out/’ and ‘out-www’ subdirectories in other places in the source tree, but these are only portions of the docs. Please do not complain about anything which is broken in those places; the only complete set of documentation is in ‘out-www/offline-root/’ from the top of the source tree.

make doc sends the output from most of the compilation to logfiles. If the build fails for any reason, it should prompt you with the name of a logfile which will provide information to help you work out why the build failed. These logfiles are not deleted with make doc-clean. To remove all the logfiles generated by the compilation process, use:

make log-clean

make doc compiles the documents for all languages. To save some compile time, the English language documents can be compiled on their own with:

make LANGS='' doc

Similarly, it is possible to compile a subset of the translated documentation by specifying their language codes on the command line. For example, the French and German translations are compiled with:

make LANGS='de fr' doc

Note that this will also compile the English version.

Compilation of documentation in Info format with images can be done separately by issuing:

make info

An issue when switching branches between master and translation is the appearance/disappearance of translated versions of some manuals. If you see such a warning from make:

No rule to make target `X', needed by `Y'

Your best bet is to delete the file Y.dep and to try again.

Building a single document

It’s possible to build a single document. For example, to rebuild only ‘contributor.pdf’, do the following:

cd build/
cd Documentation/
touch ../../Documentation/contributor.texi
make out=www out-www/contributor.pdf

If you are only working on a single document, test-building it in this way can give substantial time savings - recreating ‘contributor.pdf’, for example, takes a matter of seconds.

Saving time with CPU_COUNT

The most time consuming task for building the documentation is running LilyPond to build images of music, and there cannot be several simultaneously running lilypond-book instances, so the ‘-j’ make option does not significantly speed up the build process. To help speed it up, the makefile variable ‘CPU_COUNT’ may be set in ‘local.make’ or on the command line to the number of .ly files that LilyPond should process simultaneously, e.g. on a bi-processor or dual core machine:

make -j3 CPU_COUNT=3 doc

The recommended value of ‘CPU_COUNT’ is one plus the number of cores or processors, but it is advisable to set it to a smaller value unless your system has enough RAM to run that many simultaneous LilyPond instances. Also, values for the ‘-j’ option that pose problems with ‘make’ are less likely to pose problems with ‘make doc’ (this applies to both ‘-j’ and ‘CPU_COUNT’). For example, with a quad-core processor, it is possible for ‘make -j5 CPU_COUNT=5 doc’ to work consistently even if ‘make -j5’ rarely succeeds.

AJAX search

To build the documentation with interactive searching, use:

make doc AJAX_SEARCH=1

This requires PHP, and you must view the docs via a http connection (you cannot view them on your local filesystem).

Installing documentation

The HTML, PDF and if available Info files can be installed into the standard documentation path by issuing

make install-doc

This also installs Info documentation with images if the installation prefix is properly set; otherwise, instructions to complete proper installation of Info documentation are printed on standard output.

To install the Info documentation separately, run:

make install-info

Note that to get the images in Info documentation, install-doc target creates symbolic links to HTML and PDF installed documentation tree in ‘prefix/share/info’, in order to save disk space, whereas install-info copies images in ‘prefix/share/info’ subdirectories.

It is possible to build a documentation tree in ‘out-www/online-root/’, with special processing, so it can be used on a website with content negotiation for automatic language selection; this can be achieved by issuing

make WEB_TARGETS=online doc

and both ‘offline’ and ‘online’ targets can be generated by issuing

make WEB_TARGETS="offline online" doc

Several targets are available to clean the documentation build and help with maintaining documentation; an overview of these targets is available with

make help

from every directory in the build tree. Most targets for documentation maintenance are available from ‘Documentation/’; for more information, see Documentation work.

The makefile variable QUIET_BUILD may be set to 1 for a less verbose build output, just like for building the programs.

Building documentation without compiling

The documentation can be built locally without compiling LilyPond binary, if LilyPond is already installed on your system.

From a fresh Git checkout, do

./autogen.sh   # ignore any warning messages
cp GNUmakefile.in GNUmakefile
make -C scripts && make -C python
nice make LILYPOND_EXTERNAL_BINARY=/path/to/bin/lilypond doc

Please note that this may break sometimes – for example, if a new feature is added with a test file in input/regression, even the latest development release of LilyPond will fail to build the docs.

You may build the manual without building all the ‘input/*’ stuff (i.e. mostly regression tests): change directory, for example to ‘Documentation/’, issue make doc, which will build documentation in a subdirectory ‘out-www’ from the source files in current directory. In this case, if you also want to browse the documentation in its post-processed form, change back to top directory and issue

make out=www WWW-post

Known issues and warnings

You may also need to create a script for pngtopnm and pnmtopng. On GNU/Linux, I use this:

export LD_LIBRARY_PATH=/usr/lib
exec /usr/bin/pngtopnm "$@"

On MacOS X with fink, I use this:

export DYLD_LIBRARY_PATH=/sw/lib
exec /sw/bin/pngtopnm "$@"

On MacOS X with macports, you should use this:

export DYLD_FALLBACK_LIBRARY_PATH=/opt/local/lib
exec /opt/local/bin/pngtopnm "$@"

4.6.3 Testing LilyPond binary

LilyPond comes with an extensive suite that exercises the entire program. This suite can be used to test that the binary has been built correctly.

The test suite can be executed with:

make test

If the test suite completes successfully, the LilyPond binary has been verified.

More information on the regression test suite is found at Regression tests.

4.7 Problems

For help and questions use lilypond-user@gnu.org. Send bug reports to bug-lilypond@gnu.org.

Bugs that are not fault of LilyPond are documented here.

Compiling on MacOS X

Here are special instructions for compiling under MacOS X. These instructions assume that dependencies are installed using MacPorts. The instructions have been tested using OS X 10.5 (Leopard).

First, install the relevant dependencies using MacPorts.

Next, add the following to your relevant shell initialization files. This is ~/.profile by default. You should create this file if it does not exist.

export PATH=/opt/local/bin:/opt/local/sbin:$PATH
export DYLD_FALLBACK_LIBRARY_PATH=/opt/local/lib:$DYLD_FALLBACK_LIBRARY_PATH

Now you must edit the generated ‘config.make’ file. Change

FLEXLEXER_FILE = /usr/include/FlexLexer.h

to:

FLEXLEXER_FILE = /opt/local/include/FlexLexer.h

At this point, you should verify that you have the appropriate fonts installed with your ghostscript installation. Check ls /opt/local/share/ghostscript/fonts for: ’c0590*’ files (.pfb, .pfb and .afm). If you don’t have them, run the following commands to grab them from the ghostscript SVN server and install them in the appropriate location:

svn export http://svn.ghostscript.com/ghostscript/tags/urw-fonts-1.0.7pre44/
sudo mv urw-fonts-1.0.7pre44/* /opt/local/share/ghostscript/fonts/
rm -rf urw-fonts-1.07pre44

Now run the ./configure script. To avoid complications with automatic font detection, add

--with-fonts-dir=/opt/local/share/ghostscript/fonts

Solaris

Solaris7, ./configure

‘./configure’ needs a POSIX compliant shell. On Solaris7, ‘/bin/sh’ is not yet POSIX compliant, but ‘/bin/ksh’ or bash is. Run configure like

CONFIG_SHELL=/bin/ksh ksh -c ./configure

or

CONFIG_SHELL=/bin/bash bash -c ./configure

FreeBSD

To use system fonts, dejaview must be installed. With the default port, the fonts are installed in ‘usr/X11R6/lib/X11/fonts/dejavu’.

Open the file ‘$LILYPONDBASE/usr/etc/fonts/local.conf’ and add the following line just after the <fontconfig> line. (Adjust as necessary for your hierarchy.)

<dir>/usr/X11R6/lib/X11/fonts</dir>

International fonts

On Mac OS X, all fonts are installed by default. However, finding all system fonts requires a bit of configuration; see this post on the lilypond-user mailing list.

On Linux, international fonts are installed by different means on every distribution. We cannot list the exact commands or packages that are necessary, as each distribution is different, and the exact package names within each distribution changes. Here are some hints, though:

Red Hat Fedora

    taipeifonts fonts-xorg-truetype ttfonts-ja fonts-arabic \
         ttfonts-zh_CN fonts-ja fonts-hebrew

Debian GNU/Linux

   apt-get install emacs-intl-fonts xfonts-intl-.* \
        fonts-ipafont-gothic  fonts-ipafont-mincho \
        xfonts-bolkhov-75dpi xfonts-cronyx-100dpi xfonts-cronyx-75dpi

Using lilypond python libraries

If you want to use lilypond’s python libraries (either running certain build scripts manually, or using them in other programs), set PYTHONPATH to ‘python/out’ in your build directory, or ‘…/usr/lib/lilypond/current/python’ in the installation directory structure.

4.8 Concurrent stable and development versions

It can be useful to have both the stable and the development versions of LilyPond available at once. One way to do this on GNU/Linux is to install the stable version using the precompiled binary, and run the development version from the source tree. After running make all from the top directory of the LilyPond source files, there will be a binary called lilypond in the out directory:

<path to>/lilypond/out/bin/lilypond

This binary can be run without actually doing the make install command. The advantage to this is that you can have all of the latest changes available after pulling from git and running make all, without having to uninstall the old version and reinstall the new.

So, to use the stable version, install it as usual and use the normal commands:

lilypond foobar.ly

To use the development version, create a link to the binary in the source tree by saving the following line in a file somewhere in your $PATH:

exec <path to>/lilypond/out/bin/lilypond "$@"

Save it as Lilypond (with a capital L to distinguish it from the stable lilypond), and make it executable:

chmod +x Lilypond

Then you can invoke the development version this way:

Lilypond foobar.ly

TODO: ADD

- other compilation tricks for developers

4.9 Build system

We currently use make and stepmake, which is complicated and only used by us. Hopefully this will change in the future.

Version-specific texinfo macros

• made with scripts/build/create-version-itexi.py and
  scripts/build/create-weblinks-itexi.py
• used extensively in the WEBSITE_ONLY_BUILD version of the website (made with ‘website.make’, used on lilypond.org)
• not (?) used in the main docs?
• the numbers in VERSION file: MINOR_VERSION should be 1 more than the last release, VERSION_DEVEL should be the last online release. Yes, VERSION_DEVEL is less than VERSION.

5. Documentation work

There are currently 11 manuals for LilyPond, not including the translations. Each book is available in HTML, PDF, and info. The documentation is written in a language called texinfo – this allows us to generate different output formats from a single set of source files.

To organize multiple authors working on the documentation, we use a Version Control System (VCS) called Git, previously discussed in Starting with Git.

5.1 Introduction to documentation work

Our documentation tries to adhere to our Documentation policy. This policy contains a few items which may seem odd. One policy in particular is often questioned by potential contributors: we do not repeat material in the Notation Reference, and instead provide links to the “definitive” presentation of that information. Some people point out, with good reason, that this makes the documentation harder to read. If we repeated certain information in relevant places, readers would be less likely to miss that information.

That reasoning is sound, but we have two counter-arguments. First, the Notation Reference – one of five manuals for users to read – is already over 500 pages long. If we repeated material, we could easily exceed 1000 pages! Second, and much more importantly, LilyPond is an evolving project. New features are added, bugs are fixed, and bugs are discovered and documented. If features are discussed in multiple places, the documentation team must find every instance. Since the manual is so large, it is impossible for one person to have the location of every piece of information memorized, so any attempt to update the documentation will invariably omit a few places. This second concern is not at all theoretical; the documentation used to be plagued with inconsistent information.

If the documentation were targeted for a specific version – say, LilyPond 2.10.5 – and we had unlimited resources to spend on documentation, then we could avoid this second problem. But since LilyPond evolves (and that is a very good thing!), and since we have quite limited resources, this policy remains in place.

A few other policies (such as not permitting the use of tweaks in the main portion of NR 1+2) may also seem counter-intuitive, but they also stem from attempting to find the most effective use of limited documentation help.

Before undertaking any large documentation work, contributors are encouraged to contact the Documentation Meister.

5.2 \version in documentation files

Every documentation file which includes LilyPond code must begin with a \version statement, since the build procedure explicitly tests for its presence and will not continue otherwise. The \version statement should reference a version of LilyPond consistent with the syntax of the contained code.

Since the \version statement is not valid Texinfo input it must be commented out like this:

@c \version "2.19.1"

So, if you are adding LilyPond code which is not consistent with the current version header, you should

1. run convert-ly on the file using the latest version of LilyPond (which should, if everybody has done proper maintenance, not change anything);
2. add the new code;
3. modify the version number to match the new code.

5.3 Documentation suggestions

Small additions

For additions to the documentation,

1. Tell us where the addition should be placed. Please include both the section number and title (i.e. "LM 2.13 Printing lyrics").
2. Please write exact changes to the text.
3. A formal patch to the source code is not required; we can take care of the technical details.
4. Send the suggestions to the bug-lilypond mailing list as discussed in Contact.
5. Here is an example of a perfect documentation report:

   To: bug-lilypond@gnu.org
   From: helpful-user@example.net
   Subject: doc addition
   
   In LM 2.13 (printing lyrics), above the last line ("More options,
   like..."), please add:
   
   ----
   To add lyrics to a divided part, use blah blah blah.  For example,
   
   \score {
     \notes {blah <<blah>> }
     \lyrics {blah <<blah>> }
     blah blah blah
   }
   ----
   
   In addition, the second sentence of the first paragraph is
   confusing.  Please delete that sentence (it begins "Users
   often...") and replace it with this:
   ----
   To align lyrics with something, do this thing.
   ----
   
   Have a nice day,
   Helpful User
   

Larger contributions

To replace large sections of the documentation, the guidelines are stricter. We cannot remove parts of the current documentation unless we are certain that the new version is an improvement.

1. Ask on the lilypond-devel mailing list if such a rewrite is necessary; somebody else might already be working on this issue!
2. Split your work into small sections; this makes it much easier to compare the new and old documentation.
3. Please prepare a formal git patch.

Contributions that contain examples using overrides

Examples that use overrides, tweaks, customer Scheme functions etc. are (with very few exceptions) not included in the main text of the manuals; as there would be far too many, equally useful, candidates.

The correct way is to submit your example, with appropriate explanatory text and tags, to the LilyPond Snippet Repository (LSR). Snippets that have the “docs” tag can then be easily added as a selected snippet in the documentation. It will also appear automatically in the Snippets lists. See Introduction to LSR.

Snippets that don’t have the “docs” tag will still be searchable and viewable within the LSR, but will be not be included in the Snippets list or be able to be included as part of the main documentation.

Generally, any new snippets that have the “docs” tag are more carefully checked for syntax and formatting.

Announcing your snippet

Once you have followed these guidelines, please send a message to lilypond-devel with your documentation submissions. Unfortunately there is a strict ‘no top-posting’ check on the mailing list; to avoid this, add:

> I'm not top posting

(you must include the > ) to the top of your documentation addition.

We may edit your suggestion for spelling, grammar, or style, and we may not place the material exactly where you suggested, but if you give us some material to work with, we can improve the manual much faster.

Thanks for your interest!

5.4 Texinfo introduction and usage policy

5.4.1 Texinfo introduction

The language is called Texinfo; you can see its manual here:

http://www.gnu.org/software/texinfo/manual/texinfo/

However, you don’t need to read those docs. The most important thing to notice is that text is text. If you see a mistake in the text, you can fix it. If you want to change the order of something, you can cut-and-paste that stuff into a new location.

5.4.2 Documentation files

All manuals live in ‘Documentation/’.

In particular, there are four user manuals, their respective master source files are ‘learning.tely’ (LM, Learning Manual), ‘notation.tely’ (NR, Notation Reference), ‘music-glossary.tely’ (MG, Music Glossary), and ‘lilypond-program’ (AU). Each chapter is written in a separate file, ending in ‘.itely’ for files containing lilypond code, and ‘.itexi’ for files without lilypond code, located in a subdirectory associated to the manual (‘learning/’ for ‘learning.tely’, and so on); list the subdirectory of each manual to determine the filename of the specific chapter you wish to modify.

Developer manuals live in ‘Documentation/’ too. Currently there is only one: the Contributor’s Guide ‘contrib-guide.texi’ you are reading.

Snippet files are part of documentation, and the Snippet List (SL) lives in ‘Documentation/’ just like the manuals. For information about how to modify the snippet files and SL, see LSR work.

5.4.3 Sectioning commands

The Notation Reference uses section headings at four, occasionally five, levels.

• Level 1: @chapter
• Level 2: @section
• Level 3: @subsection
• Level 4: @unnumberedsubsubsec
• Level 5: @subsubsubheading

The first three levels are numbered in HTML, the last two are not. Numbered sections correspond to a single HTML page in the split HTML documents.

The first four levels always have accompanying nodes so they can be referenced and are also included in the ToC in HTML.

Most of the manual is written at level 4 under headings created with

@node Foo
@unnumberedsubsubsec Foo

Level 3 subsections are created with

@node Foo
@subsection Foo

Level 4 headings and menus must be preceded by level 3 headings and menus, and so on for level 3 and level 2. If this is not what is wanted, please use:

@subsubsubheading Foo

Please leave two blank lines above a @node; this makes it easier to find sections in texinfo.

Do not use any @ commands for a @node. They may be used for any @sub... sections or headings however.

not:
@node @code{Foo} Bar
@subsection @code{Foo} Bar

but instead:
@node Foo Bar
@subsection @code{Foo} Bar

No punctuation may be used in the node names. If the heading text uses punctuation (in particular, colons and commas) simply leave this out of the node name and menu.

@menu
* Foo Bar::
@end menu

@node Foo Bar
@subsection Foo: Bar

Backslashes must not be used in node names or section headings. If the heading text should include a backslash simply leave this out of the node name and menu and replace it with @bs{} in the heading text.

@menu
* The set command
@end menu

@node The set command
@subsection The @code{@bs{}set} command

References to such a node may use the third argument of the @ref command to display the texually correct heading.

@ref{The set command,,The @code{@bs{}set command}

With the exception of @ commands, \ commands and punctuation, the section name should match the node name exactly.

Sectioning commands (@node and @section) must not appear inside an @ignore. Separate those commands with a space, ie @n ode.

Nodes must be included inside a

@menu
* foo::
* bar::
@end menu

construct. These can be constructed with scripts: see Stripping whitespace and generating menus.

5.4.4 LilyPond formatting

• Most LilyPond examples throughout the documentation can be produced with:

  @lilypond[verbatim,quote]
  

  If using \book{} in your example then you must also include the papersize=X variable, where X is a defined paper size from within ‘scm/paper.scm’. This is to avoid the default a4 paper size being used and leaving too much unnecessary whitespace and potentially awkward page breaks in the PDFs.

  The preferred papersizes are a5, a6 or a8landscape.

  a8landscape works best for a single measure with a single title and/or single tagline:

  @lilypond[papersize=a8landscape,verbatim]
  \book {
    \header {
      title = "A scale in LilyPond"
    }
    \relative {
      c d e f
    }
  }
  @end lilypond
  

  and can also be used to easily show features that require page breaks (i.e. page numbers) without taking large amounts of space within the documentation. Do not use the quote option with this paper size.

  a5 or a6 paper sizes are best used for examples that have more than two measures of music or require multiple staves (i.e. to illustrate cross-staff features, RH and LH parts etc.) and where \book{} constructions are required or where a8landscape produces an example that is too cramped. Depending on the example the quote option may need to be omitted.

  In rare cases, other options may be used (or omitted), but ask first.

• Please avoid using extra spacing either after or within the @lilypond parameters.

  not:          @lilypond [verbatim, quote, fragment]
  but instead:  @lilypond[verbatim,quote,fragment]
  

• Inspirational headwords are produced with:

  @lilypondfile[quote,ragged-right,line-width=16\cm,staffsize=16]
  {pitches-headword.ly}
  

• LSR snippets are linked with:

  @lilypondfile[verbatim,quote,ragged-right,texidoc,doctitle]
  {filename.ly}
  

• Use two spaces for indentation in lilypond examples (no tabs).
• All engravers should have double-quotes around them:

  \consists "Spans_arpeggio_engraver"
  

  LilyPond does not strictly require this, but it is a useful convention to follow.

• All context or layout object strings should be prefaced with #. Again, LilyPond does not strictly require this, but it is helpful to get users accustomed to this scheme construct, i.e. \set Staff.instrumentName = #"cello"
• Try to avoid using #' or #` when describing context or layout properties outside of an @example or @lilypond, unless the description explicitly requires it.

  i.e. “...setting the transparent property leaves the object where it is, but makes it invisible.”

• If possible, only write one bar per line.
• If you only have one bar per line, omit bar checks. If you must put more than one bar per line (not recommended), then include bar checks.
• Tweaks should, if possible, also occur on their own line.

  not:          \override TextScript.padding = #3 c1^"hi"
  but instead:  \override TextScript.padding = #3
                c1^"hi"
  

  excepted in Templates, where ‘doctitle’ may be omitted.

• Avoid long stretches of input code. Nobody is going to read them in print. Create small examples. However, this does not mean it has be minimal.
• Specify durations for at least the first note of every bar.
• If possible, end with a complete bar.
• Comments should go on their own line, and be placed before the line(s) to which they refer.
• For clarity, always use { } marks even if they are not technically required; i.e.

  not:
  
  \context Voice \repeat unfold 2 \relative c' {
    c2 d
  }
  
  but instead:
  
  \context Voice {
    \repeat unfold 2 {
      \relative c' {
        c2 d
      }
    }
  }
  

• Add a space around { } marks; i.e.

  not:          \chordmode{c e g}
  but instead:  \chordmode { c e g }
  

• Use { } marks for additional \markup format commands; i.e.

  not:          c^\markup \tiny\sharp
  but instead:  c^\markup { \tiny \sharp }
  

• Remove any space around < > marks; i.e.

  not:           < c e g > 4
  but instead:   <c e g>4
  

• Beam, slur and tie marks should begin immediately after the first note with beam and phrase marks ending immediately after the last.

  a8\( ais16[ b cis( d] b) cis4~ b' cis,\)
  

• If you want to work on an example outside of the manual (for easier/faster processing), use this header:

  \paper {
    indent = 0\mm
    line-width = 160\mm - 2.0 * 0.4\in
    line-width = #(- line-width (* mm  3.000000))
  }
  
  \layout {
  }
  

  You may not change any of these values. If you are making an example demonstrating special \paper{} values, contact the Documentation Editor.

5.4.5 Text formatting

• Lines should be less than 72 characters long. (We personally recommend writing with 66-char lines, but do not bother modifying existing material). Also see the recommendations for fixed-width fonts in the Syntax survey.
• Do not use tabs.
• Do not use spaces at the beginning of a line (except in @example or @verbatim environments), and do not use more than a single space between words. ‘makeinfo’ copies the input lines verbatim without removing those spaces.
• Use two spaces after a period.
• In examples of syntax, use @var{musicexpr} for a music expression.
• Don’t use @rinternals{} in the main text. If you’re tempted to do so, you’re probably getting too close to “talking through the code”. If you really want to refer to a context, use @code{} in the main text and @rinternals{} in the @seealso.

5.4.6 Syntax survey

Comments

• @c … — single line comment. ‘@c NOTE:’ is a comment which should remain in the final version. (gp only command ;)
• @ignore — multi-line comment:

  @ignore
  …
  @end ignore
  

Cross references

Enter the exact @node name of the target reference between the brackets (eg. ‘@ref{Syntax survey}’). Do not split a cross-reference across two lines – this causes the cross-reference to be rendered incorrectly in HTML documents.

• @ref{…} — link within current manual.
• @rchanges{…} — link to Changes.
• @rcontrib{…} — link to Contributor’s Guide.
• @ressay{…} — link to Engraving Essay.
• @rextend{…} — link to Extending LilyPond.
• @rglos{…} — link to the Music Glossary.
• @rinternals{…} — link to the Internals Reference.
• @rlearning{…} — link to Learning Manual.
• @rlsr{…} — link to a Snippet section.
• @rprogram{…} — link to Application Usage.
• @ruser{…} — link to Notation Reference.
• @rweb{…} — link to General Information.

External links

• @email{…} — create a mailto: E-mail link.
• @uref{URL[, link text]} — link to an external url. Use within an @example ... @end example.

  @example
  @uref{URL [, link text ]}
  @end example
  

Fixed-width font

• @code{…}, @samp{…} —

  Use the @code{…} command when referring to individual language-specific tokens (keywords, commands, engravers, scheme symbols, etc.) in the text. Ideally, a single @code{…} block should fit within one line in the PDF output.

  Use the @samp{…} command when you have a short example of user input, unless it constitutes an entire @item by itself, in which case @code{…} is preferable. Otherwise, both should only be used when part of a larger sentence within a paragraph or @item. Do not use @code{…} or @samp{…} inside an @example block, and do not use either as a free-standing paragraph; use @example instead.

  A single unindented line in the PDF has space for about 79 fixed-width characters (76 if indented). Within an @item there is space for about 75 fixed-width characters. Each additional level of @itemize or @enumerate shortens the line by about 4 columns.

  However, even short blocks of @code{…} and @samp{…} can run into the margin if the Texinfo line-breaking algorithm gets confused. Additionally, blocks that are longer than this may in fact print nicely; it all depends where the line breaks end up. If you compile the docs yourself, check the PDF output to make sure the line breaks are satisfactory.

  The Texinfo setting @allowcodebreaks is set to false in the manuals, so lines within @code{…} or @samp{…} blocks will only break at spaces, not at hyphens or underscores. If the block contains spaces, use @w{@code{…}} or @w{@samp{…}} to prevent unexpected line breaks.

  The Texinfo settings txicodequoteundirected and txicodequotebacktick are both set in the manuals, so backticks (`) and apostrophes (') placed within blocks of @code, @example, or @verbatim are not converted to left- and right-angled quotes (‘ ’) as they normally are within the text, so the apostrophes in ‘@w{@code{\relative c''}}’ will display correctly. However, these settings do not affect the PDF output for anything within a @samp block (even if it includes a nested @code block), so entering ‘@w{@samp{\relative c''}}’ wrongly produces ‘\relative c’’’ in PDF. Consequently, if you want to use a @samp{…} block which contains backticks or apostrophes, you should instead use ‘@q{@code{…}}’ (or ‘@q{@w{@code{…}}}’ if the block also contains spaces). Note that backslashes within @q{…} blocks must be entered as ‘@bs{}’, so the example above would be coded as ‘@q{@w{@code{@bs{}relative c''}}}’.

• @command{…} — Use when referring to command-line commands within the text (eg. ‘@command{convert-ly}’). Do not use inside an @example block.
• @example — Use for examples of program code. Do not add extraneous indentation (i.e. don’t start every line with whitespace). Use the following layout (notice the use of blank lines). Omit the @noindent if the text following the example starts a new paragraph:

  …text leading into the example…
  
  @example
  …
  @end example
  
  @noindent
  continuation of the text…
  

  Individual lines within an @example block should not exceed 74 characters; otherwise they will run into the margin in the PDF output, and may get clipped. If an @example block is part of an @item, individual lines in the @example block should not exceed 70 columns. Each additional level of @itemize or @enumerate shortens the line by about 4 columns.

  For long command line examples, if possible, use a trailing backslash to break up a single line, indenting the next line with 2 spaces. If this isn’t feasible, use ‘@smallexample … @end smallexample’ instead, which uses a smaller fontsize. Use @example whenever possible, but if needed, @smallexample can fit up to 90 characters per line before running into the PDF margin. Each additional level of @itemize or @enumerate shortens a @smallexample line by about 5 columns.

• @file{…} — Use when referring to filenames and directories in the text. Do not use inside an @example block.
• @option{…} — Use when referring to command-line options in the text (eg. ‘@option{--format}’). Do not use inside an @example block.
• @verbatim — Prints the block exactly as it appears in the source file (including whitespace, etc.). For program code examples, use @example instead. @verbatim uses the same format as @example.

  Individual lines within an @verbatim block should not exceed 74 characters; otherwise they will run into the margin in the PDF output, and may get clipped. If an @verbatim block is part of an @item, individual lines in the @verbatim block should not exceed 70 columns. Each additional level of @itemize or @enumerate shortens the line by about 4 columns.

Indexing

• @cindex … — General index. Please add as many as you can. Don’t capitalize the first word.
• @funindex … — is for a \lilycommand.

Lists

• @enumerate — Create an ordered list (with numbers). Always put ‘@item’ on its own line. As an exception, if all the items in the list are short enough to fit on single lines, placing them on the ‘@item’ lines is also permissible. ‘@item’ and ‘@end enumerate’ should always be preceded by a blank line.

  @enumerate
  
  @item
  A long multi-line item like this one must begin
  on a line of its own and all the other items in
  the list must do so too.
  
  @item
  Even short ones
  
  @end enumerate
  

  @enumerate
  
  @item Short item
  
  @item Short item
  
  @end enumerate
  

• @itemize — Create an unordered list (with bullets). Use the same format as @enumerate. Do not use ‘@itemize @bullet’.

Special characters

• --, --- — Create an en dash (–) or an em dash (—) in the text. To print two or three literal hyphens in a row, wrap one of them in a @w{…} (eg. ‘-@w{-}-’).
• @@, @{, @} — Create an at-sign (@), a left curly bracket ({), or a right curly bracket (}).
• @bs{} — Create a backslash within a @q{…}, @qq{…}, or @warning{…} block. This is a custom LilyPond macro, not a builtin @-command in Texinfo. Texinfo would also allow ‘\\’, but this breaks the PDF output.
• @tie{} — Create a variable-width non-breaking space in the text (use ‘@w{ }’ for a single fixed-width non-breaking space). Variables or numbers which consist of a single character (probably followed by a punctuation mark) should be tied properly, either to the previous or the next word. Example: ‘The letter@tie{}@q{I} is skipped’

Miscellany

• @notation{…} — refers to pieces of notation, e.g. ‘@notation{clef}’. Also use for specific lyrics (‘the @notation{A - men} is centered’). Only use once per subsection per term.
• @q{…} — Single quotes. Used for ‘vague’ terms. To get a backslash (\), you must use ‘@bs{}’.
• @qq{…} — Double quotes. Used for actual quotes (“he said”) or for introducing special input modes. To get a backslash (\), you must use ‘@bs{}’.
• @var{…} — Use for metasyntactic variables (such as foo, bar, arg1, etc.). In most cases, when the @var{…} command appears in the text (and not in an @example block) it should be wrapped with an appropriate texinfo code-highlighting command (such as @code, @samp, @file, @command, etc.). For example: ‘@code{@var{foo}}’, ‘@file{@var{myfile.ly}}’, ‘@samp{git checkout @var{branch}}’, etc. This improves readability in the PDF and HTML output.
• @version{} — Return the current LilyPond version string. Use ‘@w{@version{}}’ if it’s at the end of a line (to prevent an ugly line break in PDF); use ‘@w{"@version{}"}’ if you need it in quotes.
• @w{…} — Do not allow any line breaks.
• @warning{…} — produces a “Note: ” box. Use for important messages. To get a backslash (\), you must use ‘@bs{}’.

5.4.7 Other text concerns

• References must occur at the end of a sentence, for more information see the texinfo manual. Ideally this should also be the final sentence of a paragraph, but this is not required. Any link in a doc section must be duplicated in the @seealso section at the bottom.
• Introducing examples must be done with

  . (i.e. finish the previous sentence/paragraph)
  : (i.e. `in this example:')
  , (i.e. `may add foo with the blah construct,')
  

  The old “sentence runs directly into the example” method is not allowed any more.

• Abbrevs in caps, e.g., HTML, DVI, MIDI, etc.
• Colon usage
  1. To introduce lists
  2. When beginning a quote: “So, he said,...”.

     This usage is rarer. Americans often just use a comma.

  3. When adding a defining example at the end of a sentence.
• Non-ASCII characters which are in utf-8 should be directly used; this is, don’t say ‘Ba@ss{}tuba’ but ‘Baßtuba’. This ensures that all such characters appear in all output formats.

5.5 Documentation policy

5.5.1 Books

There are four parts to the documentation: the Learning Manual, the Notation Reference, the Program Reference, and the Music Glossary.

• Learning Manual:

  The LM is written in a tutorial style which introduces the most important concepts, structure and syntax of the elements of a LilyPond score in a carefully graded sequence of steps. Explanations of all musical concepts used in the Manual can be found in the Music Glossary, and readers are assumed to have no prior knowledge of LilyPond. The objective is to take readers to a level where the Notation Reference can be understood and employed to both adapt the templates in the Appendix to their needs and to begin to construct their own scores. Commonly used tweaks are introduced and explained. Examples are provided throughout which, while being focussed on the topic being introduced, are long enough to seem real in order to retain the readers’ interest. Each example builds on the previous material, and comments are used liberally. Every new aspect is thoroughly explained before it is used.

  Users are encouraged to read the complete Learning Manual from start-to-finish.

• Notation Reference: a (hopefully complete) description of LilyPond input notation. Some material from here may be duplicated in the Learning Manual (for teaching), but consider the NR to be the "definitive" description of each notation element, with the LM being an "extra". The goal is _not_ to provide a step-by-step learning environment – do not avoid using notation that has not be introduced previously in the NR (for example, use \break if appropriate). This section is written in formal technical writing style.

  Avoid duplication. Although users are not expected to read this manual from start to finish, they should be familiar with the material in the Learning Manual (particularly “Fundamental Concepts”), so do not repeat that material in each section of this book. Also watch out for common constructs, like ^ - _ for directions – those are explained in NR 3. In NR 1, you can write: DYNAMICS may be manually placed above or below the staff, see @ref{Controlling direction and placement}.

  Most tweaks should be added to LSR and not placed directly in the ‘.itely’ file. In some cases, tweaks may be placed in the main text, but ask about this first.

  Finally, you should assume that users know what the notation means; explaining musical concepts happens in the Music Glossary.

• Application Usage: information about using the program lilypond with other programs (lilypond-book, operating systems, GUIs, convert-ly, etc). This section is written in formal technical writing style.

  Users are not expected to read this manual from start to finish.

• Music Glossary: information about the music notation itself. Explanations and translations about notation terms go here.

  Users are not expected to read this manual from start to finish.

• Internals Reference: not really a documentation book, since it is automagically generated from the source, but this is its name.

5.5.2 Section organization

• The order of headings inside documentation sections should be:

  main docs
  @predefined
  @endpredefined
  @snippets
  @seealso
  @knownissues
  

• You must include a @seealso.
  • The order of items inside the @seealso section is

    Music Glossary:
    @rglos{foo},
    @rglos{bar}.
    
    Learning Manual:
    @rlearning{baz},
    @rlearning{foozle}.
    
    Notation Reference:
    @ruser{faazle},
    @ruser{boo}.
    
    Application Usage:
    @rprogram{blah}.
    
    Essay on automated music engraving:
    @ressay{yadda}.
    
    Extending LilyPond:
    @rextend{frob}.
    
    Installed Files:
    @file{path/to/dir/blahz}.
    
    Snippets: @rlsr{section}.
    
    Internals Reference:
    @rinternals{fazzle},
    @rinternals{booar}.
    

  • If there are multiple entries, separate them by commas but do not include an ‘and’.
  • Always end with a period.
  • Place each link on a new line as above; this makes it much easier to add or remove links. In the output, they appear on a single line.

    ("Snippets" is REQUIRED; the others are optional)

  • Any new concepts or links which require an explanation should go as a full sentence(s) in the main text.
  • Don’t insert an empty line between @seealso and the first entry! Otherwise there is excessive vertical space in the PDF output.
• To create links, use @ref{} if the link is within the same manual.
• @predefined ... @endpredefined is for commands in ‘ly/*-init.ly’
• Do not include any real info in second-level sections (i.e. 1.1 Pitches). A first-level section may have introductory material, but other than that all material goes into third-level sections (i.e. 1.1.1 Writing Pitches).
• The @knownissues should not discuss any issues that are in the tracker, unless the issue is Priority-Postponed. The goal is to discuss any overall architecture or syntax decisions which may be interpreted as bugs. Normal bugs should not be discussed here, because we have so many bugs that it would be a huge task to keep the @knownissues current and accurate all the time.

5.5.3 Checking cross-references

Cross-references between different manuals are heavily used in the documentation, but they are not checked during compilation. However, if you compile the documentation, a script called check_texi_refs can help you with checking and fixing these cross-references; for information on usage, cd into a source tree where documentation has been built, cd into Documentation and run:

make check-xrefs
make fix-xrefs

Note that you have to find yourself the source files to fix cross-references in the generated documentation such as the Internals Reference; e.g. you can grep scm/ and lily/.

5.5.4 General writing

• Do not forget to create @cindex entries for new sections of text. Enter commands with @funindex, i.e.

  @cindex pitches, writing in different octaves
  @funindex \relative
  

  Do not bother with the @code{} (they are added automatically). These items are added to both the command index and the unified index. Both index commands should go in front of the actual material.

• @cindex entries should not be capitalized, i.e.

  @cindex time signature
  

  is preferred instead of “Time signature”. Only use capital letters for musical terms which demand them, e.g. “D.S. al Fine”.

• For scheme function index entries, only include the final part, i.e.

  @funindex modern-voice-cautionary
       and NOT
  @funindex #(set-accidental-style modern-voice-cautionary)
  

• Use American spelling. LilyPond’s internal property names use this convention.
• Here is a list of preferred terms to be used:
  • Simultaneous NOT concurrent.
  • Measure: the unit of music.
  • Bar line: the symbol delimiting a measure NOT barline.
  • Note head NOT notehead.
  • Chord construct NOT just chord (when referring to < ... >)
  • Staff NOT stave.
  • Staves NOT Staffs: Phrases such as ‘multiple @internalsref{Staff}s’ should be rephrased to ‘multiple @internalsref{Staff} contexts’.

5.5.5 Technical writing style

These refer to the NR. The LM uses a more gentle, colloquial style.

• Do not refer to LilyPond in the text. The reader knows what the manual is about. If you do, capitalization is LilyPond.
• If you explicitly refer to ‘lilypond’ the program (or any other command to be executed), write @command{lilypond}.
• Do not explicitly refer to the reader/user. There is no one else besides the reader and the writer.
• Avoid contractions (don’t, won’t, etc.). Spell the words out completely.
• Avoid abbreviations, except for commonly used abbreviations of foreign language terms such as etc. and i.e.
• Avoid fluff (“Notice that,” “as you can see,” “Currently,”).
• The use of the word ‘illegal’ is inappropriate in most cases. Say ‘invalid’ instead.

5.6 Tips for writing docs

In the NR, I highly recommend focusing on one subsection at a time. For each subsection,

• check the mundane formatting. Are the headings (@predefined, @seealso, etc.) in the right order?
• add any appropriate index entries.
• check the links in the @seealso section – links to music glossary, internal references, and other NR sections are the main concern. Check for potential additions.
• move LSR-worthy material into LSR. Add the snippet, delete the material from the ‘.itely’ file, and add a @lilypondfile command.
• check the examples and descriptions. Do they still work? Do not assume that the existing text is accurate/complete; some of the manual is highly out of date.
• is the material in the @knownissues still accurate?
• can the examples be improved (made more explanatory), or is there any missing info? (feel free to ask specific questions on -user; a couple of people claimed to be interesting in being “consultants” who would help with such questions)

In general, I favor short text explanations with good examples – “an example is worth a thousand words”. When I worked on the docs, I spent about half my time just working on those tiny lilypond examples. Making easily-understandable examples is much harder than it looks.

Tweaks

In general, any \set or \override commands should go in the “select snippets” section, which means that they should go in LSR and not the ‘.itely’ file. For some cases, the command obviously belongs in the “main text” (i.e. not inside @predefined or @seealso or whatever) – instrument names are a good example of this.

\set Staff.instrumentName = #"foo"

On the other side of this,

\override Score.Hairpin.after-line-breaking = ##t

clearly belongs in LSR.

I’m quite willing to discuss specific cases if you think that a tweaks needs to be in the main text. But items that can go into LSR are easier to maintain, so I’d like to move as much as possible into there.

It would be “nice” if you spent a lot of time crafting nice tweaks for users… but my recommendation is not to do this. There’s a lot of doc work to do without adding examples of tweaks. Tweak examples can easily be added by normal users by adding them to the LSR.

One place where a documentation writer can profitably spend time writing or upgrading tweaks is creating tweaks to deal with known issues. It would be ideal if every significant known issue had a workaround to avoid the difficulty.

See also

Adding and editing snippets.

5.7 Scripts to ease doc work

5.7.1 Scripts to test the documentation

Building only one section of the documentation

In order to save build time, a script is available to build only one section of the documentation in English with a default HTML appearance.

If you do not yet have a ‘build/’ subdirectory within the LilyPond Git tree, you should create this first. You can then build a section of the documentation with the following command:

scripts/auxiliar/doc-section.sh MANUAL SECTION

where SECTION is the name of the file containing the section to be built, and MANUAL is replaced by the name of the directory containing the section. So, for example, to build section 1.1 of the Notation Reference, use the command:

scripts/auxiliar/doc-section.sh notation pitches

You can then see the generated document for the section at

build/tempdocs/pitches/out/pitches.html

According to LilyPond issue 1236, the location of the LilyPond Git tree is taken from $LILYPOND_GIT if specified, otherwise it is auto-detected.

It is assumed that compilation takes place in the ‘build/’ subdirectory, but this can be overridden by setting the environment variable LILYPOND_BUILD_DIR.

Similarly, output defaults to ‘build/tempdocs/’ but this can be overridden by setting the environment variable LILYPOND_TEMPDOCS.

This script will not work for building sections of the Contributors’ Guide. For building sections of the Contributors’ Guide, use:

scripts/auxiliar/cg-section.sh SECTION

where SECTION is the name of the file containing the sections to be built. For example, to build section 4 of the Contributors’ Guide, use:

scripts/auxiliar/cg-section.sh doc-work

cg-section.sh uses the same environment variables and corresponding default values as doc-section.sh.

5.7.2 Scripts to create documentation

Stripping whitespace and generating menus

To automatically regenerate @menu portions and strip whitespace, use:

scripts/auxiliar/node-menuify.py FILENAME

If you are adding documentation that requires new menus, you will need to add a blank @menu section:

@menu
@end menu

Stripping whitespace only

To remove extra whitespace from the ends of lines, run

scripts/auxiliar/strip-whitespace.py FILENAME

Updating doc with convert-ly

Don’t. This should be done by programmers when they add new features. If you notice that it hasn’t been done, complain to lilypond-devel.

5.8 Docstrings in scheme

Material in the Internals reference is generated automatically from our source code. Any doc work on Internals therefore requires modifying files in ‘scm/*.scm’. Texinfo is allowed in these docstrings.

Most documentation writers never touch these, though. If you want to work on them, please ask for help.

5.9 Translating the documentation

The mailing list translations@lilynet.net is dedicated to LilyPond web site and documentation translation; on this list, you will get support from the Translations Meister and experienced translators, and we regularly discuss translation issues common to all languages. All people interested in LilyPond translations are invited to subscribe to this list regardless of the amount of their contribution, by sending an email to translations-request@lilynet.net with subject subscribe and an empty message body. Unless mentioned explicitly, or except if a translations coordinator contacts you privately, you should send questions, remarks and patches to the list translations@lilynet.net. Please note that traffic is high on the English-speaking list lilypond-user@gnu.org, so it may take some time before your request or contribution is handled.

5.9.1 Getting started with documentation translation

First, get the sources of branch translation from the Git repository, see Starting with Git.

Translation requirements

Working on LilyPond documentation translations requires the following pieces of software, in order to make use of dedicated helper tools:

• Python 2.4 or higher,
• GNU Make,
• Gettext,
• Git.

It is not required to build LilyPond and the documentation to translate the documentation. However, if you have enough time and motivation and a suitable system, it can be very useful to build at least the documentation so that you can check the output yourself and more quickly; if you are interested, see Compiling.

Before undertaking any large translation work, contributors are encouraged to contact the Translation Meister.

Which documentation can be translated

The makefiles and scripts infrastructure currently supports translation of the following documentation:

• the web site, the Learning Manual, the Notation Reference and Application Usage – Texinfo source, PDF and HTML output; Info output might be added if there is enough demand for it;
• the Changes document.

Support for translating the following pieces of documentation should be added soon, by decreasing order of priority:

• automatically generated documentation: markup commands, predefined music functions;
• the Snippets List;
• the Internals Reference.

Starting translation in a new language

At top of the source directory, do

./autogen.sh

or (if you want to install your self-compiled LilyPond locally)

./autogen.sh --prefix=$HOME

If you want to compile LilyPond – which is almost required to build the documentation, but is not required to do translation only – fix all dependencies and rerun ./configure (with the same options as for autogen.sh).

Then cd into ‘Documentation/’ and run

make ISOLANG=MY-LANGUAGE new-lang

where MY-LANGUAGE is the ISO 639 language code.

Finally, add a language definition for your language in ‘python/langdefs.py’.

5.9.2 Documentation translation details

Please follow all the instructions with care to ensure quality work.

All files should be encoded in UTF-8.

Files to be translated

Translation of ‘Documentation/foo/bar’ should be ‘Documentation/LANG/foo/bar’. Unmentioned files should not be translated.

Priorities:

• 1. delivery,
• 2. 3. 4. 5. 6. later,
• 7. optional.

Files of priority 1 should be submitted along all files generated by starting a new language in the same commit and thus a unique patch, and the translation of files marked with priority 2 should be committed to Git at the same time and thus sent in a single patch. Files marked with priority 3 or more may be submitted individually. Word counts (excluding LilyPond snippets) are given for each file. For knowing how to commit your work to Git, then make patches of your new translations as well as corrections and updates, see Basic Git procedures.

-1- Web site
760   web.texi
5793  web/introduction.itexi
1158  web/download.itexi
1139  macros.itexi
9     po/lilypond-doc.pot (translate to po/MY_LANGUAGE.po)
0     search-box.ihtml
---   lilypond-texi2html.init (section TRANSLATIONS)
8859  total

-2- Tutorial
1314  web/manuals.itexi
124   learning.tely
2499  learning/tutorial.itely
4421  learning/common-notation.itely
8358  total

-3- Fundamental Concepts, starting of Usage and Community
11240 learning/fundamental.itely -- Fundamental concepts
135   usage.tely
5469  usage/running.itely
2097  usage/updating.itely
2449  web/community.itexi
21390 total

-4- Rest of Learning manual and Suggestions on writing LilyPond files
16592 learning/tweaks.itely -- Tweaking output
1236  learning/templates.itely -- Templates
2793  usage/suggestions.itely -- Suggestions on writing LilyPond files
20621 total

-5- Notation reference
326   notation.tely
91    notation/notation.itely -- Musical notation
5413  notation/pitches.itely
6853  notation/rhythms.itely
1819  notation/expressive.itely
1288  notation/repeats.itely
2979  notation/simultaneous.itely
2554  notation/staff.itely
1481  notation/editorial.itely
2754  notation/text.itely
81    notation/specialist.itely -- Specialist notation
4977  notation/vocal.itely
1975  notation/chords.itely
702   notation/piano.itely
799   notation/percussion.itely
826   notation/guitar.itely
66    notation/strings.itely
242   notation/bagpipes.itely
5518  notation/ancient.itely
12853 notation/input.itely -- Input syntax
2164  notation/non-music.itely -- Non-musical notation
10982 notation/spacing.itely -- Spacing issues
17050 notation/changing-defaults.itely -- Changing defaults
5187  notation/programming-interface.itely -- Interfaces for programmers
3079  notation/notation-appendices.itely -- Notation manual tables
252   notation/cheatsheet.itely -- Cheat sheet
92311 total

-6- Rest of Application Usage
4211  usage/lilypond-book.itely -- LilyPond-book
1122  usage/converters.itely -- Converting from other formats
5333  total

-7- Appendices whose translation is optional
382   essay/literature.itely
1222  learning/scheme-tutorial.itely (should be revised first)
1604  total

In addition, not listed above, Snippets’ titles and descriptions should be translated; they are a part of the Notation Reference and therefore their priority is 5.

Translating the Web site and other Texinfo documentation

Every piece of text should be translated in the source file, except Texinfo comments, text in @lilypond blocks and a few cases mentioned below.

Node names are translated, but the original node name in English should be kept as the argument of @translationof put after the section title; that is, every piece in the original file like

@node Foo bar
@section_command Bar baz

should be translated as

@node translation of Foo bar
@section_command translation of Bar baz
@translationof Foo bar

The argument of @rglos commands and the first argument of @rglosnamed commands must not be translated, as it is the node name of an entry in Music Glossary.

Every time you translate a node name in a cross-reference, i.e. the argument of commands @ref, @rprogram, @rlearning, @rlsr, @ruser or the first argument of their *named variants, you should make sure the target node is defined in the correct source file; if you do not intend to translate the target node right now, you should at least write the node definition (that is, the @node @section_commmand @translationof trio mentioned above) in the expected source file and define all its parent nodes; for each node you have defined this way but have not translated, insert a line that contains @untranslated. That is, you should end up for each untranslated node with something like

@node translation of Foo bar
@section_command translation of Bar baz
@translationof Foo bar

@untranslated

Please think of the fact that it may not make sense translating everything in some Texinfo files, and you may take distance from the original text; for instance, in the translation of the web site section Community, you may take this into account depending on what you know the community in your language is willing to support, which is possible only if you personally assume this support, or there exists a public forum or mailing list listed in Community for LilyPond in your language:

• Bug reports: this page should be translated only if you know that every bug report sent on your language’s mailing list or forum will be handled by someone who will translate it to English and send it on bug-lilypond or add an issue in the tracker, then translate back the reply from developers.
• Help us: this page should be translated very freely, and possibly not at all: ask help for contributing to LilyPond for tasks that LilyPond community in your language is able and going to handle.

In any case, please mark in your work the sections which do not result from the direct translation of a piece of English translation, using comments i.e. lines starting with ‘@c’.

Finally, press in Emacs <C-c C-u C-a> to update or generate menus. This process should be made easier in the future, when the helper script texi-langutils.py and the makefile target are updated.

Some pieces of text manipulated by build scripts that appear in the output are translated in a ‘.po’ file – just like LilyPond output messages – in ‘Documentation/po’. The Gettext domain is named lilypond-doc, and unlike lilypond domain it is not managed through the Free Translation Project.

Take care of using typographic rules for your language, especially in ‘macros.itexi’.

If you wonder whether a word, phrase or larger piece of text should be translated, whether it is an argument of a Texinfo command or a small piece sandwiched between two Texinfo commands, try to track whether and where it appears in PDF and/or HTML output as visible text. This piece of advice is especially useful for translating ‘macros.itexi’.

Please keep verbatim copies of music snippets (in @lilypond blocs). However, some music snippets containing text that shows in the rendered music, and sometimes translating this text really helps the user to understand the documentation; in this case, and only in this case, you may as an exception translate text in the music snippet, and then you must add a line immediately before the @lilypond block, starting with

@c KEEP LY

Otherwise the music snippet would be reset to the same content as the English version at next make snippet-update run – see Updating documentation translation.

When you encounter

@lilypondfile[<number of fragment options>,texidoc]{filename.ly}

in the source, open ‘Documentation/snippets/filename.ly’, translate the texidoc header field it contains, enclose it with texidocMY-LANGUAGE = " and ", and write it into ‘Documentation/MY-LANGUAGE/texidocs/filename.texidoc’. Additionally, you may translate the snippet’s title in doctitle header field, in case doctitle is a fragment option used in @lilypondfile; you can do this exactly the same way as texidoc. For instance, ‘Documentation/MY-LANGUAGE/texidocs/filename.texidoc’ may contain

doctitlees = "Spanish title baz"
texidoces = "
Spanish translation blah
"

@example blocks need not be verbatim copies, e.g. variable names, file names and comments should be translated.

Finally, please carefully apply every rule exposed in Texinfo introduction and usage policy, and Documentation policy. If one of these rules conflicts with a rule specific to your language, please ask the Translation meister on translations@lilynet.net list and/or the Documentation Editors on lilypond-devel@gnu.org list.

Adding a Texinfo manual

In order to start translating a new manual whose basename is FOO, do

cd Documentation/MY-LANGUAGE
cp ../FOO.tely .
mkdir FOO
cp web/GNUmakefile FOO

then append FOO to variable SUBDIRS in Documentation/MY-LANGUAGE/GNUmakefile, then translate file MY-LANGUAGE/FOO.tely and run skeleton-update:

cd Documentation/
make ISOLANG=MY-LANGUAGE TEXI_LANGUTIL_FLAGS=--head-only skeleton-update

Your are now ready to translate the new manual exactly like the web site or the Learning Manual.

5.9.3 Documentation translation maintenance

Several tools have been developed to make translations maintenance easier. These helper scripts make use of the power of Git, the version control system used for LilyPond development.

You should use them whenever you would like to update the translation in your language, which you may do at the frequency that fits your and your cotranslators’ respective available times. In the case your translation is up-do-date (which you can discover in the first subsection below), it is enough to check its state every one or two weeks. If you feel overwhelmed by the quantity of documentation to be updated, see Maintaining without updating translations.

Check state of translation

First pull from Git – see Pulling and rebasing, but DO NOT rebase unless you are sure to master the translation state checking and updating system – then cd into ‘Documentation/’ (or at top of the source tree, replace make with make -C Documentation) and run

make ISOLANG=MY_LANGUAGE check-translation

This presents a diff of the original files since the most recent revision of the translation. To check a single file, cd into ‘Documentation/’ and run

make CHECKED_FILES=MY_LANGUAGE/manual/foo.itely check-translation

In case this file has been renamed since you last updated the translation, you should specify both old and new file names, e.g. CHECKED_FILES=MY_LANGUAGE/{manual,user}/foo.itely.

To see only which files need to be updated, do

make ISOLANG=MY_LANGUAGE check-translation | grep 'diff --git'

To avoid printing terminal colors control characters, which is often desirable when you redirect output to a file, run

make ISOLANG=MY_LANGUAGE NO_COLOR=1 check-translation

You can see the diffs generated by the commands above as changes that you should make in your language to the existing translation, in order to make your translation up to date.

Global state of the translation is recorded in ‘Documentation/translations.itexi’, which is used to generate Translations status page. To update that page, do from ‘Documentation/’

make translation-status

This will also leave ‘out/translations-status.txt’, which contains up-to-dateness percentages for each translated file, and update word counts of documentation files in this Guide.

See also

Maintaining without updating translations.

Updating documentation translation

Instead of running check-translation, you may want to run update-translation, which will run your favorite text editor to update files. First, make sure environment variable EDITOR is set to a text editor command, then run from ‘Documentation/’

make ISOLANG=MY_LANGUAGE update-translation