\begin[class=book]{document}
%\begin[papersize=152mm x 229mm,class=book]{document}
\script[src=packages/grid]
\script[src=packages/url]
\script[src=packages/autodoc]
\script[src=packages/verbatim]
\script[src=packages/color]
\script[src=packages/image]
\script[src=packages/frametricks]
\script[src=packages/linespacing]
\script[src=packages/lorem]
\script[src=packages/bidi]\bidi-off
\script[src=packages/pdf]
\script[src=packages/pullquote]
\include[src=documentation/macros.xml]
\define[command=silehp]{\url{http://www.sile-typesetter.org/}}
\define[command=sileversion]{\script{SILE.typesetter:typeset(SILE.version)}}
\set[parameter=document.baselineskip,value=3ex]
\font[size=11pt,family=Gentium Book Basic]
\nofolios
%\set[parameter=document.lskip,value=5em]
%\set[parameter=document.rskip,value=5em]
\par\center{\font[size=50pt,weight=600,filename=documentation/Silson-Condensed.otf]{The}\par
\center{\img[src=documentation/sile-logo.pdf,height=40mm]}\par
\font[size=50pt,weight=600,filename=documentation/Silson-Condensed.otf]{Book}}
\bigskip
\font[size=12pt,filename=documentation/Silson-Condensed.otf]{\hss for SILE version \sileversion \hss \par}
\skip[height=1in]
\skip[height=1in]
\font[size=20pt,weight=600,filename=documentation/Silson-Condensed.otf]{\hss Simon Cozens \hss \par}
%
%\set[parameter=document.lskip,value=0]
%\set[parameter=document.rskip,value=0]
\set[parameter=document.parskip,value=4pt plus 1pt]
\open-double-page
\tableofcontents
\set-counter[id=folio,value=0]
\chapter{What is SILE?}

SILE is a typesetting system. Its job is to produce beautiful printed
documents from raw content. The best way to understand what SILE is and what it
does is to compare it to other systems which you may have heard of.

\section{SILE versus Word}

When most people produce printed documents using a computer, they usually use
desktop oriented word processing software such as Microsoft Word, iWork Pages,
or LibreOffice Writer. SILE is not a word processor; it is a typesetting
system. There are several important differences.

The job of a word processor is to produce a document that looks exactly like
what you type on the screen. By contrast the job of a typesetting system is to
take raw content and produce a document that looks as good as possible. The
input for SILE is a text document that includes instructions about how the
content should be layed out on a page. In order to obtain the typeset result
the input file must be processed to render the desired output.

Word processors often describe themselves as WYSIWYG—What You See Is What You
Get. SILE is cheerfully \em{not} WYSIWYG. In fact, you don’t see what you get
until you get it. Rather, SILE documents are prepared initially in a \em{text
editor}—a piece of software which focuses on the text itself and not what it
looks like—and then run through SILE in order to produce a PDF document.

For instance, most word processors are built roughly around the concept of a
page with a central content area into which you type and style your content.
The overall page layout is controlled by the page size and margins and more
fine tuning is done by styling the content itself. You typically type
continuously and when you hit the right margin, your cursor will automatically
jump to the next line. In doing so the user interface shows you where the lines
will break.

In SILE the overall page layout is defined with a paper size and a serious of
one or more content frames. These frame descriptions provide the containers
where content will later be typeset including information about how it might
flow from one frame to the next. Content is written separately and automatically
flowed from frame to frame and page to page. When preparing content for SILE
you don't know where the lines will break until after it has been processed.
You may user your text editor to type and type and type as long a line as you
like, and when SILE comes to process your instructions, it will consider your
input (up to) three times over in order to work out how to best to break the
lines to form a paragraph. For example if after one pass it finds that it has
ended two successive lines with a hyphenated word it will go back and try
again.

The same idea applies to page breaks. When you type into a word processor, at
some point you will spill over onto a new page. When preparing content for
SILE, you keep typing, because the page breaks are determined after considering
the layout of the whole document.

In other words, SILE is a \em{language} for describing what you want to happen
and an interpreter that will make certain formatting decisions about the best
way for those instructions to be turned into print.

\section{SILE versus TeX}

“Ah,” some people will say, “that sounds very much like TeX!”\footnote{Except
that, being TeX users, they will say “Ah, that sounds very much like
T\glue[width=-.1667em]\lower[height=0.5ex]{E}\glue[width=-.125em]X!”} If you
don’t know much about TeX or don’t care, you can probably skip this section.

And it’s true. SILE owes an awful lot of its heritage to TeX. It would be
terribly immodest to claim that a little project like SILE was a worthy
successor to the ancient and venerable creation of the Professor of the Art of
Computer Programming, but… really, SILE is basically a modern rewrite of TeX.

TeX was one of the earliest typesetting systems, and had to make a lot of
design decisions somewhat in a vacuum. Some of those design decisions have
stood the test of time—TeX is still an extremely well-used typesetting system
more than thirty years after its inception, which is a testament to its design
and performance—but many others have not. In fact, most of the development of
TeX since Knuth’s era has involved removing his early decisions and replacing
them with technologies which have become the industry standard: we use TrueType
fonts, not METAFONTs (xetex); PDFs, not DVIs (pstex, pdftex); Unicode, not
7-bit ASCII (xetex again); markup languages and embedded programming languages,
not macro languages (xmltex, luatex). At this point, the parts of TeX that
people actually \em{use} are 1) the box-and-glue model, 2) the hyphenation
algorithm, and 3) the line-breaking algorithm.

SILE follows exactly in TeX's footsteps for each of these three areas that have
stood the test of time; it contains a slavish port of the TeX line-breaking
algorithm which has been tested to produce exactly the same output as TeX given
equivalent input. But as SILE is itself written in an interpreted
language,\footnote{And if the phrase \code{TeX capacity exceeded} is familiar
to you, you should already be getting excited.} it is very easy to extend or
alter the behaviour of the SILE typesetter.

For instance, one of the things that TeX can’t do particularly well is
typesetting on a grid. This is something of a must have feature for anyone
typesetting bibles. Typesetting on a grid means that each line of text will
line up between the front and back of each piece of paper producing much less
visual bleed-through when printed on thin paper. This is virtually impossible
to accomplish in TeX. There are various hacks to try to make it happen, but
they’re all horrible. In SILE, you can alter the behaviour of the typesetter
and write a very short add-on package to enable grid typesetting.

Of course, nobody uses plain TeX—they all use LaTeX equivalents plus a huge
repository of packages available from the CTAN. SILE does not benefit from the
large ecosystem and community that has grown up around TeX; in that sense, TeX
will remain streets ahead of SILE for some time to come. But in terms of
\em{core capabilities}, SILE is already certainly equivalent to, if not
somewhat more advanced than, TeX.

\section{SILE versus InDesign}

The other tool that people reach for when designing printed material on
a computer is InDesign (or similar Desktop Publishing software, such as
Scribus).

\float[rightboundary=7pt,bottomboundary=15pt]{\img[src=documentation/fig1.png,width=150]}
DTP software is similar to Word Processing Software in that they are
both graphical and largely WYSIWYG, but the paradigm is different. The focus is
usually less on preparing the content than on laying it out on the
page--you click and drag to move areas of text and images around the screen.

InDesign is a complex, expensive, commercial publishing tool. SILE is a free,
open source typesetting tool which is entirely text-based; you enter commands
in a separate editing tool, save those commands into a file, and hand it to
SILE for typesetting. And yet the two systems do have a number of common
features.

In InDesign, text is flowed into \em{frames} on the page. On the left, you can
see an example of what a fairly typical InDesign layout might look like.

SILE also uses the concept of frames to determine where text should appear on
the page, and so it’s possible to use SILE to generate page layouts which are
more flexible and more complex than that afforded by TeX.

Another thing which people use InDesign for is to turn structured data in XML
format—catalogues, directories and the like—into print. The way you do this in
InDesign is to declare what styling should apply to each XML element, and as
the data is read in, InDesign formats the content according to the rules that
you have declared.

You can do exactly the same thing in SILE, except you have a lot more control
over how the XML elements get styled, because you can run any SILE command you
like for a given element, including calling out to Lua code to style a piece of
XML. Since SILE is a command-line filter, armed with appropriate styling
instructions you can go from an XML file to a PDF in one shot. Which is quite
nice.

In the final chapters of this book, we’ll look at some extended examples of
creating a \em{class file} for styling a complex XML document into a PDF with
SILE.

\section{Conclusion}

SILE\footnote{In case you’re wondering, the author pronounces it
\font[family=Gentium Plus]{/saɪəl/}, to rhyme with “trial”.} {}takes some textual
instructions and turns them into PDF output. It has
features inspired by TeX and InDesign, but seeks to be more flexible,
extensible and programmable than either of them. It’s useful both for
typesetting documents (such as this very documentation) written in the SILE
language, and as a processing system for styling and outputting structured
data.

\chapter{Getting Started}

Now that we understand some of what SILE is about and what it seeks to do,
let’s dive into SILE itself.

\section{A Basic SILE Document}

Before we show you how to use SILE, let’s have a look at an example of what
SILE documents look like. This is the input that we’re going to feed to SILE,
which it is going to process and turn into a PDF file. Each of the examples can
be saved as a text file and then use them as input to SILE.

\note{These documents are \em{plain text} files; when you create your own SILE
files, you will need to create them in a plain text editor. Trying to create
these files in a word processor such as Word will not work as they will be
saved with the word processor’s formatting rather than in a plain text format.
Examples of good text editors include cross platform editors such as Atom and
Sublime Text which are GUI oriented or Vim and Emacs which are more keyboard
command oriented. Depending on your operating system there are other good
choices such as Notepad++ on Windows or TextMate on OS X. You can also get
started more simply with one of the basic editors built into your desktop
environment such as Notepad on Windows, TextEdit on OS X, Gedit on Gnome, Kate
on KDE, etc. For comparisons of text editors see
\url{http://alternativeto.net/tag/text-editor/} and select your platform.}

To begin with, here’s the most basic SILE file of all:

\begin{verbatim}
\line
\\begin\{document\}
Hello SILE!
\\end\{document\}
\line
\end{verbatim}

We’ll pick apart this document in the next chapter, but for now take it on
trust that this is what a SILE document looks like.

At the risk of belabouring the obvious, this is going to produce an A4-sized PDF
document, with the text \examplefont{Hello SILE} at the top left, and the page
number (1) centered at the bottom. How are we going to get to that PDF?

\section{Installing}

First of all, we need to get ahold of SILE and get it running on our computer.
Downloads of SILE can be obtained from the home page at \silehp.

\subsection{Installing Preconfigured Packages}

For OS X machines the recommended way to install SILE is through the Homebrew
package manager. Once you have Homebrew running (see \url{http://brew.sh}),
installing SILE is as easy as running:

\listitem{\code{brew install sile}}

If you have not used Lua programs before, that will prompt you to manually
install a couple dependencies. After running the commands it provides you can
run the install comand again to complete the instalation.

The formula also has instruction that can compile SILE from the current
Git HEAD version. To test the latest unreleased code you can install using:

\listitem{\code{brew install sile --HEAD}}

For Linux users, preconfigured package build files are available for Arch
Linux. The \code{sile} package has the latest stable release while the
\code{sile-git} package will build a package using the latest unreleased code
from the Git repository. If you use a package manager with AUR support you can
install either one as you would any other package:

\listitem{\code{yaourt -S sile}}

For all other systems you will need to follow the steps to download and compile
the source yourself.

\subsection{Installing From Source}

SILE requires a number of other software packages to be installed on the
computer before it can work—the Lua programming language, and the Harfbuzz text
shaping library. SILE provides its own PDF creation library, which has its own
requirements: \code{freetype}, \code{fontconfig}, \code{libz} and
\code{libpng}.

It is suggested you use your distro's package manager to install as many of the
dependencies as possible. On DEB-based Linux machines such as Debian and
Ubuntu, you should be able to install most of the needed dependencies by
issuing the command:

\listitem{\code{apt-get install liblua5.2-dev lua-expat lua-lpeg
	libharfbuzz-dev libfreetype6-dev libfontconfig1-dev libpng-dev libicu-dev}}

Here’s an incantation that will work on most Redhat-based Linux distros:

\listitem{\code{yum install harfbuzz-devel make automake libtool gcc
freetype-devel fontconfig-devel lua-devel lua-lpeg lua-expat libpng-devel libicu-devel}}

Once these dependencies are installed, you also need to install some Lua
libraries if they’re not already installed:

\listitem{\code{luarocks install lpeg}}
\listitem{\code{luarocks install luaexpat}}
\listitem{\code{luarocks install lua-zlib}}
\listitem{\code{luarocks install luafilesystem}}
\listitem{\code{luarocks install luasocket}}
\listitem{\code{luarocks install luasec OPENSSL_DIR=...}}

Once you have these requirements in place, you should then be able to unpack
the file that you downloaded from SILE’s home page, change to that directory
\footnote{If you downloaded a copy of the SILE source by cloning the git
repository rather than dowloading one of the release packages you will also
need to run \code{./bootstrap.sh} to setup the configure script at this point
before continuing to the next step.}, and run:

\listitem{\code{./configure; make}}

You can now run SILE as is, uninstalled:

\listitem{\code{./sile examples/simple.sil}}

If all has gone well, this should produce a file \code{examples/simple.pdf}.

Most users of SILE will want to install the \code{sile} command and SILE’s
library files onto their system; this can be done with:

\listitem{\code{make install}}

Now the \code{sile} command is available from any directory.

\subsection{Notes for Windows users}

Some people have reported successfully running SILE on Windows in the
\code{mingw32} environment. At the moment there is no guaranteed recipe, but
hints may be found in \url{https://github.com/simoncozens/sile/issues/82}.

\section{Running SILE}

Let’s move to a new directory, and in a text editor, create the file
\code{hello.sil}. Copy in the content above and save the file. Now at your
command line run:

\listitem{\code{sile hello}}

\note{SILE will automatically provide the extension \code{.sil} to input files
if it is not provided by the user.}

Once again, this should produce an output file \code{hello.pdf}.
Congratulations—you have just typeset your first document with SILE!

\section{Let’s Do Something Cool}

In \code{examples/article-template.xml}, you will find a typical DocBook 5.0
article. Normally turning DocBook to print involves a curious dance of XSLT
processors, format object processors and/or strange LaTeX packages. But SILE
can read XML files and it also comes with a \code{docbook} class, which tells
SILE how to render (admittedly, a subset of) the DocBook tags onto a page.

Turning \code{examples/article-template.xml} into
\code{examples/article-template.pdf} is now as simple as:

\begin{verbatim}
\line
\% \em{./sile -I docbook examples/article-template.xml}
This is SILE \sileversion
Loading docbook
<classes/docbook.sil><examples/article-template.xml>[1] [2] [3]
\line
\end{verbatim}

The \code{-I} flag loads up a \em{class} before reading the input file; after
this has been loaded, the DocBook file can be read directly and its tags
interpreted as SILE commands.

In Chapter 10, we’ll look at how the \code{docbook} class works, and how you
can define processing expectations for other XML formats.

\chapter{SILE’s Input Language}

Let’s now go back and reconsider the first SILE file we saw:

\begin{verbatim}
\line
\\begin\{document\}
Hello SILE!
\\end\{document\}
\line
\end{verbatim}

\section{Defining the paper size}

A document starts with a \code{\\begin\{document\}} command and ends with
\code{\\end\{document\}}. In between, SILE documents are made up of two
elements: text to be typeset on the page, such as “Hello SILE!” in our example,
and commands.

The default paper size is A4, although each class may override this value. To
manually change the paper size, an optional argument may be added to the document
declaration:

\begin{verbatim}
\line
\\begin[papersize=letter]\{document\}
\line
\end{verbatim}

SILE knows about the ISO standard A, B and C series paper sizes using names a4,
b5, etc. Also available are the traditional sizes letter, note, legal,
executive, halfletter, halfexecutive, statement, folio, quarto, ledger,
tabloid, the series archA through archE, as well as DL, Comm10, Monarch, flsa,
flse, csheet, dsheet, and esheet.

If you need a paper size for your document which is not one of the standards,
then you can specify it by dimensions:

\code{papersize=\em{<basic length> x <basic length>}}.

SILE knows a number of ways of specifying lengths. A \code{<basic length>} as
mentioned above can be specified as a floating-point number followed by
a dimension abbreviation. Acceptable dimensions are printer’s points
(\code{pt}), millimeters (\code{mm}), centimeters (\code{cm}) or inches.
(\code{in}) For instance a standard B-format book can be specified
\code{papersize=198mm x 129mm}.

Once some of the basic document properties have been setup using these fixed
size units, other dimensions may be specificed relative to them using relative
units. Percentage units are available relative to the page width or height, the
current frame width or height, and the line width (\code{\%pw}, \code{\%ph},
\code{\%fw}, \code{\%fw}, and \code{\%lw} respectively). You can specify
lengths in terms of the current paragraph skip (\code{ps}) and baseline skip
(\code{bs}), which will make sense later on. Additional units are available
relative to the largest or smallest value of either axis
(\code{\%pmax}, \code{\%pmin}, \code{\%fmax, \code{\%fmin}}).

Later we will meet some other ways of specifying lengths to make them
stretchable or shrinkable.

\section{Ordinary text}

On the whole, ordinary text isn’t particularly interesting—it’s just typeset.

\note{TeX users may have an expectation that SILE will do certain things with
ordinary text as well. For instance, if you place two straight-backquotes
into a TeX document (like this: \code{``}) then TeX will magically turn that
into a double opening quote (“). SILE won’t do this. If you want a double
opening quote, you have to ask for one. Similarly, en- and em-dashes have to be
input as actual Unicode characters for en- and em-dashes, rather than the
pseudo-ligatures such as \code{--} or \code{---} that TeX later transforms to
the Unicode characters.}

There are only a few bits of cleverness that happen around ordinary text.

The first is that space is not particularly significant. If you write
\code{Hello {} {} SILE!} with three spaces, you get the same output as if you
write \code{Hello SILE!} with just one. Space at the beginning of a line will
be ignored.

Similarly, you can place a line break anywhere you like in the input file, and
it won’t affect the output because SILE considers each paragraph at a time and
computes the appropriate line breaks for the paragraph based on the width of
the line available. In other words, if your input file says

\begin{verbatim}
\line
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu
fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
culpa qui officia deserunt mollit anim id est laborum.
\line
\end{verbatim}

\noindent{}…you might not necessarily get a line break after ‘eiusmod’; you’ll
get a line break wherever is most appropriate. In the context of this document,
you’ll get:

\line
\nohyphenation{\examplefont{Lorem ipsum dolor sit amet, consectetur adipisicing
elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut
enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip
ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate
velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat
cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est
laborum.}}
\line

In other words, a line break is converted to a space.

\note{Sometimes this conversion is not what you want. If you don’t want single
line breaks to be converted to a space, use a comment character \code{\%} at
the end of a line to suppress the additional whitespace.}

When you want to end a paragraph, you need to input two line breaks in a row,
like this:

\begin{verbatim}
\line
Paragraph one.

Paragraph two.
This is not paragraph three.

This is paragraph three.
\line
\end{verbatim}

The second clever thing that happens around ordinary text is that a few—four,
in fact—characters have a special meaning to SILE. All of these will be
familiar to TeX users.

We’ve seen that a \em{backslash} is used to start a command, and we’ll look
into commands in more detail soon. \em{Left and right curly braces} (\code{\{},
\code{\}}) are used for grouping, particularly in command arguments. Finally,
a \em{percent sign} is used as a comment character, meaning that everything
from the percent to the end of the line is ignored by SILE. If you want to
actually typeset these characters, prepend a backslash to them: \code{\\\\}
produces ‘\\’, \code{\\\{} produces ‘\{’, \code{\\\}} produces ‘\}’, and
\code{\\\%} produces ‘\%’.

The third clever thing is SILE will automatically hyphenate text at the end of
a line if it feels this will make the paragraph shape look better. Text is
hyphenated according to the current language options in place. By default, text
is assumed to be in English unless SILE is told otherwise. In the Latin text
above, we turned hyphenation off.

The final clever thing is that, where fonts declare ligatures (where two or
more letters are merged into one in order to make them visually more
attractive), SILE automatically applies the ligature. So if you type
\code{affluent fishing} then, (depending on your font), your output might look
like: ‘\examplefont{affluent fishing}.’ If you specifically want to break up
the ligature, then insert an empty group (using the grouping characters
\code{\{} and \code{\}}) in the middle of the ligature: \code{af\{\}f\{\}luent
f\{\}ishing}: \examplefont{af{}f{}luent f{}ishing}. See the section on
\em{OpenType Features} for more information on how to control the display of
ligatures and other font features.

\section{Commands}

Typically (and we’ll unpack that statement later), SILE commands are made up of
a backslash followed by a command name, and a document starts with
a \code{\\begin\{document\}} command and ends with \code{\\end\{document\}}.

A command may also take two other optional components: some \em{parameters},
and an \em{argument}. The \code{\\begin} command at the start of the document
is an example of this.\footnote{Strictly speaking \code{\\begin} isn’t actually
a command but we’ll pretend that it is for now and get to the details in
a moment.}

\begin{verbatim}
\line
\\begin\{document\}
\line
\end{verbatim}

The parameters to a command are enclosed in square brackets and take the form
\code{\em{key}=\em{value}};\break{}multiple parameters are separated by commas
or semicolons, as in \code{[key1=value1,key2=value2,\dots]} Spaces around the
keys are not significant; we could equally write that as \code{[key1 = value1;
key2 = value2; \dots]}. If you need to include a comma or semicolon within the
value to a parameter, you can enclose the value in quotes: \code{[key1
= "value1, still value 1", key2 = value2; \dots]}.

The optional argument (of which there can only be at most one) is enclosed in
curly braces.\footnote{TeX users may forget this and try adding a command
argument “bare”, without the braces. This won’t work; in SILE, the braces are
always mandatory.} Because the argument is optional, there is a difference between this: \code{\\command\{args\}} (which is interpreted as a command with argument \code{args}) and \code{\\command \{args\}} (which is interpreted as a command with no arguments, followed by the word \command{args} in a new group).

SILE will ignore whitespace and newlines after a command.

Here are a few more examples of SILE commands:

\begin{verbatim}
\line
\\eject                                   \% A command with no parameters or argument

\\font[family=Times,size=10pt]            \% Parameters, but no argument

\\chapter\{Introducing SILE\}               \% Argument but no parameters

\\font[family=Times,size=10pt]\{Hi there!\} \% Parameters and argument
\line
\end{verbatim}%
\section{Environments}

Commands like \code{\\chapter} and \code{\\em} (emphasises text by making it
italic) are normally used to enclose a relatively small piece of text; a few
lines at most. Where you want to enclose a larger piece of the document, you
can use an \em{environment}; an environment begins with
\code{\\begin\{\em{name}\}} and encloses all the text up until the
corresponding \code{\\end\{\em{name}\}}. We’ve already seen an example, the
\code{document} environment, which must enclose the \em{entire} document.

Here is a secret: there is absolutely no difference between a command and
an environment. In other words, the following two forms are equivalent:

\begin{verbatim}
\line
\\font[family=Times,size=10pt]\{Hi there!\}

\\begin[family=Times,size=10pt]\{font\}
Hi there!
\\end\{font\}
\line
\end{verbatim}

However, in some cases the environment form of the command will be easier to
read and will help you to be clearer on where the command begins and ends.

\section{The XML Flavour}

While we’re on the subject of alternative forms, SILE can actually process its
input in a completely different file format. What we’ve seen so far has been
SILE’s “TeX-like flavor”, but if the first character of the input file is an
angle bracket (\code{<}) then SILE will interpret its input as an XML file. (If
it isn’t well-formed XML, then SILE will get very upset.)

Any XML tags within the input file will then be regarded as SILE commands, and
tag attributes are interpreted as command parameters; from then on, the two
file formats are exactly equivalent, with one exception: instead of
a \code{<document>} tag, SILE documents can be enclosed in \em{any} tag.
(Although \code{<sile>} is conventional for SILE documents.)

In other words, the XML form of the above document would be:

\begin{verbatim}
\line
<sile>
Hello SILE!
</sile>
\line
\end{verbatim}

Commands without an argument need to be well-formed self-closing XML tags (for
instance, \code{<break/>}), and commands with parameters should have
well-formed attributes. The example above, in XML flavor, would look like this:

\break
\begin{verbatim}
\line
<font family="Times" size="10pt">Hi there!</font>
\line
\end{verbatim}

We don’t expect humans to write their documents in SILE’s XML flavor—the
TeX-like flavor is much better for that—but having an XML flavor allows for
computers to deal with SILE a lot more easily. One could create graphical user
interfaces to edit SILE documents, or convert other XML formats to SILE.

However, there is an even smarter way of processing XML with SILE. For this,
you need to know that you can define your own SILE commands, which can range
from very simple formatting to fundamentally changing the way that SILE
operates. If you have a file in some particular XML format—let’s say it’s
a DocBook file—and you define SILE commands for each possible DocBook tag, then
the DocBook file becomes a valid SILE input file, as-is.

In the final two chapters, we’ll provide some examples of defining SILE
commands and processing XML documents.

\chapter{Some Useful SILE Commands}

We’re going to organise our tour of SILE by usage; we’ll start by giving
you the most useful commands that you’ll need to get started typesetting
documents using SILE, and then we’ll gradually move into more and more
obscure corners as the documentation progresses.

\section{Fonts}

The most basic command for altering the look of the text is the \code{\\font}
command; it takes two forms:

\noindent{}• \code{\\font[\em{parameters\dots}]\{\em{argument}\}}

\noindent{}• \code{\\font[\em{parameters\dots}]}

The first form sets the given argument text in the specified font; the
second form changes the font used to typeset text from this point on.

So, for instance:

\begin{verbatim}
\line
Small text

\\font[size=15pt]%
Big text!

\\font[size=30pt]\{Bigger text\}

Still big text!
\line
\end{verbatim}

\noindent{}produces

\begin{examplefont}%
\line%
Small text

\noindent\font[size=15pt]%
Big text!

\noindent\font[size=30pt]{Bigger text}

\noindent{}Still big text!
\line
\end{examplefont}

\font[size=11pt]%

As you can see, one possible attribute is \code{size}, which can be specified as
a SILE \code{<dimension>}. A \code{<dimension>} is like a \code{<basic length>}
(described above) but with a few extra possible dimensions which are relative
to either the size of the \em{current} font (as in ex units (\code{ex}), em units
(\code{em}), and en units (\code{en})) or the page, frame, or other dimention
(as in the percent of page width (\code{\%pw}), height (\code{\%ph}), etc.).

The full list of attributes to the \code{\\font} command are:

\medskip

\noindent{}• \em{size}: as above.

\noindent{}• \em{family}: the name of the font to be selected. SILE should know
about all the fonts installed on your system, so that fonts can be specified by
their name.

\noindent{}• \em{filename}: if a filename is supplied, SILE will use the font file provided rather than looking at your system’s font library.

\noindent{}• \em{style}: can be \code{normal} or \code{italic}.

\noindent{}• \em{weight}: a CSS-style numeric weight ranging from \font[weight=100]{100},
through \font[weight=200]{200},
\font[weight=300]{300},
\font[weight=400]{400},
\font[weight=500]{500},
\font[weight=600]{600},
\font[weight=700]{700},
\font[weight=800]{800} to
\font[weight=900]{900}. Not all fonts will support all weights (many just have
two), but SILE will choose the closest.

\noindent{}• \em{language}: The two letter (ISO639-1) language code for the
text. This will affect both font shaping and hyphenation.

\noindent{}• \em{script}: The script family in use. See the section “Hyphenation and Language”
below.
\medskip

It’s quite fiddly to be always changing font specifications manually; later
we’ll see some ways to automate the process. SILE provides the \code{\\em\{\dots\}}
command as a shortcut for \code{\\font[style=italic]\{\dots\}}.
There is no shortcut for boldface, because boldface isn’t good typographic
practice and so we don’t want to make it easy for you to make bad books.

\section{Document Structure}

SILE provides a number of different \em{class}es of document (similar to
LaTeX classes). By default, you get the \em{plain} class, which has very
little support for structured documents. There is also the \em{book} class,
which adds support for right and left page masters, running headers, footnotes,
and chapter, section and subsection headings.

To use the commands in this section, you will need to request the \em{book}
class by specifying in your \code{\\begin\{document\}} command ‘\code{[class=book]}’;
for example, the document you are currently reading begins with the command
\code{\\begin[class=book]\{document\}}.

\subsection{Chapters and Sections}

You can divide your document into different sections using the commands
\code{\\chapter\{\dots\}}, \code{\\section\{\dots\}} and \code{\\subsection\{\dots\}}.
The argument to each command is the name of the chapter or section respectively;
chapters will be opened on a new right-hand page, and the chapter name will form
the left running header. Additionally the section name and number will form the right running header.

\note{Chapters, sections and subsections will be automatically
numbered starting from 1; to alter the numbering, see the documentation for the
\code{counters} package in the next chapter. To produce an unnumbered chapter,
provide the parameter \code{[numbering=no]}.}

This subsection begins with the command \code{\\subsection\{Chapters and Sections\}.}

\subsection{Footnotes}

Footnotes can be added to a book with the \code{\\footnote\{\dots\}} command.\footnote{Like this: \code{\\footnote\{Like this.\}}} The
argument to the command will be set as a footnote at the bottom of the page;
footnotes are automatically numbered from 1 at the start of each chapter.

\section{Indentation and Spacing}

Paragraphs in SILE normally begin with an indentation (by default, 20 points
in width). To turn this off, you can use the \code{\\noindent} command at the
start of a paragraph. (This current paragraph doesn’t need to call \code{\\noindent}
because \code{\\section} and \code{\\chapter} automatically call it for the
text following the heading.) A \code{\\noindent} can be cancelled by following
it with an \code{\\indent}.

To increase the vertical space between paragraphs or other elements, the commands \code{\\smallskip}, \code{\\medskip} and \code{\\bigskip} are available to add a 3pt, 6pt and 12pt
gap respectively. There will be a \code{\\bigskip} after this paragraph.

\bigskip%
There are also some commands to increase the horizontal space in a line; from
the smallest to the largest, \code{\\thinspace} (1/6th of an em), \code{\\enspace}
(1 en), \code{\\quad} (1 em), and \code{\\qquad} (2em).

\begin{center}
You can center a paragraph of text by wrapping it in the \code{center} environment.
(\code{\\begin\{center\} \dots \\end\{center\}}). This paragraph is centered on the
page.
\end{center}

\section{Breaks}

SILE automatically determines line and page breaks; in later chapters we will
introduce some \em{settings} which can be used to tweak this process. However,
SILE’s plain class also provides some commands to help the process on its way.

Between paragraphs, the \code{\\break} command requests a \em{frame break} at the given location.
(The commands \code{\\framebreak} and \code{\\eject} are also available as synonyms.) Where there
are multiple frames on a page—for instance, in a document with multiple columns—the current frame will be ended and typesetting will recommence at the top of the next frame.
\code{\\pagebreak} (also known as \code{\\supereject}) is a more forceful variant,
and ensures that a new page is opened even if there are remaining frames on the page.
A less forceful variant is \code{\\goodbreak}, which suggests to SILE that this is
a good point to break a page. The opposite is \code{\\nobreak} which requests that,
if at all possible, SILE does not break at the given point. A neutral variant is
\code{\\allowbreak}, which allows SILE to break at a point that it would otherwise
not consider as suitable for breaking.

\em{Within} a paragraph, these commands have a different meaning. The \code{\\break} command requests a \em{line} break at the given location, and, \em{mutatis mutandis},
so do \code{\\goodbreak}, \code{\\nobreak} and \code{\\allowbreak}. If you want to be
absolutely sure that you are inhibiting a \em{page} break, you can say \code{\\novbreak}.

\begin{raggedright}
SILE normally fully-justifies text—that is, it tries to alter the spacing between
words so that the text stretches across the full width of the column.\footnote{This
does not mean that text will always exactly fill the width of the column. SILE will
choose line breaks and alter the spacing between words up to a certain extent, but
when it has done its best, it will typeset the least bad solution; this may involve
some of the words jutting slightly out into the margin.} An alternative
to full justification is ragged right margin formatting, where the spacing between
words is constant but the right hand side of the paragraph may not line up. Ragged
right is often used for children’s books and for frames with narrow columns such as
newspapers. To use ragged right formatting, enclose your text in a \code{\\begin\break\{raggedright\}} environment.
This paragraph is set ragged right.
\end{raggedright}

\begin{raggedleft}
Similarly, there is a \code{raggedleft} environment, in which the right-hand
margin of the paragraph is fixed, but the left-hand margin is allowed to vary. This
paragraph is set ragged left.\par
\end{raggedleft}

\section{Hyphenation and Language}

SILE hyphenates words based on its current language. (Language is set using the
\code{\\font} command above.) SILE comes with support for hyphenating a wide
variety of languages, and also aims to encode specific typesetting knowledge about
languages.

The default hyphen character is “-”, which can be tweaked by the \code{\\font}
parameter \code{hyphenchar}. It accepts a Unicode character or Unicode codepoint
in \code{[Uu]+<code>} or Hexadecimal \code{0[Xx]<code>} format – for instance
\code{\\font[family=Rachana,language=ml,hyphenchar=U+200C]}.

SILE comes with a special “language” called \code{und}, which has no hyphenation
patterns available. If you switch to this language, text will not be hyphenated.
The command \code{\\nohyphenation\{\dots\}} is provided as a shortcut for
\code{\\font[language=und]\{\dots\}}.

Aside from hyphenation, typesetting conventions differ from language to language.
SILE has basic support for typesetting most languages and scripts. (Again, let me
know if there are languages or scripts you are using that don’t work properly and
I will add support for them.)

In some cases, languages using the same alphabet
expect the text to be typeset in different ways. For instance, Sindhi and Urdu
users will expect the Arabic letter \em{heh} to combine with other letters in
different ways to standard Arabic shaping. In those cases, you should ensure that
you set the \code{language} and \code{script} options using the \command{\\font}
command appropriately:

\bidi-on
\begin{verbatim}
\line
Standard Arabic:
\\font[family=Scheherazade,language=ar,script=Arab]\{ه{}ه{}ه\};
then in Sindi:
\\font[family=Scheherazade,language=snd,script=Arab]\{ه{}ه{}ه\};
then in Urdu:
\\font[family=Scheherazade,language=urd,script=Arab]\{ه{}ه{}ه\}.
\line
\end{verbatim}

\begin{examplefont}
\line
Standard Arabic: \font[family=Scheherazade,language=ar,script=Arab]{ههه};
then in Sindi: \font[family=Scheherazade,language=snd,script=Arab]{ههه};
then in Urdu: \font[family=Scheherazade,language=urd,script=Arab]{ههه}.
\line
\end{examplefont}
\bidi-off

(A complete list of possible values for the \code{script} option can be found at
\url{http://www.simon-cozens.org/content/duffers-guide-fontconfig-and-harfbuzz})

\section{Including Other Files and Code}

To make it easier for you to author a large document, you can break your SILE document
up into multiple files. For instance, you may wish to put each chapter into a
separate file; you may wish to develop a file of user-defined commands (see
chapter 6) and keep this separate from the main body of the document. You
will then need the ability to include one SILE file from another.

This ability is provided by the \code{\\include} command. It takes one mandatory
parameter, \code{src=\em{<path>}}, which represents the path to the file. So
for instance, you may wish to write a thesis like this:

\begin{verbatim}
\line
\\begin[class=thesis]\{document\}
\\include[src=macros]
\\include[src=chap1]
\\include[src=chap2]
\\include[src=chap3]
\dots\par
\\include[src=endmatter]
\\end\{document\}
\line
\end{verbatim}

\code{\\include}s may be nested, in that file A can include file B which includes
file C.

SILE is written in the Lua programming language, and the Lua interpreter is
available at runtime. Just as one can run Javascript code from within a HTML
document using a \code{<script>} tag, one can run Lua code from within a SILE
document using a \code{\\script} command. (It looks better in XML-flavor.) This
command has two forms: \code{\\script[src=\em{<filename>}]} which includes a Lua
file, and \code{\\script\{\dots\}} which runs Lua code inline.

Doing anything interesting inline requires knowledge of the internals of SILE,
(thankfully the code is not that hard to read)
but to get you started, the Lua function \code{SILE.typesetter:typeset(\dots)}
will add text to a page, \code{SILE.call("\dots")} will call a SILE command,
and \code{SILE.typesetter:leaveHmode()} ends the current paragraph and outputs the
text. So, for example:

\begin{verbatim}
\line
\\begin\{script\}
  for i=1,10 do
    SILE.typesetter:typeset(i .. " x " .. i .. " = " .. i*i .. ". ")
    SILE.typesetter:leaveHmode()
    SILE.call("smallskip")
  end
\\end\{script\}
\line
\end{verbatim}

produces the following output:

\begin{examplefont}
\line
\begin{script}
  SILE.call("indent");SILE.typesetter:leaveHmode()
  for i=1,10 do
    SILE.typesetter:typeset(i .. " x " .. i .. " = " .. i*i .. ". ")
    SILE.typesetter:leaveHmode()
    SILE.call("smallskip")
  end
\end{script}
\line
\end{examplefont}

There is one notable caveat when embedding Lua code documents written with the
TeX-flavor markup. Since SILE has to first parse the TeX markup to find the
start and end of such script commands \em{without} understanding what's in
between, it is strictly necessary that no end tags appear inside the Lua code.
This means that for the environment block version (\code{\\begin\{script\}})
there must be no instances of \code{\\end\{script\}} inside the Lua code block,
even inside a string that would otherwise be valid Lua code, (e.g. if it was
inside a quoted string or Lua comment). The first instance of such and end tag
terminates the block and there is currently no way to escape it. For the inline
comand form (\code{\\script}) an exact matching number of open and closing
braces may appear in the Lua code — the next closing brace at the same level as
the opening brace will close the tag, even if it happens to be inside some
quoted string in the Lua code. Including any nuber of nested opening and closing
braces is possible as long as they are balanced.

\chapter{SILE Packages}

SILE comes with a number of packages which provide additional functionality.
In fact, the actual “core” of SILE’s functionality is very small and very
extensible, with most of the interesting features being provided by add-on
packages. SILE packages are written in the Lua programming language, and can
define new commands, change the way that the SILE system operates, or indeed
do anything that it’s possible to do in Lua.

As mentioned above, loading a package is done through the \code{\\script} command,
which runs Lua code. By convention packages live in the \code{packages/} subdirectory
of either your input file's location, your current working directory or SILE’s
installation directory. For instance, we’ll soon be talking about the
\code{grid} package, which normally can be found as
\code{/usr/local/lib/sile/packages/grid.lua}. To load this, we’d say:

\begin{verbatim}
\line
\\script[src=packages/grid]
\line
\end{verbatim}

\note{\notehead{How SILE locates files} SILE searches for paths in a variety of
directories: first, in the directory in which your input file is located,
then the current wording directory; next, if the environment variable
\code{SILE_PATH} is set, it will look in that directory; then it will look in
the standard installation directories \code{/usr/lib/sile} and
\code{/usr/local/lib/sile}. Unlike TeX, it does not descend into subdirectories
when looking for a file, and so if you have arranged your personal macro, class
or package files into subdirectories, you will need to provide a full relative
path to them.}

\section{image}

As well as processing text, SILE can also include images.

Loading the \code{image} package gives you the \code{\\img} command, fashioned
after the HTML equivalent. \code{\\img} takes the following parameters:
\code{src=\dots} must be the path to an image file;
you may also give \code{height=\dots} and/or \code{width=\dots} parameters
to specify the output size of the image on the paper. If the size parameters
are not given, then the image will be output at its ‘natural’ pixel size.

\begin{note}
With the libtexpdf backend (the default), the images can be in JPEG, PNG, EPS or PDF
formats; the Pango/Cairo backend only provides support for PNG images.
\end{note}

Here is a 200x243 pixel image output with \code{\\img[src=documentation/gutenberg.png]}:\par
\img[src=documentation/gutenberg.png]

\raggedright{
Here it is with (respectively)
\code{\\img[src=documentation/gutenberg.png,width=120px]},
\code{\\img[src=documentation/gutenberg.png,height=200px]}, and
\code{\\img[src=documentation/gutenberg.png,width=120px,height=200px]}:}

\img[src=documentation/gutenberg.png,width=120px]
\img[src=documentation/gutenberg.png,height=200px]
\img[src=documentation/gutenberg.png,width=120px,height=200px]

Notice that images are typeset on the baseline of a line of text, rather like
a very big letter.

\section{rules}
\package-documentation[src=packages/rules]

\section{color}
\package-documentation[src=packages/color]

\section{background}
\package-documentation[src=packages/background]

\section{rotate}
\package-documentation[src=packages/rotate]

\section{features}
\package-documentation[src=packages/features]

\section{unichar}
\package-documentation[src=packages/unichar]

\section{bidi}
\package-documentation[src=packages/bidi]

\section{pullquote}
\package-documentation[src=packages/pullquote]

\section{raiselower}

If you don’t want your images, rules or text to be placed along
the baseline, you can use the \code{raiselower} package to move them up
and down. (The \code{footnote} package uses this to superscript the
footnote reference numbers.)

It provides two simple commands, \code{\\raise} and \code{\\lower} which
both take a \code{height=\em{<dimension>}} parameter. They will respectively
raise or lower their argument by the given height. The raised or lowered
content will not alter the height or depth of the line.

Here is some text raised by \raise[height=3pt]{three points}; here is
some text lowered by \lower[height=4pt]{four points}.

The previous paragraph was generated by:

\begin{verbatim}
Here is some text raised by \\raise[height=3pt]\{three points\}; here is
some text lowered by \\lower[height=4pt]\{four points\}.
\end{verbatim}

\section{grid}
\grid[spacing=15pt]
In normal typesetting, SILE determines the spacing between lines of type
according to the following two rules:

\noindent• SILE tries to insert space between two successive lines so that their baselines
are separated by a fixed distance called the \code{baselineskip}.

\noindent• If this first rule would mean that the bottom and the top of the lines are less
than two points apart, then they are forced to be two points apart. (This distance
is configurable, and called the \code{lineskip})

The second rule is designed to avoid the situation where the first line has a long
descender (letters such as g, q, j, p, etc.) which abuts a high ascender on the second
line. (k, l, capitals, etc.)

In addition, the \code{baselineskip} contains a certain amount of ‘stretch’, so that
the lines can expand if this would help with producing a page break at an optimal
location, and similarly spacing between paragraphs can stretch or shrink.

The combination of all of these rules means that a line may begin at practically any
point on the page.

An alternative way of typesetting is to require that lines begin at fixed points on
a regular grid. Some people prefer the ‘color’ of pages produced by grid typesetting,
and the method is often used when typesetting on very thin paper as lining up the
lines of type on both sides of a page ensures that ink does not bleed through from
the back to the front. Compare the following examples: on the left, the lines are
guaranteed to fall in the same places on the recto (front) and the verso (back) of
the paper; on the right, no such guarantee is made.

\img[src=documentation/grid-1.png,height=130]
\img[src=documentation/grid-2.png,height=130]

The \code{grid} package alters the way that the SILE’s typesetter operates so that
the two rules above do not apply; lines are always aligned on a fixed grid, and
spaces between paragraphs etc. are adjusted to conform to the grid. Loading the package
adds two new commands to SILE: \code{\\grid[spacing=\em{<dimension>}]} and \code{\\no-grid}.
The first turns on grid typesetting for the remainder of the document; the second turns it off again.

At the start of this section, we issued the command \code{\\grid[spacing=15pt]} to
set up a regular 15-point grid. Here is some text typeset with the grid set up:

\smallskip
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
\smallskip

And here is the same text after we issue \code{\\no-grid}:

\no-grid\smallskip
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non
proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

\section{linespacing}
\package-documentation[src=packages/linespacing]

\section{verbatim}
\script{SILE.process(SILE.inputs.TeXlike.docToTree(require("packages/verbatim")))}

\section{font-fallback}
\package-documentation[src=packages/font-fallback]

\section{Packages usually used by other packages}

In addition, there are certain packages that you \em{probably} don’t need
to use directly, as their main job is to provide more
basic functionality to other packages and classes. Classes such as the
\code{book} class compose functionality from different auxiliary packages.

\subsection{footnotes}

For instance, we’ve seen that the \code{book} class allows you to add
footnotes to text with the \code{\\footnote} command. This command is
actually provided by the \code{footnotes} package. The \code{book}
class loads up the package and tells it where to put the footnotes
that are typeset, and the \code{footnotes} package takes care of
formatting the footnotes. It does this by using a number of other
packages that we will describe below.

\subsection{counters}
\package-documentation[src=packages/counters]

\subsection{pdf}
\package-documentation[src=packages/pdf]

\subsection{frametricks}

As we mentioned in the first chapter, SILE uses frames as an indication
of where to put text onto the page. The \code{frametricks} package assists
package authors by providing a number of commands to manipulate frames.

The most immediately useful is \code{\\showframe}. This asks the output
engine to draw a box and label around a particular frame. It takes an optional
parameter \code{id=\em{<frame id>}}; if this is not supplied, the current
frame is used. If the ID is \code{all}, then all frames declared by the
current class are displayed.

\breakframevertical\par
The command \code{\\breakframevertical} breaks the current frame in two
at the specified location into an upper and lower frame. If the frame initially had the ID
\code{main}, then \code{main} becomes the upper frame (before the command)
and the lower frame (after the command) is called \code{main_}. We just
issued a \code{\\breakframevertical} command at the start of this paragraph,
and now we will issue the command \code{\\showframe}. \showframe As you can
see, the current frame is called
\code{\script{SILE.typesetter:typeset(SILE.typesetter.frame.id)}}
and now begins at the start of the paragraph.

Similarly, the \code{\\breakframehorizontal} command breaks the frame in two
into a left and a right frame.
The command takes an optional argument \code{offset=<dimension>}, specifying
where on the line the frame should be split. If it is not supplied, the
frame is split at the current position in the line.

The command \code{\\shiftframeedge} allows you to reposition the current
frame left or right. It takes \code{left=} and/or \code{right=} parameters,
which can be positive or negative dimensions. It should only be used at the
top of a frame, as it reinitializes the typesetter object.

\float[bottomboundary=3ex]{\font[size=60pt]{C}}ombining all of these commands, the \code{\\float}
command breaks the current frame, creates a small frame to hold a floating
object (like the dropcap at the start of this sentence), flows text into
the surrounding frame, and then, once text has descended past the floating object,
moves the frame back into place again. It takes two optional parameters, \code{bottomboundary=\em{<dimension>}} and/or \code{rightboundary=\em{<dimension>}}, which
open up additional space around the frame. At the start of this paragraph, I issued
the command \code{\\float[bottomboundary=5pt]\{\\font[size=60pt]\{C\}\}}.

Finally, it’s possible to define frames such as sidebars which are not connected
to the main text flow of a page. We’ll see how to do that in a later chapter, but
this raises the obvious question: if they’re not part of the text flow, how do we
get stuff into them? \code{frametricks} provides the \command{\\typeset-into}
command, which allows you to write text into a specified frame:

\begin{verbatim}
\line
\\typeset-into[frame=sidebar]\{ ... frame content here ... \}
\line
\end{verbatim}

\subsection{insertions}

The \code{footnotes} package works by taking auxiliary material (the
footnote content), shrinking the current frame and inserting it into the
footnote frame. This is powered by the \code{insertions} package; it doesn’t
provide any user-visible SILE commands, but provides Lua functionality to
other packages. TeX wizards may be interested to realise that insertions are
implemented by an external add-on package, rather than being part of the SILE core.

\subsection{twoside}

The \code{book} class described in chapter 4 sets up left and right mirrored
page masters; the \code{twoside} package is responsible for swapping between
the two left and right frames, running headers and so on. It has no user-serviceable
parts.

\subsection{masters}

The masters functionality is also itself an add-on package. It allows a class to
define sets of frames and switch between them either temporarily or permanently.
It defines the commands \command{\\define-master-template} (which is pattern
on the \command{\\pagetemplate} function we will meet in chapter 8), \command{\\switch-master}
and \command{\\switch-master-one-page}. See \code{tests/masters.sil} for more
about this package.

\subsection{infonode}

\note{This package is only for class designers.}

While typesetting a document, SILE first breaks a paragraph into lines, then
arranges lines into a page, and later outputs the page. In other words,
while it is looking at the text of a paragraph, it is not clear what page
the text will eventually end up on. This makes it difficult to produce
indexes, tables of contents and so on where one needs to know the page number
for a particular element.

To get around this problem, the \code{infonode} allows you to insert \em{information
nodes} into the text stream; when a page is outputted, these nodes are collected into
a list, and a class’s output routine can examine this list to determine which nodes
fell on a particular page. \code{infonode} provides the \code{\\info} command
to put an information node into the text stream; it has two required parameters,
\code{category=} and \code{value=}. Categories are used to group similar sets of
node together.

As an example, when typesetting a Bible, you may wish to display which range
of verses are on each page as a running header. During the command which starts
a new verse, you would insert an information node with the verse reference:

\begin{verbatim}
\line
SILE.Commands["info"]({ category = "references", value = ref }, {})
\line
\end{verbatim}

During the \code{endPage} method which is called at the end of every page,
we look at the list of “references” information nodes:

\begin{verbatim}
\line
local refs = SILE.scratch.info.thispage.references
local runningHead = SILE.shaper.shape(refs[1] .. " - " .. refs[#refs])
SILE.typesetNaturally(rhFrame, runningHead);
\line
\end{verbatim}

\subsection{inputfilter}
\package-documentation[src=packages/inputfilter]

\chapter{SILE Macros and Commands}

\noindent One of the reasons that we use computers is that they are
very good at doing repetitive jobs for us, so that we don’t have to.
Perhaps the most important skill in operating computers, and particularly
in programming computers, is noticing areas where an action is being
repeated, and allowing the computer to do the work instead of the human.
In other words, Don‘t Repeat Yourself.

The same is true in operating SILE. After you have been using the
system for a while, you will discover that there are patterns of input
that you need to keep entering again and again.

\section{A simple macro}

\define[command=SILE]{S\lower[height=0.5ex]{I}L\glue[width=-.2em]\raise[height=0.6ex]{\font[size=0.8em]{E}}}
For instance, let’s suppose that we want to design a nice little
“bumpy road” logo for SILE. (Afficionados of T\glue[width=-.1667em]\lower[height=0.5ex]{E}\glue[width=-.125em]X and friends will be familar with the concept of
bumpy road logos.) Our logo will look like this: \SILE. It’s not a great
logo, but we’ll use it as \SILE’s logo for the purposes of this section.

To typeset
this logo, we need to ask \SILE to: typeset an ‘S’; typeset an ‘I’ lowered by
a certain amount (half an ex, as it happens); typeset an ‘L’; walk backwards
along the line a tiny bit; typeset a smaller-sized ‘E’ raised by a certain amount.

In \SILE code, that looks like:

\begin{verbatim}
\line
S\\lower[height=0.5ex]\{I\}L\\glue[width=-.2em]\\raise[height=0.6ex]\{\\font[size=0.8em]\{E\}\}
\line
\end{verbatim}

(Don‘t worry about the \code{\\glue} command for the moment; we’ll come back
to that later.)

We’ve used our logo four times already in this chapter, and we don‘t want to
have to input that whole monostrosity each time we do so. What we would like
to do is tell the computer “this is \SILE’s logo; each time I enter \code{\\SILE},
I want you to interpret that as \code{S\\lower[height=0.5ex]\{I\}L\\glue[width=-.2em]\\raise[height=0.6ex]\{\\font[size=0.8em]\{E\}\}}”.

In other words, we want to define our own commands.

SILE\footnote{Let’s give up on the logo at this point.} allows you to define
your own commands in two ways. The simplest commands of all are those like
\code{\\SILE} above: “when I write \code{\\x}, I
want SILE to pretend that I had written \code{X \\Y Z} instead.” These are
called \em{macros}, and the process of pretending is called \em{macro expansion}.

You can define these kinds of macros within a SILE file itself. In this
very file, we entered:

\begin{verbatim}
\line
\\define[command=SILE]\{
S\\lower[height=0.5ex]\{I\}L\\glue[width=-.2em]\\raise[height=0.6ex]\{\\font[size=0.8em]\{E\}\}
\}
\line
\end{verbatim}

We're using the built-in SILE command \code{\\define}. \code{\\define} takes
an option called \code{command}; its value is the name of the command we
are defining. The content of the \code{\\define} command is a series of SILE instructions to
be executed when the command is used.

\begin{note}
At this point it’s worth knowing the precise rules for allowable
names of SILE commands.

Commands in XML-flavor input files must be allowable XML tag names or else your
input files will not be well formed. Command names in TeX-flavor input files
may consist of any number of alphanumeric characters, hyphens or colons.
Additionally, any single character is a valid TeX-flavor command name. (Hence
\code{\\\\} for typesetting a backslash.)

When it comes to defining commands, commands defined by an XML-flavor file can actually have
any name that you like—even if they are not accessible from XML-favour! (You may define oddly-named commands in a XML-flavor SILE file
and then use them in a TeX-flavor SILE file.) Commands defined in TeX-flavor
obviously have to have names which are valid parameter values, or else they will
not parse correctly either; parameter values happen to consist
of any text up until the nearest comma, semicolon or closing square bracket.
\end{note}

\section{Macro with content}

Now let’s move on to the next level; sometimes you will want to create
commands which are not simply replacements, but which have arguments of their
own. As an example, let’s say we use the \code{color} package to turn a bit of
text red \color[color=red]{like this}. The usual way to do that is to say

\begin{verbatim}
\line
\\color[color=red]\{like this\}
\line
\end{verbatim}

However, we‘re not always going to want to be highlighting the words ‘\code{like this}’.
We might want to be able to highlight other text instead. We need the ability
to wrap the command \code{\\color\break{}[color=red]\{ ... \}} around our chosen content.
In other words, we want to be able to define our own commands which take arguments.

The way we express this in SILE is with the \code{\\process} command. \code{\\process}
is only valid within the context of a \code{\\define} command (you’ll mess everything
up if you try using it somewhere else), and it
basically means ‘do whatever you were planning to do with the arguments to this
command.’ So if we want to a command which makes things red, we can say:

\begin{verbatim}
\line
\\define[command=red]\{\\color[color=red]\{\\process\}\}

…

Making things red is a \\red\{silly\} way to emphasise text.
\line
\end{verbatim}

\begin{note}
You can’t call \code{\\process} more than once within the same macro.

In the definition of the \code{\\chapter} command, we want to 1) display the
chapter name in a big bold font, and 2) use the chapter name as the left
running header. If you try writing the \code{\\chapter} command as a macro,
you will get stuck—once you’ve \code{\\process}ed the chapter name to display
it in bold, you won’t be able to process it again to set the running header.

So the \code{\\chapter} command cannot be written as a simple macro. The other
way to implement your own commands is to write them in the Lua programming
language, which is what happens for \code{\\chapter}. We will see how to do
this in later chapters.

The \code{\\define} command really is meant to be used just for simple things.
\end{note}

\section{Nesting macros}

That said, one thing you can do is to call a macro within a macro. This should
be obvious, because a macro is just a replacement for the current processing
step—when SILE reads a macro command, it behaves as if you had entered the
definition of the macro instead, and of course such a definition can contain
other commands.

So it is possible even within the simple scope of macro processing to achieve
quite a lot of automation.

For instance, within this book, there have been a number of notes—italicized
paragraphs between two heavy lines with a left margin. These have been typeset
with the \code{\\note} command; this is not a built-in command but a macro
specified within the \code{documentation/macros.sil} file included by this document.
Here is one way to define \code{\\note}, in XML flavour:

\begin{verbatim}
\line
<define command="line">
  <par/><smallskip/><noindent/>
  <hrule width="450pt" height="0.3pt"/><par/>
  <novbreak/><smallskip/><novbreak/>
</define>

<define command="narrower">
  <set parameter="document.lskip" value="24pt"/>
  <process>
  <set parameter="document.lskip" value="0pt"/>
</end>

<define command="notefont"><font style="italic" size="10pt"><process/></font></notefont>

<define command="note">
  <narrower>
    <line/>
    <notefont><process/></notefont>
    <line/>
  </narrower>
</define>
\line
\end{verbatim}

The only command we have not yet met here is \code{\\set}, which we will now
investigate.

\chapter{SILE Settings}

As well as commands, SILE offers a variety of knobs and levers which affect
how it does its job. Changing these parameters can have anything from a
subtle to a dramatic effect on the eventual document. External packages may
declare their own settings (although none of the packages which ship with
SILE happen to do so), but here we will run through the settings which are
built into the SILE system itself.

Settings in SILE are \em{namespaced} so that 1) the name of the setting gives you
some kind of clue as to what area of the system it will affect, and 2) packages
can define their own settings without worrying that they will be interfering
with other packages or the SILE internals. Namespacing of settings takes the
form \code{\em{area.name}}—so for instance, \code{typesetter.orphanpenalty} is
the setting which changes how the typesetter penalizes orphan (end-of-paragraph)
lines.

The interface to changing settings from within a SILE document is the
\code{\\set} commmand. It takes two options: a \em{parameter} option which
expresses which setting is being changed, and a \em{value} option which expresses
the value to which the setting is being changed. As an example:

\begin{verbatim}
\line
\\set[parameter=typesetter.orphanpenalty, value=250]
\line
\end{verbatim}

If the \code{\\set} command is provided with an argument, then the change of
setting is localised to the content of the argument. In other words, this code:

\begin{verbatim}
\line
\\set[parameter=typesetter.orphanpenalty, value=250]\{ \\lorem \}
\line
\end{verbatim}

will change the orphan penalty to 250, typeset 50 words of dummy text, and then
return the orphan penalty to its previous value.

Now, let’s begin looking at what each of the built-in settings does, starting
from the most obvious and moving towards the most subtle.

\section{Spacing Settings}

In our \code{\\note} example, we saw the setting \code{document.lskip}.
This is a \em{glue} parameter which is added to the left side of every line.
Setting this to a positive length effectively increases the left margin of
the text. Similarly, \code{document.rskip} adds some space to the right side of
every line.

Note that these skip settings are not the same as page margins. The
\code{document.lskip} and \code{document.rskip} values are applied inside of the
current frame and are relative to the edge of the frame, not to the edge of the
page. They are best used for temporary adjustments to the margins relative to the
normal margins, such as to indented a pull-quote. They can also be negative,
pulling the effective margin outside of the current frame.

\begin{note}
\notehead{Glue} A \em{glue} parameter is slightly different from an ordinary dimensioned length. Glue basically means ‘space,’ but as well as signifying a length,
it also has two additional optional components: \em{stretch} and \em{shrink},
specified as \code{<dimension> plus <dimension> minus <dimension>}. The
first dimension is the basic length; the stretch is the maximum length that
can be added to it, and the shrink is some length that can be taken away from it.
For instance, \code{12pt plus 6pt minus 3pt} specifies a space that would
ideally by 12 points, but can expand or contract from a minimum of 9 points to a maximum
of 18 points.
\end{note}

Let’s think about how the \code{center}ing environment is implemented. First,
we will add incredibly stretchable glue to the left and right margins, like so:

\begin{verbatim}
\line
\\set[parameter=document.lskip,value=0pt plus 100000pt]
\\set[parameter=document.rskip,value=0pt plus 100000pt]
\line
\end{verbatim}

This produces the following:

\medskip%
\begin{examplefont}%
\set[parameter=document.lskip,value=0pt plus 500pt]% Lying for didactic purposes
\set[parameter=document.rskip,value=0pt plus 500pt]
\set[parameter=document.spaceskip,value=0.5en plus 120pt minus 0.3en]

Here is some text which is almost centered. However, there are three problems:
first, the normal paragraph indentation is applied, meaning the first line of
text is indented.
Second, the space between words is stretchable, meaning that the lines are
stretched out so they almost seem justified.
Finally, by default SILE adds very large glue at the
end of each paragraph so that when the text is justified, the spacing of the
last line is not stretched out of proportion. This makes the centering of the
last line look a bit odd. We will deal with these three issues in the following
paragraphs.\par
\end{examplefont}
\medskip

\set[parameter=document.lskip,value=0pt]%
\set[parameter=document.rskip,value=0pt]%
\set[parameter=document.spaceskip]%
The indentation at the start of each paragraph is controlled by the
setting \code{document.parindent}; this is a glue parameter, and by default it’s
set to 20pt with no stretch and shrink. Actually, the amount added to the
start of the paragraph is \code{current.parindent}. After each paragraph,
\code{current.parident} is reset to the value of \code{document.parindent}. The
\code{\\noindent} command works by setting \code{current.parindent} to zero.

\medskip%
\set[parameter=current.parindent,value=-20pt]%
\set[parameter=document.lskip,value=20pt]%
How would you make a paragraph like this with a ‘hanging’ indentation? We’ve
set the \code{document.lskip} to 20 points, and the \code{current.parindent} to
\em{minus} 20 points. (In other words, we called:\break\code{\\set[parameter=document.lskip,value=20pt]} and \code{\\set[parameter=current.parindent,\break{}value=-20pt]}.)

\medskip%
\set[parameter=document.lskip,value=0pt]%
The space between \em{paragraphs} is set with the glue parameter
\code{document.parskip}. It’s normally set to five points with one point of stretchability.

As we mentioned in the section on grid typesetting, the rules for spacing between
\em{lines} within a paragraph is determined by two
rules. Let’s reiterate those rules now in terms of settings:

\noindent• SILE tries to insert space between two successive lines to make their
baselines exactly \code{document.baselineskip} apart.

\noindent• If this first rule would mean that the bottom and the top of the lines are less
than \code{document.lineskip} apart, then they are forced to be \code{document.lineskip} apart.

\note{This linebreaking method is fiddly, and book designers may prefer to
work with the tools provided by the \code{linespacing} package.}

The final spacing setting is \code{document.spaceskip}. Normally the size of
the space between \em{words} is determined by the width of the space character
in the current font. To help with justifying the text, the spaces are shrinkable
and stretchable. Specifically, if the width of a space in the current font settings is \code{<space>}, then the default value of \code{document.spaceskip} is
\code{1.2 <space> plus 0.5 <space> minus 0.333 <space>}\footnote{This somewhat
arbitrary convention is arguably a bug.}.

If you want to set it explicitly, you can set the
\code{document.spaceskip} setting. If you want to go back to the default (measuring
the space character of the font), then you need to \em{unset} the setting. To
unset it, just call \code{\\set} with no \code{value} parameter:
\code{\\set[parameter=document.spaceskip]}.

\section{Typesetter settings}

The settings which affect SILE’s spacing controls have the most obvious effect
on a document; the typesetter itself has some knobs that can be twiddled:

\code{typesetter.widowpenalty} and \code{typesetter.orphanpenalty}\footnote{TeX
users, please notice the renaming.} affect how strongly SILE is averse to
leaving stray lines at the start and end of pages. A \em{widow} happens when
a page is broken leaving one line at the bottom of a page; an \em{orphan} line
is the last line in a paragraph broken off at the top of the page. By default,
the \em{penalty} attached to breaking the page at one of these places is 150
penalty points. This value can be any number up to \code{10000}, which means
“never break at this point.”

\set[parameter=typesetter.parfillskip,value=0pt]
SILE automatically inserts a piece of massively stretchable
glue at the end of each paragraph; without this, the justification algorithm
would apply justification to the entire paragraph, including the last line,
and produce a fully justified paragraph. (Normally we want the last line of a justified paragraph to be
left-aligned.)
The size of this glue is defined in the setting
\code{typesetter.parfillskip}. Its default value is \code{0pt plus 10000pt} but
for this current paragraph, we have unset it.

Now we can finally complete our implementation of centering:

\begin{verbatim}
\line
\\set[parameter=document.lskip,value=0pt plus 100000pt]
\\set[parameter=document.rskip,value=0pt plus 100000pt]
\\set[parameter=document.spaceskip,value=0.5en]
\\set[parameter=current.parindent,value=0pt]
\\set[parameter=document.parindent,value=0pt]
\\set[parameter=typesetter.parfillskip,value=0pt]
\line
\end{verbatim}

\medskip
\noindent\begin{examplefont}%
\set[parameter=document.lskip,value=0pt plus 100000pt]%
\set[parameter=document.rskip,value=0pt plus 100000pt]%
\set[parameter=document.spaceskip,value=0.5en]%
\set[parameter=current.parindent,value=0pt]%
\set[parameter=typesetter.parfillskip,value=0pt]

And this is (more or less) how the \code{center} environment is defined in
the plain class: we make the margins able to expand but the spaces not able
to expand; we turn off indenting at the start of the paragraph, and we turn
off the filling glue at the end of the paragraph.
\par
\end{examplefont}

\medskip

\set[parameter=typesetter.parfillskip,value=0pt plus 10000pt]

\section{Linebreaking settings}

SILE’s linebreaking algorithm is lifted entirely from TeX, and so maintains the
same level of customizability as TeX. Here is a quick run-down of the settings
applicable to the line-breaking algorithm. You are expected to know what you
are doing with these.

\noindent• \code{linebreak.tolerance}: How bad a breakpoint is before it is rejected
by the algorithm. (Default: 500)

\noindent• \code{linebreak.pretolerance}: If there are no breakpoints better than
this, the paragraph is considered for hyphenation. (Default: 100)

\noindent• \code{linebreak.adjdemerits}: Additional demerits which are accumulated in the course of paragraph building when two consecutive lines are visually incompatible. In these cases, one line is built with much space for justification, and the other one with little space. (Default: 10000)

\noindent• \code{linebreak.looseness}: How many lines the current paragraph should
be made longer than normal. (Default: 0)

\noindent• \code{linebreak.prevGraf}: The number of lines in the paragraph last added to the vertical list.

\noindent• \code{linebreak.emergencyStretch}: Assumed extra stretchability in lines of a paragraph. (Default: 0)

\noindent• \code{linebreak.linePenalty}: Penalty value associated with each line break. (Default: 10)

\noindent• \code{linebreak.hyphenPenalty}: Penalty associated with break at a hyphen. (Default: 50)

\noindent• \code{linebreak.doubleHyphenDemerits}: Penalty for consecutive lines ending with a hyphen. (Default: 10000)

\section{Settings from Lua}

Most of the time you will not be fiddling with these settings at the SILE layer,
because complex layout commands are expected to be implemented in Lua. The
following SILE functions access the settings system from inside Lua:

\noindent• \code{SILE.settings.set(\em{<parameter>}, \em{value})}: sets a setting.

\note{
You should note that, while in the SILE layer, the \code{\\set} command does its
best to turn the textual description of a type into the appropriate Lua type
for the value. \code{SILE.settings.set} does not do that; it expects the value
to be of the appropriate type: lengths need to be a \code{SILE.Length} object,
glue must be \code{SILE.Glue} and so on.}

\noindent• \code{SILE.settings.get(\em{<parameter>})}: retrieves the current
setting of the parameter.

\noindent• \code{SILE.settings.temporarily(\em{function})}: Saves all settings,
runs the function and then restores all settings afterwards.

\noindent• \code{SILE.settings.declare(\em{<specification>})}: Declares a new
setting. See the base settings in \code{settings.lua} for examples of how to
call this. A class or package should namespace its settings with \code{<package>.<setting>}.

\chapter{The Nitty Gritty}

\noindent{}We are finally at the bottom of our delve into SILE‘s commands
and settings. Here are the basic building blocks out of which all of the
other operations in SILE are created; in fact, they are the basic
building blocks of SILE's operation.

\note{At this point, it is expected that
you are a class designer, and will be able to follow the details of how
SILE implements these commands and features; we will also explain how to
interact with these components at the Lua level.}

\section{Boxes, Glue and Penalties}

SILE’s job is, looking at it in very abstract terms, all about arranging
little boxes on a page. Some of those boxes have letters in them, and
those letters are such-and-such a number of points wide and
such-and-such a number of points high;
some of the boxes are empty but are there just to take up space; when a
horizontal row of boxes has been decided (i.e. when a line break is determined)
then the whole row of boxes is put into another box and the vertical list
of boxes are then arranged to form a page.

Conceptually, then, SILE knows about a few different basic components:
horizontal boxes (such as a letter); horizontal glue (the stretchable, shrinkable
space between words); vertical boxes (a line of text); vertical glue (the space
between lines and paragraphs); and penalties (information about where and when
not to break lines and pages).\footnote{Additionally there are three more types of box
that SILE cares about: N-nodes, unshaped nodes, and discretionaries.}

The most immediately useful of these are horizontal and vertical glue. It
is possible to explicitly add horizontal and vertical glue into SILE‘s processing
stream using the \code{\\glue} and \code{\\skip} commands. These take a
\code{width} and a \code{height} parameter respectively, both of which are
glue dimensions. So, for instance, the \code{\\smallskip} command is
the equivalent of \code{\\skip[height=3pt plus 1pt minus 1pt]}; \code{\\thinspace}
is defined as being \code{\\glue[width=0.16667em]}.

Similarly, there is a \code{\\penalty} command for inserting penalty nodes;
\code{\\break} is defined as \code{\\penalty[penalty=-10000]} and \code{\\nobreak}
is \code{\\penalty[penalty=10000]}.

You can also create horizontal and vertical boxes from within SILE. One obvious
reason for doing so would be to explicitly avoid material being broken up by a
page or line break; another reason for doing so would be that once you box some
material up, you then know how wide or tall it is. The \code{\\hbox} and \code{\\vbox}
commands put their contents into a box; when called from Lua, they also \em{return}
the new box.

\subsection{From Lua}

SILE’s Lua interface contains a \code{nodefactory} for creating boxes and glue.
Before we get into that, however, you need to know that glue measurements in SILE
should always be specified in terms of \code{SILE.length} objects; these are
“three-dimensional” lengths, in that they consist of a base length plus stretch and
shrink. To construct a \code{SILE.length}:

\begin{verbatim}
\line
local l = SILE.length.new(\{ length = x, stretch = y, shrink = z\})
\line
\end{verbatim}

Now we can construct horizontal and vertical glue:

\begin{verbatim}
\line
local glue  = SILE.nodefactory.newGlue (\{ width =  l\})
local vglue = SILE.nodefactory.newVglue(\{ height = l\})
\line
\end{verbatim}

SILE’s typesetting is organised by the \code{SILE.typesetter} object; it maintains
two queues of material that it is working on—the node queues (\code{SILE.typesetter.state.nodes}) contains new horizontal boxes and glue that are going to be broken up into lines soon; and the output queue (\code{SILE.typesetter.state.outputQueue}) which consists
of vertical material (lines) which are going to be broken up into pages. Line breaking
and page breaking happens when the typesetter moves between horizontal and vertical
mode; you can force this to happen by calling the function \code{SILE.typesetter:leaveHmode()}.
The SILE-level command for forcing a paragraph end is \code{\\par}.

So, if you want to manually add a vertical space to the output, first ensure that material in the current paragraph
has been all properly boxed-up and moved onto the output queue
by calling
\code{SILE.typesetter:leaveHmode()}; then add your
desired glue to the output queue.

Adding boxes and glue to the typesetter’s queues is
such a common operation that the typesetter has some utility methods to construct
the nodes and add them for you:

\begin{verbatim}
\line
SILE.typesetter:leaveHmode()
SILE.typesetter:pushVglue(\{ height = l \})
\line
\end{verbatim}

Adding boxes yourself is a little more complicated, because boxes need to
know how to display themselves on the page. To facilitate this, they normally
store a \code{value} and an \code{outputYourself} member function. For instance,
the \code{image} package actually does something very simple; it adds a horizontal
box to the node queue which knows the width and height of the image, the source,
and instructions to the output engine to display the image:

\begin{verbatim}
\line
SILE.typesetter:pushHbox(\{
  width= \dots,
  height= \dots,
  depth= 0,
  value= options.src,
  outputYourself= function (this, typesetter, line)
    SILE.outputter.drawImage(this.value,
      typesetter.frame.state.cursorX, typesetter.frame.state.cursorY-this.height,
      this.width,this.height
    );
    typesetter.frame:advanceWritingDirection(this.width)
end\});
\line
\end{verbatim}

Adding horizontal and vertical penalties to the typesetter’s queues is similarly
done with the \code{SILE.typesetter:pushPenalty(\{penalty = x\})} and
\code{SILE.typesetter:pushVpenalty(\{penalty = y\})} methods.

\section{Frames}

As we have previously mentioned, SILE arranges text into frames on the page.
The overall layout of a page including the apparent margins between content
and the page edge and other content regions is controlled by defining the
position of the frame or frames into which the content will be flowed.

Normally those frames are defined by your document class, but you can actually
create your own frames on a per-page basis using the \code{\\pagetemplate}
and \code{\\frame} commands. There are very few situations in which you will
actually want to do this, but if you can understand this, it will help you
to understand how to define your own document classes.

For instance, in a couple of pages time we’re going to implement a two-column layout.
SILE uses a \em{constraint solver} system to declare its frames, which means
that you can tell it how the frames relate to each other and it will compute
where the frames should be physically placed on the page.

Here is how we will go about it. We need to start with a page break, because
SILE will not appreciate you changing the page layout after it’s started to
determine how to put text onto that page.\footnote{Of course you can use
the \code{frametricks} package to get around this limitation—split the current
frame and start fiddling around with the positions of the new frames that
\code{frametricks} created for you.} How do we get to the start of a new
page? Remember that the \code{\\eject} (another word for \code{\\break} in
vertical mode) only adds a penalty to the end of the output queue; page breaking
is triggered when we leave horizontal mode, and the way to do that is \code{\\par}.
So we start with \code{\\eject\\par} and then we will begin a \code{\\pagetemplate}.
Within \code{\\pagetemplate} we need to tell SILE which frame to being typesetting
onto.

\begin{verbatim}
\line
\\eject\\par
\\begin[first-content-frame=leftCol]\{pagetemplate\}
\line
\end{verbatim}

Now we will declare our columns. But we’re actually going to start by declaring
the gutter first, because that’s something that we know and can define; we’re
going to stipulate that the gutter width will be 3\% of the page width:

\begin{verbatim}
\line
\\frame[id=gutter,width=3\%pw]
\line
\end{verbatim}

\begin{note}%
Declarations of frame dimensions are like ordinary SILE \code{<dimension>}s,
except with three additional features:

\noindent• You can refer to properties of other frames using the \code{top()},
\code{bottom()}, \code{left()}, \code{right()}, \code{height()} and \code{width()}
functions. These functions take a frame ID. SILE magically pre-defines the frame
\code{page} to allow you to access the dimensions of the whole page.

\noindent• You can use arithmetic functions: plus, minus, divide, multiply, and
parenthesis have their ordinary arithmetic meaning. To declare that frame \code{b}
should be half the height of frame \code{a} plus 5 millimeters, you can say
\code{height=5mm + (height(b) / 2)}. However, as we will see later, it is usually
better to structure your declarations to let SILE make those kind of computations for
you.

\noindent• Since book design is often specified in terms of proportion of a page,
you can use the shortcut \code{width=5\%pw} instead of \code{width=0.05 * width(page)}
and \code{height=50\%ph} instead of \code{height=0.5 * height(page)}.

\end{note}

Next we declare the left and right column frames. The \code{book} class
gives us some frames already, one of which, \code{content}, defines a typeblock
with a decent size and positioning on the page.
We will use the boundaries of this frame to declare our columns: the left
margin of the left column is the left margin of the typeblock; the right margin of
the right column is the right margin of the typeblock. But we also want
a few other parameters to ensure that:

\noindent• the gutter is placed between our two columns

\noindent• the two columns have the same width (We don’t know what that width is, but SILE
will work it out for us.)

\noindent• after the left column is full, typesetting should move to the right
column.

\begin{verbatim}
\line
\\frame[id=leftCol, left=left(r), right=left(gutter),
       top=top(r), bottom=bottom(r),
       next=rightCol]
\\frame[id=rightCol, left=right(gutter), right=right(r),
       top=top(r), bottom=bottom(r),
       width=width(leftCol)]
\line
\end{verbatim}

And now finally we can end our \code{pagetemplate}.

\begin{verbatim}
\line
\\end\{pagetemplate\}
\line
\end{verbatim}

Let’s do it.
\eject\par%
\begin[first-content-frame=leftCol]{pagetemplate}
\frame[id=gutter,width=3%pw]
\frame[id=leftCol,left=left(content),right=left(gutter),top=top(content),bottom=bottom(content),next=rightCol]
\frame[id=rightCol,left=right(gutter),right=right(runningHead),top=top(content),bottom=bottom(content),width=width(leftCol)]
\end{pagetemplate}
\showframe[id=leftCol]\showframe[id=rightCol]

So there we have it: a two-column page layout.

In the next chapter we’ll use the knowledge of how to declare frames to
help us to create our own document class files. In the meantime, here is
some dummy text to demonstrate the fact that text does indeed flow between
the two columns naturally:

\lorem[words=500]

\chapter{Designing Basic Class Files}

Now we know how to define a frame layout for a single page, let’s try
to define one for an entire document.

Document classes are Lua files, and live somewhere in the \code{classes/}
subdirectory of either where your input file is located, your current working
directory or your SILE path (typically \code{/usr/local/share/sile}). We’re
going to create a simple class file which merely changes the size of the
margins and the typeblock. We’ll call it \code{bringhurst.lua}, because it
replicates the layout of the Hartley & Marks edition of Robert Bringhurst’s
\em{The Elements of Typographical Style}.

We are designing a book-like class, and so we will inherit from SILE’s
standard \code{book} class, \code{classes/book.lua}.

Let’s briefly have a look at \code{book.lua} to see how it works;
after the initial class definition, we see that it loads the \code{masters}
package, and defines a master the following frames:
(wrapped for legibility)

\begin{verbatim}
\line
book:defineMaster(\{
  id = "right",
  firstContentFrame = "content",
  frames = \{

    content = \{
      left = "8.3\%pw", right = "86\%pw",
      top = "11.6\%ph", bottom = "top(footnotes)"
    \},

    folio = \{
      left = "left(content)", right = "right(content)",
      top = "bottom(footnotes)+3\%ph",bottom = "bottom(footnotes)+5\%ph"
    \},

    runningHead = \{
      left = "left(content)", right = "right(content)",
      top = "top(content) - 8\%ph", bottom = "top(content)-3\%ph"
    \},

    footnotes = \{
      left ="left(content)", right = "right(content)",
      height = "0", bottom="83.3\%ph"
    \}
  \}
\})
\line
\end{verbatim}

So there are four frames directly declared; the first is the
content frame, which by SILE convention is called
\code{content}. Directly abutting the \code{content} frame at the bottom is
the \code{footnotes} frame. The top of the typeblock
and the bottom of the footnote frame have fixed positions, but the
boundary between typeblock and footnote is variable. Initially
the height of the footnotes is zero (and so the typeblock takes up
the full height of the page) but as footnotes are inserted
into the footnote frame, its height will be adjusted; its bottom
is fixed and therefore its top will be adjusted, and the bottom
of the main typeblock frame will also be correspondingly adjusted.

The folio frame (the page number) lives below the footnotes, and the
running headers live above the \code{content} frame.

Next, we use the \code{twoside} package to mirror our right-page
master into a left-page master:

\begin{verbatim}
\line
book:loadPackage("twoside", \{ oddPageMaster = "right", evenPageMaster = "left" \});
book:mirrorMaster("left", "right")
\line
\end{verbatim}

The book class also sets up commands for sectioning, and declares various
things that need to be done at the start and end of each page.
Since we will be inheriting from the book class, we will have all
these definitions already available to us. All we need to do is set
up our new class, and then define what is different about it. Here is
how we set up the inheritance:

\begin{verbatim}
\line
local book = SILE.require("book", "classes")
local bringhurst = book \{ id = "bringhurst" \}
...
return bringhurst
\line
\end{verbatim}

Now we need to define our frame masters.

The LaTeX memoir class’ \em{A Few Notes
On Book Design} tells us that Bringhurst’s book has a spine margin one thirteenth
of the page width; a top margin eight-fifths of the spine margin; a front margin
and bottom margin both sixteen-fifths of the spine margin. We can define this in SILE
terms like so:

\begin{verbatim}
\line
bringhurst:defineMaster(\{
  id = "right", firstContentFrame = "content",
  frames = \{
    content = \{
      left = "width(page)/13",
      top = "left(content) * 8 / 5",
      right = "100\%pw - 2*top(content)",
      bottom = "top(footnotes)"
    \},
    folio = \{ ... as before ... \},
    footnotes = \{ ... as before ... \},
    runningHead = \{ ... as before ... \},
    \},
  \}
\})
bringhurst:mirrorMaster("right", "left")
\line
\end{verbatim}

We are nearly finished!

If we try this class as-is, we’ll
actually find that the running headers are too high, because the typeblock is
higher on the page than the standard book class, and the running heads are
defined relative to them.

So, we need to change the definition the running header frame to bring them
down a bit lower:

\begin{verbatim}
\line
  runningHead = \{
    left = "left(content)", right = "right(content)",
    top = "top(content) - 4\%ph", bottom = "top(content)-2\%ph"
  \},
\line
\end{verbatim}

If all we want to do in our new class is to create a different page shape,
this is all we need.

\section{Defining Commands}

However, it’s usually the case that a class will want to do more than
just change the page shape; a class will typically want to do some other things
as well: define additional commands, alter the output routine, store and
investigate bits of state information, and so on. We’ll look briefly
at some of the principles involved in those things here, and in the next
chapters will run through some worked examples.

To define your own command at the Lua level, you use the \code{SILE.registerCommand}
function. It takes three parameters: a command name, a function to implement
the command, and some help text. The
signature of a function representing a SILE command is fixed: you need
to take two parameters, \code{options} and \code{content} (of course you
can name your parameters whatever you like, but these are the most common
names). Both of these parameters are Lua tables. The \code{options} parameter
contains the command’s parameters or XML attributes as a key-value table,
and the \code{content}
is an abstract syntax tree reflecting the input being currently processed.

So in the case of \code{\\mycommand[size=12pt]\{Hello \\break world\}},
the first parameter will contain the table \code{\{size = "12pt"\}} and
the second parameter will contain the table

\begin{verbatim}
\line
\{
  "Hello ",
  \{
    attr = \{\},
    id = "command",
    pos = 8,
    tag = "break"
  \},
  " world"
\}\line
\end{verbatim}

Most commands will find themselves doing something with the \code{options}
and/or calling \break\code{SILE.process(content)} to recursively process the
argument. Here’s a very simple example; an XML \code{<link>} tag may take
an XLink \code{xl:href} attribute\footnote{Yes, I know the document author
might choose a different XML namespace to refer to XLink. Let’s keep things
simple.} We want to render \code{<link xl:href="http://...">\goodbreak{}Hello</link>}
as \examplefont{Hello (\code{http://...})}. So, first we need to render
the content, and then we need to do something with the attribute:

\begin{verbatim}
\line
SILE.registerCommand("link", function(options, content)
  SILE.process(content)
  if (options["xl:href"]) then
    SILE.typesetter:typeset(" (")
    SILE.call("code", \{\}, \{options["xl:href"]\})
    SILE.typesetter:typeset(")")
  end
end)
\line
\end{verbatim}

We use the usual \code{SILE.typesetter:typeset} and \code{SILE.call}
functions to output text and call other commands.

\note{If you do need to do something with a dimension, you can use
\code{SILE.toPoints} to parse a basic length and
\code{SILE.parseComplexFrameDimension} to parse a frame dimension, and turn
them into points.}

\section{Output Routines}

As well as defining frames and packages, different classes also alter
the way that SILE performs its output—what it should do at the start or
end of a page, for instance, which controls things like swapping between
different master frames, displaying page numbers, and so on.

The key methods for defining the \em{output routine} are:

\noindent{}• \code{newPar} and \code{endPar} are called at the start and end of each
paragraph.

\noindent{}• \code{newPage} and \code{endPage} are called at the start
and each of each page.

\noindent{}• \code{init} and \code{finish} are called at the start and
end of the document.

Once again this is done in an object-oriented way, with derived classes
overriding their superclass’ methods where necessary.

When you are loading packages which affect the output routine, the composition of
such packages into a class is not completely automatic\footnote{Because the
order of events is sometimes significant.}; in other words, loading the package
will not necessarily by itself change the output routines. You need to
explicitly plug the various features provided by those packages into the output routine
yourself.

So, for instance, the \code{footnote} or \code{insertions} packages
provide a \code{outputInsertions} method which needs to be called at the end
of each page. If you want to build a document class that inherits from \code{plain}
but also has footnote functionality, you will want your \code{endPage} method
to look like this:

\begin{verbatim}
\line
myClass.endPage = function(self)
  myClass:outputInsertions()
  plain.endPage(self)
end
\line
\end{verbatim}

Let’s demonstrate roughly how the \code{tableofcontents} package works. We’ll
be using the \code{infonodes} package to collect the information about which
pages contain table of content items.

First, we set up our infonodes by creating a command that can be called
by sectioning commands. In other words, \code{\\chapter}, \code{\\section}, etc.
should call \code{\\tocentry} to store the page reference for this section.

\begin{verbatim}
\line
SILE.registerCommand("tocentry", function (options, content)
  SILE.call("info", \{
    category = "toc",
    value = \{
      label = content, level = (options.level or 1)
    \}
  \})
end)
\line
\end{verbatim}

Infonodes work on a per-page basis, so if we want to save them throughout
the whole document, at the end of each page we need to move them from the
per-page table to our own
table. We also need to make sure we store their page numbers!

\note{SILE provides the \code{SILE.scratch} variable for you to store
global information in; you should use a portion of this table namespaced to
your class or package.}

So, here is a routine we can call at the end of each page to move the
TOC nodes.

\begin{verbatim}
\line
SILE.scratch.tableofcontents = \{ \}

-- Gather the tocentries into a big document-wide TOC
local moveNodes = function(self)
  local n = SILE.scratch.info.thispage.toc
  for i=1,#n do
    n[i].pageno = SILE.formatCounter(SILE.scratch.counters.folio)
    table.insert(SILE.scratch.tableofcontents, n[i])
  end
end
\line
\end{verbatim}

We’re going to take the LaTeX approach of storing these items out as a
separate file, then loading them back in again when typesetting the TOC.
So at the end of the document, we serialize the \code{SILE.scratch.tableofcontents}
table to disk. Here is a function to be called by the \code{finish} output
routine.

\begin{verbatim}
\line
local writeToc = function (self)
  local t = std.string.pickle(SILE.scratch.tableofcontents)
  saveFile(t, SILE.masterFilename .. '.toc')
end
\line
\end{verbatim}

And then the \code{\\tableofcontents} command reads that file if it is
present, and typesets the TOC nodes appropriately:

\begin{verbatim}
\line
SILE.registerCommand("tableofcontents", function (options, content)
  local toc = loadFile(SILE.masterFilename .. '.toc')
  if not toc then
    SILE.call("tableofcontents:notocmessage")
    return
  end
  SILE.call("tableofcontents:header")
  for i = 1,#toc do
    local item = toc[i]
    SILE.call("tableofcontents:item", \{level = item.level, pageno = item.pageno\}, item.label)
  end
end)
\line
\end{verbatim}

And the job is done. Well, nearly. The \code{tableofcontents} package now contains a
couple of functions—\code{moveNodes} and \code{writeToc}—that need to be called at
various points in the output routine of a class which uses this package. How do we
do that?

\section{Exports}

Packages which are primarily used for providing functionality to other classes
and packages need a way of supplying these composible bits of functionality to
the code which is going to use them. This is called the \em{export mechanism}.

As well as defining commands, each package may also return a Lua table consisting
of two entries, \code{init} and \code{exports}.

\code{init} allows you to perform some initialization actions, optionally
based on arguments supplied by the loading class. When the
package is loaded with \code{class:loadPackage(package, args)}, the initializer
is called with two arguments, \code{class} and \code{args}. For instance, the
\code{twoside} package receives information about the IDs of the main right
and left master frames so that it can set up the code to switch masters on page change.
In our case, we will want to ensure that the \code{infonode} package is loaded into
our caller:

\begin{verbatim}
\line
return \{
  init = function (caller)
    caller:loadPackage("infonode")
  end,
\line
\end{verbatim}

The other entry to be returned from the package is \code{exports},
which contains names and functions to be mixed into the caller’s namespace.
In other words, after:

\begin{verbatim}
\line
  exports = \{writeToc = writeToc, moveTocNodes = moveNodes\}
\}
\line
\end{verbatim}

\noindent{}any class which loads \code{tableofcontents} can call \code{self:writeToc()}
and \code{self:moveTocNodes()} (note that we renamed this function when exporting it).
It is the class’s responsibility for calling these methods at the appropriate point into
the output routine.

\chapter{Advanced Class Files 1: SILE As An XML Processor}

Now we are ready to look at a worked example of writing a class to turn an arbitrary
XML format into a PDF file. We’ll be looking at the DocBook processor that ships with
SILE. DocBook is an XML format for structured technical documentation. DocBook itself
doesn’t encode any presentation information about how its various tags should be rendered
on a page, and so we shall have to make all the presentation decisions for ourself.

The first thing you should know is that it makes your life significantly easier if you
consider writing the class file in \em{two} files; the first being a SILE file in TeX
format, and the second as Lua code. This allows you to dispose of all the easy jobs in a
convenient format, and then deal with the hard jobs in Lua. When you use the
\code{-I \em{classname}} command line option to SILE, SILE first looks for \code{\em{classname}.sil}
and uses that as a wrapper around your file to be processed. If that then begins

\begin{verbatim}
\line
\\begin[class=\em{classname}]\{document\}%
\line
\end{verbatim}

\noindent{}then SILE will also load up \code{classes/\em{classname}.lua} as normal.

Now we can start defining SILE commands to render XML elements. Most of these are
fairly straightforward so we will not dwell on them too much. For instance,
DocBook has a tags like \code{<code>}, \code{<filename>}, \code{<guimenu>} which should
all be rendered in a monospaced typewriter font. To make it easier to customize the
class, we abstract out the font change into a single command:

\begin{verbatim}
\line
\\define[command=docbook-ttfont]\{\\font[family=Inconsolata,size=2ex]\{\\process\}\}
\line
\end{verbatim}

Now we can define our \code{<code>} (etc.) tags:

\begin{verbatim}
\\define[command=code]\{\\docbook-ttfont\{\\process\}\}
\\define[command=filename]\{\\docbook-ttfont\{\\process\}\}
\\define[command=guimenu]\{\\docbook-ttfont\{\\process\}\}
\\define[command=guilabel]\{\\docbook-ttfont\{\\process\}\}
\\define[command=guibutton]\{\\docbook-ttfont\{\\process\}\}
\\define[command=computeroutput]\{\\docbook-ttfont\{\\process\}\}
\end{verbatim}

If an end user wants things to look different, they can redefine the \code{docbook-ttfont}
command and get a different font.

\section{Handling Titles}

So much for simple tags.
Things get interesting when there is a mismatch between the simple format of
SILE macros and the complexity of DocBook markup.

We have already seen an example of the \code{<link>} tag where we also need to process
XML attributes, so we will not repeat that here. Let’s look at another area of complexity:
the apparently-simple \code{<title>} tag. The reason this is complex is that it occurs
in different contexts, sometimes more than once within a context; it should often be
rendered differently in different contexts. So for instance \code{<article><title>...}
will look different to \code{<section><title>...}. Inside an \code{<example>} tag,
the title will be preferenced by an example number; inside a \code{<bibliomixed>}
bibliography entry the title should not be set off as a new block but should be
part of the running text, and so on.

What we will do to deal with this situation is provide a very simple definition of
\code{<title>}, but when processing the containing elements of \code{<title>}
(such as \code{<article>}, \code{<example>}), we will process the title ourselves.

For instance, let’s look at \code{<example>}, which has the added complexity of needed
to keep track of an article number.

\begin{verbatim}
\line
SILE.registerCommand("example", function(options,content)
  SILE.call("increment-counter", \{id="Example"\})
  SILE.call("bigskip")
  SILE.call("docbook-line")
  SILE.call("docbook-titling", \{\}, function()
    SILE.typesetter:typeset("Example".." ".. SILE.formatCounter(SILE.scratch.counters.Example]))
\end{verbatim}

\code{\\docbook-line} is a command that we’ve defined in the \code{docbook.sil} macros
file to draw a line across the width of the page to set off examples and so on.
\code{\\docbook-titling} is a command similarly defined in \code{docbook.sil} which sets
the default font for titling and headers; once again, if people want to customize the
look of the output we make it easier for them by giving them simple, compartmentalized
commands to override.

So far so good, but how do we extract the \code{<title>} tag from the \code{content}
abstract syntax tree? SILE does not provide XPath or CSS-style selectors to locate
content form within the DOM tree;\footnote{Patches, as they say, are welcome.} instead
there is a simple one-level function called \code{SILE.findInTree} which looks for
a particular tag or command name within the immediate children of the current tree:

\begin{verbatim}
    local t = SILE.findInTree(content, "title")
    if t then
      SILE.typesetter:typeset(": ")
      SILE.process(t)
\end{verbatim}

We’ve output \examplefont{Example 123} so far, and now we need to say \examplefont{: \em{Title}}.
But we also need to ensure that the \code{<title>} tag doesn’t get processed again
when we process the content of the example:

\begin{verbatim}
      docbook.wipe(t)
\end{verbatim}

\code{docbook.wipe} is a little helper function which nullifies all the content of a
Lua table:

\begin{verbatim}
function docbook.wipe(tbl)
  while((#tbl) > 0) do tbl[#tbl] = nil end
end
\end{verbatim}

Let’s finish off the \code{<example>} example by skipping a big between the title and
the content, processing the content and drawing a final line across the page:

\begin{verbatim}
    end
  end)
  SILE.call("smallskip")
  SILE.process(content)
  SILE.call("docbook-line")
  SILE.call("bigskip")
\line
\end{verbatim}

Now it happens that the \code{<example>}, \code{<table>} and \code{<figure>} tags
are pretty equivalent: they produce numbered titles and then go on to process their
content. So in reality we actually define an abstract \code{countedThing} method
and define these commands in terms of that.

\section{Sectioning}

DocBook sectioning is a little different to the SILE \code{book} class. \code{<section>}
tags can be nested; to start a subsection, you place another \code{<section>} tag
inside the current \code{<section>}. So in order to know what level we are currently on,
we need a stack; we also need to keep track of what section number we are on at \em{each}
level. For instance:

\begin{verbatim}
  <section><title>A</title> : \examplefont{1. A}
    <section><title>B</title>: \examplefont{1.1 B}
    </section>
    <section><title>C</title>: \examplefont{1.2 C}
      <section><title>D</title>: \examplefont{1.2.1 D}
      </section>
    </section>
    <section><title>E</title>: \examplefont{1.3 E}
  </section>
  <section><title>F</title>: \examplefont{2. F}
\end{verbatim}

So, we will keep two variables: the current level, and the counters for all of the
levels so far. Each time we enter a \code{section}, we increase the current level counter:

\begin{verbatim}
\line
SILE.registerCommand("section", function (options, content)
  SILE.scratch.docbook.seclevel = SILE.scratch.docbook.seclevel + 1
\end{verbatim}

We also increment the count at the current level, while at the same time wiping out
any counts we have for levels above the current level (if we didn’t do that, then
\examplefont{E} in our example above would be marked \examplefont{1.3.1}):

\begin{verbatim}
  SILE.scratch.docbook.seccount[SILE.scratch.docbook.seclevel] =
    (SILE.scratch.docbook.seccount[SILE.scratch.docbook.seclevel] or 0) + 1
  while #(SILE.scratch.docbook.seccount) > SILE.scratch.docbook.seclevel do
    SILE.scratch.docbook.seccount[#(SILE.scratch.docbook.seccount)] = nil
  end
\end{verbatim}

Now we find the title, and prefix it by the concatenation of all the \code{seccount}s:

\begin{verbatim}
  local title = SILE.findInTree(content, "title")
  local number = table.concat(SILE.scratch.docbook.seccount, '.')
  if title then
    SILE.call("docbook-section-"..SILE.scratch.docbook.seclevel.."-title",\{\},function()
      SILE.typesetter:typeset(number.." ")
      SILE.process(title)
    end)
    docbook.wipe(title)
  end
\end{verbatim}

Finally we can process the content of the tag, and decrease the level count as we leave the
\code{</section>} tag:

\begin{verbatim}
  SILE.process(content)
  SILE.scratch.docbook.seclevel = SILE.scratch.docbook.seclevel - 1
end)
\line
\end{verbatim}

\section{Other Features}

SILE’s DocBook implementation is a work in progress, and there is more that can be
done. For instance, there is a basic implementation of lists, which equally need to
be able to handle nesting; we implement another stack to take care of the type of
list and list counter at each level of nesting.

How would you handle a tag like \code{<xref>} which renders a cross-reference to
another part of the document? For instance, \code{<xref linkend="ch02"/>} should
generate something like \examplefont{Chapter 2, “The Second Chapter”}. This is
another problem which can be handled using the \code{infonode} package to collect
and store cross-reference information about chapter numbers and titles.

\chapter{Further Tricks}

We’ll conclude our tour of SILE by looking at some tricky situations which require
further programming.

\section{Parallel Text}

The file \code{examples/parallel.sil} contains a rendering of chapter 1 of
Matthew’s Gospel in English and Greek. It uses the \code{diglot} class to align
the two texts side-by-side. \code{diglot} provides the \code{\\left} and \code{\\right}
commands to start entering text on the left column or the right column respectively,
and the \code{\\sync} command to ensure that the two columns are in sync with each
other. It’s an instructive example of what can be done in a SILE class, so we
will take it apart and see how it works.

The key thing to note is that the SILE typesetter is an object. (in the object-oriented
programming sense) Normally, it’s a singleton object—i.e. one typesetter is used for
typesetting everything in a document. But there’s no reason why we can’t have more
than one. In fact, for typesetting parallel texts, the simplest way to do things is
to have two typesetters, one for each column, and have them communicate with each
other at various points in the operation.

Let’s begin \code{diglot.lua} as usual by setting up the class and declaring our
frames:

\begin{verbatim}
\line
local plain = SILE.require("plain", "classes");
local diglot = std.tree.clone(plain);
SILE.require("packages/counters");
SILE.scratch.counters.folio = \{ value = 1, display = "arabic" \};
SILE.scratch.diglot = \{\}
diglot:declareFrame("a",    \{left = "8.3\%pw", right = "48\%pw",
            top = "11.6\%ph", bottom = "80\%ph" \});
diglot:declareFrame("b",    \{left = "52\%pw",  right = "100\% - left(a)",
            top = "top(a)", bottom = "bottom(a)" \});
diglot:declareFrame("folio",\{left = "left(a)", right = "right(b)",
            top = "bottom(a)+3\%ph",bottom = "bottom(a)+8\%ph" \});
\end{verbatim}

Now we create two new typesetters, one for each column, and we tell each one
how to find the other:

\begin{verbatim}
diglot.leftTypesetter = SILE.defaultTypesetter \{\}
diglot.rightTypesetter = SILE.defaultTypesetter \{\}
diglot.rightTypesetter.other = diglot.leftTypesetter
diglot.leftTypesetter.other = diglot.rightTypesetter
\end{verbatim}

Each column needs its own font, so we provide commands to store this information.
The \code{\\leftfont} and \code{\\rightfont} macros simply store their options to
be passed to the \code{\\font} command every time \code{\\left} and \code{\\right}
are called. (Because the fonts are controlled by global settings rather than being
typesetter-specific.)

\begin{verbatim}
SILE.registerCommand("leftfont", function(options, content)
  SILE.scratch.diglot.leftfont = options
end, "Set the font for the left side")

SILE.registerCommand("rightfont", function(options, content)
  SILE.scratch.diglot.rightfont = options
end, "Set the font for the right side")
\end{verbatim}

Next come the commands for sending text to the appropriate typesetter. The current
typesetter object used by the system is stored in the variable \code{SILE.typesetter};
many commands and packages call methods on this variable, so we need to ensure
that this is set to the typesetter object that we want to use. We also want to
turn off paragraph detection, as we will be handling the paragraphing manually
using the \code{\\sync} command:

\begin{verbatim}
SILE.registerCommand("left", function(options, content)
  SILE.settings.set("typesetter.parseppattern", -1)
  SILE.typesetter = diglot.leftTypesetter;
  SILE.Commands["font"](SILE.scratch.diglot.leftfont, \{\})
end, "Begin entering text on the left side")

SILE.registerCommand("right", function(options, content)
  SILE.settings.set("typesetter.parseppattern", -1)
  SILE.typesetter = diglot.rightTypesetter;
  SILE.Commands["font"](SILE.scratch.diglot.rightfont, \{\})
end, "Begin entering text on the right side")
\end{verbatim}

The meat of the \code{diglot} package comes in the \code{sync} command,
which ensures that the two typesetters are aligned. Every time we call
\code{sync}, we want to ensure that they are both at the same position on
the page. In other words, if the left typesetter has gone further down the page
than the right one, we need to insert some blank space onto the right typesetter’s
output queue to get them back in sync, and vice versa.

SILE’s page builder has a method called \code{SILE.pagebuilder.collateVboxes}
which bundles a bunch of vertical boxes into one; we can use this to bundle up
each typesetter’s output queue and measure the height of the combined vbox.
(Of course it’s possible to sum the heights of each box on the output queue
by hand, but this achieves the same goal a bit more cleanly.)

\begin{verbatim}
SILE.registerCommand("sync", function()
  local lVbox = SILE.pagebuilder.collateVboxes(diglot.leftTypesetter.state.outputQueue)
  local rVbox = SILE.pagebuilder.collateVboxes(diglot.rightTypesetter.state.outputQueue)
  if (rVbox.height > lVbox.height) then
    diglot.leftTypesetter:pushVglue(\{ height = rVbox.height - lVbox.height \})
  elseif (rVbox.height < lVbox.height) then
    diglot.rightTypesetter:pushVglue(\{ height = lVbox.height - rVbox.height \})
  end
\end{verbatim}

Next we end each paragraph (we do this after adding the glue so that \code{parskips}
do not get in the way), and go back to handling paragraphing as normal:

\begin{verbatim}
  diglot.rightTypesetter:leaveHmode();
  diglot.leftTypesetter:leaveHmode();
  SILE.settings.set("typesetter.parseppattern", "\\n\\n+")
end)
\end{verbatim}

Now everything is ready apart from the output routine. In the output routine
we need to ensure, at the start of each document and the start of each page,
that each typesetter is linked to the appropriate frame:

\begin{verbatim}
diglot.init = function(self)
  diglot.leftTypesetter:init(SILE.getFrame("a"))
  diglot.rightTypesetter:init(SILE.getFrame("b"))
  return SILE.baseClass.init(self)
end
\end{verbatim}

\noindent{}(\code{SILE.getFrame} retrieves a frame that we have declared.)

The default \code{newPage} routine will do this for one typesetter every
time we open a new page, but it doesn’t know that we have another typesetter
object to set up as well; so we need to
to make sure that, no matter which typesetter causes an new-page
event, the other typesetter also gets correctly initialised:

\begin{verbatim}
diglot.newPage = function(self)
  plain.newPage(self)
  if SILE.typesetter == diglot.leftTypesetter then
    SILE.typesetter.other:initFrame(SILE.getFrame("b"))
    return SILE.getFrame("a")
  else
    SILE.typesetter.other:initFrame(SILE.getFrame("a"))
    return SILE.getFrame("b")
  end
end
\end{verbatim}

And finally, when one typesetter causes an end-of-page event, we need to ensure
that the other typesetter is given the opportunity to output its queue to the
page as well:

\begin{verbatim}
diglot.endPage = function(self)
  SILE.typesetter.other:leaveHmode(1)
  plain.endPage(self)
end
\end{verbatim}

Similarly for the end of the document, but in this case we will use the
emergency \code{chuck} method; whereas \code{leaveHmode} means “call the
page builder and see there’s enough material to build a page”, \code{chuck}
means “you must get rid of everything on your queue now.” We add some infinitely
tall glue to the other typesetter’s queue to help the process along:

\begin{verbatim}
diglot.finish = function(self)
  table.insert(SILE.typesetter.other.state.outputQueue, SILE.nodefactory.vfillGlue)
  SILE.typesetter.other:chuck()
  plain.finish(self)
end
\end{verbatim}

And there you have it; a class which produces balanced parallel texts using two
typesetters at once.

\section{Sidenotes}

One SILE project needed two different kinds of sidenotes, margin notes and gutter
notes.

\line
\img[src=documentation/discovery.png,width=width(content)]
\line

Sidenotes can be seen as a simplified form of
parallel text. With a true parallel, neither the left or the right typesetter
is “in charge”—either can fill up the page and then inform the other typesetter
that they need to catch up. In the case of sidenotes, there’s a well-defined
main flow of text, with annotations having to work around the pagination of the
typeblock.

There are a variety of ways that we could implement these sidenotes; as it
happened, I chose a different strategy for the margin notes and the gutter notes.
Cross-references in the gutter could appear fairly frequently, and so needed to
“stack up” down the page—they need to be \em{at least} on a level with the verse
that they relate to, but could end up further down the page if there are a few
cross-references close to each other. Markings in the margin, on the other hand,
were guaranteed not to overlap.

We’ll look at the margin marking first. We’ll implement this as a special
zero-width hbox (what TeX would call a \code{\\special}) which, although it lives in the output stream of the main
typeblock, actually outputs itself by marking the margin at the current vertical
position in the typeblock. In the example above, there will be a special hbox
just before the word “there” in the first line.

First we need to find the appropriate margin frame and, find its left boundary:

\begin{verbatim}
\line
discovery.typesetProphecy = function(symbol)
  local margin = discovery:oddPage() and
                  SILE.getFrame("rMargin") or SILE.getFrame("lMargin")
  local target = margin:left()
\end{verbatim}

Next, we call another command to produce the symbol itself; this allows the
book designer to change the symbols at the SILE level rather than having to
mess about with the Lua file. We use the \command{\\hbox} command to wrap the
output of the command into a hbox. \command{\\hbox} returns its output, but also
puts the box into the typesetter’s output node queue; we don’t want it to appear
in the main typeblock, so we remove the node again, leaving our private copy
in the \code{hbox} variable.

\begin{verbatim}
  local hbox = SILE.call("hbox",\{\}, function()
    SILE.call("prophecy-"..symbol.."-mark")
  end)
  table.remove(SILE.typesetter.state.nodes)
\end{verbatim}

What we \em{do} want in the output queue is our special hbox node which will
put the marking into the margin. This special hbox has no impact on the
current line—it has no width, height, or depth—and it contains a copy of the
symbol that we stored in the \code{hbox} variable.

\begin{verbatim}
  SILE.typesetter:pushHbox(\{
    width= 0,
    height = 0,
    depth= 0,
    value= hbox,
\end{verbatim}

Finally we need to write the routine which outputs this hbox. Box
output routines receive three parameters: the box itself, the current
typesetter (which knows the frame it is typesetting into, and the frame
knows whereabouts it has got to), and a variable representing the stretchability
or shrinkability of the line. (We don’t need that for this example.)

What our output routine should do is: save a copy of our horizontal position,
so that we can restore it later as we carry on outputting other boxes; jump
across to the left edge of the margin, which we computed previously; tell
the symbol that we’re carrying with us to output \em{itself}; and then jump
back to where we were:

\begin{verbatim}
    outputYourself= function (self, typesetter, line)
      local saveX = typesetter.frame.state.cursorX;
      typesetter.frame.state.cursorX = target
      self.value:outputYourself(typesetter,line)
      typesetter.frame.state.cursorX = saveX
    end
  \})
\line
\end{verbatim}

This was a quick-and-dirty version of sidenotes (in twenty lines of code!)
which works reasonably well
for individual symbols which are guaranteed not to overlap. For the gutter notes,
which are closer to true sidenotes, we need to do something a bit more intelligent.
We’ll take a similar approach to when we made the parallel texts, by employing a
separate typesetter object.

As before we’ll create the object, and ensure that at the start of the document
and at the start of each page it is populated correctly with the appropriate frame:

\begin{verbatim}
\line
discovery.innerTypesetter = SILE.defaultTypesetter \{\}

discovery.init = function()
  local gutter = discovery:oddPage() and
                 SILE.getFrame("rGutter") or SILE.getFrame("lGutter")
  discovery.innerTypesetter:init(gutter)
  ...
  return SILE.baseClass:init()
end

discovery.newPage = function ()
  ...
  discovery.innerTypesetter:leaveHmode(1)
  local gutter = discovery:oddPage() and
                 SILE.getFrame("rGutter") or SILE.getFrame("lGutter")
  discovery.innerTypesetter:init(gutter)
  ...
  return SILE.baseClass.newPage(discovery);
end
\line
\end{verbatim}

Now for the function which actually handles a cross-reference. As with the
parallels example, we start by totting up the height of the material processed
on the current page by both the main typesetter and the cross-reference typesetter.

\begin{verbatim}
\line
discovery.typesetCrossReference = function(xref)
  discovery.innerTypesetter:leaveHmode(1)
  local innerVbox = SILE.pagebuilder.collateVboxes(discovery.innerTypesetter.state.outputQueue)
  local mainVbox = SILE.pagebuilder.collateVboxes(SILE.typesetter.state.outputQueue)
\end{verbatim}

This deals with the material which has already been put into the output queue: in
other words, completed paragraphs. The problem here is that we do not want to
end a paragraph between two verses; if we are mid-paragraph while typesetting
a cross-reference, we need to work out what the height of the material \em{would have
been} if we were to put it onto the output queue at this point. So, we take
the \code{SILE.typesetter} object on a little excursion.

First we take a copy of the current node queue, and then we call the typesetter’s
\code{pushState} method. This initializes the typesetter anew, while saving
its existing state for later. Since we have a new typesetter, its node queue
is empty, and so we feed it the nodes that represent our paragraph so far. Then
we tell the typesetter to leave horizontal mode, which will cause it to go away
and calculate line breaks, leading, paragraph height and so on. We box up its
output queue, and then return to where we were before. Now we have a box which
represents what would happen if we set the current paragraph up to the point
that our cross-reference is inserted; the height of this box is the distance
we need to add to \code{mainVbox} to get the vertical position of the cross-reference
mark.

\begin{verbatim}
  local unprocessedNodes = std.tree.clone(SILE.typesetter.state.nodes)
  SILE.typesetter:pushState()
  SILE.typesetter.state.nodes = unprocessedNodes
  SILE.typesetter:leaveHmode(1)
  local subsidiary = SILE.pagebuilder.collateVboxes(SILE.typesetter.state.outputQueue)
  SILE.typesetter:popState()
  mainVbox.height = mainVbox.height + subsidiary.height
\end{verbatim}

\note{The \code{1} argument to \code{leaveHmode} means “you may not create
a new page here.”}

In most cases, the cross-reference typesetter hasn't got as far down the page
as the body text typesetter, so we tell the cross-reference typesetter to shift itself
down the page by the difference. Unlike the parallel example, where either typesetter
could tell the other to open up additional vertical space, in this case it’s OK if
the cross-reference appears a bit lower than the verse it refers to.

\begin{verbatim}
  if (innerVbox.height < mainVbox.height) then
    discovery.innerTypesetter:pushVglue(\{ height = mainVbox.height - innerVbox.height \})
  end
\end{verbatim}

At this point the two typesetters are now either aligned, or the cross-reference
typesetter has gone further down the page than the verse it refers to. Now we
can output the cross-reference itself.

\begin{verbatim}
  SILE.settings.temporarily(function()
    SILE.settings.set("document.baselineskip", SILE.nodefactory.newVglue("7pt"))
    SILE.Commands["font"](\{size = "6pt", family="Helvetica", weight="800"\}, \{\})
    discovery.innerTypesetter:typeset(SILE.scratch.chapter..":"..SILE.scratch.verse.." ")
    SILE.Commands["font"](\{size = "6pt", family="Helvetica", weight="200"\}, \{\})
    discovery.innerTypesetter:typeset(xref)
    discovery.innerTypesetter:leaveHmode()
    discovery.innerTypesetter:pushVglue(\{ height = SILE.length.new(\{length = 4\}) \})
  end)
end
\line
\end{verbatim}

\note{We haven’t used \code{SILE.call} here because it performs all its operations
on the default typesetter. If we wanted to make things cleaner, we could swap
typesetters by assigning \code{discovery.innerTypesetter} to \code{SILE.typesetter}
and then calling ordinary commands, rather than doing the settings and glue
insertion “by hand.”}

In the future it may make sense for there to be a standard \code{sidenotes}
package in SILE, but it has been instructive to see a couple of ‘non-standard’
examples to understand how the internals of SILE can be leveraged to create such a
package. Your homework is to create one!

\section{SILE As A Library}

So far we’ve been assuming that you would want to run SILE as a processor for
an existing document. But what if you have a program which produces or munges
data, and you would like to produce PDFs from within your application?
In that case, it may be easier and provide more flexibility to use SILE as a library.

In the \code{examples/} directory of the SILE distribution, you will find an
example of a Lua script which produces a PDF from SILE. It’s actually fairly
simple to use SILE from within Lua; the difficult part is setting things up.
Here’s how to do it.

\begin{verbatim}
\line
require("core/sile")
SILE.outputFilename = "byhand.pdf"
local plain = require("plain", "classes")
SILE.documentState.documentClass = plain;
local ff = plain:init()
SILE.typesetter:init(ff)
\line
\end{verbatim}

Loading the SILE core library also loads up all the other parts of SILE. We
need to set the output file name and load the class that we want to use to
typeset the document with.  We then need to tell SILE what class we are
actually using, call \code{init} on the class to get the first frame for
typesetting, and then initialize the typesetter with this frame. This is all
that SILE does to get itself ready to typeset.

After this, all the usual API calls will work: \code{SILE.call},
\code{SILE.typesetter:typeset} and so on.

\begin{verbatim}
SILE.typesetter:typeset(data)
\end{verbatim}

The only thing to be careful is the need to call the \code{finish} method
on your document class at the end of processing to finish off the final page:

\begin{verbatim}
plain:finish()
\line
\end{verbatim}

\section{Debugging}

When you are experimenting with SILE and its API, you may find it necessary
to get further information about what SILE is up to. SILE has a variety of
debugging switches that can be turned on by the command line or by Lua code.

Running SILE with the \code{--debug \em{facility}} switch will turn on debugging
for a particular area of SILE’s operation:

\noindent{}• \code{typesetter} provides general debugging for the typesetter:
turning characters into boxes, boxes into lines, lines into paragraphs, and
paragraphs into pages.

\noindent{}• \code{pagebuilder} helps to debug problems when determining page
breaks.

\noindent{}• \code{vboxes} provides even more information about page break decisions, showing you what vboxes were in SILE's queue when considering the breaking.

\noindent{}• \code{break} provides (copious) information about the line breaking
algorithm.

\noindent{}• \code{frames} puts red boxes around the frames on the page.

\noindent{}• \code{profile} turns on Lua profiling, which gives a report on where the Lua interpreter is spending its time while processing your document. It also makes SILE go really, really slow.

\noindent{}• \code{versions} gives a report on the versions of libraries and fonts in use; please include this information when reporting bugs in SILE!

\noindent{}• Any package or other area of SILE’s operation may define their own debugging facility; the \code{insertions} package does this, as does the Uyghur language support system (\code{--debug uyghur}).

Multiple facilities can be turned on by separating them with commas:
\code{--debug typesetter,break} will turn on debugging information for the
typesetter and line breaker.

From Lua, you can add entries to the \code{SILE.debugFlags} table to turn on
debugging for a given facility. This can be useful for temporarily debugging
a particular operation:

\begin{verbatim}
\line
SILE.debugFlags.typesetter = 1
SILE.typesetter:leaveHmode()
SILE.debugFlags.typesetter = nil
\line
\end{verbatim}

From a package’s point of view, you can write debugging information by
calling the \code{SU.debug} function (\code{SU} stands for SILE Utilities,
and contains a variety of auxiliary functions used throughout SILE):

\begin{verbatim}
\line
SU.debug("mypackage", "Doing a thing")
\line
\end{verbatim}

Sometimes it’s useful for you to try out Lua code within the SILE environment;
SILE contains a REPL (read-evaluate-print loop) for you to enter Lua code and
print the results back to you. If you call SILE with no input file name, it
enters the REPL:

\begin{verbatim}
This is SILE 0.9.0
> \em{l = SILE.length.parse("22mm")}
> \em{l.length}
62.3622054
\end{verbatim}

At any point during the evaluation of Lua commands, you can call \code{SILE.repl()}
to enter the REPL and poke around; hitting Ctrl-D will end the REPL and
return to processing the document.

\section{Conclusion}

We’ve seen not just the basic functionality of SILE but also given you some
examples of how to extend it in new directions; how to use the SILE API to
solve difficult problems in typesetting. Go forth and create your own SILE
packages!
\end{document}
