Text merges in TEX and L^{A}TEX^{∗}

Mike Piff

January 4, 1996

Abstract In this article the author explains how to do some standard and not so standard word processor text merges in TEX documents, using no other tools than TEX itself. A common application is to the mail merge or form letter, where names and addresses are stored in a file, together with other bits of information, and a standard letter with variable fields embedded in it is customized for every name from this file. Another application is to the pretty-printing of the contents of a database.

The macros described in textmerg.sty work equally in both plain TEX and L^{A}TEX. However, this has meant heavy use of \def where \newcommand would have been preferable.

Contents

1 Introduction 1

2 A simple example 2

3 A few complications 3

4 A complicated example 5

1 Introduction

It is often said that although L^{A}TEX is good at typesetting mathematics, it is wholly unsuitable for common word processor functions such as mail merges. The latter are easy to achieve in most ordinary word processors, but in its raw state L^{A}TEX is incapable of doing a mail merge, or, indeed, of generating the same block of text over and over again but with different parameters in each block, those parameters having been read from a subsidiary merge file. The latter file might possibly be the output from a database or any other program.

This article aims to show the reader that such a repetitive task need not be as difficult as it at first appears. In TEX, it is possible to hide many details of a facility inside a subsidiary style file, so that the user is unaware of what fearful processes are going on in the background. It is then possible to present the enduser with an extremely simple interface, perhaps simpler and more powerful than is available in other systems.

∗This article originally appeared in TUGboat Vol 13(4), 1992, p518–523.

In earlier TUGboat articles [Bel87, Gar87, Lee86, McK87] it was shown how a standard letter could be customized by adding names and addresses from a separate file. I aim to show that it is possible to achieve far more than this with a fairly compact but general set of macros.

2 A simple example

Suppose that we have a list of student names and examination grades, one per student, and that we wish to send a letter to each student giving his/her exam grade. We must decide first what bits of information must be prepared in our subsidiary file, by looking at an example letter and finding out which items change from letter to letter.

Suppose that one instance of our letter is the following, a L^{A}TEX example.

\begin{letter}{Miss Iusta Mo\\
   34 Winchester Road\\
   Sheffield\\
   England}
   \opening{Dear Miss Mo,}
   This letter is to inform you
   that you obtained grade A in
   your recent examinations.
   \closing{Yours faithfully,}
\end{letter}

We can see that we need to know the student’s title, forename(s), surname, address and grade to compose such a letter.

One of the simplest ways of achieving this effect is to prepare a file with lines of the form

\MyLetter{Mr}...{C}

for each student and then simply \input it into a L^{A}TEX file in which \MyLetter has been defined as having five parameters. A problem with this approach is that we may not be able to coax the student database into producing such a file. Another problem is that we need something more subtle if there are fifty parameters. For example, we might want to print out the contents of the student database with one page per student, but it could be that there are fifty information fields per student. Even worse, the number of pieces of information per student might not be a constant number, because, say, we are printing out fields from a related file in which marks on individual examination papers are held.

We shall tackle our simple example in a way that lends itself to more generality later on, and in a form that most database programs should be capable of handling.

We thus prepare a subsidiary file results.dat with records of five fields in it. Each student is represented by five lines of this file, 1 ⟨∗results⟩

 2 Miss
 3 Iusta
 4 Mo
 5 34 Winchester Road\\Sheffield\\England
 6 A
 7 Mr

 8 Arthur
 9 Minit
10 43 Sheffield Road\\Winchester\\England

11 C

12 ⟨/results⟩ and the student records appear one after another in this file. Thus both the field and record separators are carriage returns.

\Merge TEX itself needs to know three bits of information:

\Fields

1. the name of the subsidiary file,

2. the fields to read, and

3. the template of the letter.

We pass it this information in the following form 13 ⟨∗examp⟩

14 \documentclass[12pt]{letter}
15 \usepackage{textmerg}
16 \begin{document}
17 \Fields{\Title\Forenames\Surname
18 \Address\Grade}
19 \Merge{results.dat}{%
20 \begin{letter}{\Title\ \Forenames\
21 \Surname\\\Address}
22 \opening{Dear \Title\ \Surname,}
23 This letter is to inform you
24 that you obtained grade \Grade\ in
25 your recent examinations.
26 \closing{Yours faithfully,}
27 \end{letter}}
28 \end{document}

29 ⟨/examp⟩ L^{A}TEX should open the subsidiary file and, for each set of five parameters, generate a letter in the dvi file. When it reaches the end of the merge file, L^{A}TEX should terminate execution of the \Merge command and presumably finish the document.

3 A few complications

Looking at the above example in a bit more generality, we see that we are reading records of n fields from the merge file and placing them into a TEX document in such a way that they replace n preassigned control sequences. However, it may happen that the merge file is prepared by humans, who might possibly have inserted some extra blank lines into the file. Again, it could be that certain sorts of fields might be blank, whereas others can never be blank. Perhaps it would be better to build in some degree of error recovery.

\Fields We shall make the assumption that the first field in any record is definitely a non-blank one and that we know beforehand whether each of the others might conceivably be blank. We make a modification to our \Fields statement. It can contain not only the field name control sequences but also the tokens + and -, with the following interpretation. A + indicates that all following fields should be re-read until a non-blank result is obtained. A - indicates that any following fields could conceivably be blank, subject to the restriction that the very first field is always non-blank.

Thus the command

\Fields{\a+\b\c-\d}

would indicate that only \d is allowed to be blank, because the + token has no effect. In

\Fields{-\a\b+-\c+\d}

the initial - token enables blank reading of data tokens, but the very first data token is not permitted to be blank anyway. Thus \a is read as a non-blank token and \b as a possibly blank token. The sequence +- now switches non-blank reading on and off again, so \c is read as possibly blank. Finally \d is non-blank.

Another complication we allow is that the \Fields command can appear several times in our file. The interpretation is that the last occurrence of \Fields before we encounter the \Merge command will indicate the fields to be read for every record. Any occurrences of \Fields within the merged text indicate a new list of fields to be read when that command is encountered. This lets us do some conditional processing, such as^{1}

   \ifx\Title\Mrs
      \Fields{\MaidenName}
   \fi

and also gives us some flexibility about the field order later on.

It should also be stressed that the undefined control sequences appearing in the template need not correspond exactly to the fields in the subsidiary file. An example might be that the subsidiary file contains the text

Spriggs, Mr Abraham L

and one field read is \FullName. TEX would then have to pre-process this name to generate its several components as used in the template. The command \PreProcess could be included at the start of the template.

\def\parse#1, #2 #3\endparse{%
   \def\Surname{#1}\def\Title{#2}%
   \def\Forenames{#3}}
\def\PreProcess{\expandafter
   \parse\FullName\endparse}

An alternative and simpler looking approach to reading fields from a file \fil might be to define each such field as follows.

\def\Field#1{\def#1{\read\fil to#1#1}}
\Field\Name \Field\Address \Field\Mark

The first time \Name is encountered, it reads its own expansion from \fil and then expands itself. Henceforth, it has acquired its new expansion. The disadvantage is that \Name must appear in the text before any subsidiary field such as \Surname can be used.

1It is assumed that \Mrs expands to Mrs.

Finally we should consider the possibility that the second parameter of \Merge might be too large to fit into memory. We can clearly handle this problem by allowing the second parameter merely to consist of the text \input template, so that the root file handles two subsidiary files, one containing the template and the other containing the fields.

4 A complicated example

We will next look at an example in which the template contains a table of indeterminate length, albeit fixed width. So far our macros work in either plain TEX or in L^{A}TEX, but the way in which these two packages handle tables is slightly different. However, the only difference that need concern us is that L^{A}TEX uses \\ where plain TEX uses \cr.

\MultiRead The example given here is in L^{A}TEX, but our style will work equally well in plain TEX. In our student letter we wish to insert a table of course codes and marks. Since each student did a different number of courses, we need some way of recognizing the end of the course list in the merge file. The default will be to insert a blank line at the end of such a sub-list. Thus, the following text appears before the close of the letter template.

Here are your marks on individual papers.
\begin{center}
   \begin{tabular}{|lr|}\hline
      Code&Mark\\\hline
      \MultiRead{2}\\\hline
   \end{tabular}
\end{center}

The merge file now has the following structure.

Title
...
Grade
Code
Mark
...
Code
Mark

⟨blank⟩

Title
...

\MarkEnd In other applications some of the fields in the table might possibly be blank. We then let the user change the ⟨blank⟩ line marking the end of a list to some other string of his own choosing.

\MarkEnd{***}

There might be multiple tables in the same template, with their data intermingled in the merge file with main fields. The generalized \Fields command allows us to order the merge file however we want. Thus we could have main fields, then a table, followed by more main fields, and so on.

\Process A final complication is that the fields appearing in a table are essentially anonymous. By this I mean that they are transferred into the table as they are, without any pre-processing possible through appearing in the template as control sequences. If we wish what appears in the table to be different from what appears in the file, a mechanism is needed to tell TEX that a certain column has to be treated in a certain way. The command

\Process{n}{\foo}

will replace every field ⟨f⟩ read into column n by \foo{⟨f⟩}. It is even possible to do some numerical calculations by this method.

Here is a L^{A}TEX example to illustrate the table processing features of

textmerg.sty.

30 ⟨∗example⟩

31 \documentclass[12pt]{article}
32 \usepackage{textmerg}
33 \MarkEnd{***}
34 \Process{2}{\Advance}
35 \def\Advance#1{#1\addtocounter{page}{#1}}
36 \Fields{+\Name\Verb}
37 \begin{document}
38 \Merge{silly.dat}{%
39 Dear \Name,\par
40 Here is a table to \Verb\ at:
41 \Fields{\Width}%
42 \begin{tabular}{*{\Width}c}
43  \MultiRead\Width
44 \end{tabular}.\par
45 \Fields{\Adj}%
46 That was \Adj!
47 \clearpage}
48 \end{document}

49 ⟨/example⟩ The effect of this file is not apparent until we see silly.dat. It is listed here. 50 ⟨∗silly⟩

51 Mike
52 look

53 3 54 1 55 2 56 3

57 11
58 12
59 13
60 ***
61 good
62 Shelagh
63 gaze

64 2

65 21
66 22
67 23
68 24

69 ***
70 horrid

71 ⟨/silly⟩

The same can be done in plain TEX. 72 ⟨∗plainexample⟩

73 \input textmerg
74 \MarkEnd{***}
75 \Process{2}{\Advance}
76 \def\Advance#1{#1\global\advance\count0by#1}
77 \Fields{+\Name\Verb}
78 \Merge{silly.dat}{%
79 Dear \Name,\par
80 Here is a table to \Verb\ at:
81 \Fields{\Width}%
82 \vbox{\halign{\hfil{} ## {}\hfil&&\hfil{} ## {}\hfil\cr
83  \MultiRead\Width\cr
84 }}.\par
85 \Fields{\Adj}%
86 That was \Adj!
87 \vfill\eject}
88 \end

89

90 ⟨/plainexample⟩

References

[Bel87] Edwin V. Bell, II. AutoLetter: A TEX form letter procedure. TUG-Boat, 8(1):54, April 1987.

[Gar87] John S. Garavelli. Form letter macros. TUGBoat, 8(1):53, April 1987.

[Lee86] John Lee. Form letters. TUGBoat, 7(3):187, October 1986.

[McK87] Graeme McKinstry. Form letters. TUGBoat, 8(1):60, April 1987.