A Tool to Manipulate BibTEX Files

Version 2.33

Gerd Neugebauer

Abstract

BibTEX provides an easy to use means to integrate citations and bibliographies into L^{A}TEX documents. But the user is left alone with the management of the BibTEX files. The program BibTool is intended to fill this gap. BibTool allows the manipulation of BibTEX files which goes beyond the possibilities — and intentions — of BibTEX. The possibilities of BibTool include sorting and merging of BibTEX data bases, generation of uniform reference keys, and selecting of references used in a publication.

— This documentation is still in a rudimentary form and needs additional efforts. —

\underline{2}

This file is part of BibTool Version 2.33

Copyright \begin{matrix}c \\ ◯ \end{matrix}1996 Gerd Neugebauer

BibTool is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version.

BibTool is distributed in the hope that it will be useful, but WITHOUT ANY WAR-RANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this documentation; see the file COPYING. If not, write to the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.

Gerd Neugebauer Mainzer Str. 8 56321 Rhens (Germany)

Net: gerd@informatik.uni-koblenz.de
      gerd@imn.th-leipzig.de
        gerd@intellektik.informatik.th-darmstadt.de

Contents

1 Introduction 5 1.1 Related Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Using BibTool — Some Instructive Examples . . . . . . . . . . . . . . . . . 6

1.2.1 Sorting and Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2.2 Key Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2.3 Extracting Entries for a Document . . . . . . . . . . . . . . . . . . . 9 1.2.4 Extracting Entries Matching a Regular Expression . . . . . . . . . . 9 1.2.5 Translating ISO8859-1 Characters . . . . . . . . . . . . . . . . . . . 9

A Reference Manual 11 A.1 Command Line Usage and Resource Files . . . . . . . . . . . . . . . . . . . 11 A.2 Input File Specification and Search Path . . . . . . . . . . . . . . . . . . . . 13 A.3 Output File Specification and Status Reporting . . . . . . . . . . . . . . . . 14 A.4 Parsing and Pretty Printing . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 A.5 Key Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 A.6 Format Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

A.6.1 Constant Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 A.6.2 Formatting Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 A.6.3 Expanding TEX/L^{A}TEX Macros . . . . . . . . . . . . . . . . . . . . . 21 A.6.4 Conjunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 A.6.5 If-Then-Else . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 A.6.6 Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 A.6.7 Grouping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

A.7 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 A.8 Selecting Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

A.8.1 Extracting by aux Files . . . . . . . . . . . . . . . . . . . . . . . . . 24 A.8.2 Extracting with Regular Expressions . . . . . . . . . . . . . . . . . . 25

A.9 Regular Expression Matching . . . . . . . . . . . . . . . . . . . . . . . . . . 26 A.10 Field Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

A.10.1 Adding or Deleting Fields . . . . . . . . . . . . . . . . . . . . . . . . 27 A.10.2 Field Rewriting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 A.10.3 Field Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

A.11 Semantic Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 A.11.1 Finding Double Entries . . . . . . . . . . . . . . . . . . . . . . . . . 30 A.11.2 Regular Expression Checks . . . . . . . . . . . . . . . . . . . . . . . 30

A.12 Strings — also called Macros . . . . . . . . . . . . . . . . . . . . . . . . . . 31 A.13 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

B Limitations 33 B.1 Limits of BibTool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 B.2 Bugs and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

C Sample Resource Files 35 C.1 The Default Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 C.2 Useful Translations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 C.3 Other Resource Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Introduction

1.1 Related Programs

BibTEX [Lam94, Pat88a, Pat88b] is a system for integrating bibliographic information into L^{A}TEX [Lam94] documents. BibTEX is designed to serve exactly this purpose. It has shown that various tasks in relation with managing bibliographic databases are not covered by BibTEX. Usual activities on bibliographic databases include

• inserting new entries

• editing

• using citations in documents

• sorting and merging of bibliographic data bases

• extraction of bibliographic data bases

Since only the integration in documents is covered by BibTEX several utilities emerged to fill the gaps. We will sketch some of them shortly.

BibTEX is a program by Oren Patashnik to select publications used in a L^{A}TEX document and format them for inclusion into this document.. This program should be part of each TEX installation.

bibclean is a program by Nelson H.F. Beebe to prettyprint BibTEX files. It also can act as syntax checker. The C sources can be compiled on several systems.

bibindex/biblook is a pair of program by Nelson H.F. Beebe to generate an index for a BibTEX file and use it to perform a fast lookup of certain entries. The programs so far run only under UNIX. bibsort is a UNIX shell script by Nelson H.F. Beebe to sort a BibTEX file.

bibclean is a UNIX shell script by Nelson H.F. Beebe to extract entries from a BibTEX file which are used in a L^{A}TEX document.

lookbibtex/bibdestringify are Perl scripts by John Heidemann to extract entries from a BibTEX file which are used in a L^{A}TEX document and to remove strings from a BibTEX file.

bibtools is a collection of UNIX shell scripts by David Kotz to add and extract records to bibliographic databases. Several small programs are provided to perform special tasks.

bibview is a Perl script by Dana Jacobsen to extract entries from a BibTEX file which are used in a L^{A}TEX document.

BibCard is a program by William C. Ogden running under X11/xview which provides a means to edit bibliographic databases.

hyperbibtex Something similar for Macintosh computers.

xbibtex/bibprocess/bibsearch are programs by Nicholas J. Kelly and Christian H. Bischof running under X11 which provides a means to edit bibliographic databases, add fields to a BibTEX file and extract certain entries from a BibTEX file.

bibview is an X11 program by Holger Martin, Peter Urban, and Armin Liebl to search in and manipulate BibTEX files.

BibTEX-Mode is an extension of the editor GNU-Emacs to provide means to edit BibTEX files. Several useful operations are provided.

Some additional information can be found in [GMS94, Chapter 13].

Most of those utilities are tailored towards a special operating system and thus they are not available on other platforms. Most of these program are made to perform a single task. Often they can not be configured to suit a personal taste of a user.

Still there are some points not covered by the utilities mentioned above. BibTool tries to provide the missing features and integrate others into a single tool.

1.2 Using BibTool — Some Instructive Examples

BibTool has been developed on UN*X and UN*X-like machines. This has influenced many of the design decisions. Version 1 has been controlled using numerous command line options. This way of controlling has been supplemented in version 2 by the concept of a resource file. This resource file allows the modification of various internal parameters determining the behavior of BibTool.

When BibTool has been compiled correct there should be an executable file named bibtool^{1}. We will assume that you are running BibTool from a command line interpreter. There you can simply issue the command

       bibtool

Now BibTool will start reading from the standard input lines obeying the rules of a BibTEX file.^{2} The entries read are pretty-printed on the standard output. It is obvious that this behavior is not very useful in itself. The origin of this kind of interface lies in the concepts of UN*X where many commands can act as filters.

Usually we do not intend to use BibTool in this way. Thus we need a way to specify an input file. This is simply done by adding the file name as argument after the command name like in

       bibtool file.bib

The result of this command can at once be seen on the screen. The contents of the file file.bib is pretty printed.

After we have seen the simplest case of the application of BibTool we will the the simplest case of a useful application of BibTool. This application is the syntax check and pretty printing

1.2.1 Sorting and Merging

BibTEX files can be sorted by specifying the the command line option -s. The given files are sorted according to the reference key. Several files can be given at once. If those files are already sorted then the result of BibTool is merging the files given.

bibtool -s file1.bib file2.bib

1.2.2 Key Generation

Once you have a reference and you insert it into a BibTEX file you have to assign a reference key to it. The problem is to find a key which is unique and meaningful, i.e. easy to remember. The easiest way to remember a key is to use an algorithm to create it and remember the algorithm — which is the same for all keys.

One algorithm which comes to mind is to use the author and (an initial part) of the title. Alternatively we can use the author and the year. But the problem are industrious authors writing more than one publication per year. The necessary disambiguation of such references is not very intuitive.

BibTool contains the possibility to describe desired keys. Thus the alternatives described above can be realized.

For this section we want to use the following BibTEX entry as our example:^{3} Suppose it is contained in a file named sample.bib.

1Maybe with an additional extension.

2We assume that no resource file can be found. Resource files will be described later.

3Shamelessly stolen from the BibTEX xamples.bib file.

@ARTICLE{article-full,
   author = {L[eslie] A. Aamport},
   title = {The Gnats and Gnus Document Preparation System},
   journal = {\mbox{G-Animal's} Journal},
   year = 1986,
   volume = 41,
   number = 7,
   pages = "73+",
   month = jul,
   note = "This is a full ARTICLE entry",
}

First, we want to see how we can make keys consisting of author and title. This is one of my favorite algorithms thus it is rather easy to use it. You simply have to run the command

       bibtool -k sample.bib -o sample1.bib

Afterwards the following entry can be found in the file sample1.bib:

@Article{         aamport:gnats,
  author        = {L[eslie] A. Aamport},
  title         = {The Gnats and Gnus Document Preparation System},
  journal       = {\mbox{G-Animal's} Journal},
  year          = 1986,
  volume        = 41,
  number        = 7,
  pages         = "73+",
  month         = jul,
  note          = "This is a full ARTICLE entry"
}

You see that the reference key has been changed. It now consists of the last name and the first relevant word of the title, separated by a colon. Sometimes it might be desirable to incorporate the initial names as well. This can be achieved by the command

       bibtool -K sample.bib -o sample1.bib

The resulting reference key is aamport.la:gnats. The initials are appended after the first name. Thus the usual lexicographic order on the keys will (hopefully) bring together the publications of the same first author.

Another alternative is to use the author and the year This can be achieved with the following command:^{4}

       bibtool -f %n(author):%2d(year) sample.bib -o sample1.bib

The resulting key is Aamport:86. Note that the last example work as desired for our sample file. But for a real application of this technique a deep understanding of the key generation mechanism as described in section A.5 is necessary.

4Note that some command line interpreters (like the UN*X shells) require the format string to be quoted (enclosed in single quotes).

1.2.3 Extracting Entries for a Document

BibTool can be used to extract the references used in a document. For this purpose BibT ool analyzes the .aux file and takes the information given there. This includes the names of the BibTEX files. Thus no BibTEX files have to be given in the command line. Instead the .aux file has to specified — preceded by the option -x.

bibtool -x document.aux -o document.bib The second option -o followed by a file name specifies the destination of the output. I.e.

instead of writing the result to the standard output stream the result is written into this file.

1.2.4 Extracting Entries Matching a Regular Expression

BibTool can be used to extract the references which fulfill certain criteria. Those criteria can be specified utilizing regular expressions.^{5} As a special case we can extract all entries containing a certain substring of the key:

bibtool -X tex all.bib -o some.bib This instruction selects all records containing the substring tex in the key. The second

option -o followed by a file name specifies the destination of the output. I.e. instead of writing the result to the standard output stream the result is written into this file.

Next we want to look up all records containing a substring in some of its fields. For this purpose we search for the string in all fields first:^{6}

bibtool -- select{”tex”} all.bib -o some.bib Note that the comparison is not done case sensitive. Nevertheless this can be customized

(see page 25).

Finally we want to select only those records containing the substring in one of certain fields. For this purpose we simply specify the names of those fields in the select instruction:

bibtool -- select{title booktitle $key ”tex”} all.bib -o some.bib This example extracts all records containing the substring tex in the title field, the

booktitle field, or the reference key.

After we have come so far we can say that the first example in this section is in fact a short version of the following command:

bibtool -- select{$key ”tex”} all.bib -o some.bib

1.2.5 Translating ISO8859-1 Characters

Sometimes you need to translate some special characters into BibTEX sequences. Suppose you have edited a BibTEX file and by mistake used those nice characters that are incompatible with standard ASCII as used in BibTEX. You can use BibTool to do the trick:

5Those features are only usable if the regular expression library has been enabled during the configuration of BibTool—which is the default.

6Note that some command line interpreters (e.g the UN*X shells) might need additional quoting of the select instruction since it contains special characters.

10 1. Introduction

bibtool -r iso2tex -i iso.bib -o ascii.bib

Reference Manual

A.1 Command Line Usage and Resource Files

BibTool cam be controlled either by arguments given in the command line or by commands given in a file. Those command files are called resource files. If BibTool is installed correctly you should have the executable command bibtool (maybe with an additional extension). Depending on your computer and operating system you can start This command in different ways. This can be done either by issuing a command in a command line interpreter (shell), by clicking an icon, or by selecting a menu item. In the following description we will concentrate on the use in a UN*X like shell. There you can type simply

    bibtool

Now BibTool is waiting for your input. As you type BibTool read what you type. This input is interpreted as data conforming BibTEX file rules. The result is immediately printed when an entry is finished. You can terminate the reading phase with your End-Of-File character (e.g. Control-C on UN*X, or Control-Z on MS-D*S) This application in itself is rather uninteresting. Thus we come to the possibility to give arguments to BibTool. The simplest argument is -h as in

    bibtool -h

This command should print the version number and a short description of the command line arguments to the screen. The next application is the specification of resources. Resource files can be given in the command line after the flag -r.

bibtool -r resource file In this way an arbitrary number of resource files can be given. Those resource files are read in turn and the commands contained are evaluated. If no resource file is given in the command line BibTool tries to find one in standard places. First of all the environment variable BIBTOOLRSC is searched. If it is defined then the value is taken as a resource file name. Otherwise the default resource file (usually the file .bibtoolrsc) is tried to be read in the home directory (determined by the environment variable HOME) or the current directory. The resource files are searched similar to the searching mechanism for BibTEX files (see section A.2). The extension .rsc is tried and a search path can be used. This search path is initialized from the environment variable BIBTOOL. Initially only the current directory is on the search path. The search path can also be set in a resource file (for following resource file reading). This can be achieved by setting the resource resource.search.path.

resource.search.path = path When an explicit resource file is given in the command line the defaults are not used. To incorporate the default resource searching mechanism the command line option -R can be used:

    bibtool -R

Now we have to describe the commands allowed in a resource file. The general form of a resource command is of the form

name = value name is the resource name which conforms the rules of BibTEX reference keys. I.e. name can be composed of all characters but whitespace characters and the exceptions

    "  #  %  '  (  )  ,  =  {  }

Our resource names are currently composed of letters and the period. The next component is the equality sign (=). This equality sign is optional. Nevertheless it helps detecting syntax problems. Whitespace characters surrounding the equality sign or separating resource name and resource value are ignored. The resource value can be of the following kind:

• A number composed of digits only.

• A string conforming the rules of resource names, i.e. made up of all but the forbidden characters described above.

• An string containing arbitrary characters delimited by double quotes (”) not containing double quotes. Parentheses and curly brackets have to come in matching pairs.

• An string containing arbitrary characters delimited by curly brackets ({}). Parentheses and curly brackets have to come in matching pairs.

You can think of resource names as variables or functions in a programming language. Resource commands simply set the variables to the given value, add the value to the old value, or initiate a action. There are different types of resources

• Boolean resources can take only the values on and off. on, t, true, 1, and yes are interpreted the same. Everything else is interpreted as off.

• Numeric resources can take numeric values only.

• String resources can take arbitrary strings.

Usually whitespace characters are ignored. There is one exception. The characters % and # act as comment start characters if given between resource commands. All characters to the end of the line are ignored afterwards. Now we come the description of the first resource available. To read in additional resource files the resource file may contain the resource

resource = {additional/resource/file}

Thus the resource given above has the same functionality as the command line option -r described above. One resource command useful for debugging is the print resource. The resource value is immediately written to the error stream. The output is terminated by a newline character. Certain translations are performed during the reading of a resource which can be observed when printing. Each sequence of whitespace characters is translated into a single space. To end this subsection we give an example of the print resource. In this sample we also see the possibility the omit the equality sign and use quotes as delimiters.

print ”This is a stupid message.”

Finally we can note that the commands given in a resource file can also be specified on the command line. This can be achieved with the command line option -- The next command line argument is taken as a resource command.

bibtool -- resource command

Be aware that command line interpreters have different ideas about what to do with a command line before passing the arguments to a program. Thus it might be necessary to carefully quote the arguments. Especially if the command contains spaces it is very likely that quoting is needed.

A.2 Input File Specification and Search Path

An arbitrary number of input files can be specified. Input files can be specified in two ways. The command line option -i is immediately followed by a file name.

    bibtool -i input file

The resource name input can be used to specify additional input files.

input = {input file}

Input files are processed in the order they are given. If no input file is specified the standard input is used to read from. The files are searched in the following way. The file is can’t be opened as given then the extension .bib is appended and another read is tried. In addition directories can be given which are searched for input files. The search path can be given in two different ways. First, the resource name bibtex.search.path can be set to contain a search path specification.

bibtex.search.path = {directory1:directory2:directory3} The elements of the search path are separated by colons. Thus colons are not allowed as parts of directories. Another source of the search path is the environment variable BIBINPUTS. This environment variable is usually used by BibTEX to specify the search path. The syntax of the specification is the same as for the resource bibtex.search.path. To check the appropriate way to set your environment variable consult the documentation of your shell, since this is highly dependent on it. To allow adaption to operating systems other than UN*X the following resources can be used. The name of the environment bibtex.env.name overwrites the name of the environment variable which defaults to BIBINPUTS.

bibtex.env.name = {ENVIRONMENT VARIABLE} The first character of the resource env.separator is used as separator of directories in the resource bibtex.search.path and the environment variable given as bibtex.env.name.

env.separator = {:} The default character separating directories in a file name is the slash (/). The first character of the resource dir.file.separator can be used to change this value.

dir.file.separator = {\} Note that the defaults for env.separator and dir.file.separator are set at compile time to a value suitable for the operating system. Usually you don’t have to change them at all. E.g. for MSDOS machines the env.separator is usually set to ; and the dir.file.separator is usually set to \.

A.3 Output File Specification and Status Reporting

Several output streams can be used to write output to. The processed BibTEX entries are written to the standard output. This output can be redirected to a file using the command line option -o as in

    bibtool -o output file

The resource name output.file can be used for the same purpose.

output.file = {output file} No provisions are made to check if the output file is the same as a input file. A second output stream is used to display error messages and status reports. The standard error stream is used for this purpose. The messages can roughly be divided in three categories: error messages, warnings, and status reports. Error messages indicate severe problems. They can not be suppressed. Warnings indicate possible problems which could (possibly) have been corrected. Usually the are displayed but they can be suppressed. Status reports are messages during the processing which indicate actions currently performed. Usually the are suppressed but they can be enabled. Warning messages can be suppressed using the command line option -q. This option toggles the boolean quiet value.

    bibtool -q

The same effect can be obtained by assigning the value on or off to the resource quiet:

quiet = on Status reports are useful to see the operations performed. They can be enabled using the command line option -v. This option toggles the boolean verbose value.

    bibtool -v

The same can also be achieved with the boolean resource verbose:

verbose = on Another output stream can be used to select the string definitions. This is described in section A.12 on macros. For completeness we can also mention that the internal symbol table can be printed using the command line option -$ or the boolean resource dump.symbols. This is mainly meant for debugging purposes. Please send me a bug report and the diffs to fix it :-)

A.4 Parsing and Pretty Printing

The first and simplest task we have to provide on BibTEX files is the parsing and pretty printing. This is not superfluous since BibTEX is rather pedantic about the accepted syntax. Thus I decided to try to be generous and correct as may errors as I can. Each input file is parsed and stored in an internal representation. BibTEX simply ignores any characters between entries. Thus BibTool mainly adopts this behavior. This kind of comments is not represented internally. Normally anything between entries is simply discarded and a warning printed. The boolean resource pass.comments can be used to change this behavior.

pass.comments = on If this resource is on then the characters between entries are directly passed to the output file. This transfer starts with the first non-space character after the end of an entry. The standard BibTEX styles support a limited number of entry types. Those are predefined in BibTool. Additional entry types can be defined using the resource new.entry.type as in

new.entry.type = {Anthology} This option can also be used to redefine the appearance of entry types which are already defined. Suppose we have defined Anthology as above. Afterwards we can redefine this entry type to be printed in upper case with the following option:

new.entry.type = {ANTHOLOGY} Each undefined entry type leads to an error message. The internal representation is printed in a format which can be adjusted by certain options. Those options are available through resource files or by specifying resources on the command line.

print.line.length This numeric resource specifies the desired width of the lines. lines which turn out to be longer are tried to split at spaces and continued in the next line. The value defaults to 77. print.indent This numeric resource specifies indentation of normal items, i.e. items in entries

which are no strings or comments. The value defaults to 2.

print.align This numeric resource specifies the column at which the ’=’ in non-comment and non-string entries are aligned. This value defaults to 18.

print.align.key This numeric resource specifies the column at which the ’=’ in non-comment and non-string entries are aligned. This value defaults to 18.

print.align.string This numeric resource specifies the column at which the ’=’ in string entries are aligned. This value defaults to 18.

print.align.preamble This numeric resource specifies the column at which preamble entries are aligned. This value defaults to 11.

print.align.comment This numeric resource specifies the column at which comment entries are aligned. This value defaults to 10.

print.newline This numeric resource specifies the number of newlines between records. This value defaults to 1.

print.use.tab This boolean resource specifies if the TAB character should be used for indenting. This use is said to cause portability problems. Thus it can be disabled. If disabled then the appropriate number of spaces are inserted instead. This value defaults to on.

The resource values described above are illustrated by the following examples. First we look at a string entry.

@STRING{macro   = "This is a rather long replacement text which exceeds one
                  line"}

print.align.string print.line.length

Next we look at an unpublished entry. It has a rather long list of authors and a long title. It shows how the lines are broken.

@Unpublished{     unpublished-key,
  author        = "First A. U. Thor and Seco N. D. Author and Third A. Uthor
                  and others",
  title         = "This is a rather long title of an unpublished entry which
                  exceeds one line",
  note          = "Some useless comment"
}

print.indent  print.align  print.line.length

The field names of an entry are usually printed in lower case. This can be changed with the resource new.field.type. The argument of this resource is an equation where left of the ’=’ sign is the name of a field and on the right side is it’s print name. They should only contain allowed characters.

new.field.type { author = AUTHOR } This feature can be used to rewrite the field types. Thus it is completely legal to have a different replacement text than the original field:

new.field.type { OPTauthor = Author }

String names are used case insensitive by BibTEX. BibTool normalizes string names before printing. Per default string names are translated to lower case. Currently two other types are supported: translation to upper case and translation to mixed case, i.e. the first letter upper case and the others in lower case. The translation is controlled by the resource symbol.type. The value is one of the strings lower, upper, and cased. The resource can be set as in

symbol.type = upper

The macro names are passed through the same normalization apparatus as field types. Thus you can force a rewriting of macro names with the same method as described above. You should be careful when choosing macro names which are also used as field types. The reference key is usually translated to lower case letters unless a new key is generated (see section A.5). In this case the chosen format determines the case of the key. Sometimes it can be desirable to preserve the case of the key as given (even so BibTEX does not mind). This can be achieved with the boolean resource preserve.key.case. Usually it is turned off (because of backward compatibility and the memory used for this feature). You can turn it on as in

preserve.key.case = on

If it is turned on then the keys as they are read are recorded and used when printing the entries.

A.5 Key Generation

The key generation facility provides a mean to uniformly replace the reference keys by generated values. Two algorithms are compiled in. Additionally a free formatting facility is provided. This can be used to specify your own algorithm to generate keys. The generation of new keys can be enabled using the command line option -f in the following way:

    bibtool -f format

This command adds format disjunctively to the formatting instructions already given. The same effect can be achieved with the resource key.format.

key.format = {format}

Some values of format have a special meaning. Fixed formatting rules are used when one of them is in effect. The special values are described below.

short If an author or editor field are present, then this field is used. The short version uses last names only. Afterwards a title or booktitle field is appended, after the fmt.name.title separator has been inserted. Finally if all else fails then the default key default.key is used. The result is disambiguated. long The long version acts like the short version but incorporates initials when formatting

names.

short.need This version formats like short but only if the given key field is empty.

long.need This version formats like long but only if the given key field is empty.

empty The empty version clears the key entirely. The result does not conform to the BibTEX syntax rules. This feature can be used to generate keys only if required.

In contrast to the command line option the resource instruction only modifies the formatting specification. The key generation has to be activated explicitly. This can be done using the command line option -F as in

    bibtool -F

Alternatively the boolean resource key.generation can be used in a resource file:

key.generation = on As we have seen there are options to adapt the behavior of formatting. Before we will explain the free formating specification in section A.6 we will present the formatting options. Those options can be activated from a resource file exclusively.

default.key The value of this resource is used by the styles short and long if nothing else can be used. The default value is **key*.

key.base The value of this resource is used to determine the kind of formatting the disambiguating number. Possible values are upper, lower, and digit. Uppercase letters, lower case letters, or digits are used respectively.

key.number.separator The value of this resource is used to separate the disambiguating number from the rest of the key. The default value is *.

key.expand.macros The value of this boolean resource is used to indicate whether macros should be expanded while generating a key. The default value is off.

fmt.name.title The value of this resource is used by the styles short and long to separate names and titles. The default value is :.

fmt.title.title The value of this resource is used to separate words inside titles. The default value is :.

fmt.name.name The value of this resource is used to separate different names (where the BibTEX file has and) when formatting names. The default value is ..

fmt.inter.name The value of this resource is used to separate parts of multi-word names when formatting names. The default value is -.

fmt.name.pre The value of this resource is used to separate names and first names when formatting names. The default value is .. fmt.et.al The value of this resource is used to format and others parts of a name list. The

default value is .ea.

The key style short can be formulated in terms of the format specification given in section A.6 as follows:

{
  { %-2n(author)
  # %-2n(editor)
  }
  { %s($fmt.name.title) %-1T(title)
  # %s($fmt.name.title) %-1T(booktitle)
  #
  }
}
#
{ { %s($fmt.name.title) %-1T(title)
  # %s($fmt.name.title) %-1T(booktitle)
  }
}
# %s($default.key)

The syntax and meaning of such format specifications is explained in section A.6.

A.6 Format Specification

A.6.1 Constant Parts

The simplest component of a format is a constant string. Such strings are made up of any character except whitespace and the following ten characters

    "  #  %  '  (  )  ,  =  {  }

This choice of special characters is the same as the special characters of BibTEX. Since no means is provided to include a special character into a format string we guarantee that the resulting key string is conform to the BibTEX rules. For example the following strings are legal constant parts of a format:

  Key
  the_name.of-the-@uthor-is:

Now we come to explain the meaning of the special characters. The first case consists of the whitespace characters. They are simply ignored. Thus the following format strings are equal:^{1}

  Author Or Editor
  AuthorOrEditor
  A u t h o r   O r   E d i t o r

1Well, this is not the whole truth. Internally it makes a difference whether there is a space or not. In the presence of spaces more memory is used. But you shouldn’t worry too much about this.

A.6.2 Formatting Fields

The next component of formats are made up formating instructions which are starting with a % character. The general idea has been inspired by formating facilities of C. Since there are several different types of information in a BibTEX entry we provide several primitives for formating. The simplest form is for instance

  %N(author)

The % character is followed by a single character — here N — which indicates the way of formating and the name of the field to be formatted enclosed in parenthesis. The example above requests to format the field author according to formatting rules for names (N). The general form is

%sign pre.post letter(field)

In this specification sign is + or -. + means that all characters will be translated to upper case. - means that all characters will be translated to lower case. If no sign is given, the case of the field is preserved. pre and post are positive integers whose meaning depends on the format letter letter. letter is a one letter specification indicating the desired formatting type. Possible values are as described in the following list:

n Format last names only.

In a list of names at most pre last names are used. If there are more names they are treated as given as and others. If post is greater than 0 then at most post characters per name are used. Otherwise the whole name is considered.

pre defaults to 2 and post defaults to 0.

N Format names with last names and initials.

In a list of names at most pre last names are used. If there are more names they are treated as given as and others. If post is greater than 0 then at most post characters per name are used. Otherwise the whole name is considered.

pre defaults to 2 and post defaults to 0.

d Format a number, e.g. a year.

The first number in the field is searched. At most pre digits — counted from the right — are used. E.g. the field "june 1958" formatted with %2d results in 58.

pre defaults to a large number.

s Take a field as is (after translation of special characters). At most pre characters are used.

pre defaults to a large number.

T Format sentences. Certain words are ignored.

At most pre words are used. The other words are ignored. If pre is 0 then no artificial limit is forced. If post is positive then at most post letters of each word are considered. Otherwise the complete words are used.

New words to be ignored can be added with the resource ignored.word.

pre defaults to 1 and post defaults to 0.

t Format sentences. In contrast to the format letter T no words are ignored. At most pre words are used. The other words are ignored. If pre is 0 then no artificial limit is forced. If post is positive then at most post letters of each word are considered. Otherwise the complete words are used.

pre defaults to 1 and post defaults to 0.

W Format word lists.

This specifier acts like T except that nothing is inserted between words.

w Format word lists.

This specifier acts like t except that nothing is inserted between words.

The field specification (field) selects the field of the entry to be formatted. As usual in BibTEX the case of the letters is ignored. If the field does not exist then the formatting fails and continues at the next alternative (see below). In addition to the ordinary fields of an entry there are several pseudo fields.

$key This pseudo field contains the old reference key — before generating a new one. If none has been given then the access fails.

$default.key This pseudo field contains the value of the resource default.key similarly the resources fmt.name.title, fmt.title.title, fmt.name.name, fmt.inter.name, fmt.name.pre, and fmt.et.al can be accessed.

$type This pseudo field contains the type of the entry, i.e. the string following the initial @ of an BibTEX entry. E.g. article. It is always present.

@type This pseudo field is matched against the type of the entry. If they are identical (ignoring cases) then the type is returned. Otherwise the access fails.

In an article item the specification %s(@Article) succeeds and returns Article whereas %s(@Book) fails.

A.6.3 Expanding TEX/L^{A}TEX Macros

When fields are formatted certain L^{A}TEX macros may be replaced by pure text. Each macro not defined is simply ignored. Initially no L^{A}TEX macro is defined. The resource tex.define can be used to define L^{A}TEX macros. The syntax is very close to L^{A}TEX. The simplest form is the following definition.

tex.define = {macro=replacement text}

This resource defines a simple macro which is replaced by the replacement text. This replacement text may in turn contain macros. In addition to this simple macro also macros involving arguments can be defined. As in L^{A}TEX’s \newcommand the number of arguments is appended after the macro name.

tex.define = {macro[arg]=replacement text}

The number of arguments may not exceed 9. The actual parameters are addressed by writing #n, where n is the number of the argument. For instance, this feature can be used to ignore certain arguments of macros. Note that there should be no spaces between the macro head and the equality sign (=). Similarly any unwanted spaces after the equality sign may have strange effects. Usually the macro name starts with a backslash (\). If the macro name starts with another character then this character is made active (cf. [Knu89]). This feature is especially useful for translating characters with an extended ASCII code (≥ 128) to the appropriate TEX macros. For instance the following definition forces the expansion of the macro \TeX to the string

TeX.

tex.define = {\TeX=TeX}

Without this definition the title The \TeX{}book would result in book. With this definition the same title results in TeXbook. Suppose you have an input file containing 8-bit characters (e.g. ISO 8859-1 encoding). The following definition can be used to map this character into a pure ASCII string^{2}

tex.define = {ü=ue}

Some useful definitions can be found in the libraries distributed with BibTool (see also appendix C).

A.6.4 Conjunctions

Conjunctions are formatting instructions evaluated in sequence. The conjunctions are simply written by successive formatting instructions. A conjunction succeeds if every part succeeds. The empty conjunction always succeeds. Suppose an BibTEX entry contains fields for editor and year. Then the following conjunction succeeds:

     %-3n(editor) : %2d(year)

If the value of the editor field is "E.D. Itor" and the year field contains "1992" then the result is itor:92.

2To add an e is the German convention for umlaut characters.

A.6.5 If-Then-Else

Depending on the presence of a (pseudo-) field formatting instructions can be issued. This corresponds to an if-then-else statement in a Pascal-like language. The syntax is as follows:

(field){then-part} {else-part}

If the access to the (pseudo-) field as described in A.6.2 succeeds then the then-part is evaluated. Otherwise the else-part is evaluated. Both parts may be empty. Nevertheless the braces are required. Let us look at an example. The following construction can be used to format a field author if it is present or print an constant string.

     (author){%N(author)}{--no-author--}

A.6.6 Alternatives

Alternatives (disjunctives) are separated by the hash mark (#). The general form is

alternative-1 # alternative-2 # ... # alternative-n

The alternatives are evaluated from left to right. The first one that succeeds terminates the processing of all alternatives with success. If no alternative is successful then the whole construct fails. An alternative can be empty. The empty alternative succeeds without any other effect. The example given in subsection A.6.5 can be also written as

     %N(author) # --no-author--

If the author field is accessible the the first alternative succeeds and terminates the construct. Otherwise the constant string is used. This constant string always succeeds.

A.6.7 Grouping

Any number of constructs can be enclosed in braces ({}) for grouping. Thus the precedence of operators can be bypassed. Coming back to our example from the previous subsection. To complicate the example we want to append an optional title, or an constant string. This is accomplished as follows.

     { %N(author) # --no-author-- }  { %T(title) # --no-title-- }

The grouping allows to restrict the range of the alternative operator # in this example.

A.7 Sorting

The entries can be sorted according to a certain sort key. The sort key is by default the reference key. Sorting can enabled with the command line switches -s and -S as in

    bibtool -s
    bibtool -S

The first variant sorts in ascending ASCII order (including differentiation of upper and lower case). The second form sorts in descending ASCII order. The same effect can be achieved with the boolean resource values sort and sort.reverse respectively.

sort = {on}

sort.reverse = {on}

An alternate format of the sort key can be specified like for the key format described in section A.5. The associated resource name is sort.format. Several formats are combined as alternatives.

sort.format = {%N(author)}

sort.format = {%N(editor)}

Those two lines are equivalent with the single resource

sort.format = {%N(author) # %N(editor)}

A.8 Selecting Items

A.8.1 Extracting by aux Files

BibTool includes a module to extract BibTEX entries required for a document. This is accomplished by analyzing the aux file of the document. The aux file is usually produced by L^{A}TEX. It contains the information which BibTEX files and which references are used in the document. Only those entries mentioned in the aux file are selected for printing. Since the BibTEX files are already named in the aux file it is not necessary to specify an input file. To use an aux file the command line option -x can be given. This option is followed by the name of the aux file.

    bibtool -x file.aux

Multiple files can be given this way. As always the same functionality can be requested with a resource. The resource extract.file can be used for this purpose.

extract.file = {file.aux}

One special feature of BibTEX is supported. If the command \nocite{*} is given in the L^{A}TEX file then all entries of the bibliography files are included in the bibliography. The same behavior is imitated by the extracting mechanism of BibTool.

A.8.2 Extracting with Regular Expressions

Another selecting mechanism uses regular expressions to select items. This feature can be used in addition to the selection according to aux files. The regular expression syntax is identical to the one used in GNU Emacs. For a description see section A.9. The resource select allows to specify which fields should be used to select records. The general form is as follows:

select = {field_{1} … field_{n} ”regular expression”}

If no field is specified then the regular expression is searched in each field. Any number of selection rules can be given. A record is selected if one of those rules selects it. The select rule selects a record if this record has a field named field which has a substring matching regular expression. The field can be missing in which case the regular expression is tried to match against any field in turn. The pseudo fields $key, $type, and @type can be used to access the key and the type of the record. See page 21 for details. The routines used there are the same as those used here. The boolean resource select.case.sensitive can be used to determine whether the selection is performed case sensitive or not:

select.case.sensitive = off

A regular expression can be specified in the command line using the option -X as in

bibtool -X regular expression

The fields compared against this regular expression are given in the string valued resource select.fields. Initially this resource has the value $key. In general the value is a list of fields and pseudo fields to be considered. The elements of the list are separated by spaces. If the list is empty then all fields and the key are considered for comparison. Thus the following setting means that the regular only the fields author and editor are considered when doing a selection.

select.fields = ”author editor”

Without changing the resource select.fields the command line given previously is equivalent to the (longer) command

bibtool -- {$key ”regular expression”}

Note that the resources select.case.sensitive and select.fields are used for all regular expressions following their definition until they are redefined. This means that it is possible to specify that some comparisons are done case sensitive and others are not done case sensitive. Finally the resource extract.regex can be used as in

extract.regex = {regular expression}

This is equivalent to specifying a single regular expression to be matched against the key. This feature is kept for backward compatibility only. It is not encouraged and will vanish in a future release.

A.9 Regular Expression Matching

A concise description of regular expressions is contained in the document Regex/regex.texi contained in the BibTool distribution. In any cases of doubt this documentation is preferable. The remainder of this section contains a short description of regular expressions.

Ordinary characters match only to themself or their upper or lower case counterpart. Any character not mentioned as special is an oridinary character. Amoung others letters and digits are ordinary characters.

E.g. the regular expression abc matches the string abc.

The period (.) matches any single character.

E.g. the regular expression a.c matches the string abc but it does not match the string abbc.

The star (*) ist used to denote any number of repetitions of the preceeding regular expression. If no regular expression preceeds the star then it is an ordinary character.

E.g. the regular expression ab*c matches any string which starts with a followed by an arbitrary number of b and ended by a c. Thus it matches ac and abbbc. But it does not match the string abcc.

The plus (+) is used to denote any number of repetitions of the preceeding regular expression, but at least one. Thus it is the same as the star operator except that the empty string does not match. If no regular expression preceeds the plus then it is an ordinary character.

E.g. the regular expression ab+c matches any string which starts with a followed by one or more b and ended by a c. Thus it matches abbbc. But it does not match the string ac.

The question mark (?) is used to denote an optional regular expression. The preceeding regular expression matches zero or one times. If no regular expression preceeds the question mark then it is an ordinary character.

E.g. the regular expression ab?c matches any string which starts with a followed by at most one b and ended by a c. Thus it matches abc. But it does not match the string abbc.

The bar (|) separates two regular expressions. The combined regular expresion matches a string if one of the alternative separated by the bar does.

E.g. the regular expression abc—def matches the string abc and the string def.

Parentheses (()) can be used to group regular expressions. A group is enclosed in parentheses. It matches a string if the enclosed regular expression does.

E.g. the regular expression a(b—d)c matches the strings abc and adc.

The dollar ($)matches the empty string at the end of the string. It can be used to anchor a regular expression at the end. If the dollar is not the beginning of the regular expression then it is an ordinary character.

E.g. the regular expression abc$ matches the strings aaaabc but does not match the string abcdef.

The hat (^) matches the empty string at the beginning of the string. It can be used to anchor a regular expression at the beginning. If the hat is not the beginning of the regular expression then it is an ordinary character. There is one additional context in which the hat has a special meaning. This context is the list operator described below.

E.g. the regular expression ^abc matches the strings abcccc but does not match the string aaaabc.

The brackets ([]) are used to denote a list of characters. If the first character of the list is the hat (^) then the list matches any character not contained in the list. Otherwise it matches any characters contained in the list.

E.g. the regular expression [abc] matches the single letter strings a, b, and c. It does not match d.

The regular expression [ˆ abc] matches any single letter string not consisting of a, b, or c.

The backslash (\) is used for several purposes. Primarily it can be used to quote any special character. I.e. if a special character is preceded by the backslash then it is treated as if it were an ordinary character.

If the backslash is followed by a digit d then this construct is the same as the dth matching group.

E.g. the regular expression (an)\1as matches the string ananas since the first group matches an.

If the backslash is followed by the character n then this is equivalent to entering a newline.

If the backslash is followed by the character t then this is equivalent to entering a single tab character.

A.10 Field Manipulation

A.10.1 Adding or Deleting Fields

Certain fields can be added or deleted. This feature can be used to update time stamps. For this purpose it is important to know that deletion is done before addition. Two resources are provided to accomplish adding and deleting fields: add.field and delete.field

add.field = {field=value}

delete.field = {field} Suppose a time stamp is stored in the field time. With these resources the update of a time-stamp can be achieved using the resource instructions

delete.field = {time}

add.field = {time=”jun 13, 1995”}

A.10.2 Field Rewriting

To modify the value of fields can be used to optimize or normalize the appearance of a BibTEX data base. The powerful facility of regular expression matching is used for this purpose. The resource rewrite.rule can be used to specify rewrite rules. The general form is as follows:

rewrite.rule { field_{1} … field_{n} # pattern # replacement text} field_{1} … field_{n} is a list of field names. The rewrite rule is only applied to those fields which have one of those names. If no field name is given then the rewrite rule is applied to all fields.

rewrite.rule { pattern # replacement text} Next there is the separator ’#’. This separator is optional. It can also be the equality sign ’=’. pattern is a regular expression enclosed in double quotes (”). This pattern is matched against substrings of the field value — including the delimiters. If a match is found then the matching substring is replaced by the replacement text or the field deleted if no replacement text is given. replacement text is the string to be inserted for the matching substring of the field value. The backslash ’\’ is used as escape character. ’\n’ is replaced by the nth matching group of pattern. n is a single digit (1-9). Otherwise The character following the backslash is inserted.^{3} Thus it is possible to have double quotes inside the replacement text. Other specials are

\$ which is replaced by the key of the current record.

\@ which is replaced by the type of the current record.

If no replacement text is given then the whole field is deleted. This is illustrated in the following examples:

rewrite.rule { field # pattern }

rewrite.rule { pattern } Usually the matching is not done case sensitive. This means that any upper case letter matches its lower counterpart and vice versa. This behavior is controlled by the boolean resource rewrite.case.sensitive which is ON by default. Changing this variable influences only rewrite rules specified later.

3Future releases may use backslash followed by letters for special purposes. It is not save to rely on escaping letters.

rewrite.case.sensitive = off A problem occurs e.g. when a string is replaced by a string containing the original one. To avoid infinite recursion in such cases the numeric resource rewrite.limit controls the number of applications of each rewrite rule. If the number given in rewrite.limit is not negative and this limit is exceeded then a warning is printed and further applications of this rule are stopped. A negative value of the resource rewrite.limit indicates that no limitation should be used. Next we will investigate some concrete examples. Note that in these examples the character ’␣’ denotes a single space. It is used to highlight places where spaces have to be used which would be hard to recognize otherwise.

• Empty entries are composed of delimiters — either double quotes or curly braces which enclose an arbitrary number of spaces. If we want to delete empty entries we can use the following two rules.

rewrite.rule { "^\"␣*\"$" } rewrite.rule { "^{␣*}$" }

The caret ’^’ denotes the beginning of the whole string and the dollar is its end. The star is an operator which says that an arbitrary number of the preceding regular expression — i.e. the space — can occur at this point.

• Ranges of pages should usually be composed of numbers separated by an n-dash (--). The next example shows how the pages field can be normalized. Spaces are deleted and a single minus sign is replaced by a double minus.

         rewrite.rule { pages # "\([0-9]+\)␣*-␣*\([0-9]+\)" = "\1--\2" }

A.10.3 Field Ordering

Fields can be reordered within an entry. This feature is controlled by the presence of a specification for the order to use. The order is specified with the resource sort.order. The general form is as follows:

sort.order { entry = field1 # field2 # ... } entry is the name of an entry like book. The fields are an arbitrary number of field names like author. This specification says that field1 should precede field2 etc. Fields which are not in this list are arranged after the specified ones. The are left in the same order as they appear in the entry. Another possibility is to specify the entry *. Such a sorting order is applicable to any kind of entry. If no specific sort order is found then this general order is used if one has been specified. Any sorting order is added to a list of sorting orders if it has not been defined before. If a sorting order is specified again, the old one is simply overwritten. Consider the following part of a resource file:

sort.order = {* = author # title}

sort.order = {misc = author # title # howpublished # year # month # note} This means that the author field goes before the title field in any entry type. For the misc entries additional specifications are made. The library sort fld.rsc contains a sample sorting order for the standard entry types.

A.11 Semantic Checks

Semantic checks can be enabled in addition to the syntactic checks performed during parsing.

A.11.1 Finding Double Entries

When merging several bibliographic data bases a common problem is the occurrence of doubled entries in the resulting data base. When searching for double entries several problems arise. Which entries should be considered equal and what should happen to double entries. The first question is answered as follows. Two entries are considered equal if their sort key is identical and they are adjacent in the final output. The first condition of identical sort keys allows the user to specify which criteria should be used when comparing entries. This can be achieved with the resource sort.format (see A.7). The second condition can easily be achieved by additionally turn on sorting when requesting checking of doubles. It remains the question what to do with the doubles. Usually it is not desirable to keep double entries in one data base. Thus only one entry which is considered equal is kept. The others are printed as comments, i.e. the initial ’@’ is replaced by ’###´ . Thus all information is still present but inactive in the BibTEX file. The very first entry found is put into the data base file. The resource check.double can be used to turn on the checking of doubles. This feature is turned off initially.

check.double = ON Checking of doubles can also be turned on with the command line option -d:

    bibtool -d

A.11.2 Regular Expression Checks

The regular expressions (see section A.9) which are used to rewrite fields (see section A.10.2) can also be used to perform semantic checks on fields. For this purpose the resource check.rule is provided. The syntax of check.rule is the same as for rewrite.rule.

check.rule { field # pattern # message} Again field and message is optional. The separator # can also be written as equality sign (=) or omitted. Each field is processed as follows. Each check.rule is tried in turn until one rule is found where field (if given) is identical to the field name and pattern matches a substring of the field value. If such a rule is found then the message is written to the error stream. If no message is given then nothing is printed and the processing of the current field is ended. message is treated like the replacement text in rewrite.rule, I.e. the special character combinations described in section A.10.2 are expanded. Usually the matching is not done case sensitive. This means that any upper case letter matches its lower counterpart and vice versa. This behavior is controlled by the boolean resource check.case.sensitive which is ON by default. Changing this variable influences only rewrite rules specified later.

check.case.sensitive = off Consider the following example. We want to check that the year field contains only years from 1800 to 1999. Additionally we want to allow two digit abbreviations.

    check.rule { year "^[\"{]1[89][0-9][0-9][\"}]$" }
    check.rule { year "^[\"{][0-9][0-9][\"}]$" }
    check.rule { year "" "\@ \$:  Year has to be a suitable number"}

The first rule matches any number starting with 1 followed by 8 or 9 and finally two digits. The whole number may be enclosed in double quotes or curly braces.^{4} The hat at the beginning and the dollar at the end force that the pattern matches against the whole field value only. The next rule covers years consisting of two digits. The first two rules produce no error message but end the search for further matches. Thus is something suitable is found then one of the first two rules finds it. Otherwise we have to produce an error message. This is done with the third rule. The empty pattern matches against any value of the year field. In this case we print an error message. \@ is replaced by the current type and \$ by the current key.

A.12 Strings — also called Macros

Strings in BibTEX files play an important role when managing large bibliographic data bases. Thus the deserve special treatment. If the resource macro.file is defined then the macros are written to this file. The argument is a file name as in

macro.file = {macro/file/name} The boolean resource print.all.strings indicates if all macros defined in the BibTEX file should be printed or only those macros actually used.

print.all.strings = ON The appearance of string names is controlled by the resource symbol.type (see 17). Strings can be expanded when printing records. This feature of BibTool is controlled by the resource expand.macros as in

expand.macros = ON The effect is that all known strings in normal records are replaced by their values. If the values are not defined at the time of expansion then the macro name remains untouched. As a side effect strings concatenations are simplified. Imagine the following BibTEX file.

4In fact the regular expression allows also strings starting with a quote and ending in a curly brace. But this syntactical nonsense is ruled out by the parser already.

  @string{ WGA = " World Gnus Almanac" }
  @Book{ almanac-66,
          title =  1967 # WGA,
          month = "1~" # jan
  }

If BibTool is applied with expand.macros turned on this results in the following output — if the default settings are used for any other resource.

  @STRING{wga   = " World Gnus Almanac" }
  @Book{          almanac-66,
    title       = {1967 World Gnus Almanac},
    month       = {1~} # jan
  }

The macro WGA has been expanded and merged with 1967. Note that the string jan has not been expanded since the value should be defined in a BibTEX style file (.bst). The delimiters of entries is normalized, i.e. only one style is used. In this example braces are used. The alternative would be to use double quotes. This behavior is controlled by the resource print.braces. If this resource is on then braces are used otherwise double quotes are taken. It can be changed like in

print.braces = OFF The delimiters of the whole record are recommended to be braces. For compatibility with Scribe it is also allowed that parentheses are used for those delimiters. This behavior can be achieved with the boolean resource print.parentheses. Initially this resource is off. It can be set like in the following instruction:

print.parentheses = ON

A.13 Statistics

Some information can be obtained at the end of a BibTool run. The number of BibTEX items read and written is printed. To enable this feature the resources count.all and count.used are provided.

count.all = ON

count.used = ON count.all indicates that all known types of BibTEX items should be listed. count.used forces only those types of BibTEX items to be listed which have been found in the input files.

Limitations

B.1 Limits of BibTool

BibTool has been written with dynamic memory management wherever possible. Thus BibTool should be limited by the memory available only.

B.2 Bugs and Problems

Problems currently known are the following ones. They are not considered to be bugs.

• The referencing feature of BibTEX is not supported. \cite macros can be contained in fields (e.g. notes). Such things can be confused.

• The memory management uses dynamic memory. This memory is reused but not returned to the operating system. Thus BibTool may run out of memory even if a more elaborated memory management may find free memory. This is a design decision and I don’t think that I will change it.

• The TEX reading apparatus is only imitated to a certain limit. But this should be enough for most applications to produce satisfactory results.

• In several modules ASCII encoding is assumed. I do not know to which extend this influences the functionality since I don’t have access to non-ASCII machines.

• Macro expansion uses a dynamic array which can turn out to be too short. This will be corrected as soon as I have an example where this bug shows up.

34 B. Limitations

Sample Resource Files

Sample resource files are included in the distribution of BibTool in the directory Lib. Only some of them are reproduced in this section.

C.1 The Default Settings

bibtex.env.name        = "BIBINPUTS"
check.double           = off
check.case.sensitive   = on
count.all              = off
count.used             = off
default.key            = "**key*"
dir.file.separator     = "/"
dump.symbols           = off
env.separator          = ":"
expand.macros          = on
fmt.et.al              = ".ea"
fmt.inter.name         = "-"
fmt.name.name          = "."
fmt.name.pre           = "."
fmt.name.title         = ":"
fmt.title.title        = "-"
ignored.word           = "a"
ignored.word           = "an"
ignored.word           = "the"
ignored.word           = "le"
ignored.word           = "les"
ignored.word           = "la"
ignored.word           = "un"
ignored.word           = "une"
ignored.word           = "el"
ignored.word           = "il"
ignored.word           = "der"
ignored.word           = "die"
ignored.word           = "das"
ignored.word           = "ein"
ignored.word           = "eine"

key.base               = lower
key.expand.macros      = off
key.format             = short
key.generation         = off
key.number.separator   = "*"
new.entry.type         = "Article"
new.entry.type         = "Book"
new.entry.type         = "Booklet"
new.entry.type         = "Conference"
new.entry.type         = "InBook"
new.entry.type         = "InCollection"
new.entry.type         = "InProceedings"
new.entry.type         = "Manual"
new.entry.type         = "MastersThesis"
new.entry.type         = "Misc"
new.entry.type         = "PhDThesis"
new.entry.type         = "Proceedings"
new.entry.type         = "TechReport"
new.entry.type         = "Unpublished"
preserve.key.case      = off
print.align            = 18
print.align.string     = 18
print.align.preamble   = 11
print.align.comment    = 10
print.align.key        = 18
print.braces           = on
print.all.strings      = off
print.indent           = 2
print.line.length      = 77
print.newline          = 1
print.parentheses      = off
print.use.tab          = on
rewrite.case.sensitive = on
rewrite.limit          = 512
quiet                  = off
select.case.sensitive  = off
select.field           = "$key"
sort                   = off
sort.format            = "$key"
sort.reverse           = off
symbol.type            = lower
verbose                = off

C.2 Useful Translations

This resource file translates international characters into plain text representations. Especially the German umlaut sequences are translated. E.g. the letter Ä which is written as {\"A} in a BibTEX file is translated to Ae. Additionally some logos are defined.

tex.define {\"[1]=#1e}
tex.define {\ss=ss}
tex.define {\AE=AE}
tex.define {\OE=OE}
tex.define {\aa=aa}
tex.define {\AA=AA}
tex.define {\o=o}
tex.define {\O=O}
tex.define {\l=l}

tex.define {\L=L}
tex.define {\TeX=TeX}
tex.define {\LaTeX=LaTeX}
tex.define {\LaTeXe=LaTeX2e}
tex.define {\BibTeX=BibTeX}
tex.define {\AMSTeX=AMSTeX}

C.3 Other Resource Files

The distribution contains additional resource files. Some of them are sketched here. Others may be contained in the distribution as well. Look in the appropriate directory.

iso2tex

define rewrite rules to translate ISO8859-1 characters into BibTEX compatible sequences.

iso def

define macro equivalents for ISO8859-1 characters into TEX compatible sequences.

sort fld

defines a sort order for the common BibTEX entry types.

check y

contains a sample for semantic checks. The year field is checked to be a suitable number.

month

tries to introduce BibTEX strings for month names. Provisions are made to preserve other information contained in the month field.

opt

copes with OPT prefixes as introduced e.g. by bibtex-mode.

38 C. Sample Resource Files

Bibliography

[GMS94] Michel Goosens, Frank Mittelbach, and Alexander Samarin. The L^{A}TEX companion. Addison-Wesley Publishing Company, 1994.

[Knu89] Donald E. Knuth. The TEXbook. Addison-Wesley Publishing Company, 15th edition, 1989.

[Lam94] Leslie Lamport. L^{A}TEX: A Document Preparation System. Addison-Wesley Publishing Company, 2nd edition, 1994.

[Pat88a] Oren Patashnik. BibTEXing, 1988.

[Pat88b] Oren Patashnik. Designing BibTEX Styles, 1988.

Index

-- . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9, 13, 25 -$ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

add.field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27, 28

BIBINPUTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 bibtex.env.name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 bibtex.search.path . . . . . . . . . . . . . . . . . . . . . . . . 13, 14 BIBTOOL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 BIBTOOLRSC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

check.case.sensitive . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 check.double . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 check.rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30, 31 count.all . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 count.used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

-d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 default.key . . . . . . . . . . . . . . . . . . . . . . . . . . . .17, 18, 21 delete.field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27, 28 digit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 dir.file.separator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 dump.symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

empty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 env.separator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 expand.macros . . . . . . . . . . . . . . . . . . . . . . . . . . . .31, 32 extract.file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 extract.regex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

-F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 -f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8, 17 fmt.et.al . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19, 21 fmt.inter.name . . . . . . . . . . . . . . . . . . . . . . . . . . . .18, 21 fmt.name.name . . . . . . . . . . . . . . . . . . . . . . . . . . . 18, 21 fmt.name.pre . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18, 21 fmt.name.title . . . . . . . . . . . . . . . . . . . . . . . . .17, 18, 21 fmt.title.title . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18, 21

-h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 HOME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

-i . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 ignored.word . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 -K . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 -k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 key.base . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 key.expand.macros . . . . . . . . . . . . . . . . . . . . . . . . . . . .18 key.format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 key.generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 key.number.separator . . . . . . . . . . . . . . . . . . . . . . . . . 18

long . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 long.need . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 lower . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

macro.file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 new.entry.type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 new.field.type . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16, 17

-o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9, 14 off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 output.file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

pass.comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 preserve.key.case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 print . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 print.align . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 print.align.comment . . . . . . . . . . . . . . . . . . . . . . . . . . 16 print.align.key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 print.align.preamble . . . . . . . . . . . . . . . . . . . . . . . . . . .16 print.align.string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 print.all.strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 print.braces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 print.indent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 print.line.length . . . . . . . . . . . . . . . . . . . . . . . . . . . 15, 16 print.newline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 print.parentheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 print.use.tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

-q . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 quiet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

-R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 -r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10, 11, 13 resource . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

\underline{INDEX 41}

resource.search.path . . . . . . . . . . . . . . . . . . . . . . . . . . 12 rewrite.case.sensitive . . . . . . . . . . . . . . . . . . . . . . 28, 29 rewrite.limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 rewrite.rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28–31

-S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 -s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7, 24 select . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 select.case.sensitive . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 select.fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 short . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17–19 short.need . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 sort.format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24, 30 sort.order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29, 30 sort.reverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 symbol.type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17, 31

t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 tex.define . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21, 22 true . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

upper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

-v . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 verbose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

-X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9, 25 -x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9, 24

yes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12