Managing X in a Large Distributed Environment

George D. M. Ross^{∗}

CS–TN–39; April 1994

Abstract

The Department of Computer Science at the University of Edinburgh operates a distributed computing network supporting over 1000 registered users on around 300 assorted workstations and X terminals. This paper describes how our X startup scripts, resource handling mechanisms and the window manager combine to provide an environment which meets the needs of even the most naïve of these users, while at the same time being easily managed and capable of being tailored to suit both course requirements and individuals’ own preferences.

1 Introduction

The Department of Computer Science at the University of Edinburgh supports over 1000 registered users, ranging in ability from complete beginners to sophisticated and experienced programmers, on almost 300 assorted workstations and X-terminals. Only a very few actually write any X code; the vast majority are end-users of X-based applications, or simply find a window system a useful productivity aid.

A variety of hardware is in use in the Department, including Sun-4 workstations, computeservers and multi-user machines, Sun-3 workstations and X-terminals, HP 300-series workstations, HP 700/X X-terminals, some NCD X-terminals and several DEC compute-servers. In addition we make use of a number of special-purpose machines operated by the central Computing Services and by other Departments.

As far as possible we provide a uniform view of the global filesystem from all our machines. Users’ home directories are automounted on all systems, while local binaries, including those for X, are replicated on some 20 main fileservers and a dozen or so local disc caches, each of which is automatically updated nightly from a central master copy (the details of the mechanism by which this is done are discussed in the paper and technical notes listed in the “references” section). Users can therefore log in to any workstation or terminal on the Department’s network and find more-or-less the same environment available.

The environment described here was first put together around three years ago, based around X11R3, though parts of it existed in prototype form before that.

2 X Software

Where possible we use the MIT X distribution in preference to the versions bundled by the manufacturers. We have the MIT core clients and libraries installed on all architectures, with the R5 server used on all Sun machines, including those transformed into X-terminals, and the R4 server used on the HP 300-series workstations (our compute servers and special-purpose machines are generally operated headless). There were several reasons for this decision:

∗Department of Computer Science, University of Edinburgh, Kings Buildings, Mayfield Road, Edinburgh EH9 3JZ, Scotland. email: gdmr@dcs.ed.ac.uk. This is a slightly-modified version of the article of the same title published in Volume 7 of The X Resource (O’Reilly and Associates, 1993), which was in turn based on a talk given at the European X User Group’s 1992 conference, held at Brunel University, Uxbridge, West London in September 1992.

• It results in a consistent setup across platforms. This has benefits for both application writer and X maintainer, as utilities, libraries and configuration files are always found in the same locations.

• Building from source makes it possible to conform to local conventions regarding location of binaries, data and configuration files. Common automatic procedures then apply.

• Server and library security hooks are available in a uniform fashion.

• New library features and server extensions are available more quickly than would be the case if manufacturers’ own servers and libraries were used. Non-standard server extensions can be incorporated as required.

• Libraries and (Sun) servers can be kept at the latest revision level.

There are, of course, cases where a manufacturer’s version might be “better” than the MIT distribution: for example, it might be faster on some platform, or it might have some some useful extension such as Display PostScript or a more fully developed PEX. However we have to weigh the support cost that is generally incurred by having to install copies on fileservers individually (or by adapting the software to our automatic distribution system) and any additional tailoring of our scripts against the benefits of the additional features provided by the particular non-MIT implementation, with the cut-off point varying according to the needs of the Department at the time and the actual functionality provided by the version in question. We might also consider making the package available with a lower level of support such as with fewer fonts or a reduced level of security.

We operate an automatic nightly update procedure, by which most of our locally-built software, and X in particular, is distributed from one master NFS server to (currently) around 20 slave servers. This server carries both architecture-specific and shared non-specific master copies of all the local filesystems. Overnight the slave servers compare their copies against the master, updating themselves as appropriate with the previous day’s changes. (The details of how this is done are described in the the paper and technical notes listed in the “references” section below.)

Installing new X clients, libraries or whatever therefore consists of building the particular package and then copying the relevant files to the architecture-specific and shared non-specific master filesystems. From there they will propagate overnight to the slave NFS servers, We have set up our site.def X configuration file so as to simplify this procedure for imake-generated Makefiles, including the core MIT distribution and most contributed clients. Unfortunately the R5 configuration process assumes that the path by which a file is installed is the same as the path by which it will eventually be used, which is not true in our case. We currently solve this problem by using a modified version of the install utility which knows that paths of the form

        /usr/local/bin/...

should be transformed to

        /export/local/`arch`/bin/...

i.e., on the master copy of the filesystem, and similarly for lib and etc, and that

        /usr/local/share/...

should be transformed to

        /export/local/share/...

We should also really have a similarly-modified version of ranlib, but new libraries are installed sufficiently infrequently that its absence has not been other than a minor annoyance. When the files propagate to the slave servers the transformation will be undone again and the files will end up where the site.def said they should. (It would be nice if this were cleaned up for X11R6, possibly by having separate installation configuration variables—for example, InstallUsrLibDir with default value UsrLibDir, or something like InterViews’ “relative” mechanism.)

In addition to the core MIT distribution, we have a considerable amount of contributed and commercial X software installed. With the exception of our DEC compute servers, which are not used as general-purpose machines, this is installed on as many platforms as it will build for, so as to maintain as uniform a user environment as possible.

3 The User Environment

Our user population is generally not interested in X configuration details, though a minority do require the ability to tailor almost every aspect of the environment. Our scripts and configuration files therefore supply suitable defaults at all stages during a session’s initialisation. These defaults can be modified on a group basis to suit particular courses, while experienced individual users are able to augment or bypass both the system-wide and group-specific defaults.

The mechanism by which this is done has been made as unobtrusive as possible. In particular, the system and group defaults require no files to be present in a user’s home directories in order to take effect. This is for two reasons:

• As students proceed from one course to another, and indeed as courses are revised, the default resources, applications, and menus may change. It is much easier to edit one central file than it is to devise a script to edit files in users’ own directories, particularly when those files may have been mangled in unpredictable ways.

• Naïve users (and supposedly sophisticated ones too!) have been known to break their environment by editing or deleting files in their home directories which they do not recognise. It is safer not to have such files visible in the first place.

At the same time we have tried to preserve compatibility with existing practice as far as possible, both for the benefit of users arriving with experience of X from elsewhere, and for those following suggestions in books or instructions provided with some packages.

3.1 Common Startup Functions

X session initialisation is complicated somewhat by the need to support both xdm and shell-level startup scripts. It is not an option for us to use xdm everywhere, as some users and courses require access to other window systems or directly into raw framebuffers.

It is also desirable to preserve as much commonality as possible in the startup scripts, both from a maintenance point of view and to minimise the amount of effort required to tailor a user’s environment. In particular, it is not acceptable for the user to require one set of configuration files when starting an X session using xdm and another different set when starting from a shell command prompt.

We have therefore factored out most of the required functionality into a shared session script which is called by small shell- and xdm-specific scripts. This common script:

• tidies up the $DISPLAY variable;

• loads any available local fonts;

• loads the resource database;

• tidies up the key mapping for some keyboards; • runs the user’s preferred set of clients, or else those defined for the user’s group, or else the

global default set;

• opens the console window, if appropriate; and finally

• runs the window manager^{1} as the distinguished session-defining client.

The mechanism used to load local fonts is quite simple: any sub-directories of <LibDir>/fonts.local are assumed to be font directories, and are appended to the font path.

3.2 The Resource Database

xrdb is used to load the resource database in four steps:

1. a handful of “required” resources are loaded, which for some reason can not go into the app-default files (for example, they may be dependent on the screen dimensions) but which should not easily be overridden by the user;

2. the remainder of the global default resources are merged;

3. any group-specific^{2} resources are merged;

4. the contents of the user’s .Xdefaults and .Xresources files, if they exist, are merged.

This provides an easy way for system management, course organisers and users to modify the defaults, either for personal preference or as instructed in the documentation for specific packages. Merging in the user’s own resource files at the end is an upwardly-compatible extension to the way these are usually handled.

Alternatively, the user may bypass steps 2 and 3 completely, but is of course then responsible for tracking any modifications to the system and group defaults. The session script checks for the existence of a .Xdefaults-load file in the user’s home directory, and if one is found it is merged in place of the usual defaults. (The name is somewhat unsatisfactory, but was chosen to reflect the actions of xdm’s -load option.)

There are several advantages to defining global defaults in this way over modifying applications’

app-defaults files:

• it makes screen-specific tailoring easier than through the “customization” mechanism, particularly when several combinations of depths and sizes are involved;

• it separates local preferences from those definitions required for applications to run properly— as well as being clearer for users who might want to tailor further an application’s appearance, there is less likelihood of changes being lost when new versions are installed; and

• group-specific defaults can be defined more easily and reliably than would be the case if they had to be conditionally included in some global file or, worse, incorporated through the “customization” mechanism.

3.3 Specific Startup Scripts

The startup script called by the user from shell level to start the X session: decides which X server to run based on host architecture and framebuffer availability; generates the “cookies” for the authorisation mechanism; inserts the X-specific directories into the user’s $PATH and $LD LIBRARY PATH^{3}; runs xinit using the chosen server and common session script; and when that terminates, resets the keyboard and clears the screen.

1twm by default, though another window manager, or indeed any X client such as xterm or xquitbutton, can be used by setting the $XWM environment variable in the user’s own .x11rc file.

2These are now selected based on the XENV environment variable, which is defaulted by user-group in the common bash startup scripts.

3This latter has been dropped.

xdm-specific startup is complicated by the requirement for the resulting environment to be the same as that obtained from a shell-level startup. The particular problem is that simply modifying the user’s $PATH and $LD LIBRARY PATH and then running the session script means that the user’s default shell is never run as a login shell. As a result, the user’s once-only login-time initialisation does not take place. It is not desirable to require users to place all shell initialisation in the “everytime” rc-files, as this would both unnecessarily complicate these files and penalise the majority of shell startups, and in any case it would run counter to existing practice and documentation. Furthermore, it is not sufficient to have xterm clients run login shells, as other clients will still have an incomplete environment—in particular, the window manager and any clients started from its menus will not have a full environment defined.

Our solution to this problem is to have xdm run the user’s favourite shell as a login shell, thus establishing the correct environment. This shell is given a one-line script as its input, whose sole purpose is to exec another script to perform the xdm-specific processing before running the common session script. The reason for this rather convoluted method is that although most of our users have bash as their default shell, there are still a few using csh variants with a different syntax for environment variables and I/O-redirection. Fortunately the Bourne-shell and C-shell exec commands are sufficiently similar that a common script containing only this is acceptable to both. The script which is finally run can, of course, use the shell of its choice.

3.4 Window Managers

Our standard window manager is a slightly modified twm, with similarly modified versions of vtwm and tvtwm available (and quite popular). The modification simply feeds the rc-file through a filter, allowing us to pre-process the configuration details and to provide group-specific variants and easy user-customisation. Our scheme could have been implemented for twm, or indeed for any other window manager, by generating suitable configuration files as part of the session startup, but there are a couple of small advantages to filtering:

• the purpose of the rc-file, if any, in the user’s home directory remains as before, so printed documentation and manual pages are less likely to cause confusion; and

• the configuration files can be changed and immediately reloaded without having to remember to regenerate an intermediate version first.

The default filter is a shell script wrapper which passes the .twmrc file through cpp along with some useful macro definitions. (A previous version of the filter used m4 in place of cpp, but users were much more familiar with the latter so we switched when we moved from X11R3 to X11R4.) This use of cpp does mean that ‘#’ can no longer be used as a comment indicator, which is one of the few incompatibilties between our version and “standard” twm, but this has not caused users any major problems. In fact, the filter can also be substituted by the user, but almost no-one actually does this.

We split the default rc-file into several sub-files, which are #included into the main file. cpp is set up to search the following directories in order:

  1. $HOME/lib/twm
  2. <LibDir>/twm/<group>^{4}
  3. <LibDir>/twm

This allows course organisers to override the global defaults, with individual preferences taking priority over either. The sub-files are arranged by logical function: one controls the host menus, one the mouse bindings, one the keypad bindings, and so on. In particular, different categories of user are presented with different host menus, listing only those hosts to which they have a legitimate reason to connect.

4Now XENV-specific

The default session script runs twm as the last, session-defining client, though this too is subject to user preference. If the environment variable $XWM is defined in the user’s .x11rc file then this is used instead of twm. Normally this would be to another window manager, though it is sometimes useful to define it instead as the quit-button client or as an xterm.

3.5 Default Clients

In contrast to resource definition and window manager configuration, the initial set of clients is taken either completely from the user’s own .x11rc file or from the group-specific file or from the global default file. There were a couple of reasons for this:

• resources can easily be over-ridden when the user’s preferences are merged, but it is easier not to start a client than it is to kill it once it has started (and perhaps to have to tidy up after it); and

• in general only a handful of clients are started, so there is no advantage in breaking down the configuration files into smaller functional units, in contrast to the window manager configuration, which is both quite complex and amenable to subdivision.

The actual mechanism by which this is implemented is simple: if there is a .x11rc file in the user’s home directory then use it; else if there is a group-specific^{5} file then use that; else use the global default file.

A special “quit button” client, xquitbutton, is usually provided as part of these defaults, as this is a more obvious route to terminating the session than would be a window-manager menu-option (though this is also provided) or a distinguished xterm client. A dialog-box asks for confirmation; if given, the session-defining process is killed, returning the user to the “login” prompt or the xdm login or chooser window. Other default clients typically include an xterm or two, an editor and a mail-watcher.

4 Security

In our Department, with potentially mischievous or untrustworthy users and full Internet access for many of our systems, X security is of some concern. Our physical network topology is such that passive snooping on our ethers is generally not a problem, while breached password or filesystem security would open up possibilities far beyond those available by attacking an X server. At the same time, a simple host-based mechanism would be insufficient, as this would not adequately protect X terminals served off multi-user machines.

We have therefore adopted the token-based mit-magic-cookie-1 authentication scheme. All of our workstations and most of our X-terminals are secured in this way, with the cookie being generated either by xdm, or in the shell-specific startup script by a purpose-written utility which seeds a random-number generator with some “hard-to-predict” values. Because home directories are automounted on all (Departmental) systems, once generated the cookie is available to all clients no matter which system they are being run on.

One minor complication arises with su-like commands which change both of a user’s real and effective user-IDs. Setuid utilities can either revert to their real user-ID^{6} while extracting the cookie from the authorisation file, or obtain access to system resources under their effective user-ID before reverting to their real user-ID to access the authorisation file and open the display. This possibility is, however, no longer available to clients started after an su, as they have lost all information about their previous user-ID. Our local nsu utility gets round this by extracting all the relevant cookies from the caller’s authorisation file under the caller’s real ID, saving them in memory, changing real and effective IDs, and finally storing the saved cookies into the authorisation file corresponding to the new ID.

5Again now XENV-specific

6This is complicated somewhat by the semantics of the SysV setuid() system call when the effective user-ID is root. The current workaround involves a vfork() hack.

5 Remote Clients

As well as the host they initially logged in to, there are a number of other hosts available to most of our users. These include the Department’s own multi-user machines and compute-servers, as well as systems operated by the University’s Computing Services and other Departments and by outside bodies. Access to these may involve intermediate hosts, and since users are generally not interested in the means used to reach any particular host, we provide a shell script (“rterm”) to hide the details of the access mechanisms.

In fact, most users never call this script directly. Instead it is usually invoked indirectly from a window manager menu. This is set up, often on a per-group basis though there is a system-wide default, so as to contain most of the hosts which the user is likely to want (or be authorised!) to connect to, and with the appropriate access-mechanism selection flags set.

The script usually invokes a local or remote xterm, but can optionally run any other suitable client. It:

• decides which machine is to host the xterm client, depending on whether that xterm is to provide an interactive shell or to run some other program such as telnet, tip or pad;

• constructs a suitable $DISPLAY variable, depending on whether the xterm is to run on the same machine as the X server, on another machine in our local domain, or on a remote machine in another domain;

• constructs an application instance-name and a title for the window manager’s titlebar;

• propagates the user’s $PATH and $LD LIBRARY PATH^{7}, unless this is disabled by a commandline option;

• for alien hosts, extracts and propagates the user’s authorisation cookie, or alternatively runs “xhost +” for the remote host, as directed by a comand-line option; and finally

• runs the xterm client from a purpose-written utility which redirects stdout and stderr to a log file, redirects stdin to /dev/null and closes all other possible file descriptors.

The purpose of closing or redirecting the xterm client’s file descriptors is to dissociate it from any dæmon which might have been required to invoke it. A special-purpose utility was found to be more reliable than attempting to deal with the vagaries of various systems’ shell syntax and dæmon options.

6 Conclusions

Whether or not you consider our environment “large,” it is certainly big enough that X system management is qualitatively different from that necessary at “small” sites or where users are mainly X-literate. Although there may be one or two rough edges in the actual implementation, the underlying framework is sound and has proven to be extremely useful to our users, course organisers and system support staff alike.

There are three main ways in which our environment succeeds:

• it makes installation on 20 (or even 200) fileservers as straightforward as on one;

• it is unobtrusive, so is less likely to be broken by careless or inexperienced users; and

• customisation, where necessary, is both flexible and easy.

Indeed, whether or not you choose to adopt any of the ideas described here, we would suggest that some kind of systematic treatment of configuration files is essential for any site with more than a fairly small number of competent users.

7No longer propagated.

7 References

[1] Paul Anderson: Managing program binaries in a heterogeneous UNIX network. In Proceedings of LISA V Conference, pp 1–9, USENIX, 1991.

[2] Paul Anderson and Alastair Scobie: The Local UNIX directory hierarchy. Technical note CS–TN–21, Department of Computer Science, University of Edinburgh, August 1991.

[3] Paul Anderson: Installing software on the Computer Science Department network. Technical note CS–TN–24, Department of Computer Science, University of Edinburgh, August 1991.

8 Anonymous FTP

The paper and technical notes mentioned above are available by anonymous FTP from ftp.dcs.ed.ac.uk in the file pub/lfu/doc.shar.Z, with the automatic nightly-update software in pub/lfu/lfu.tar.Z.

The components of the environment described in this paper can be obtained by anonymous FTP from:

   • ftp.dcs.ed.ac.uk, in the file pub/X11R5/dcs-environment.tar.Z, or

• export.lcs.mit.edu and its usual shadows, in the file

     contrib/edinburgh-environment.tar.Z.

This file contains our local clients and utilities, the twm patch, the scripts described here, copies of our twm and xdm configuration files, and our site.def installation configuration file.

9 Acknowledgements

Several of the support staff of the Department contributed to the environment described here, in particular Paul Anderson who wrote the original versions of many of the scripts. Helpful comments from our users also resulted in useful modifications and enhancements.