Previous Next Contents Index Doc Set Home


Managing SunOS Development With TeamWare


This case study describes how the Sun WorkShop TeamWareTM code management software is being used in a large software development project. The study is taken from the experience of a Sun Microsystems® business unit, in its ongoing development of the SunOSTM operating system component of the Solaris® operating environment.


Project Description

The source base for the SunOS project is large -- more than one million lines of code contained in over twelve thousand files. The staffing for the project is correspondingly large, with more than 100 developers working at sites across California and Colorado.

Management recognized during the project's planning stages that developers would need parallel access to the most current source -- the classic one-developer-at-a-time access allowed by SCCS (the Source Code Control System) was too limiting for their needs. As a more capable alternative, they adopted Configuring, the TeamWare component that manages groups of files for groups of developers.

Configuring provides an isolated workspace for each developer and automatically records the change history of each source file. By judiciously setting up their hierarchy of Configuring workspaces, the group allows each developer access to the newest stable source code at any time. If two developers edit copies of the same file, Configuring detects the differences and presents them graphically in the Merging tool. One or both of the developers merge the changes into a single file that is then returned to the common source base.

Five integration engineers coordinate and maintain the Configuring workspace hierarchy for the project. Only a fraction of their time is spent on Configuring issues, however. Their additional duties include:

The integration engineering staff generally agrees that TeamWare tools have reduced time-to-market for the SunOS product by lowering the overhead required to manage source code during the product's development phase. The tools are also being used to maintain earlier product releases.


Case Study Overview

This case study details the following aspects of the strategy used by the SunOS group in adopting and deploying Configuring and the other TeamWare tools:


Basic Configuring Concepts

This section defines five Configuring terms that will help you understand the case study.

Workspace -- A directory (and its subdirectories) that has been designated for use by Configuring. Configuring identifies a workspace by the presence of a subdirectory named Codemgr_wsdata.

Workspace hierarchy -- An organization of parent and child workspaces, much like a standard directory hierarchy. Each workspace can have a parent and one or more children, in the same way that a directory has a parent directory and subdirectories.

Putback -- A transaction between a child workspace and its parent, in which changes to files from the child are copied ("put back") into the parent.

Bringover -- A transaction between a child workspace and its parent, in which changes to files from the parent are copied ("brought over") into the child.

Reparent -- An operation that changes the parent of a child workspace.


Organizing the Workspace Hierarchy

SunOS development is an ongoing project, so the workspace hierarchy was organized to allow for successive, sometimes overlapping, releases. Central to the organization is the concept of "the train," a constantly evolving top-level workspace that always represents the current group development effort. The train analogy results from the way product-release workspaces are generated, and is explored more fully on pages 14-16.

Beneath the top-level workspace, called train, is a group of sibling integration workspaces (see Figure 1). Developers bring files into their private workspaces from these integration workspaces. They also put back to the integration workspaces, never to train directly. Because the integration workspaces restrict access to the release-level workspace, the SunOS group calls them gates. In each group, a senior-level developer is designated as the gatekeeper. The gatekeeper is responsible for maintaining the group's gate workspace, making sure that the source it contains is without error before it is put back to the release-level workspace.

Click for closeup.

Figure  1 Basic Workspace Hierarchy

Most integration workspaces are organized functionally: developers working on a related group of commands or libraries transact with a single integration workspace. Other integration workspaces represent physical organization: developers working at a remote site transact with a single integration workspace, which puts back to train overnight. (Remote integration workspaces communicate with train through the NFS® distributed file system by means of a dedicated communications link, which slows down transactions somewhat.)

The organization shown in Figure 1 is an ideal; the actual hierarchy at any given time may look quite different because the SunOS group usually reparents workspaces during building and testing, as described in the following sections. The ability to reparent workspaces results in a great deal of flexibility. The hierarchy is a statement of group policy, not a rigid organizational structure, and can easily be revised.1

Building and Testing Executables

Each developer in the SunOS development group is expected to build (compile and link) and test the new code he writes. Custom File List Programs (FLPs) and hierarchical makefile structures help ensure that every development workspace builds with the most recent header files and libraries. After a successful build and test, the developer puts back his new source files into the integration workspace. If a conflict occurs because another developer's changes were put back while the first developer was working, the first developer resolves the conflicts with the Merging tool, and the resulting source files are once again compiled, linked, and tested.

A large software development project is almost always organized into functional modules, and the SunOS project is no exception. Because of this modularity, the code that has been put back to an integration workspace cannot be guaranteed to build and test successfully even though each developer tests new code in his own workspace. While a development workspace usually tests one bug fix at a time, an integration workspace can accumulate dozens or even hundreds of putbacks between each build-test cycle. To make certain that the interaction between all these changes is correct, a senior developer is given responsibility for building and testing at the integration workspace level. Only when the testing is completed is the new code put back into the release workspace to become part of the product.

Building in a parent workspace, whether it be integration or release, requires that no child workspaces put back during the build. If putbacks were allowed, the source code base could change in the midst of compilation, almost assuring errors. This requirement clashes with the goal of letting each developer transact with his integration workspace at any time -- a goal that increases productivity and decreases wait time for developers. Configuring solves the problem by allowing reparenting of workspaces, as discussed in the next section.

Isolating Test Workspaces

The SunOS development group creates identical copies ("clones") of integration workspaces in which to build and test. After a successful test of the software, reparenting is generally used to put back changes to the release workspace. Depending on the demands of the moment, testing is done in a clone of one gate or in a temporary composite workspace formed by cloning one gate, then reparenting the clone to a second gate and updating it with files from the second gate. This section describes both approaches.

Testing in a Single Integration Workspace

Consider the gatekeeper who is responsible for maintaining the integration workspace gate_1. The workspace has accumulated enough changes to warrant putting them back into train. Before putting back, however, the gatekeeper must do three things:

1. Bring over any new source files from train that have been put back from other integration workspaces. This step also involves resolving any conflicts that may result from the bringover.

2. Compile and link the source code in gate_1.

3. Test the resulting executables.

If the gatekeeper brings over directly from train into gate_1, she is likely to encounter conflicts. As she resolves these conflicts (by merging source files), she will be forced to lock gate_1 to keep developers from putting back while she works.

Instead of locking the gate workspace, the SunOS group's gatekeepers use a cloning technique that allows developers access to the gate while they resolve conflicts and fix bugs in the source base. The following figure (Figure 2) shows the steps the gatekeeper would perform to integrate gate_1 with the release workspace train.



Figure  2 Integrating an Integration Workspace Into a Release Workspace 

Create a clone of gate_1 with a bringover create command (\xac ).

Click for closeup.

Reparent to train (\xad ), and then bring over into the clone (\xae ).

After the gatekeeper resolves the conflicts that result from bringing over updates from train, gate_1_clone is no longer an exact image of gate_1 -- it has been modified by the most recent changes other integration workspaces made in train.

Click for closeup.

The gatekeeper must now build and test the source code in gate_1_clone to make sure it is correct (\xaf ). Building in gate_1_clone requires locking it against putbacks, but this constraint inconveniences no one because gate_1_clone has no children.

The build and test may reveal inconsistencies and bugs that will have to be fixed. As the gatekeeper fixes the errors, the contents of gate_1_clone diverge from gate_1.

When the gatekeeper is satisfied with the results of her testing, she puts back from gate_1_clone to train (\xb0 ) rather than reparenting and putting back to gate_1.

When changes are put back from the clone directly into train, they become available immediately to all other integration workspaces, including gate_1. A bringover from train to gate_1 brings gate_1 up to date (\xb1 ).

Click for closeup.

Note that in the day or two the gatekeeper spent building and testing in gate_1_clone, developers may have been putting new changes into gate_1. The best method for synchronizing gate_1 and gate_1_clone in that event involves three steps, as described in Figure 3.



Figure  3 Synchronizing the Integration Environment With its Clone 

Reparent gate_1_clone to gate_1 (\xb1 ).

Bring over to update gate_1_clone (\xb2 ), resolving conflicts as required.

Click for closeup.

Put back to gate_1, (\xb3 ).

Click for closeup.

The cloning technique just outlined allows development to move forward at the same time a large, complex workspace is being tested and integrated into the release level. It's disadvantage comes when a developer puts back a large functional module into the integration workspace while testing is going on in the clone. In such a case, the final bringover and putback (\xb2 and \xb3 , Figure 3) can result in many conflicts and a great deal of work for the gatekeeper. These cases can be avoided by polling developers about their work in progress and scheduling the build and test operations for lull times following a developer milestone putback event.

Testing in Merged Workspaces

With a few additional steps, the process discussed in the last section can be applied to merging several gates, building and testing, and then reparenting and putting back to the release workspace. If the gates do not contain a lot of new code, gatekeepers can save time by merging the gates before building and testing them: instead of building and testing a clone of each gate, the gatekeepers build and test once for all the merged workspaces.

Figure 4 shows the process for two integration workspaces.



Figure  4 Merging Two Workspaces and Putting Back to the Release Level 

A gatekeeper creates a clone of gate_1 by bringing over into a new workspace called gate_merge (\xac ).

Click for closeup.

The gatekeeper reparents to gate_2 (\xad ) and brings over changes, which may create conflicts that must be resolved in gate_merge (\xae ).

After the conflicts are resolved, the gatekeeper builds and tests in gate_merge (\xaf ).

Click for closeup.

When testing is done, the gatekeeper reparents to train (\xb0 ) and puts back into the release workspace (\xb1 ).

Click for closeup.

To synchronize gate_1 and gate_2 with gate_merge, the gatekeeper could bring over from train. However, that bringover could create conflicts that would need to be resolved in gate_1 and gate_2, which would lock out developers from those integration workspaces. A better approach is separately to reparent and put back from gate_merge into gate_1 (\xb2 ) and gate_2 (\xb3 ).

If the putbacks are blocked, the gatekeeper can perform new bringovers and resolve conflicts in gate_merge, leaving the integration workspaces free for developer transactions.

Click for closeup.

Clearly, more than two integration workspaces can be merged in the manner just described in Figure 4. Projects of lesser complexity may extend workspace merging to include all integration workspaces in the project, periodically merging, building, testing, and putting back to the release level. See the TW Topic "Workspace Hierarchy Strategies for Software Development and Release on page 37" for a discussion of this approach.

Managing Release-Level Workspaces

Many of the considerations that apply to integration-level workspaces also apply to the release level -- providing transaction access for child workspaces during build and test cycles, for example. Additional issues at the release level involve providing for alpha, beta, and customer releases, multiple simultaneous customer releases, and maintenance (patch) releases.

As a workspace nears release, the group relies on a senior developer who is assigned to the workspace. This person decides when to merge changes into the workspace and when testing has been completed satisfactorily. The position is one of considerable responsibility. Integrators at the release level are aided by Configuring and network features that control access to source files and email notifications when workspace transactions occur.

Posting Notifications

Configuring can send email messages when an interworkspace transaction occurs (bringover, putback) or when a workspace is deleted, moved, or reparented. Integrators at the release level set up these notifications so that they are alerted whenever an interesting transaction occurs in workspaces for which they have responsibility.

The SunOS group arranged to have a special email message sent to team members who bring over files from release-level workspaces. The message is a policy statement outlining the restrictions on putting back files to the release level and explaining the release-gateway-developer workspace hierarchy. This message helps ensure that new developers do not bring over source directly from the release level into their private workspaces, from which they will not be allowed to put back to the release level.

Controlling Access to Source Files

Controlling access to workspaces is important, particularly at the release level. Configuring, through the access_control file in the Codemgr_wsdata directory, allows only specified users and net groups to perform transactions with a workspace (bringovers and putbacks) and operations on a workspace (reparentings, deletions, and so on).

Although workspace access can be controlled through Configuring commands, there is no special access control for workspaces at the file system level. Configuring does not limit the ability of a user to change directories to the workspace across the NFS distributed file system, check out a file from SCCS, edit it, and check it back in. This possibility is especially worrisome at the release workspace level, where a developer might accidentally check in a file.

To overcome this problem, the SunOS group selectively sets the permissions of the directories in its workspaces. For example, the Codemgr_wsdata directory is writable by anyone because workspace transactions require the ability to write to a locks file in that directory. However, the usr directory, which is the root of the source base, is read-only on most file systems so that it cannot be written to casually. It is read-write to only one machine, which developers must log into remotely (rlogin) in order to put back. The machine itself can be password protected from remote logins by unauthorized users, but more important is the fact that qualified developers must perform the extra step of logging into the machine before putting back. This extra step protects from accidental source file checkins by ensuring a conscious step on the part of developers before they put back source files.

Building and Testing at the Release Level

At the release level, building and testing is performed in a clone workspace just as it is at the integration level. However, procedures at the release level are more formalized because risks to the code base are higher. Figure 5 illustrates the process. By cloning a workspace (\xac ) for test activity, the release workspace itself is free for transactions with integration workspaces.

Click for closeup.

Figure  5 Release Level Building and Testing

The clone workspace is write-only -- it never puts back to the release workspace. If a bug arises during testing, it is evaluated in the clone workspace and narrowed down to an area of functionality corresponding to an integration workspace. The bug is fixed at the integration level and put back to the release level.

A typical bug fix scenario is shown in Figure 6.



Figure  6 Release-level Bug Fixing 

Here, a bug has been identified as having originated in the gate_1 workspace. A clone is made of gate_1 (¨) so that gate_1 itself can be free for developer transactions.

The bug is fixed and tested in gate_1_clone (\xad ).

Click for closeup.

The clone is reparented to train, and the new code is put back to the release level (\xae ).

From release level, the new code is brought over into train_clone (\xaf ) where it is tested again (\xb0 ).

Click for closeup.

Because the SunOS operating system is compiled for more than one architecture (SPARC® and Intel® x86 architectures at this writing), a build clone is created for each architecture.

Managing Test Releases

If the train clone shown in Figure 6 is created for a test release, it can be named Alpha or Beta and maintained for the period of the test release. If it is created only to establish a stable release workspace, it can be deleted after testing is completed.

Managing Multiple Customer Releases

The SunOS group publishes an ongoing set of releases, handling at least two releases concurrently. As one release enters beta testing and First Customer Ship (FCS), development and alpha testing begins on the next release. A modified cascade workspace structure is used to support the releases (see the TW Topic "Workspace Hierarchy Strategies for Software Development and Release on page 37" for a more general discussion of workspace hierarchies).

The release-level workspace train is always the top of the current development hierarchy. For each release, train is stabilized and cloned. The old train workspace is renamed to designate its release number and the clone is renamed train. Active integration workspaces are reparented to the new train, and development continues on the next release. Figure 7 illustrates the process for Release 3.0.



Figure  7 Leaving a Release Behind While the train Moves On

Before Release 3.0

Release-level workspace train is cloned and
named train_clone (\xac ).

Click for closeup.

After Release 3.0

Workspace train is renamed Release_3.0, and workspace train_clone is renamed train (\xad ).

The integration-level children of Release_3.0 are reparented to train (\xae ).

An alternative reparenting technique, for hierarchies with many integration workspaces, is to name the train clone Release_3.0, then reparent Release_3.0 to Release_2.0 and train to Release_3.0.

Click for closeup.

By reparenting all the existing development and integration workspaces to the new train, any work in progress that was not finished for the last release (the development of a new feature, for example) will automatically be applied to the next release.2

The analogy to a train is apt when the workspace hierarchy is viewed over time. Each release begins as a snapshot of the current train, and a string of releases takes on the appearance of a series of boxcars left behind by train, which proceeds along the track. From a developer's perspective, "the train" is always the active development workspace with the most recent code, always moving on to the next release (Figure 8).

Click for closeup.

Figure  8 A String of Releases Left Behind by train

Managing Maintenance Releases

Of course, each release does not remain frozen in time. As bugs are reported against a particular release, child workspaces are created in which to fix the bug. Moreover, a bug fix applied to a shipping release may also be applied to later release workspaces, including train. This is the normal case for the SunOS project because each succeeding release is a superset of the previous release.

Note that the arrows between the release workspaces in Figure 8 show a one-way information flow. Bug fixes from earlier releases are brought over into later releases, but no code is ever put back into an earlier release. The major reason for this formality is to avoid introducing incompatible features from a recent release into older releases.

In a similar vein, a child of a late release is never allowed to reparent into the hierarchy of an earlier release because of the risk of putting back a late feature into an earlier release. Figure 9 illustrates what can go wrong.



Figure  9 Problems in Reparenting From a Recent Release to an Earlier Release

A senior developer creates a child of Release 3.0 in order to fix a bug (\xac ).

Some of the code he brings over implements Feature X, unknown to him. Feature X has been developed for Release 3.0, but is not intended for Release 2.0.

He fixes the bug, which is entirely unrelated to Feature X, and decides to apply his bug fix to Release 2.0.

Click for closeup.

He reparents to an integration workspace under Release 2.0 (\xad ) and puts back his changes, inadvertently putting back Feature X at the same time.

The next time the integration workspace puts back to Release 2.0, it will include Feature X in the files it puts back.

Click for closeup.

The inadvertent contamination of earlier releases illustrated in Figure 9 is difficult to guard against and can be difficult to fix. The best defense is to strongly discourage such reparenting, as the SunOS group does. Instead, they fix the bug in the earliest release to which it applies and bring the changes forward along the release chain.

Bringing Changes Into a Recent Release

When changes are brought over from an early release workspace into a later release, they are almost always brought over into a clone of the later release workspace (not into the later release directly). For example, when the person responsible for release-level integration wants to bring changes from the Release 2.0 workspace into Release 3.0, she creates a clone of Release 3.0, reparents to Release 2.0, and brings over the changes into the clone (Figure 10). After resolving conflicts and thoroughly testing the changes in the Release 3.0 clone, she puts back to the Release 3.0 workspace.

If the build reveals a major problem, the bug is investigated in the clone and narrowed down to one area of functionality in one of the release workspaces. The bug is fixed in the integration workspace where the functionality originated, and the change is put back into the release workspace. The change is then brought over into the build clone, and build and test begins again.



Figure  10 Bringing Changes from Release 2.0 into Release 3.0 

To bring Release 2.0 changes into Release 3.0, a release-level integrator first creates a clone of Release 3.0 (\xac ).

Click for closeup.

Release_3.0_clone is reparented to Release 2.0, and a bringover update is performed on Release_3.0_clone (\xad ).

If the bringover creates conflicts, they are resolved in Release_3.0_clone.

After the bringover, Release_3.0_clone becomes a merge of Release 2.0 and Release 3.0.

Click for closeup.

After building and testing the source in Release_3.0_clone, the clone is reparented to Release_3.0 and changes are put back.

Click for closeup.

By merging Releases 2.0 and 3.0 in a clone of Release 3.0, the release-level integrator leaves Release 3.0 free for putbacks from its child bug-fix workspaces. The penalty for allowing these putbacks is that the putback from the clone into Release 3.0 (\xae ) may be blocked. In that case, the integrator must bring over updates from Release 3.0 into the clone, resolve conflicts, build, test, and put back again.

When the early release workspaces are active with bug fixes, the SunOS group merges changes into the current release every two or three days because the group is concerned about bringing bug fixes into recent releases as quickly as possible. And, as with all such merging, the more time that elapses between merges, the more difficult the merge becomes.


Automating Builds Within Configuring

Maintaining logical, consistent source and makefile hierarchies is important to any large project, whether or not it uses Configuring. Like most large projects, the SunOS project organizes its source files into directories along functional lines. The makefile hierarchy follows the source file organization: each directory contains one default makefile, and each makefile may include other makefiles from parent directories.

Staff engineers established guidelines that encourage developers to observe certain conventions when writing makefiles. For example, developers are requested to use designated environment variables in their makefiles and to include a master makefile that defines system-wide make targets.

To smoothly automate the build process in any workspace, SunOS integration engineers used two mechanisms to provide Configuring with insight into the source and makefile structures: they wrote an initialization script called ws, and they provided custom File List Programs (FLPs).

Organizing the Workspace Directory Structure

The root of each Configuring workspace is defined by the environment variable CODEMGR_WS. This variable contains the absolute path name of the workspace, and can be thought of as the relative root of the workspace. Beneath this relative root are three top-level directories:

The proto directory has no makefiles because no builds are performed directly in it or its subdirectories.

Organizing the Source Directory Structure

Source files are contained in a directory hierarchy under $CODEMGR_WS/usr/src. However, /usr/src is never referred to directly by any makefile; instead, it is the value of the SRC environment variable, which is used instead. The full path name of the top-level source tree in an active workspace is therefore $CODEMGR_WS/$SRC. This indirection allows the entire source tree to be moved without altering any makefile or script.

Figure 11 shows part of the top-level source tree. Besides directories for commands, device drivers, header files, and so on, the top-level directory contains a makefile that is used to build the entire product, and a master makefile that is included in other makefiles in the source tree.

Click for closeup.

Figure  11 Partial Directory Tree for SunOS Source Hierarchy

Coordinating Makefiles

The SunOS group had the following goals in mind when it wrote its guidelines for writing makefiles:

An important guideline that realizes these goals is to have makefiles share as much code and as many rules as possible. Because all makefiles include Makefile.master, any change there (a change to the value of the CFLAGS variable, for example) will affect all other makefiles.

The following example shows how the policy of including parent makefiles helps keep each makefile simple. The example shows the build makefile for the ls command. It includes two higher-level makefiles, Makefile.cmd and Makefile.targ.

Makefile.cmd contains variables and rules common to the builds of all commands in the $SRC/cmd directory. It includes the top-level makefile, Makefile.master. The second included makefile, Makefile.targ, specifies system-wide install targets.

#
#ident "@(#)Makefile 1.7 92/12/15 SMI"
#
# Copyright (c) 1992 by Sun Microsystems, Inc.
#
PROG= ls
include ../Makefile.cmd
LDLIBS += -lgen -lw -lintl
.KEEP_STATE:
all: $(PROG)
install: all $(ROOTPROG)
clean:
lint: lint_PROG
include ../Makefile.targ

Figure  12 Makefile for the ls Command

Using DMake

Large software projects typically consist of multiple independent modules that can be built in parallel. DMake supports concurrent processing of targets on a single machine; this concurrence can significantly reduce the time required to build a large project.

Table 1 shows typical times required to build the SunOS source base on three different machines. On a SPARCstationTM 2 system, the use of DMake improves performance by over 30 percent. Larger improvements are realized on higher-performance machines.

Table  1 Build Times for SunOS Operating System, Make vs. DMake

Build Machine
Make
DMake
Percent Improvement

SPARCstation 2

15 hours

10 hours

33

SPARCstation 10

10 hours

4.5 hours

55

SPARCcenterTM 2000

7 hours

1 hour

86

The ws Utility

The SunOS group wrote the ws utility to automatically initialize the build environment within a Configuring workspace. It configures an environment to build the SunOS executables, sets environment variables for the workspace, and spawns a shell for the environment that has been set up.

During a build, make relies on makefiles to define search paths for the proper header files, libraries, and installation directories. Makefiles construct these search paths by absorbing environment variables. For the makefiles to work properly, each developer must set several environment variables. Setting them manually is an error-prone chore for the developer, a chore that is eliminated by ws.

Header and library files undergoing development are installed in a workspace directory named proto. When ws configures the environment, it sets environment variables that define the proto directory to be used during the build. If the build involves installing files, ws also defines a (perhaps different) proto directory as the install target.

The ws utility was written as a shell script, which makes it both portable and easy to modify.

Using ws

Consider the developer responsible for the ls command. Only a few source files are required to build the ls executable, and these are the files she brings over into her workspace when she begins work (Figure 13). CODEMGR_WS is the environment variable that contains the absolute path name of the current workspace.

Click for closeup.

Figure  13 Directory Structure for ls Development Workspace

When the developer builds the ls executable, she does not want to use the native system library and header files. Instead, she wants to use the project-specific files that have been installed in proto. If proto is empty or does not exist in her workspace, she must direct make to search for the directory in ancestor workspaces during the build. The ws utility automates setting of these search paths to ensure that a build always uses the proper libraries and headers.

By setting four environment variables (named PROTO1, PROTO2, PROTO3, and TERMPROTO), the developer can set compiler flags to control the build search paths. For the ls case, she might want to search for header files and libraries first in the proto directory in her development workspace, then in the integration workspace used by the library group, then in her development workspace's parent (integration) workspace, and finally in the release workspace. She would set her PROTO variables to search as shown in Figure 14.

Click for closeup.

Figure  14 Example Search Path for Files in proto Directory

The search path shown in Figure 14 might provide the developer with the latest versions of the libraries -- those produced by the libraries group -- but might not provide her with the most stable versions. Those versions would presumably be in the release workspace. Or, if she and her fellow developers in the commands group decide to build with the same libraries, she would want those libraries to be in her parent workspace ("commands" in Figure 14).

If the necessary files cannot be found in any proto directory, the search can default to the native file system libraries, issue a warning message, or both. Whatever search sequence she decides on, ws provides an easy way to specify it.

The following fragment from the ws script first shows the search for a protodefs file, which the user can edit to set up her PROTO environment variables. If no protodefs file exists, the PROTO variables are set to default values. The script then constructs the ENVCPPFLAGS and ENVLDLIBS environment variables from the PROTO variables. The ENVCPPFLAGS and ENVLDLIBS variables are absorbed by makefiles to direct the compiling and linking of the program.



Figure  15 Part of the ws Script That Sets Environment Variables 

...
CM_DATA=Codemgr_wsdata
wsosdir=$CODEMGR_WS/$CM_DATA/sunos
protofile=$wsosdir/protodefs

if [ ! -f $protofile ]; then
	if [ ! -w $CODEMGR_WS/$CM_DATA ]; then
		#
		# The workspace doesn't have a protodefs file
		# and this script cannot create one.
		# Tell user and use /tmp instead.
		#
		echo "Unable to create the proto defaults file 
				($protofile)."

		# Create a protodefs file in in /tmp
		wsosdir=/tmp
		protofile=$wsosdir/protodefs
	fi

	if [ ! -d $wsosdir ]; then
		mkdir $wsosdir
	fi

	cat << PROTOFILE_EoF > $protofile
#!/bin/sh
#
#	Set default proto areas for this workspace
#	NOTE: This file was initially automatically generated.
#
#	Feel free to edit this file.  If this file is removed
#	it will be rebuilt containing default values.
#
#	The variable CODEMGR_WS is available to this script.
#
#	PROTO1 is the first proto area searched
#	and is typically set to a proto area associated
#	with the workspace.  The ROOT environment variable
#	is set to the same as PROTO1.  If you will be doing
#	make installs, this proto area needs to be writable.
#
#	PROTO2 and PROTO3 are set to proto areas to search
#	before the search proceeds to the local machine
#	or the proto area specified by TERMPROTO.
#
#	TERMPROTO (if specified) is the last place searched.
#	If TERMPROTO is not specified the search will end
#	at the local machine.
#

PROTO1=\$CODEMGR_WS/proto

if [ -f "\$CODEMGR_WS/Codemgr_wsdata/parent" ]; then
	#
	# If this workspace has a Configuring parent,
	# then set PROTO2 to point to the parent's
	# proto space.
	#
	parent=\\Qworkspace parent \$CODEMGR_WS\\Q
	if [ "\$parent" != "" ]; then
	   PROTO2=\$parent/proto
	fi
fi
PROTOFILE_EoF

fi

. $protofile

#	The next line means you don't have to type make -e
#	each time you build

MAKEFLAGS=e; export MAKEFLAGS

#
#	Set up the CPPFLAGS and LDLIBS environment variables
#
MACH=\Quname -p\Q
ROOT=/proto/root_${MACH}                             # default

ENVCPPFLAGS1=
ENVCPPFLAGS2=
ENVCPPFLAGS3=
ENVCPPFLAGS4=
ENVLDLIBS1=
ENVLDLIBS2=
ENVLDLIBS3=

if [ "$PROTO1" != "" ]; then       # first proto area specifed
	ROOT=$PROTO1
	ENVCPPFLAGS1=-I$ROOT/usr/include
	export ENVCPPFLAGS1
	ENVLDLIBS1="-L$ROOT/usr/ccs/lib -L$ROOT/usr/lib"
	export ENVLDLIBS1

	if [ "$PROTO2" != "" ]; then
                                   # second proto area specifed
		ENVCPPFLAGS2=-I$PROTO2/usr/include
		export ENVCPPFLAGS2
		ENVLDLIBS2="-L$PROTO2/usr/ccs/lib -L$PROTO2/usr/lib"
		export ENVLDLIBS2

		if [ "$PROTO3" != "" ]; then
                                    # third proto area specifed
			ENVCPPFLAGS3=-I$PROTO3/usr/include
			export ENVCPPFLAGS3
			ENVLDLIBS3="-L$PROTO3/usr/ccs/lib \
							-L$PROTO3/usr/lib"
			export ENVLDLIBS3
		fi
	fi
fi

export ROOT

if [ "$TERMPROTO" != "" ]; then	# fallback area specifed
	ENVCPPFLAGS4="-Y I,$TERMPROTO/usr/include"
	export ENVCPPFLAGS4
	ENVLDLIBS3="$ENVLDLIBS3 -Y 
P,$TERMPROTO/usr/ccs/lib:$TERMPROTO/usr/lib"
	export ENVLDLIBS3
fi
...

Shielding Developers From Unstable Files

If a developer sets up his build search path to refer to a copy of a library or header file located in another workspace's proto directory, he is exposed to the changes other developers make to the file. Any change in the file will be immediately visible the next time he performs a build. However, there are times when a developer does not want to see such changes -- when he is diagnosing a bug, for example. A developer can do one of two things to shield himself from changes:

Other Custom Utilities

The SunOS group wrote several other scripts to simplify manipulation of source files controlled by Configuring. The most interesting are sccsrm and sccsmv.

sccsrm

Files in workspaces controlled by Configuring can be renamed but not deleted. The reason for this seeming paradox is obvious if you consider that a file that has been deleted from one workspace will reappear when it is put back from a child workspace where it has not been deleted. By renaming a file to a hidden file name -- .del-filename, for example -- the file appears to be deleted, and the rename propagates to other workspaces, where the file will also appear to be deleted.

The sccsrm script initially renames a file to a .del* name in the current directory; another script is run periodically to move .del* files to an area in a directory hierarchy parallel to (and with the same structure as) the source hierarchy. See the TW Topic "Deleting Files From the Configuring Tool Workspaces on page 89" for details on this strategy.

Because users are not required to remember the path name of the directory that stores deleted files when they rename a file they want to delete, the sccsrm utility helps avoid mistakes. Figure 16 shows the entire script for sccsrm.

Figure  16 Script for the sccsrm Utility

 
#!/usr/bin/sh 
USAGE="usage: sccsrm [-f] <filename> ..."
#
#ident "@(#)sccsrm								1.4			93/03/30 SMI"
#
# This script is to be used to remove files from any CodeManager
# workspace. It will do this by moving the specified file,
# and its corresponding s-dot file, to a .del-<file>-\Qdate\Q
# format.
#
# The only way to remove files under the CodeManager is
# through the rename mechanism - it is not enough to
# simply \Qrm' the file.
#


message() {
	if [ ${F_FLAG} -eq 0 ]; then
	echo "$*"
	fi
} 

#
# LC_ALL=C is set so that the this script will work no matter
# which localization you have installed on your machine. Some
# localizations can cause the output of \Qdate' and other commands
# to vary.
#
LC_ALL="C"; export LC_ALL

date=\Q/usr/bin/date +%h-%d-%y\Q
F_FLAG=0


#
# Parse options...
#
set -- \Qgetopt f $*\Q
if [ $? != 0 ]; then
 echo $USAGE
 exit 2
fi


for i in $*
do
 case $i in
 -f) F_FLAG=1; shift;;
 --) shift; break;;
 esac
done

if [ $# -eq 0 ]; then
 message $USAGE 
 exit 1
fi

#
# Process s-dot files.
#
for file in $*
do
 new_file="${file}-${date}"
 #
 # If there is a deleted file of the same name, append the pid
 # to the name.
 if [ -f SCCS/s..del-${new_file} -o -d .del-${new_file} ]; then
 new_file="${new_file}.$$"
 fi
 if [ -f SCCS/s.$file ]; then
 if [ -f SCCS/p.${file} ]; then
 if [ ${F_FLAG} -eq 0 ]; then
	echo "warning: ${file} is checked out for editing, \
	all edits will be lost - continue (y/n)"
	read ans
	while [ \Qexpr $ans : "^[YyNn]"\Q -eq 0 ]
	do
	echo "warning: ${file} is checked out for editing, \
	all edits will be lost - continue (y/n)"
	read ans
	done
	else
	ans="y"
	fi
	if [ \Qexpr $ans : "^[Yy]"\Q -eq 1 ]; then
 rm -f SCCS/p.${file}
	rm -f ${file}
	else
	continue
	fi
 fi
 if [ -f ${file} ]; then
 mv ${file} .del-${new_file}
 fi
 mv SCCS/s.${file} SCCS/s..del-${new_file}
 elif [ -d ${file} -a ${file} != "SCCS" ]; then
 mv ${file} .del-${new_file}
 else
 message "${file}: not an SCCS file"
 fi
done

sccsmv

Because moving SCCS-controlled files is so similar to deleting them in the Configuring environment, the utility sccsmv was written as a companion to sccsrm.

sccsmv renames files the same way sccsrm does, but with sccsmv a user can specify the new name of the file. In practical terms, this utility merely saves the extra step of moving both the g-file (sometimes called the clear file) and the SCCS history file when performing a rename. However, because Configuring acts only on SCCS history files, errors are eliminated by moving them automatically whenever a g-file name is given to sccsmv.

Using File List Programs

When a developer brings over a file into her workspace, she explicitly specifies the file she wants to move. When she specifies a directory, a default File List Program (FLP) sees to it that all the files in the directory are brought over. The default FLP accomplishes this feat by listing the contents of the directory and writing them to stdout, from which the bringover and putback commands read them. Users can replace the default FLP with a custom FLP that writes any file names they want to stdout, enabling them to bring over (and put back) files without naming them explicitly.

The SunOS group has found custom FLPs to be an easy way to ensure that all the files needed to build an executable are available in a development workspace.

Default File List Program

Once again, consider the developer responsible for the ls command. With the default FLP (named def.dir.flp) in force, the developer would explicitly bring over the ls and proto directories, resulting in a development workspace directory structure like the one shown in Figure 17. (The Codemgr_wsdata directory is present in every workspace.)

Click for closeup.

Figure  17 Development Workspace for the ls Command (Default FLP)

One shortcoming of the workspace shown in Figure 17 is obvious when you consider the makefile for ls shown on page 22. That makefile includes two other makefiles, Makefile.cmd and Makefile.install from the cmd directory. Makefile.cmd, in turn, includes Makefile.master. None of these files were brought over by the default FLP, so an attempt to build the ls executable will fail in the workspace shown in Figure 17.

A less obvious problem is that other files in the source base may need to be built before ls, as determined by the ls makefile. If such files exist, they must be brought over explicitly when using the default FLP.

Custom File List Program

The SunOS group has customized the default FLP so that it executes scripts that identify the files that are required for a specific build. The required files are listed by scripts named inc.flp and req.flp, which are stored in the same directory as the source files required for the build.

Using the custom FLP, the developer specifies only the /usr/src/cmd/ls directory in his bringover command. The custom FLP looks in the directory and lists its contents recursively just as the default FLP does. In addition, it looks for a script named inc.flp or req.flp. If one or both of these scripts is among the contents of the directory, it is executed. The executed scripts list additional files required to build the executable, and the names are read by the bringover command. In the case of ls, the scripts would name the proto directory and the makefiles required for the build. Figure 18 shows the resulting workspace.

Click for closeup.

Figure  18 Development Workspace for the ls Command (Custom FLP)

The inc.flp script lists files that do not reside under the current directory but are required in order to build the current directory. Header (include) files are the best examples of such files. The req.flp script lists other files that are not included but are nevertheless required to build ls -- most importantly, makefiles. The inc.flp script is associated with the source directory structure, while the req.flp script is associated with the makefile structure.

When a developer brings over /usr/src/cmd/ls, the custom FLP starts in the current directory and searches for the req.flp and inc.flp scripts. It searches for req.flp in ancestor directories and for inc.flp in descendant directories, executing the scripts when it finds them (Figure 19). The req.flp script lists files that are required for any build in a subdirectory. For example, the req.flp script in the directory /usr/src/cmd writes out Makefile.cmd and Makefile.install, which are used to build all commands.

Click for closeup.

Figure  19 The Custom FLP Searches Directories for req.flp and inc.flp

Similarly, when the custom FLP executes the req.flp script in the /usr/src directory, it writes out the name Makefile.master, the makefile that is used to build every executable in the source base.

If the developer brings over /usr/src/lib/lib.c, the custom FLP encounters a different req.flp. The req.flp in /usr/src/lib writes out Makefile.lib, which is required to build every library. Makefile.master is brought over when the custom FLP executes the req.flp in the ancestor directory /usr/src.

The inc.flp script performs a similar function for files elsewhere in the file system. When a build depends on files in the brought-over directory, its subdirectories, or in distant directories, inc.flp provides the file names to the bringover command.

The custom FLP and the scripts it executes help automate complicated Configuring transactions. However, the scripts themselves are static descriptions of the dependencies of build targets and must be edited when the directory structure or build target dependencies change.


Registering Public Workspaces With NIS

The SunOS group takes advantage the automounter and the Network Information Service (NIS), both standard parts of the Solaris operating environment, to reduce the amount of information developers must remember daily.

The automounter (automount(8)) is a daemon that automatically mounts an NFS directory as needed. It monitors attempts to access directories that are associated with an NIS automount map, along with any directories or files that reside under them. When a user attempts to access a file or directory, the daemon mounts the required NFS file system.

By registering the SunOS public workspaces (integration and release-level) in the automount map, the SunOS group is able to assign them abstract, net-wide names. The abstract names relieve users from having to remember the true path name of workspaces that must be accessed often.

For example, a workspace physically located at
server:/export/here/there/everywhere/sunos1
is normally accessed by the automounter with the command
cd /net/server/export/here/there/everywhere/sunos1
In contrast, after a map name for the workspace has been entered in the automounter map, SunOS developers can access the same workspace with the command
cd /workspace/sunos1

If the physical location of a workspace must be changed because of disk space limitations or other reasons, users do not need to learn the new path name. The new physical location is entered into the automounter map, but the abstract name does not change.



1 Developers working in close collaboration can even reparent to each other's workspaces in order to solve a problem jointly, then put back to the integration workspace when the problem is solved.

2 And, if it is not finished for the next release, it can be applied to the one after that.


Previous Next Contents Index Doc Set Home