Previous Next Contents Index Doc Set Home


Remote Software Development Using the Configuring Tool


Software projects are sometimes developed by groups in which all the developers cannot share a common file system. Developers in these groups may be separated geographically and/or electronically. This separation can further complicate the already complex task of coordinating the work of development groups. Configuring tool is exceptionally well suited to assist groups to coordinate the flow of code developed concurrently under these circumstances.

This topic discusses how a group of developers at a U.S. software company successfully use Configuring to coordinate concurrent compiler development between the United States and Russia.


Scenario

The U.S. company has contracted with a group of Russian computer scientists in Moscow to assist with compiler development. Most new development is done at the company's U.S. facility, while the Russian group focuses largely on fixing bugs. As a result of this arrangement, both groups are often working simultaneously on the same files.

The compiler development group decided that Configuring provides the means by which they can keep work at both sites synchronized and manage the transfer of files and merging of changes made simultaneously at both sites.


Implementation

This section describes the process the compiler group uses to coordinate remote development between the U.S. and Russian sites. The first section discusses how the group decided on a remote development strategy and the second section discusses tactics they considered for transferring data between sites.

The Remote Development Process

After analyzing the situation in great detail, the group came to the following conclusions. These conclusions formed the basis of their final remote development process.

Maintain Equivalent Source Bases at Both Sites

In order to coordinate their work the group realized that it was necessary for both sites to contain complete source hierarchies and for the hierarchies to be kept as synchronized as possible. They recognized that it is important to update each site with changes made by the other site as frequently as possible.

Use Bridge Workspaces for Data Transfer

In order to make it quicker and easier for developers to access the changes made by the other site, the group decided that each local site treat the other (remote) site as if it were (hierarchically) another local developer. To accomplish this they created bridge workspaces at each site at the same hierarchical level as the other developers. Bridge workspaces are regular Configuring workspaces that are dedicated exclusively to transferring data between the sites. Bridge workspaces are used to:

The primary goal is to keep both bridge workspaces synchronized.

Click for closeup.

Figure  12 Workspace Hierarchies With Bridge Workspaces

Update Each Site Sequentially

For the sake of reliability, the group realized that data must move between the sites sequentially -- only one site is updated at a time. At first glance, it seemed seductively more efficient to have both sites ship their updates to the other site at the same time. Upon further analysis they determined that this method does not ensure that changes are propagated correctly.

To maintain consistency, Configuring enforces a single-writer model -- only one transaction that alters data can proceed in a workspace at a time. Concurrency is achieved by the developers working concurrently in their own workspaces and then updating a central integration workspace sequentially, one at a time. From Configuring's perspective, both Bridge workspaces in this model function together as a single virtual workspace.

If both sites execute bringover transactions to their Bridge workspaces simultaneously, it is equivalent to executing two simultaneous bringover transactions from two different workspaces into a single workspace. This breaks the Configuring single-writer paradigm; under normal (non-remote) circumstances Configuring does not permit this to happen.

Click for closeup.

Figure  13 A Remote Update Cycle

Consider the typical update cycle pictured in Figure 13. At the conclusion of Step  and Step  the Bridge workspaces at both sites must be identical. The putback transaction in Step  (A) is exactly the same as if it were executed directly from the Russian Bridge workspace and the putback transaction in Step  (A) is exactly the same as if it were executed directly from the U.S. Bridge workspace -- from a Configuring perspective, both Bridge workspaces function together as a single workspace.

Transferring File Changes Between Sites

After the group determined the fundamental model they would use, they had to decide how they intended to transfer the updates between the two sites. After analyzing the situation they realized that there is an inherent trade-off -- the less information they transferred between sites, the more additional overhead was required.

After careful evaluation they determined that they had the following three options:

Table  0-1 Data Transfer Options

Option
Description

1

Transfer the entire Bridge workspace between the sites.

2

Transfer a workspace that contains only the files that have changed (files that are updated, newly created or renamed).

3

Transfer only:

Differences (created using the diff command) created in the SCCS history files that have changed

The entire SCCS histories of files that were newly created

A list of the names of files that had been renamed

A workspace "checksum" created using the Freezepointing command that is used to verify that the two Bridge workspaces are identical after each update

The group decided that it was crucial for them to be able to transfer data via email -- this eliminated Option 1. They felt it important to reduce as much as possible the amount of data they send between the sites, even at the added cost of additional administrative work at both sites -- for this reason they chose Option 3.

The following three sections describe the three different options in some detail and present advantages and disadvantages of each.

Option 1

Transferring the entire Bridge workspace makes the process virtually the same as if the Russian site's Bridge workspace was actually a child of the U.S. Integration workspace (and vice versa). With this option, except for the transit delay, Configuring is used normally.

Click for closeup.

Figure  14 Transferring the Entire Bridge Workspace (Option 1)

1. The Russian bridge workspace is updated from the integration workspace (A), copied to tape and shipped to the U.S. site (B).


Note - As mentioned in the previous section it is crucial that the two Bridge workspaces stay synchronized. After it has been updated with changes from the Integration workspace, the Russian Bridge workspace must not be altered until it is in turn updated with changes from U.S. The Configuring access control facility can be used to lock the workspace against any changes.

2. The update is received at the U.S. site and the Russian Bridge workspace is extracted.

3. Using the Configuring reparenting facility, the newly arrived Russian Bridge workspace is reparented to the U.S. Bridge workspace.1

Note that this step is necessary even if the names of the workspaces are the same at both sites because the fully qualified path names of the parent workspaces are probably different both sites. Configuring stores the names of a workspace's parent and child workspaces as fully qualified path names.

4. The U.S. bridge workspace is updated with the work of Russian developers using the putback transaction (C).

At this point the U.S. Bridge workspace is identical to the current Russian Bridge workspace. After the update is complete, the copy of the Russian workspace can be deleted.

5. The information in the newly updated bridge workspace is putback into the U.S. integration workspace (D).

The information is now available to the U.S. developers.

6. Reverse the previous steps to update the Russian site with changes made in the U.S.

Option 2

Transfer a workspace that contains only the recent updates to the Bridge workspace.

The advantage of this method over Option 1 is that you can greatly reduce the amount of data you transfer between sites. This reduction in data size might enable you to transfer updates via email, saving time and expense over other methods such as shipping tapes.

The disadvantage of this method over Option 1 is that more human intervention is required. A few additional steps are added to the process and people are required to do more work.

Click for closeup.

Figure  15 Transferring Updated, Created or Renamed Files (Option 2)

1. A Reference workspace is created as a child of the Russian Bridge workspace (A).

This workspace is used to preserve the original state of the Bridge workspace.

2. The Russian Bridge workspace is updated from the integration workspace (B).

3. The Bringover transaction (with the Preview option selected) is used to compare the Bridge workspace to the Reference workspace.

This produces a list of files that have been updated, created or renamed.

4. The files that are listed as updated, created, or renamed are brought over into a newly created Transfer workspace (C).

The Transfer workspace can be compressed and encoded using standard UNIX utilities such as compress, uuencode, and tar. The resulting file can then be sent via email to the U.S. site.

5. The Transfer workspace is emailed to the U.S. where it is uncompressed (D).

6. The Transfer workspace is reparented to the U.S. Bridge workspace.

At this point the contents of the U.S. Bridge workspace are still identical to the Russian Bridge workspace before its most recent update.

7. The contents of the Transfer workspace are put back into the Bridge workspace (E).

At this point the U.S. Bridge workspace is identical to the current Russian Bridge workspace. The Transfer workspace can be deleted after the update is complete.

8. The contents of the U.S. Bridge workspace are put back to the integration workspace (F).

Option 3

It is possible to reduce even further the amount of data transferred between sites. There are however, costs for this reduction:

Click for closeup.

Figure  16 Transferring Only diffs of Changed SCCS History Files (Option 3)

1. A Reference workspace is created as a child of the Russian Bridge workspace (A).

This workspace is used to preserve the original state of the Bridge workspace.

2. The Russian Bridge workspace is updated from the integration workspace (B).

3. The Bringover transaction (with the Preview option selected) is used to compare the Bridge workspace to the Reference workspace.

This produces a list of files that have been updated, created, or renamed.

4. The TeamWare Freezepointing utility is used to create a "checksum" from the Russian Bridge workspace.

This checksum is used at the U.S. site to confirm that the U.S. and Russian Bridge workspaces are identical after they are updated.

5. The diff utility is used to determine the differences between SCCS history files in the Bridge workspace and the Reference workspace.

The SCCS history file of each file that has been updated is compared to its previous state in the Reference workspace. The output of the diff utility is applied (using the patch utility) to the same files in the U.S. Bridge workspace to recreate the current state of the Russian Bridge workspace.

6. The data required to make the U.S. Bridge workspace identical to the Russian Bridge workspace is packaged into an email file (compressed, encoded, etc.) (C) and mailed to the U.S. site (D).

The information required for this process includes:

  • The diff results for all updated files
  • 7. The U.S. Bridge workspace is updated using the information sent from Russia (E).

    8. The updated U.S. Bridge workspace is verified to be identical to the Russian Bridge workspace.

    Freezepointing is used to create a freezepoint file from the U.S. Bridge workspace. The U.S. freezepoint file is compared to the Russian freezepoint file; if they match, then the update was performed correctly.

    9. The contents of the U.S. Bridge workspace are put back to the integration workspace (F).

    Final Notes



    1 You can save a step in the process by simply replacing the U.S. bridge workspace with the Russian bridge workspace, and then reparenting the Russian bridge workspace directly to the integration workspace. However, by circumventing the putback transaction, Configuring cannot protect you from losing any data that may have been inadvertently copied to the U.S. bridge workspace. In addition, the Configuring history facility will not record the transactions for later inspection.


    Previous Next Contents Index Doc Set Home