IBM Rational ClearCase 7.0: Master the Tools That Monitor, Analyze, and Manage Software Configurations

Take a deep dive into extending ClearCase 7.0 to ensure the consistency and reproducibility of your software configurations

(For more resources on IBM, see here.)

Java

ClearCase was mostly (at least originally) written in C++, and its build model is well suited (with some historical adjustments to cope with templates in two generations of compilers) to development using this language. Java, although already old at the time of its wide success, broke a few significant assumptions of the model.

Problems with the Java build process

The traditional build model is stateless, and therefore easily reproducible: running the same build command in the context of static sources (leaves of the dependency tree, seen upwards from the products) produces the same results, but doesn't alter the context. This is not the case anymore with javac. The reason is trivial: javac integrates into the compiler a build tool function. The compiler reads the Java source as a build script and uses the information to build a list of dependencies, which it verifies first, using traditional time stamp comparison between the sources and class files produced, and rebuilding missing or out-dated class files. It doesn't, however, perform a thorough recursive analysis, nor attempt to validate jars, for instance.

This behavior is highly problematic from a clearmake point of view, as it results in the list of derived objects produced with a given rule (siblings of the target) being variable from one invocation to the next, and conversely, in a given derived object potentially being produced by several different rules. Both of these effects result in incorrect dependency analysis, and in spurious invalidation of previous build results.

Let's note that since javac performs time stamp comparisons, the default behavior of cleartool to set the timestamp at checkin time is inadequate for Java sources, and results in needlessly invalidating classes produced before checkin: set up a special element type manager defaulting to the -ptime (preserve time) checkin option.

The second traditional assumption broken by Java is a practical one: the language has been designed to optimize compilation speed, which means that build time stops being a primary issue. This is of course obtained by using a single target, the Java virtual machine, and at the expense of run-time performance; but history has already clearly validated this choice, in the context of favorable progresses in hardware.

This is obviously not a problem in itself, but it had two clear consequences:

• The winkin behavior of clearmake cannot be sold to users anymore on the argument of its saving build time (by side-effect). As we know, the argument of the accuracy of management appeals only to users having experienced its importance, and reward following investment is a known recipe for failure in evolution.
• It encourages carelessness among developers: throw away (clean) and start again from scratch.

Of course, the gain in speed is mostly felt in small configurations, and at the beginning of projects: this strategy doesn't scale, as the total build time still depends on the overall size of the component, instead of on this of the increment (the number of modified files). It is however often late to change one's strategy when the slowness becomes noticeable.

.JAVAC support in clearmake

Support for Java was added relatively late to clearmake (with version 2003.06.00), in terms of a .JAVAC special target (and a javaclasses makefile macro). The idea (to which your authors contributed) was to use the build audit to produce a .dep file for every class, which would be considered by clearmake in the next invocation, thus giving it a chance to preempt the javac dependency analysis. Of course, the dependency tree would only be as good as this of the previous compile phase, but it would get refined at every step, eventually converging towards one which would satisfy even the demanding catcr -union -check.

Special care was needed to handle:

• • Inner classes (producing several class files per java source, some of them being prefixed with the name of the enclosing class, with a dollar sign as separator—not a friendly choice for Unix shells)

.

• Cycles, that is, circular references among a set of classes: a situation which clearmake could only process by considering all the set as a common target with multiple siblings.

This solution should be very satisfactory, from the point of view of ensuring correctness (consistency of the versions used), sharing of objects produced, and thus managing by differences. It should offer scalability of performance, and therefore present a breakeven point after which it would compete favorably with from scratch building strategies.

One might add that a makefile-based system is likely to integrate with systems building components written in other languages (such as C/C++), as well as with performing other tasks than compiling Java code.

Let us demonstrate how the dependency analysis and derived objects reuse are working using the .JAVAC target in the makefiles, testing exactly the aspects mentioned above—inner classes and cycles.

In our small example, Main.java implements the main class, FStack.java implements another independent class, which the Main class is using. Finally, the FStack class also contains an inner class Enumerator, which results after the compilation in a file of name FStack$Enumerator.class:

# Main.java
public class Main {
public static void main(String args[]) {
FStack s = new FStack(2);
s.push("foo");
s.push("bar");
}
};

# FStack.java
public class FStack {
Object array[];
int top = 0;
FStack(int fixedSizeLimit) {
array = new Object[fixedSizeLimit];
}
public void push(Object item) {
array[top++] = item;
}
public boolean isEmpty() {
return top == 0;
}
public class Enumerator implements java.util.Enumeration {
int count = top;
public boolean hasMoreElements() {
return count > 0;
}
public Object nextElement() {
return array[--count];
}
}
public java.util.Enumeration elements() {
return new Enumerator();
}
}

We create a tiny Makefile making use of the .JAVAC target. Note that we do not have to describe any dependencies manually; we just mention the main target Main.class, leaving the rest to the javac and the ClearCase Java build auditing:

# Makefile
.JAVAC:
.SUFFIXES: .java .class
.java.class:
rm -f $@
$(JAVAC) $(JFLAGS) $<
all: /vob/jbuild/Main.class

The first run of the clearmake does not look very spectacular: it just executes the javac compiler, submitting the Main.java source to it, and all the three class files (FStack.class, FStack$Enumerator.class, and Main.class) get generated. The same would have been produced if we used the "default" Makefile (the same, but without the .JAVAC target):

$ clearmake -f Makefile
rm -f /vob/jbuild/Main.class
/usr/bin/javac /vob/jbuild/Main.java

Note though that one thing looks different from the default Makefile execution: our ".JAVAC" Makefile produces the following dependency (.dep) files:

$ ll *.dep
-rw-r--r-- 1 joe jgroup 654 Oct 19 14:45 FStack.class.dep
-rw-r--r-- 1 joe jgroup 514 Oct 19 14:45 Main.class.dep

But their contents are somewhat puzzling at the moment:

So, it looks as if the FStack class was depending on the Main class, and the other way around as well. But that's what one can only figure out after a single javac execution—The Main class was produced and, in order to compile it, two more classes were needed: FStack and FStack$Enumerator.

But we can do better. Let's try the second subsequent clearmake execution, without any real changes (for our purpose: in a real work scenario, a new build would of course be motivated by a need to test some changes). It does not yield all is up to date, as one would expect when using the default Makefile, but instead it does something interesting:

$ clearmake -f Makefile
rm -f /vob/jbuild/FStack.class
/usr/bin/javac /vob/jbuild/FStack.java

rm -f /vob/jbuild/Main.class
/usr/bin/javac /vob/jbuild/Main.java

Note that it does not even execute the default script, but rather some other one (/usr/bin/javac /vob/jbuild/FStack.java). Where did it come from? Actually from the FStack.class.dep dependency file mentioned above. And what about the dependency files themselves?-They have somewhat changed:

And now this looks right! The FStack class depends on FStack$Enumerator, but it does not depend on the Main class, and this is noted in the modified FStack.class.dep. The Main class, on the other hand, does depend on FStack, and that is stated correctly in Main.class.dep.

Now, if we try to run clearmake once again, it yields 'all' is up to date:

$ clearmake -f Makefile
'all' is up to date.

But this time it means that all the dependencies have been analyzed and recorded in the dep files.