RE: incremental linking

[Andrew Kennedy <akenn@microsoft.com>]

<---- MLj users ---- http://www.dcs.ed.ac.uk/home/mlj/doc/users.html ---->

David,

Thanks very much for your comments. Before discussing incremental linking,
I'd like to point out that in a future release of MLj (probably not ready
for end September, but soon after that) the small amount of "separate
compilation" that happens already (parsing, type checking, translation to
intermediate language, limited optimisation) will be made "persistent". That
is to say, the intermediate language term and signature for each module will
be pickled in files so that modules don't need to be recompiled between MLj
sessions.

Even so, there's an awful lot that happens "post-link". When designing MLj
we did consider module-to-class compilation, but after a certain amount of
experimentation rejected it as having too detrimental an impact on runtime
performance. Suppose that a structure is compiled down to a class.
Presumably each value binding would translate to a static final field (or
possibly a static method for function values). Obviously we lose
cross-module optimisation, but modern JITs do quite a bit of inlining so we
might make up some lost ground (note that this wasn't the case when we
started designing MLj). However, the biggest issue is uniform data
representation, in particular *polymorphism*. The easiest compilation
technique for polymorphism (still used by O'Caml) is for values to be given
uniform representations so only one version of a polymorphic function is
required. For native-code compilers, this entails using pointers for all
values, "boxing" non-pointer values such as ints and floats where necessary.
For Java, it would mean using Object for all values. This is very expensive,
requiring extra allocations on the heap *and* many downcast operations to
get code past the verifier. 

Monomorphisation is easy and obviously more efficient, but relies on (a)
there being a finite, statically-known number of instantiations of each
polymorphic function (true for ML but not for Haskell) and (b) having the
whole program available. 

One intriguing alternative might be to simulate parametric polymorphism in
the Java classes by inventing an extended class format that describes
parameterized classes and polymorphic methods. GJ does this already. These
could be instantiated to real Java classes either statically by a special
class linker, or dynamically at class load time (code generation on the
fly!).

- Andrew Kennedy.

> -----Original Message-----
> From: gurr@pop.mrs.med.ge.com [mailto:gurr@pop.mrs.med.ge.com]
> 
> Currently MLj links & compiles an entire project at a time.  Would it
> not be posible to do incremental linking?  The virtues of incremental
> linking are both the obvious but also the problem that currently the
> only ways to modify (extend, patch, etc) a MLj generated 
> application are
> 
>   a) to recompile the whole thing from scratch or,
>   b) bumble arround trying to partition the application into parts 
>      with Java interfaces
>