Dynamic Aspects

Interactive terms and evaluation stores

noreply@blogger.com (Roly Perera) — Mon, 16 Nov 2009 00:18:00 +0000

Here's the final version of my PADL paper. I've hived off all the technical spec to a separate technical report, and devoted the space thereby liberated to a clearer explanation of how it works: interactive terms, lifting and unlifting, value stores, evaluation traces and evaluation stores, reification, memo-reification and synchronisation. The mutually recursive relationship between reification and synchronisation is one of the things I've tried to clarify.

I went to FP lunch at Nottingham on Friday. The main reason for the trip was to hook up with Henrik, but I also gave a talk. For the talk I knocked up some slides that animate the evaluation store for the example programming scenario in the paper, using output generated by my F# implementation. It's much easier to visualise how the system works this way, so I've included them below. I've also include a rather crude hand-crafted animation of the mock-up UI from the paper. As mentioned in the paper, I only allow edits to program constants, which in this setting, means terms built purely out of constructors.

Motivating example

These slides animate the motivating example from the paper. The programmer interactively discovers and fixes two bugs in a simple program which compare two lists. The UI shown is only a mock-up, intended to be representative of what one could build on top of my current system.

In the scenario, the programmer takes her program through 5 editing states. For some states, the slides also show some alternative view configurations. For example, at state 4, the programmer collapses the recursive call to equal, once she realises the problem actually lies with the definition of equal_nat. This isn't an edit to the program, but just to the view. Browsing features like this will be essential to a practical implementation of interactive programming, but are orthogonal to the work presented in the paper.

The following diagram illustrates some of the features of this hypothetical UI; they're not essential to understanding the paper, but are useful for making sense of the mock-ups:

Interactive terms

The second set of slides show how edits are enabled. A transformation called lifting replaces constants in the original program by references to store locations initialised to those constants. (I call this a value store, to distinguish it from an evaluation store, below.) The result of lifting is called an interactive term (shown on the right), whence we can recover a non-interactive term (on the left) by unlifting at a particular store state. Edits are modelled by mutations to the value store, and so an interactive term is associated with a family of unliftings, one for each possible state of the value store. Lifting thus converts a term into a mutable representation, and unlifting does the reverse.

Evaluation stores

Because the unlifting of an interactive term changes as the value store changes, how it executes may also change. It can compute a different value; or it can compute the same value, but "for a different reason". In other words the intensional behaviour of the program can change, even if extensionally it is unchanged. This behaviour is reified into a mutable representation called an evaluation store. These slides show the evaluation store evolving through the 5 states of our scenario. There are two further states which show how fragments of reified evaluation persist and are reused if they arise in subsequent states. The notation used in the slides is explained in the paper.

So what next? The main limitation of the current system is that only data values can be changed. This in turn effectively requires a restriction to a first-order language. The reason is this. In an interactive programming system, functions are not purely extensional beasts. Their intensional behaviour is observable too. We can't therefore allow modification of functional values without permitting arbitary code modification, since the intensional behaviour of a function is given by its definition. This doesn't arise in standard FRP systems, which are higher-order, but where functions behave purely extensionally.

Interestingly, and as Thorsten mentioned, once you permit code changes, you obtain something of a higher-order feel, even in a first-order setting. So code modification is the key thing I want to enable, from which higher-order functions should naturally follow.

The whys and what ifs of programming

noreply@blogger.com (Roly Perera) — Sat, 05 Sep 2009 06:56:00 +0000

I'm submitting this paper to PADL this coming Friday. Conveniently, the organisers extended the deadline by a week, so I have a few days to solicit some comments. It's not as coherent as I would like, but for the first time I've got something approaching an end-to-end story: a problem, a motivating example, a solution (with formal spec), and an implementation. It's still restricted to a first-order, call-by-value setting, allowing me to side-step the issue of code modification. (If values can have function type, and values can change, then there's no side-stepping the issue. Ditto call-by-name, which allows values to have code in them.) I'm planning to tackle code modification next, and with that in place it should be easy enough to accommodate higher-order functions too.

I've narrowed my motivation down to focus on an "interactive" style of programming, where the programmer is engaged in an ongoing dialogue with the development environment, and all programming activities, including writing new code, are framed in terms of the programmer asking questions and the development environment responding. It took me a while to realise that having a narrow motivation is really the only sane methodology. If there are other applications or benefits (as I think there are) they're best treated separately, else the whole thing gets too unwieldy. For now it's nice to have a narrow problem to solve and a reasonably simple story to tell.

There's not much yet in the way of theorems about formal properties of the system. So far, the formalism is really just serving as a dependently-typed programming language, only one I don't happen to have a compiler for. Guess I should probably start using Agda.

There is some source code though. I was planning to host it on GitHub, but unfortunately I'm still running on Windows, and Git on Windows looks a bit sketchy. So I'm using Google Code. (Annoyingly its source browser doesn't like the apostrophes I've used in my identifiers, so I may think again.) The code is now in F# instead of Haskell, which will hopefully make experimenting with UI ideas a bit easier. For now there's no UI, though.

So: what are the biggest problems with this paper? I'm glossing lots of important issues, time and space performance being obvious ones, and how to recover something like normal effectful programming.

Full vs. partial reification

noreply@blogger.com (Roly Perera) — Thu, 09 Jul 2009 13:05:00 +0000

My research project is called interactive programming. The central concepts are (1) reification, where a big-step semantics is taken to characterise a data type, and evaluation to produce instances of this data type; (2) reactivity, the interpretation of a program as a transition system whose states are such instances; and (3) persistence, a means for sharing state across transitions so that common substructure is not duplicated. Since none of these ideas is particularly novel, I need to be careful in my treatment of related work.

Reified evaluation arises, in a restricted form, in Umut Acar's self-adjusting computation, as well as in debuggers that record execution traces or similar structures. Reactivity is central to visual programming, spreadsheet languages, functional reactive programming (FRP), and again, self-adjusting computation. Persistence also arises in self-adjusting computation, as a form of memoisation. Self-adjusting computation is therefore of special importance: it is the only prior work which combines all three concepts. What I want to do here is compare Acar's (mature, well-worked out) system with what I'm hoping to do.

Before I start, I should point the reader to this interim technical report. My report has a number of problems, which I won't attempt to enumerate, and unfortunately it doesn't use the term "reification". (That was Tom Harke's idea.) It should shed some light on what I mean by a persistent evaluation graph, which is essentially Acar's notion of a "trace". My notation is intentionally similar to Acar's.

Self-adjusting computation

Self-adjusting computation is a language and runtime system for incremental computation. After an initial evaluation, the inputs of a program can be repeatedly modified, and the resulting changes to the output observed. During the initial evaluation, the runtime records a trace identifying how parts of the computation depend on other parts. When an input is modified, the output is re-calculated by a change propagation algorithm, which exploits the information in the trace to perform the update efficiently.

Early formulations required the programmer to deal explicitly with much of the technical machinery involved in making a computation self-adjusting. Although the resulting programs were still much easier to write than hand-coded dynamic algorithms, the approach was difficult and error-prone.

The most recent version, a self-adjusting extension of Standard ML called ΔML, provides more direct language support. This brings it closer in concept to interactive programming. Nevertheless it differs in several important respects, thanks to its focus on incremental performance. The following discussion is organised around the common features of reification, reactivity and persistence. The figure show some typical ΔML code.


structure ModList =
struct
  datatype 'a cell = NIL | CONS of 'a * 'a modlist
  withtype 'a modlist = 'a cell box
  type 'a t = 'a modlist

  afun lengthLessThan (l: 'a modlist) : bool box =
  let putM = mkPut $ ()
      afun len (i,l) =
        if i >= n then false
        else case get $ l of
               NIL => true
             | CONS(h,t) => len $ (i+1,t)
  in putM $ (NONE, len $ (0,l)) end

  afun map (f: 'a -> 'b) (l: 'a modlist) : 'b modlist =
  let val putM = mkPut $ ()
      mfun m l =
        case get $ l of
          NIL => NIL
        | CONS(h,t) => CONS (f h, putM $ (SOME h, m t))
  in putM $ (NONE, m l) end
end

Reification

In both systems, the initial evaluation is reified into a data structure capturing the dependencies between parts of the computation.

The key difference seems to be in whether the reification is "full", or only "partial". In self-adjusting computing, the trace only captures the aspects of evaluation relevant to efficient incremental update. In interactive programming, the entire evaluation is reified.

Self-adjusting computation	Interactive programming
The programmer must explicitly identify the adaptive aspects of the computation. The type constructor `box` distinguishes changable data, which may change after the initial evaluation, from stable data, which cannot. Keywords `afun` and `mfun` identify adaptive functions, which produce and consume changeable data.	All data is 'changeable'; all functions are 'adaptive'.
Computations are partly reified into traces. Traces are data structures which record the dependencies between adaptive parts of the computation, and the code fragments associated with them.	Computations are fully reified into evaluations. Evaluations are data structures "all the way down", recording individual steps of the big-step semantics.
No direct requirement that traces reflect in any way the derivation tree given by the original semantics, but only that the final computed values are consistent.	Evaluation graph can at any state be unravelled into the derivation tree given by the original semantics.
Runtime interface exports operations for querying and updating changeable data, but not for observing traces. But trace information could in principle be exported.	Runtime interface exports evaluation graphs as well.

Reactivity

In both systems, data can be modified after the initial evaluation, causing the output to be updated. Synchronisation updates the trace as well as the output, producing a result consistent with an initial evaluation, and leaving the system ready to respond to another input change. Both implementations are imperative, to allow state to be reused across transitions.

The differences seem to follow from the different treatment of traces: partial reification means that self-adjusting computation has an (efficient, but potentially problematic) hybrid execution model.

Self-adjusting computation	Interactive programming
Partial reification means that change propagation must re-execute, rather than merely synchronise, adaptive computations when the modifiables they read have changed.	Full reification of evaluation means no notion of "re-execution". Synchronisation is the reactive mutation of structure, "all the way down".
Re-execution interacts poorly with imperative features such as I/O (effects may be duplicated) and memory allocation (effects may differ). The occurrences of the `mkPut` primitive in the figure above are a symptom of the latter problem.	Full reification implies a purely functional treatment of effects: the evaluation can include descriptions of effects, and these descriptions can be updated by synchronisation. Invites the question of how interactive programming accommodates "traditional" I/O, such as launching missiles or writing snapshots of execution to a file.
Modification to data values only. Since ΔML is higher-order, these values may be of function type. However, functional values are opaque, rather than represented as changable data structures.	Intention is to permit modification to code as well as data, as a separate consideration from higher-order vs. first-order. Requires programs to have a similar representation to user-defined data types: in Acar's terms, modifiable code, as well as modifiable data.
Functional specification; mutable traces not part of observable behaviour.	Imperative specification; mutable evaluation nodes essential part of observable behaviour.
Imperative, bottom-up implementation, based on priority queue. No proof that algorithm conforms to functional specification.	Top-down implementation, corresponding closely to imperative specification.
Implementation shown to be optimally efficient for programs which are monotone with respect to a particular class of input changes.	Inefficient. No performance guarantees.

Persistence

In both systems, sub-computations are shared across states by a generalised memoisation facility which stores not just computed values but associated traces.

Here, the differences arise because interactive programming makes the identities of evaluation nodes part of the observable structure. This shifts memoisation from a performance trick, to playing an essential role in the treatment of evaluation graphs as purely functional data structures.

Self-adjusting computation	Interactive programming
Traces are not exported by runtime interface, so memoisation can only be observed indirectly, as incremental performance.	Evaluations are exported by runtime interface, so memoisation can be observed as an evaluation subgraph shared by adjacent states.
Only adaptive functions can be memoised, not arbitrary terms. Programmer specifies when adaptive functions are memoised, via keyword `mfun`.	Evaluation of every term is memoised, effectively by memoising the `eval` function of the interpreter. Common substructure is therefore always shared by adjacent states; evaluation graph is a purely functional data structure.
Memoisation only occurs across evaluations, not within a single evaluation.	Memoisation occurs within evaluations too, meaning that common sub-computations are shared.
Essential role of memoisation in performance restricts language to CBV. Unclear what kind of incremental performance is possible under CBN.	Not limited to CBV, because performance is disregarded. Intention is to support CBN.

The $64,000 question is: can full reification ever be done efficiently?

Sleepless near Seattle

noreply@blogger.com (Roly Perera) — Sun, 12 Apr 2009 20:26:00 +0000

View Larger Map

I'm off to Vancouver on Sunday to spend five weeks at UBC, thanks to the people at Universitas 21, who were kind enough to part with the required cash. It'll be a chance to immerse myself a bit in Gregor Kiczales' current thinking on fluid AOP. I'm banking on fluid AOP being the "killer app" for interactive programming languages. Every paradigm needs its killer app!

Unfortunately I'm still miles away from useful applications, but I finally have something to report. I even have some software that works, almost 800 lines of entirely uninteractive Haskell.

I'm planning to submit this paper to Onward! at the end of the week. In terms of technical maturity (or lack of) I feel stranded half-way between "applied" conferences like OOPSLA and more formal ones like ICFP, so I'm not sure if this is the right forum. The paper is rather rough in parts, particularly towards the end. I've also gratuitously ignored the topic of bidirectionality, which is essential to practical interactive programming. I would welcome comments on which bits are hardest to make sense of.

The road to effective informality

noreply@blogger.com (Roly Perera) — Wed, 18 Feb 2009 00:11:00 +0000

...is paved with meta-formalisms.

A while back (here and here), I argued that realising so-called fluid AOP will require a more formal approach to real-world languages and tools. This emphasis on formality seemed to be at odds with Gregor Kiczales' recent advocation of a less formal approach to software. Gregor's goal seemed reasonable; it was the methodology which perturbed me a little.

Having had several months to digest Gregor's response to my blog post, I can see now that most of the apparent incompatibility between our claims goes away if we construe them at the appropriate meta-level. In particular if we distinguish the content of a view from the computation of that content, we can see that informality as a (first-order) goal is in no way incompatible with identifying formality as a (second-order) means to that goal. One potential source of confusion, as Matt Hellige noted, is that this second-order formality can itself be viewed in both formal and informal ways. Nothing a programmer can't handle, though. First a recap.

Fluid AOP

As I touched on in my last post, fluid AOP frees the programmer from having to work with a single canonical view of a program, instead providing a variety of dynamically generated views, allowing her to use the view most suited to the task at hand. Changes made to these views are "put back" to a persistent representation called the primary decomposition. Fluid AOP, at least in any kind of generality, is largely vapourware, a great idea waiting for the enabling technology which renders it feasible.

Viewing the same thing in multiple ways is what we already do, in a language such as Java, when we have a class implement multiple interfaces. But with fluid AOP, these static facts about program organisation are no longer absolute, instead being relativised to particular views. Some organisational characteristics will made salient by a particular view, others will not. Abstractions become ephemeral things, visualised or made explicit when they are needed, remaining latent (Gregor's word) the rest of the time.

First-order informality

The big challenge facing fluid AOP is making these flexible views effective. Effective means bidirectional, so that the view can mediate changes to the underlying model ("view" and "model" to be understood relatively). Effectiveness is trivial if we work directly with the primary decomposition, but becomes arbitrarily non-trivial as we work with increasingly abstract views. The more information the view abstracts away, the harder it is - all other things being equal - to usefully interpret a change to the view as a change to the model. The kind of "informal" or fuzzy views that Gregor discussed in his OOPSLA talk are simply ones where the view disregards a large amount of information.

We can't expect any magic resolution to the tension between effectiveness and abstraction, of course, but what we can hope for, as both Gregor and Matt suggested, is more access to the trade-off, so that the situation becomes "parametric", rather than "binary", as Gregor put it. We would like the difficulty of achieving bidirectionality to be broadly correlated with the gain in abstraction. If we're not interested in abstraction or informality, we might be able to obtain automatic or near-automatic effectiveness; if we don't care about effectiveness, we may be able to make (limited) use of some very abstract or informal views. We might even be able to achieve both, if we're prepared to settle for a part-manual, part-automated process.

(A quick aside: bidirectional views have been studied since at least the '80s. The so-called "constant complement" approach to relational views involves defining a "complement" of a view, which determines how view changes are put back to the underlying tables. There can be multiple complements of a given view, each such complement defining a particular put-back policy. More recent developments include Foster, Pierce et al's work on the bidirectional language Boomerang, and this paper by Wang and Gibbons on bidirectional transformations between algebraic data types.)

Second-order formality

But how to obtain practical access to the effectiveness vs. informality trade-off? I think we have little choice but to take a more formal (but ultimately simpler) approach to computing. I'm not talking about the formality of the views themselves, but of the mechanism used to compute the views. With a caveat (which I'll come to), we will need much more of this "second-order" formality if we are to have any hope of making progress on the bidirectionality issue. Backward computation is forward computation made a lot harder, and so to succeed at the former will require a non-ad-hoc solution to the latter. Trying to make the jump to "informal" computing without getting "formal" computing right is like trying to make the jump to lightspeed on a tricycle.

Once again I'm out of time, so I will have to defer the discussion of how our current approach is ad hoc to my next post. But it should at least be clear that my claim that we need more formality, and Gregor's that we need less, are not really incompatible; we're simply talking about different meta-levels.

The caveat is just the point made by Matt Hellige that I mentioned in the introduction. The higher-order formality that I suggest should characterise the relationship between model and view will itself ultimately be viewable in various ways, some formal, some informal. Once we have fluid languages, we'll be able to use them to implement fluid languages, and that's where the fun begins.

Modularity in many dimensions

noreply@blogger.com (Roly Perera) — Sun, 02 Nov 2008 20:02:00 +0000

I recently read Doug Janzen and Kris De Volder's paper Programming with Crosscutting Effective Views. I grok a little more clearly now what Gregor Kiczales meant by the word "effective" in the OOPSLA 2007 talk I discussed in my last post. In Janzen and De Volder's terms, a view is effective when it can mediate changes to the underlying model. The problem of making a view effective, sometimes called the problem of bidirectionality, is a tricky one, but won't concern us here. Instead I'm going to talk about what Kiczales and others call fluid AOP, for which effective views are essential.

One program, many faces

Aspect-oriented programming (AOP) is the latest development in our ongoing battle with the problem of modularity. But although AOP, in its standard form, represents a useful step forward, it ultimately succumbs to the very problem it sets out to address: the fact that that some aspects or features of a system cross-cut others. By rearchitecting an OO system into a set of aspect declarations, one often merely inverts the modularity problem, distributing the behaviour of the original classes over the implementation of various aspects. What would previously have been a localised change to the class structure may now involve a change to several aspects.

Fluid AOP addresses this issue by taking a much less static view of programs. The idea is to treat aspects as editable -- "effective" -- views which are generated on demand for particular coding activities. For persistence purposes, one view is nominated as the "primary decomposition". (With a legacy language this may just be the regular source view, but in principle the primary decomposition can be any view that suffices to determine all the others.) Making these views effective is where bidirectionality comes in: each view must define how edits are "put back" as edits to the primary decomposition.

Before we come to fluid AOP, I want to revisit its motivation, and in particular the modularity challenge which is sometimes called The Expression Problem. My claim will be that, although some languages may be able to solve the Expression Problem, the more general modularity problem requires more than just fancy language features. Some kind of "fluidity", it seems, is essential. I'll close by suggesting that fluidity be understood in meta-linguistic terms: views must be seen as syntax, and inspection and manipulation of views as a form of meta-programming, often refactoring. "Effectiveness" (bidirectionality) is a challenge that needs to be tackled within this framework.

But I'm getting ahead of myself. Let's start with modularity. I'll understand modularity to mean, roughly, the extent to which a change to or extension of functionality requires only local changes. I want to make the case that the pursuit of "absolute" modularity is doomed: that we must always ask "modularity with respect to what kind of change?" In effect the question "how modular is my software?" is one that should be qualified by a class of transformation. If you're already familiar with the Expression Problem, you can skip the next section.

Modularity with respect to what?

Imagine we're writing an interpreter in, say, Java, for a simple language for integer arithmetic. We crank out the usual OO class hierarchy:

abstract class SyntaxNode {
    public abstract int eval ();
}

class Add extends SyntaxNode {
    public SyntaxNode left, right;

    public int eval () {
        return left.eval() + right.eval();
    }
}

class IntegerLiteral extends SyntaxNode {
    public int val;

    public int eval () {
        return val;
    }
}

(Don't worry about the fact that the fields are public; that's just me being lazy.)

We can make the observation here that adding a new kind of node involves only local changes: we just add a new node class and implement eval:

class Multiply extends SyntaxNode {
    public SyntaxNode left, right;

    public int eval () {
        return left.eval() * right.eval();
    }
}

What we have identified is a modularity property: modularity with respect to the addition of new node types.

Now let's consider a different kind of change. Instead of a new kind of node, we want to provide a new kind of operation on nodes, say a toString method. One might be tempted to add the following code to the SyntaxNode class:

public final String toString () {
    if (this instanceof IntegerLiteral) {
        IntegerLiteral op = (IntegerLiteral)this;
        return op.val;
    }
    else
    if (this instanceof Add) {
        Add op = (Add)this;
        return op.left.toString() + " + " + op.right.toString();
    }
    else
        throw new AbstractMethodError();
}

The problem with this approach, as we all know from standard OO doctrine, is that our über-method has to know about every kind of syntax node; in particular if we want to add a new kind of syntax node, we need to update this method as well to add a new clause:

    ...
    if (this instanceof Multiply) {
        Multiply op = (Multiply)this;
        return op.left.toString() + " * " + op.right.toString();
    }
    ...

So what we don't like about this approach is that it violates modularity with respect to the addition of new node types.

In a functional language like Haskell this particular kind of modularity failure isn't considered bad; in fact it's normal! Our class hierarchy would correspond to an algebraic data type:

data SyntaxNode =
    Add SyntaxNode SyntaxNode |
    IntegerLiteral Int

All our functions would start with a case analysis, the functional counterpart of a type switch:

eval :: SyntaxNode -> Int
eval x = case x of
    Add a b -> eval a + eval b
    IntegerLiteral n -> n

toString :: SyntaxNode -> String
toString x = case x of
    IntegerLiteral n -> show n
    Add a b -> toString a ++ "+" ++ toString b

and adding a new kind of node would require modifying every function to add the corresponding clause.

But returning to the world of OO, the "right" way to define toString is as follows:

class SyntaxNode {
    ...
    public abstract String toString();
}

class Add {
    ...
    public String toString () {
        return left.toString() + " + " + right.toString();
    }
}

This allows us to maintain modularity with respect to the addition of new node types. As before, we need only define a new class and provide an implementation of the various abstract methods:

class Multiply extends SyntaxNode {
    public SyntaxNode left, right;

    public int eval () {
        return left.eval() * right.eval();
    }

    public String toString () {
        return left.toString() + " * " + right.toString();
    }
}

But clearly this kind of modularity actually came at the expense of modularity of a different kind. To make it easy to add node types, we had to spread our toString implementation across the class hierarchy. In other words adding new operations entails modifying each class; the design thus lacks modularity with respect to the addition of new node operations.

FPers usually inhabit the other corner of the trade-off space: they enjoy modularity with respect to new operations, but not with respect to new node types. Although we could probably achieve a similar inversion of modularity in Haskell:

class SyntaxNode a where
    eval :: a -> Int
    toString :: a -> String

data IntegerLiteral = IntegerLiteral Int
instance SyntaxNode IntegerLiteral where
    eval (IntegerLiteral n) = n
    toString (IntegerLiteral n) = show n

data SyntaxNode a => Add a = Add a a
instance SyntaxNode a => SyntaxNode (Add a) where
    eval (Add a b) = eval a + eval b
    toString (Add a b) = toString a ++ "+" ++ toString b

data SyntaxNode a => Multiply a = Multiply a a
instance SyntaxNode a => SyntaxNode (Multiply a) where
    eval (Multiply a b) = eval a * eval b
    toString (Multiply a b) = toString a ++ "*" ++ toString b

In either paradigm, it seems that we're torn between operations cross-cutting data types, and data types cross-cutting operations (although OO prefers the former, and FP the latter). Which of these two alternatives is the "more modular" simply depends on whether it's operations, or data types, that we intend to modify. That's not to say that whether a design is modular is a mere matter of taste or opinion, but just that modularity is relative to what you're trying to do.

The multi-dimensional modularity problem

It's tempting to look for a "linguistic" solution to this problem. Perhaps with the right language feature (or the right design pattern), it would be possible to make a program modular with respect to both kinds of transformation simultaneously. Some time ago Philip Wadler dubbed this challenge the Expression Problem. (Although discussion of the problem itself goes back further: see for example this chapter of SICP. Thanks Noel!) CLOS-style multi-methods are one linguistic approach to this problem; design patterns such as Extensible Visitor are another.

But although there certainly are linguistic approaches to the Expression Problem, there is usually a cost in terms of design-time cohesion. Multi-methods and similar approaches work precisely by making programs more granular (less cohesive), and then relying on a dispatch mechanism to compose the fragments (restore cohesion) at run-time. The point of the granularity is precisely to achieve modularity: behaviour can often be added or modified by just dropping in a new code fragment. But the loss of any unified view of individual features or behaviours means the programmer has to mentally execute the composition rules in order to know what will happen when the program runs. And the cohesion problem only gets worse when you try to generalise these approaches to multiple dimensions of composition.

Standard "static" AOP is not really a response to the Expression Problem but to this more general problem of multi-dimensional composition. It avoids the granularity problem associated with multi-methods and similar approaches by allowing us to specify aspects as cohesive design-time modules. But the very cohesion of these definitions means that they are inevitably cross-cut by other features of the system. These features can now only be understood by reasoning about how the aspects are woven into them. The benefit of increased modularity in some respects has, as with our interpreter example earlier, been bought at the price of reduced modularity in others.

The basic problem is that there is no static language feature that can universally resolve the tension between modularity and cohesion. What we really need is greater fluidity. Since no design can anticipate its evolution, the capacity to "remodularise" a program quickly is much more valuable than any particular contingent modularity property. Our programming tools should allow us to eliminate degrees of freedom which we no longer wish to exploit (so they don't distract us from the task at hand), and introduce, in their place, the degrees of freedom required to achieve that task, safely and easily. This is the promise of tool-based approaches such as fluid AOP: "just-in-time cohesion".

Cohesion-on-demand turns weaving and unweaving into a browsing activity. The programmer no longer has to reason about complex composition rules à la multi-methods or static aspects, as she can simply inspect the content of the relevant view.

Where fluid tools become fluid languages

So it seems like fluid AOP is the only plausible approach to the problem of multi-dimensional modularity. It's hugely exciting, but also rather daunting when one thinks about what's involved in making the vision a practical reality. The words "indistinguishable" and "magic" come to mind. I'm well out of time and space resources, so I'll wrap up for now by just sketching what I see as the way forward.

Shortcomings aside, linguistic approaches have one indispensible property: they offer well-defined semantics for multi-dimensional composition. (As an example, see this paper, which extends Hindley-Milner type inference with static advice weaving.) The future of fluid AOP lies I think in harnessing this linguistic precision in an interactive tool.

After all, if we hope to rely on "fluid aspect" views to visualise key features of our software, as well as mediate critical changes to it, then they really are "the source files of the future". How the contents of the various views are combined, how edits are put back, and so forth, will have to be precisely defined. In turn, this will require fluid aspects to have well-defined syntax.

Indeed the difference between fluid aspects and traditional syntax trees is not one of kind, but only of lifecycle: they are built on the fly in response to the user's changing goals, and may co-exist with other effective views that share the same underlying model. But otherwise they are just plain ol' syntax. We see a taste of this in the "virtual source files" of Janzen and De Volder, as well as in the "fluid source code views" of Desmond et al.

(Of course I don't mean that we should necessarily store such syntactic artifacts in text files, or present them to users as flat, uninterpreted text; but in an uncontroversial sense they are "just syntax". This was the point of my post Why syntax won't go away.)

A corollary of this perspective is that we must regard our interactions with these views as a form of meta-programming. Matt Hellige expressed the same idea when he suggested that we think of "programming-time" as a computational stage. To interact is to effect a syntactic transformation. Switching to a new view of a program is really just a refactoring.

A mature AOP, then, is not a static language feature as such, but an interactive meta-programming activity: the process of refactoring a program into a view that provides a single locus of change for the edit you want to apply. Refactoring back to the original view distributes ("puts back") your change into all the relevant places. We could for example implement the toString method we defined earlier as a single type switch, for convenience; expand it over a class hierarchy, in order to add a new node type like Multiply; collapse it back to a single method again, perhaps to modify its signature; expand it again, to add more node types; and so on.

The toggling, in this hypothetical scenario, between two different points in modularity space is little more than the iterated application of the refactoring Inline method and its inverse. For an overriding method, Inline method injects the body of the method to be inlined into a new clause of a type switch in the overridden method. There are really just two syntactic presentations of the "same" system here, and a function that computes one view from the other, that happens to be able to compute backwards as well as forwards. (Ok, ok: maybe there's just an incey, wincey little bit of work to do to distinguish this from magic.)

The art of the science of software

noreply@blogger.com (Roly Perera) — Mon, 07 Jan 2008 18:45:00 +0000

Gregor Kiczales gave one of the keynotes at last year's OOPSLA (slides and audio). The abstract was promising, and it sounds like the talk was well received. Nonetheless, I think Kiczales is off target. But as the talk has a number of important themes running through it, and the subject matter is extremely pertinent to my own research interests, I will attempt some course correction, rather than just pick holes. As with my last post, I intend these remarks as friendly constructive criticism. The talk was in the exploratory spirit, after all, and it deserves a response in the same spirit. Unfortunately the various strands of the talk are woven together in ways that are both confusing, and confused.

Pluralism in software, pluralism in science

Kiczales' central observation is this: software abstractions are transient beasts, subservient to our equally transient purposes. Abstractions in a sense represent points of view, and thus flex and mutate as we shift our perspective. Philosophers sometimes call this kind of "perspective-friendly" meta-perspective pluralism. Pluralism acknowledges that there can be a plurality of distinct descriptions of a system - perhaps with radically different ontologies - without requiring that at most one of these descriptions is "correct". Put like this, pluralism sounds like plain common sense. And it seems inevitable that one day our languages and tools will reflect this pluralism. What's less clear of course is how painful the journey will be.

Kiczales also makes the interesting point that the shift towards perspectivalism in software in many respects mirrors a similar shift in how we think about natural systems. (At one point he asks the audience, leadingly, whether anyone believes in "scientific objectivity".) Here it seems that Kiczales has been heavily influenced by Brian Smith's book On the Origin of Objects. Smith was in the office next door to Kiczales for several years, and some of his ideas seem to have cross-pollinated. I'll come back to Smith's book shortly. But first let's consider this analogy between software systems and natural systems. (I apologise for the brief philosophical digression, but I think it's one of the threads of the talk which hits on an important point.)

Twentieth-century philosophy of science, certainly since the decline of logical empiricism, was dominated by various flavours of a metaphysical position called realism. The basic supposition of realism is that there exist theory-independent facts about the macroscopic organisation of the physical world, and that these facts determine whether a given successful theory is really true, or "merely" capable of accounting for all the empirical data one could, in principle, obtain. A realist might, for example, claim that there is an objective fact about exactly how many hairs there are on my head, and that this fact exists independently of - is ontologically prior to - any particular theory of what hairs are, or indeed what heads are, or what it is for a hair to be attached to a head. Some such commitment to an objective, theory-independent, "natural kind" ontology - what Nelson Goodman famously called a "ready-made world" - is the cornerstone of any realist world-view.

What's wrong with the realist picture is that there is something smelly about the idea of a fact which is in principle beyond the reach of empirical science. Scientific theories typically "parse" low-level ontologies into higher-level ontologies; these macro-ontologies are really nothing more than patterns swirling in the low-level structure, and the "truth", or otherwise, of such theories is, in scientific terms, fully exhausted by the empirical success of that theory. From science's point of view, "molecules" are just patterns in the quantum-mechanical substrate (or whatever) which satisfy a certain behavioural or structural description, and the extent to which the theory of molecules is more (or less) objectively true than any other theory is just the extent to which that theory successfully (or unsuccessfully) systematises the phenomena. There are no further scientific facts - no "trans-empirical" facts - which determine which theory is "the one true theory".

I can't hope to have done justice to this topic in that one paragraph, even if I knew enough about the subject to give it proper treatment (and I don't). But my aim here is only to concede to Kiczales and Smith what I think is fair concession: that any plausible alternative to realism must be pluralistic. It must allow for there to be multiple descriptions of the same natural system - perhaps with radically differing ontologies - without imposing the requirement that at most one of them is "correct". A theory of quantum gravity, if we ever find one, will not reveal General Relativity to have been "false" - but mysteriously successful - all along. It will just be a better theory.

So let us grant the point that realism is a metaphysical red herring. And I think we can also agree that the analogy with what's wrong with our traditional conception of software is compelling. We tend to think of there as being a unique objective fact about what a piece of sofware "does" - a unique theory of its behaviour. Our awareness of the existence of some underlying source code tends to fuel this intuition. But really we need to be much more pluralistic, and accept that what a piece of software "does" inescapably depends on your point of view. A security engineer might have a completely different view of a system than an end user. Each end user probably has a different view than other users, in as much as she can't see what other users are doing. Reports or audits produced for management are really nothing more than abstractions of how the system behaves. Even a bug-fix, without too much of a stretch of the imagination, is just a view of an erroneous program that applies a correcting delta to its behaviour. And many of these views and perspectives aren't just design-time artifacts, but are live perspectives on a running program. This pluralistic way of thinking about software is even more dynamic and fluid than fluid AOP: let's call it superfluid AOP.

An exciting possibility, then, is that fixing our philosophy of the natural world and learning how to think properly about software might end up dovetailing rather nicely. God's not a mathematician, he's a programmer, right?

Deconvolve this

So far so good. But where Kiczales' talk goes awry is in its leap from pluralism, to the ushering in of a new era of "formality-free computing". In this fluffy new future, we will sit around engaging in social negotiation and situated action, interactions which will somehow manage not to be "formal all the way down". Unfortunately, these are just Smith's bad memes at play. I guess I can see how the kind of pluralism just discussed might be innocently mistaken for post-modernism of the sort offered by Smith in his cryptic book, but it's a serious mistake. All I know is that if there is a place for post-modernist, lit-crit, social constructivist thinking in the modern world, it's nowhere near the field of computing.

The following excerpt from the Amazon "review" of Smith's book (presumably written by his publisher) captures the sickly flavour of Smith's vision:

Critics of programming practice have compared it to alchemy and Smith recalls the characterisation of Newton as the last of the magicians. Is this a pre-Newtonian phase, lacking "Laws", awaiting the differential calculus? Another position is suggested:

"... that we are post-Newtonian, in the sense of being inappropriately wedded to a particular reductionist form of scientism, inapplicable to so rich an intentional phenomenon. Another generation of scientists may be the last thing we need. Maybe, instead, we need a new generation of magicians". [p362]

Magician? Magus? Seeking the secret of how it is we "deconvolve the deixis" - plus ça change, plus c'est la même chose. The Alchemist: not a charlatan, but one possessed of much empirical wisdom stumbling after the scheme of things; as this new Science of the Artificial must do, self constructed, self referential, post-post-modern, a metaphysics for the 21st century.

I'm sorry, what?? When exactly did Gary Gygax get together with Jacques Derrida? It's somewhere between uninformative and downright misleading to attach significance to the idea that software is intentional (in the philosophical sense originally popularised by Dennett, and somewhat misappropriated by Smith). We can, too, skip gaily past Smith's notions of registration and zest with no fear that we're missing any useful insights. Like it or not, the bedrock of computing is the scientific world view, and Smith's anti-scientistic stance and vaguely Continental-style philosophy are about as compatible with this world view as creationism. Indeed, with the situation arguably inverting - and computation gradually becoming the conceptual foundation upon which science is built - it is even more important that we keep computing free of this kind of pretentious twaddle. It matters too much.

And while it may be true that interfaces are, unsurprisingly, often socially negotiated, we must be careful what we infer from this. So is the spelling of identifiers, the pattern of whitespace in a source file, the arrangement of plant pots in an office, after all. What we must cleanly demarcate are the forces that define a particular technical problem, and any particular solution to that problem. The problem that Kiczales has quite rightly identified is just this: abstractions are essentially dynamic and context-sensitive. There is no unique "correct" ontology for any man-made system, any more than there is a unique correct theory of any natural system. And one of the key forces that happens to drive this dynamism and context-sensitivity - but only one among many - is social interaction. ("One man's constant is another man's variable", as Alan Perlis nicely put it.) But it is a mistake to think that any observations about computing as a social activity offer insight into potential solutions to this problem.

Formality all the way down

This leads us to the final Smithesque strand we need to extract from Kiczales' talk and lay to one side. We are all familiar with the observation that simple interactions between parts often give rise to "emergent" phenomena, behaviours that are somehow novel or surprising, such as the macroscopic behaviour of ant colonies or eBay shoppers, but which are not in any way mystical or magical. As Figure 1 attempts to show, emergent behaviours are in a sense dual to the requirements on a solution. Requirements are known and obligate the system in certain ways, whereas emergent behaviours ("emergents", one could call them) are those which are permitted by the system, but which were not known a priori.

Figure 1: Required behaviours vs. emergent behaviours

Emergence is an important topic. But again, we must be careful not to make the leap from the uncontroversial phenomenon of emergence, to the highly controversial idea that reality (and by analogy software) might not be "formal all the way down", as Kiczales, following Smith, suggests. Formal all the way down is exactly what reality is. What else could it possibly be?

Smith's new-age version of emergentism is just an invalid inference from the failure of the reductionist programme in science. In the 1960's, many scientists, as well as philosophers such as Ernest Nagel, were optimistic that we would eventually be able to deductively derive all of science from fundamental physics, by establishing the right "bridge laws" between theories. Half a century later, this optimism looks naive. There has been only limited success, for example, in deriving much of chemistry from quantum mechanics on a "first principles" basis.

But the failure of this kind of reductionist programme does not mean giving up on formalism. We simply need a more mature perspective on the relationship between two theories, perhaps seeing the relationship as closer to one of computational abstraction than one of deductive derivation. During the Q&A session after the talk, someone asked whether "biology rather than formalism" was a better model of software. Yet much of the recent success in biology has come from developing new technical perspectives and formalisms. Witness the fast-evolving population of process calculi, equipped with ambients, stochasticity, branes, and whatnot, which are fast becoming mainstream tools in systems biology. Biological reductionism may no longer be plausible, but there is no inference to the inadequacy of formalism.

Once we concede this, then as with social negotiation, we can see that emergence is only indirectly related to the technical problem of enabling "perspectival programming". We don't need to design for emergence; what we mean by "emergent" is, after all, just that which doesn't come built-in. There are no insights we can export from emergence itself to the foundations of computing. Emergence comes about from the way we use things, the way things contingently interact, not from the mechanisms of interaction per se, which can and indeed can only be strictly formal.

The technical challenge: a new paradigm for interactive computing

So at last, I think we can distill the central challenge lurking at the heart of Kiczales' talk. How do we expect to realise the task-centric, perspectival model of programming that we know is coming? If abstractions indeed need only exist only in the service of specific interactions the programmer or user has with the program, then in the future we may be abstracting and unabstracting as frequently as we switch between edit buffers today. In their various ways, systems like Mylyn, fluid AOP, and Subtext offer a glimpse of what this world might look like (although Subtext is the only one of these that offers a glimpse of just how fluid the new paradigm might be). But do we have the technical maturity to realise this superfluid, aspects-on-steroids vision?

I suspect that Kiczales would agree that the answer is no. We simply lack a compelling paradigm for building robust interactive systems. But contra Kiczales, and as I argued in my last post, working out this new paradigm will require us to to embrace the formal, not reject it. The answer is not going to be to make things less effective (in his semi-technical sense of the word - roughly synonymous with "executable", I think, or perhaps "live" or "connected") but precisely the opposite. My own suspicion is that to make this sort of fancy stuff work properly, we will ultimately need a paradigm where software components are intrinsically, persistently and bidirectionally connected, and where interactive computation is the automatic and incremental synchronisation of distributed state. I've talked about this before, and will hopefully have more to say about it in the future, but for now I only wish to suggest that this, or something like it, is where we should focus our attention. It's where the real challenge lies.

Conclusion: less pop, more sci

To sum up, I sincerely doubt that there is an impending "post-formalist" reconstruction of the foundations of computing. If we want things to be fluffy, they're damn well going to have to be fluffy in some kind of technical, mathematically robust sense, not in some...well, fluffy sense. We should embrace pluralism and perspectivalism, both in science and computing, but not at the price of sloppy pop sci or new-age philosophy. The member of the audience who wondered whether Kiczales' "radical thesis" had more in common with quantum mechanics than classical mechanics should be sent to bed without any tea.

What this kind of question, the popularity of Smith's book, and to a lesser degree, Kiczales' talk, ultimately brings home is perhaps this: that if it's socially negotiated artifacts we're after, we need look no further than the world of technical ideas, mediated by the situation action of conference attendance. Smart people often believe strange things; maybe that's the object lesson.

Why syntax won't go away

noreply@blogger.com (Roly Perera) — Sun, 11 Nov 2007 11:35:00 +0000

Some time ago Jonathan Edwards wrote:

Let’s face it: syntax is a simple and intuitive way to represent structure. But a fundamental principle of Subtext is to avoid syntactically encoding programs so that their semantics are explicit and immediate. It should be possible to use syntactic techniques as a visualization in the UI instead of as a representation of the source.

Jonathan's hope is that we will one day extricate ourselves from the muddy waters of syntax and float free in the semantic stratosphere. It's hard to argue with the spirit of this hope, and his recent achievement with Subtext 2.0 is an exciting glimpse of the kinds of virtual realities for programming we might inhabit in 5 to 10 years' time. (Let's recap...he did all that thinking about schematic tables on paper - ok, electronic paper - and then just went ahead and implemented it, only to have it actually work...I'm jealous.)

The demo itself is a nice reminder of how we are still in the pre-dawn of the computing age, fumbling forlornly in the dark, largely ignorant of our own ignorance. (Paraphrasing Jonathan's OOPSLA presentation: "In the beginning was the static void...") Subtext is far from a fumbling exercise, though; or at least it's clever fumbling, fumbling in the direction of the light switch. But Subtext is both nascent, and philosophically radical, and like most radical youth it's still a little confused: not all of its ideas quite make sense yet. My own research interests are largely compatible with Jonathan's, and I have a few thoughts on how Subtext might come of age...a little friendly constructive criticism (although I guess Jonathan would agree with much of what I have to say), but also some more general considerations for the long-term future of these sorts of highly visual, highly interactive languages.

So where is Jonathan trying to go with his idea of "explicit and immediate semantics"? Just what is a UI visualisation of a program, after all, if it is not a representation of "the source"? Conversely, what modern tools really use unstructured text - character sequences interspersed with line-terminators - as representations for programs? Compilers certainly don't, and nor do any non-trivial editors. Most work with richer representations. (How successfully they do so is another question. The fact that they are often flaky and unrobust is perhaps what feeds our intuition that they are still just working with flat files "under the hood".) Flat files endure as an interoperability format, but they are not how anything that does anything with programs represents programs. So the thought must be something deeper: that we should be able to somehow present the "semantics" of the program directly to the user - for example using direct links in place of textual names - reaping significant ergonomic benefits by so doing.

Appealing though this idea obviously is, it is problematic. The problem is not with the claim that there are significant ergonomic gains to be had from advances in UIs, but with the idea that non-textual UIs are doing something different - deeply different - from textual UIs. Because user interfaces are syntactic...and not just in some academic sense, but quite straightforwardly. They have a recursive, tree-like structure; they can be described inductively by a term algebra over a fixed set of constructors. They have semantically irrelevant flourishes, like rounded corners and drop shadows; like most textual syntaxes, in fact, with their curly braces and fancy keywords. UI's must have syntax because that's how they convey meaning: there's no way to represent a set or a function (qua mathematical object), or whatever it is you take your program to mean, "directly"; you have to denote it in some way. (And unless you're some kind of mathematical Platonist, perhaps there are only denotations. If you are you need help.)

I'm being pedantic, I admit, but I'm not just being pedantic. I think it's crucial that we start to treat fancy UIs as fancy syntaxes, for the following reason. As UIs become more powerful and interactive - and as VPLs become more compelling - they need to become better behaved. We need to stop thinking of our programming environments as (metaphorically, as well as often quite literally) little "windows" through which we squint awkwardly at the "real" program underneath. Instead our UIs need to become our programs, and our programs our UIs. This is exactly the vision Jonathan has for Subtext, of course. But it can only be practically achieved if we understand UIs as bona fide syntactic structures in their own right, so that we can treat interaction (by end-users or other software processes) as the creation and manipulation of views and transformations of that structure. Think of those cool projections that approximate "method views" in Subtext: if they are to be any use for building real-world programs (whose [partial] meaning we need to be able to grasp from a [partial] visualisation), then these views need to both have a well-defined structure - yes, a syntax - and a well-defined interpretation. To borrow from Jonathan again: notation matters.

So here's my main thesis: only if we take the syntactic nature of UIs seriously we can turn advanced browsing and editing features into the powerful multi-dimensional programming constructs that they should be. Such features become robust and well-defined transformations: sometimes preserving behaviour, sometimes changing behaviour in well-defined ways. In this hypothetical future, refactoring, browsing and AOP become closely interrelated: browsing becomes a kind of refactoring, where the "factors" are something like aspects. This may sound fanciful, but this is just what the Subtext 2.0 UI gives us a tantalising glimpse of. Refactoring is already an interactive activity, but is nothing like robust enough: for refactoring to come of age it needs to be as formally reliable as the kind of batch-mode transformations that a (well-behaved) optimising compiler might implement. The same applies to advanced browsing features, particularly if they support "putback" editing - otherwise we won't be able rely on these views or trust edits mediated by them. And conversely, traditional AOP is a robust batch-mode transformation, but lacks interactivity: and so for AOP to come of age, we need to turn aspect-weaving into a live, interactive browsing activity - a UI, in other words, that allows programs to be woven and unwoven while they are being edited [HoK05].

So VPLs have the potential to turn UIs, in an important and I hope non-contrived sense, into rich and full-featured languages for meta-programming. This is potentially in tension with the fact that, to be practical to design, describe, and implement, programming languages need to be simple, or at least modular. Are schematic tables part of a programming language or part of a UI? Think about all the advanced features that schematic tables provide: rendering of predicates in Blake canonical form, enforcement of partioning. Are these features part of the Subtext language or the Subtext UI? And if this distinction is never to be made, how is anyone else to implement Subtext? Must we require bona fide implementations to be isomorphic to the full Subtext UI? (Must it be possible to encode the entire state of the Subtext UI in the state of the candidate implementation, and vice versa?) If we agree this is almost certainly an unreasonable burden to place on implementors, then it is not clear where to draw the line. The engineering risk to which Subtext is thus exposed is non-modularity: neatly sidestepping all the horrible complexity of maintaining a live mapping to textual syntax, only to gradually mire itself in a new kind of complexity that arises from pushing all those fancy views and direct manipulation features into a single "language".

The way to avoid this potential tarpit is, I suggest, not to shun syntax, but to embrace it. Visual programming languages do not get rid of syntax; they enrich it and make it come alive. But to think of powerful UIs as syntaxes and interactions as syntactic transformations that satisfy certain semantic properties presents a significant architectural challenge. We need be able to layer views of top of views on top of views...so that it's views all the way down, until we reach a "primitive" view. (In MVC language, we could just as well call them models: the point is that "model" and "view" are, as with "syntax" and "UI", just relative notions.) The primitive view, or model, is simply the lowest level of syntax for the language, but it is not otherwise distinguished ontologically. These syntaxes are related by semantic functions (homomorphisms), and ideally those mappings would execute both automatically and incrementally, so that changes in one syntactic structure are interpreted as changes in any dependent syntactic structures. The biggest challenge facing this paradigm is perhaps that many of these semantic functions will need to be bidirectional - i.e. functional in both directions, like Pierce's lenses [FGMP05] - so that we can edit at our chosen perspective...and it is this that really blurs the distinction between syntax and user interface from the end-user's perspective, providing the rich, integrated experience required for a programming tool, without compromising the modularity of the language. In fact each tier, of a system so conceived, is a separate language and each homomorphism on languages is a denotational semantics which interprets the syntax of one language in the syntax of another. But it's syntax all the way down. These languages are live, like visual programmings languages, but they need not be visual; my preferred term is interactive programming language (IPL).

So do schematic tables belong in a programming language or a UI? The question turns out to be ill-posed, because these are really relative terms. A similar analysis can be applied to Subtext's proposed treatment of names (or lack thereof). Theoreticians are familiar with the formal redundancy of names (or at least of bound names): they prefer to work with terms "up to alpha conversion of bound variables", or to rely on encodings like de Bruijn indices to sidestep name-related issues. But for programming purposes, does life without names get easier or harder? There are no doubt ergonomic benefits to be had from using direct links instead of names. But there are almost certainly ergonomic benefits associated with names too. Imagine a Subtext-like language with only direct links equipped with comment-like annotations. For any non-trivial programming exercise it would probably not be long before we started embedding naming conventions into comments - in other words treating them as structured, rather than flat. We might for example adopt an informal rule whereby no two sibling program elements can have the same comment, so that we can unambiguously find an element by finding its parent and then searching the comments of its children. We might then want to be able to "repoint a link" by overtyping on the comment. At other times, we might expect to be able to maintain the identity of the referent and update the text of the comment instead.

I don't propose any resolution to the tension between names and links here, because I suspect the ultimate requirement is going to be to have characteristics of both. Methodologically, at any rate, it seems unwise to rule out forever the use of textual names on a priori grounds. So the risk for Subtext as it stands is that if its goal of emancipating us from the world of textual names turns out to be untenable for the kinds of reasons just mentioned, it may be forced to reintroduce names on the sly, by providing various naming-like features in its UI (and effectively embedding a complex language extension). It then faces the very "creeping complexity" dilemma that I suggested arises with schematic tables. On the one hand, if we are not careful to design the language to be layered and modular, implementations become unmanageably complex and lack clarity on exactly what they must provide. Yet if we accept the challenge of designing our language to be layered and modular, and propose language extensions to deal specifically with names - new syntaxes, plus semantics which map terms in our "nominal algebra" to direct links - then we still seem to be lacking a robust implementation paradigm. (And no, MVC doesn't touch the sides.)

As Alan Perlis said, "There's no such thing as a free variable." Or as the counterpart of Fred Brooks probably said in a nearby possible world, no syntactic silver bullets. Maybe we're forced to accept that there's a significant bootstrapping issue here: to build mature IPLs, we will need powerful IPLs. Closing the bootstrap gap is one of the focuses of my own research, so maybe one day Jonathan and I will meet in the middle.

From the sublimity of Subtext to...the Charles Petzoldiness of Charles Petzold. I have the disturbing (but in a good way) feeling that this guy is completely serious; as a precautionary measure, however, I shall be taking his observation with a suitably large pinch of monosodium glutamate. Clearly this dude is a comic genius, and the joke is on me for not spotting it. He writes:

Here’s another example of perhaps the most notorious syntactical absurdity in all of traditional C# — the for loop:
for (i=0, j=0; i < 1000; i++)
if (IsPrime(i))
j++;
In CSAML, that jumble of symbols and semicolons is abandoned for a structural beauty that can almost induce the modern programmer to weep with joy:


<ForLoop>
    <ForLoop.Initializer>
        <StatementExpressionList>
            <Assignment LValue="i">
                <Assignment.Expression>
                    <Literal Type="{x:Type Int32}"
                             Value="0" />
                </Assignment.Expression>
            </Assignment>
        </StatementExpressionList>
    </ForLoop.Initializer>
    <ForLoop.Condition>
        <BooleanExpression>
            <LessThanExpression LeftSide ="i">
                <LessThanExpression.RightSide>
                    <Literal Type="{x:Type Int32}"
                             Value="1000" />
                </LessThanExpression.RightSide>
            </LessThanExpression>
        </BooleanExpression>
    </ForLoop.Condition>
    <ForLoop.Iterator>
        <StatementExpressionList>
            <PreIncrementExpression Identifier="i" />
        </StatementExpressionList>
    </ForLoop.Iterator>
    <ForLoop.EmbeddedStatement>
        <IfStatement>
           <IfStatement.Condition>
               <BooleanExpression>
                     <InvocationExpression MemberAccess="IsPrime">
                         <InvocationExpression.ArgumentList>
                             <Variable Identifier="i" />
                         </InvocationExpression.ArgumentList>
                    </InvocationExpression>
               <BooleanExpression>
        </IfStatement.Condition>
           <IfStatement.EmbeddedStatement>
               <StatementList>
                   <PreIncrementExpression Identifier="j" />
               </StatementList>
           </IfStatement.EmbeddedStatement>
              </IfStatement>
    </ForLoop.EmbeddedStatement>
</ForLoop>

I'm not sure that weeping with joy is quite the response the CSAML code would elicit (but I think I'm in agreement that it would involve some kind of bodily function).

Sweet Home West Midlands

noreply@blogger.com (Roly Perera) — Mon, 01 Oct 2007 21:49:00 +0000

Not my actual home, obviously. I'll still be living in Bristol. But Birmingham is going to be my spiritual home: where I go when I lose the faith and stop believing in life after Dynamic Aspects. My first day left me feeling refreshingly (if inaccurately) only mildly stupid. My supervisor Paul Levy seems to be an all round nice bloke; handy, as we've got to hang out for the next three years.

I'm currently learning modal logic. It's turning out to be awesome. I'm hoping, sometime (oh, in the next hundred years or so) to be able to use it to formalise the possible worlds view of interactive systems which I've been informally mincing about with for the last 6 years or so. The basic idea of modal logic is that a modal formula is a little automaton that can wander about in a structure of worlds which are mutually accessible through various relations, taking on different values at different worlds. Different access relations between worlds induce different kinds of structural properties on the space of worlds (or contexts, states, instants or points, depending on how you want to construe the relata).

Any kind of labelled transition system can be talked about modally, but so can many other things, such as time. A simple example: defining two modalities F and P (future and past) plus their respective duals G and H) such that F and P are inverses means that theorems such as

p -> GPp

and

p -> HFp

are universally true of such systems. (Respectively: if something is true now then at all future times it will have been true in the past; if something is true now, then at all past times it was going to be true in the future.) Temporal modal logic is obviously relevant to interaction [Pnu92], but modality in general seems to offer a nice framework for thinking about lots and lots of things: state, control, and dynamic scoping, to mention some existing applications, but also incremental computation, transactional concurrency, declarative concurrency, mobile code; the list goes on. And on.

That's the theory side of things, for now. On the practical front, I'm trying to understand monads and monad transformers well enough to implement a backtracking interpreter in Haskell that addresses the famously poor interaction between a normal-order or lazy reduction strategy and function caching or memoisation. The problem is simple: to be any good as an incrementalisation technique, memoisation needs to know the values of arguments to functions up front. But normal-order reduction defers the evaluation of arguments until they're needed. A prima facie conflict of interests.

I have a vague notion of a solution, an idea gleaned from Avi Pfeffer's work on IBAL [Pfe06], but much more investigation is required. The nice thing about implementing the interpreter using monads is that it should transfer easily into a modal language, as it turns out monads themselves are closely related to modal logic [Kob97][Pfe01].

(Actually call-by-need - the technique usually used to implement call-by-name - uses something like memoisation to avoid evaluating arguments more than once. But call-by-need is still a just-in-time evaluation strategy. What I'm looking for is something with the useful termination characteristics of call-by-name, for unneeded arguments, but the eagerness, for needed arguments, of call-by-value, without requiring a full global strictness analysis.)

Whither domain/j?

noreply@blogger.com (Roly Perera) — Sat, 10 Mar 2007 21:56:00 +0000

One or two people have wondered what happened to our self-styled refactoring tool to end all refactoring tools, domain/j. It no longer really exists; it is a mere flash demo of its former self. This demo and associated papers and abstracts is pretty much all that remains.

FInCo 2007

noreply@blogger.com (Roly Perera) — Fri, 02 Mar 2007 23:40:00 +0000

Here is the draft of a FInCo 2007 paper which elaborates (although still entirely informally) the programming model called declarative interaction which I mentioned in my first posting. FInCo is one of the satellite workshops of ETAPS 2007.

The review comments were useful, and reasonably positive. I'm looking forward to some interesting discussions. I conspicuously failed to give even a passing nod to agent-oriented programming; given the likely focus at the workshop, this is something I need to learn about.

Programming languages for interactive computing

noreply@blogger.com (Roly Perera) — Tue, 02 Jan 2007 23:50:00 +0000

Almost all software systems today are interactive, in that they maintain ongoing interactions with their environments, rather than simply producing a result on termination [GSW06]. Indeed, a consistent trend since the beginning of the digital era has been towards increasingly interactive systems. The trend has progressed on at least two fronts: enhancements to end-user interactivity, and increasingly integrated systems. The trend began with the first teletype and textual interfaces and continued through early GUIs and LAN-based operating systems. It continues with today's 3D virtual worlds and applications deployed over the wide-area network.

With the Internet emerging as the "global operating system", the pressure on our software to be interactive is greater than ever before. Consider how the following requirements can be understood in terms of enhanced interactivity:

Ability to reconfigure or repair applications without taking them offline → interaction with code as well as data
Long-running, continously-available applications → interaction must be robust
Sessions resumable from wherever we happen to be located → persistence of interactions
Transparent recovery from latency problems and intermittent connectivity → interaction should not be semantically sensitive to the network
Mobile code whose behaviour depends on execution context → dynamically scoped interactions

A variety of process algebras and other formalisms have been developed for modelling and reasoning about interactive systems. Yet despite the trend towards greater interactivity, we continue to lack a simple and coherent paradigm for building robust interactive systems. The main bugbear that has faced us has been what we might characterise as an "impedance mismatch" between traditional algorithmic programming languages and the way interactive systems abstractly work. Whereas an algorithmic language treats a program as a black box which produces a final value on termination, an interactive system allows other systems to observe and influence its behaviour as it executes, and must adjust its internal state in response to each interaction to maintain the consistency of the computation.

In current desktop systems, the mismatch is usually resolved by representing the state of the system as a set of mutable stores and then employing a notification scheme to maintain the consistency of the state. Rather than the host language being used to execute a single sequential program to termination, it is employed to execute fragments of imperative code as interactions occur. Each executed fragment must produce exactly the side-effects required to synchronise the state of the system correctly.

Unfortunately this near-universal reliance on the imperative to support interaction has come at an enormous cost in complexity. It will be useful here to recall Brooks' distinction between essential and accidental (or inessential) complexity [Bro87]. Complexity inherent in the problem itself (from a user's perspective), or which can be attributed to human factors, is essential; what remains is accidental or inessential. Interactive systems as currently implemented are dominated by inessential complexity, the main culprits being this ad hoc management of state, explicit concern with flow of control, and unnecessary use of non-deterministic concurrency. Imperative languages (ab)used in this way are the "Turing tar-pit in which everything is possible but nothing of interest is easy" [Per82]. Web applications only complicate things further, by adding a variety of ill-defined staging policies and interaction models into the mix.

Ironically, what unifies much of this inessential imperative complexity is that it exists in the service of essentially declarative ends. Indeed, I suggest that the reason the current paradigm works at all is that systems implemented in this way approximate a much simpler, declarative model of interactive computation. Experienced software designers rely tacitly on something like this declarative model in order to make decisions about what counts as reasonable behaviour. Rather than being ill-suited to interaction, as is sometimes assumed, perhaps because of the somewhat arcane feel of techniques such as monads [JW93][Wad99] for modelling effectful computation, the declarative paradigm - with a simple orthogonal extension to support interactivity - fits the abstract behaviour of interactive, stateful systems surprisingly well. But today, because this declarative behaviour is achieved only indirectly, interactive systems are significantly less reliable and understandable than they need to be, despite the widespread use of design patterns such as Observer [GHJV95] to manage some of the inessential complexity. State-related dysfunctions in particular, such as the gradual degradation of a system over time, spurious sensitivities to the order in which two operations take place, deadlocks and race conditions, are common. I propose that the best way to address the impedance mismatch between algorithmic languages and interactive systems is not to wallow in the tar-pit of imperative languages, but to lift declarative languages into a computational model which supports interaction directly.

The aim of this blog is to explore the conceptual foundations of such a programming model, which I shall refer to as the declarative interaction model. Developing a formal semantics will be a topic for later treatment. Like some other declarative approaches to interaction, the declarative interaction model maintains a clean separation of (stateless, deterministic) declarative computation and (stateful, non-deterministic) reactive computation. The model is distinguished by its modal construal of interaction, whereby an interactive system is taken to be a space of canonically represented "possible worlds", each expressing a purely declarative computation, along with an index into that space indicating the "actual world", which represents the system's current state. To interact with such a system is to select a new actual state from this space of possible states. Interactions (effectful operations) are lifted to meta-programs that manipulate values of modal type; crucially, they are unable to interfere with the purely declarative semantics of each possible state, and in any case are only required when they are essential to application logic. Non-determinism arises from concurrent interactions, which are handled transactionally.

The declarative interaction model relates to many active research areas, including modal type systems, incremental computation, meta-programming, declarative concurrency, transactional concurrency, dataflow computing, interactive computing, and wide-area computing. As we shall see, four closely related concepts in particular are central to the model: modality, incrementality , concurrency, and persistence. A key validation of the model will be the development of an interactive programming language which supports the model directly.