This page contains a writeup of a discussion between Antranig Basman and Simon Bates on 15/11/16, during which the idea was floated to eliminate some of our most fundamental and long-standing framework primitives (invokers, events, and possible some others) in favour of a new, unified primitive with improved authorial values and greater generality.
It has become increasingly apparent that invokers (which are modelled on the very well-established paradigm of methods as part of object orientation, which themselves were derived from the even more basic abstraction of function calls) fail to meet our increasingly clear criteria for open authoring, articulated in our paper introducing the Open Authorial Principle (OAP).
The improvement that invokers offer with respect to methods/function calls is at least to allow the signatures of callers and callees to be freely detached - by means of Infusion's reference resolution system. However, whilst arbitrary numbers of contributions of configurations to a particular invoker site can be arbitrated by the framework in a fine-grained way (especially by means of options distributions), the final authorial result mostly remains a crude "winner-takes-all system" whereby the winner of the arbitration gets the right to author the invoker definition. There is a minor exception to this whereby the funcName and args of an invoker might result from different authorial expressions - but in practice this is a freedom which has never been seen taken advantage of. It's overwhelmingly likely that, for a given invoker definition, the same author will want to determine both the implementation's global function name and argument resolution at the same time.
As we were writing the OAP paper, the contrast to affordances of other systems became clearer. In Aspect-Oriented Programming (AOP), it is possible to issue "around advice" which can interpose arbitrary many layers of gearing between callers and callees. The ultimate prior art for all of these systems is the method combination facility provided in CLOS.
Whilst AOP advice is a powerful system (and strictly more powerful than we can currently express using invokers), it creates an opaque authorial system. In most models for AOP, each element entering method combination is only allowed to interpose on the outside of any existing method combination (note that CLOS does feature a multiple inheritance system with a complex model for specificity which does allow method definitions to be interleaved amongst each other - however, this is hardwired to the class inheritance structure and features rules so complex that authors would be nervous in practice that the order they obtained in practice would be the one they wanted - as well as privileging certain authors as a result of their position in the inheritance hierarchy, cutting off interception possibilities from those who are unable to arrange for their rule to appear in the necessary place). In AOP method combination, once the order of combination has been settled, arguments and return values become hardwired within the call graph, by means of an earlier method dispatching to a later one via the primitive call_next_method with its choice of arguments, and making its choice of return value. This only allows local control of the flow of data - a return value or an argument which is not forwarded by a predecessor becomes permanently lost.
In Favour:
Against:
The Infusion Event system began life as a very straightforward implementation of the Observer/Listener pattern with free argument lists - but gradually began to outstrip the invoker system in power and authorial affordances as the implementation quality of the rest of the framework improved. An early innovation, inspired by misunderstanding the corresponding feature in jQuery's event system, was the ability to label particular listeners to events with strings representing namespaces. This allowed the listeners to be identified after the fact, and particular listeners to be targetted for overriding as configuration for grades was merged together. However, this feature didn't obtain its full power until the innovation of priorities in the 2015-era framework (FLUID-5506). This allowed the namespaces attached to listeners to be used to derive "positional constraints", giving authors the ability for fine-grained targetting against the listener notification order. Whilst these were called "constraints", an early realisation was that these did not precisely fit the traditional model of constraint as seen in constraint programming, which would find any position which did not violate a constraint as suitable as any other, and would lead to the algorithm determining the notification order being a topological sort. Instead, a "constraint" in our system of the form before:other does not signify that "this element should appear anywhere before other" but instead that "this element should appear directly before other unless some other element interposes". In this sense our "constraints" should be seen rather as positional directives, which specify the progress of the assembly algorithm, rather than abstract mathematical constraints. This leads not only to a more efficient algorithm but also clearer and more deterministic authorial results.
However, the model underlying listeners is fundamentally limited in several ways. To start with, the observer pattern itself lacks any form of reciprocity. The dataflow is "push-only" - out from the observable to the observers. It is also a strictly multicast model - each observer is enforced to receive exactly the same payload (in our model, this is mitigated, as with observers, by the possibility for each event listener to specify its own boiled signature by means of Infusion's IoC reference resolution system). As a result of this, the ability to finely control listener notification order falls somewhat upon deaf ears - since the results of having such fine control could only be useful if in some sense the listeners had strong side-effects on some other part of the system - signifying a somewhat broken overall design.
These limitations are effectively baked into the source design pattern that our feature was modelled on, and are somewhat part and parcel of each other - it's precisely because traditional events are forcibly multicast, that the idea of a "return value" seems faulty - since it would be impossible to abstract over the multiplicity of listeners in order to determine whether one or many values should be returned, and/or which one(s) of the collection should be responsible for computing it.
In Favour:
Against:
A somewhat improved primitive, which so far only has a "pseudo-framework" status (whilst it is supported by some utilities delivered along with parts of the framework, it does not have the status of a first-class framework feature - it is delivered solely using current "userland" features) is that of a transforming promise chain. This accepts configuration syntactically identical to that of a standard Infusion event, but misuses the resulting collection of listeners by firing them in a custom workflow, with a particular stereotypical argument set. It is intended that each listener may choose to return a promise rather than a plain value, leading to an interpretation of the listener as a task. These tasks would then be run back-to-back, implementing the promise sequence algorithm (note that our hardwired signature allows for a blend of the sequence and pipeline algorithms - in that argument 1 represents the cascaded value operating the pipeline, and argument 2 represents material invariant across the task array).
In Favour:
Against:
The "transforming promise chain" feature seems to point the way, especially when situated in the current discussion, to a single, generalised feature which serves all of these use cases as well as some others. Some implementation hazards/considerations in the way of such a feature are:
{elements}.<name>.args, say, in addition to the ability to make use of the special name previous in order to refer to their immediate predecessor in the sequence if this is meaningful. We would adopt the CLOS-like conventions that any listener that made no such declaration would receive the argument list of its predecessor (in the "chain" form) or of the first element (in the "event" form), and would pass on the return value of its predecessor by default.sequence algorithm implied by the default definitions in which each listener implicitly referred to previous. This then removes the clunky hardwiring in fluid.promise.fireTransformEvent that restricts us to purely sequential dataflows (we would be freed to express dataflows outside the convex hull of sequence and pipeline).fluid.identity(value). This then promises to be a generalised replacement for the members directive resulting in data, etc.An important topic surfaced during the discussion related to the user experience of users both new to the framework and those experienced with it. Presenting new users with an over-generalised solution risks having them lose their bearings, whilst making excessive efforts to preserve forms of configuration accepted by previous versions of the framework risks increasing their learning burden, and requirement to internalise the complete history of the system (as seen with users of C++) before they could become proficient with it. An excellent example of this "generalised new features for old" can be seen in the evolution of the C++ class primitive out of the former C primitive of a struct. This put old features and new features in parity - in that it could be said that "a struct is retrospectively interpreted as a class in which the definition of all the members has implicitly been prefixed with the keyword public:".
This would be one useful way to cast the old features in terms of the new - that is, by considering them as "configurational sugar" for longer definitions with some elements retrospectively considered implicit being interpolated. The question then remains whether the old constructs should be deprecated, or else retained as "learning aids" to the framework.
"elements"?
We should be able to model all of the old primitives described in terms of the new, unified primitive. Given our discussion on learnability, it's not clear whether we would abolish the old primitives or not - but we should design the new syntax in such a way that there is sufficient static information that will allow any instances of the new primitive to be optimised to use no more resources than the old. The use of the old primitives might then become a matter of convention - that is, they represent a more compact encoding of the user's intention in a particular situation, since they would represent a different choice of defaults - but all of the full syntax would be available in each location.
We have choices as to whether to expose more context names to give a more natural fit where these can't correspond to concretely manifest data (e.g. previous) or else to reduce our intrusion on the limited resource of resolvable context names.
Original invoker definition:
invokers: {
anInvoker: {
args: ["{arguments}.1", "{exterior}.value"],
funcName: "myNamespace.funcName"
}
} |
Represented as element:
elements: {
anInvoker: {
body: {
args: ["{element}.top.args.1", "{exterior}.value"]
funcName: "myNamespace.funcName"
}
top: {
return: {element}.body.return
}
}
} |
We reserve two namespaces, body representing the (normally single) listener forming the method invocation, and top representing the element's dataflow as a whole (that is, its input arguments and return value).
Original event definition:
events: {
anEvent: null
},
listeners: {
"anEvent.aNamespace": {
funcName: "myNamespace.funcName"
}
}
|
Represented as element:
elements: {
anEvent: {
aNamespace: {
args: "{element}.top.args"
funcName: "myNamespace.funcName"
}
}
} |
We get rid of the clumsy split of events/listeners being declared in separate blocks.
We've got several hazards stemming from the traditional packaging of events as specially treated top-level elements.
events also gives rise to a top-level definition which is within events in the resulting componentaddListener, removeListener, fire both creates a support burden as well as a garbage burden - we are forced to create some kind of closure to close over the internal state required for this supportaddListener/removeListener API is actually sufficient for configuration of complex model elements, and that actually operating them will always require some kind of external machine such as fireTransformEvent etcOriginal chain definition:
events: {
aChain: null
},
listeners: {
"aChain.firstNamespace": {
funcName: "myNamespace.firstFuncName"
},
"aChain.secondNamespace": {
priority: "after:firstNamespace"
funcName: "myNamespace.secondFuncName" // This function actually return a promise
}
} |
Represented as element:
elements: {
aChain: {
firstNamespace: {
funcName: "myNamespace.firstFuncName",
args: ["{element}.previous.args.0", "{element}.top.args.0"],
},
secondNamespace: {
taskName: "myNamespace.secondFuncName",
args: ["{element}.previous.args.0", "{element}.top.args.0"]
},
top: {
return: "{element}.final.return"
}
}
} |
We need to reserve two more special namespaces, previous and final in order to replicate the original functionality.
The original inspiration for this feature emerged whilst contemplating the markup generation pipeline for the fluid authoring/debugging system, which still needs to be implemented without anything in the way of a "renderer". Written out by hand, there are various functions which require to generate pieces of markup, and then integrate them into wider assemblages of markup, and then inject them into various places into the document.
This original use case suggests that we have a further space of requirements, which is also suggested by the fireTransformEvent workflow that we implemented for promise chains - that of subsetting the workflow at execution time. In that API, this makes use of the somewhat cumbersome filterNamespaces options that allows just some elements of the sequence to be selected for execution. This is not a very "open" API - in that its cost scales with the number of authors. More natural would be the ability to select custom "input" and "output" points for the workflow, and expect that the system would arrange that all points between them would be executed. This would handle our use cases within Kettle dataSources as well as with markup generation. However, it's unclear how this could be fitted in to a system which allowed completely free signatures. The "options" system for fireTransformEvent provides a clear place to put such per-request policy information. The way out of this is perhaps to allow the entire input argument set to be "wrapped" by some routes and not others. event.fire would be retained as a native entry point that allowed no space for policy, whereas we would then support a fuller element.operate(args, options) API that left space for them in arg 2. The declarative form of invocation would then then support options at every site where invocation was possible, in addition to args. At the cost of yet more garbage, we could accommodate this in our existing recordToApplicable utility.
There's a close analogy with the Model Relay definitions that can be attached to model material within our new ChangeApplier model scheme. These express dataflow elements where data is ingested from a named path within a document, undergoes some processing, and is output to another path.
The contrasts are:
return, whereas the output of a model relay is configured via target and can be freely ranged over the model material. Whilst model relay rules can, like elements, freely range over the document for their input material (via IoC references within the body of the transform), they traditionally draw input from a distinguished path named source.previous and last - all the listeners in an element are sorted into a single, stable sequence before they are operated, whereas model relay rules operate in a free graph, triggered by model invalidationThat said, there are routes through which we could close up the gap between these features.
top.return directive - whereas the model area of a component is persistent. However, we could make an analogy between an element firing and a very short-lived dynamic component