This page describes various incarnations of a curiously recurring pattern which emerges during our move towards place-based (coordinatised, externalised, address-oriented) programming, away from the traditional orientation towards functions (pure or otherwise) operating on values.
A key, and faulty, idiom underlying both functional and object-oriented programming is that a freshly produced value may not be inspected until it has been fully processed. In OO, this moment is held to be the point at which its constructor (in practice, a mostly ordinary function) has returned. In FP, naturally any value produced within a function may not be inspected until its point of return - any violation of this principle is deemed a "side-effect".
Our new orientation is away from placeless values - every piece of state in a design should be given a more-or-less stable public address, rather than being hidden within impenetrable units of design such as functions or objects. Unfortunately we are faced with the challenge of scaffolding the new world using the techniques of the old, and even when fully incarnated we can't expect a address-oriented architecture to be fully homogeneous - there will always be "impedance boundaries" where it has to interact with traditional architectures, for reasons of efficiency or interoperability.
The places where these "new values" are brought into existence appear rather strange, from the point of view of traditional programming, and we still lack any comprehensive, general techniques for expressing them, either as traditional utilities or else as idioms within Infusion itself.
Prompting Situation
The situation which prompted this article was some work on the BioGaliano map-based species record visualisations, an early prototype of which can be seen at http://biogaliano.org/map-prototype/ . The situation was awkward, and quickly brought into mind numerous other situations where similar issues had cropped up, either within the design of Infusion itself, or even reaching back into the Spring-oriented RSF days of the 00s.
Notably the situation involves interaction with some external, rather unenlightened object-reference oriented library, the https://leafletjs.com/ library currently popular for presenting web-based maps. We have expressed our initialisation of the Leaflet map structure as a member, as so:
fluid.defaults("hortis.leafletMap", {
gradeNames: "fluid.viewComponent",
...
members: {
map: "@expand:L.map({that}.dom.map.0, {that}.options.mapOptions)"
},
...
});
Now, there is a bunch of further Leaflet setup that needs to have occurred with respect to the map before we can proceed to consume the overall component model and use it to render into Leaflet's DOM structures. Under the new post-FLUID-6148 framework, model initialisation occurs "apparently atomically" across a patch of instantiating component tree. This implies that we need to somehow tie the model-oriented consumers of the map into the Leaflet post-processing surrounding the delivery of this value. It may also be helpful to see this situation in the context of the refactoring that led into it - before the consumers of the map were model-oriented, it was sufficient to schedule the Leaflet actions into the component's in the following traditional way:
// Prior, old-fashioned scheduling of Leaflet actions in component's onCreate
fluid.defaults("hortis.leafletMap", {
...
listeners: {
"onCreate.fitBounds": "hortis.leafletMap.fitBounds({that}.map, {that}.options.fitBounds)",
"onCreate.createTooltip": "hortis.leafletMap.createTooltip"
},
...
});
Now under "new rendering" idioms, this is no longer good enough. onCreate is far too late to influence model-based rendering actions, and so we need to be much more specific about when exactly we expect these actions to occur.
In practice, we would like to express that "these actions should occur after the initialisation of the member map, and before any other part of the implementation attempts to access the value of map".
Here the intention expressed in bold is our "in-place conundrum". In a function-oriented, value-oriented, placeless architecture, this intent would be honoured by putting the actions inside the function body that computed "map", after it had been generated by its internal means (a further function call), and before it had been returned to the caller. Unfortunately, in our desired open authorial idiom, the elimination of these unaddressable, "dead spaces" in our designs is our main goal - see the following functional snippet for a sketch .
// Cursed, functional idiom
function produceMap () {
var map = upstreamProduceMap();
// authorial "dead space"
map.fitBounds(); // etc.
return map;
}
So, what to do? Some initial thoughts centred on the use of fluid.promise.fireTransformEvent, or perhaps more widely, the "gyres" promised in Plan to Abolish Invokers and Events. This seemed pretty annoying, since the function contract expects each member of the transform chain to return the value being generated - we have no desire to create silly wrapping functions for perfectly ordinary (if obnoxiously stateful) functions such as fitBounds or createTooltip.
Further thoughts considered creating some further piece of pseudostate, e.g. "initialisedMap", which depended both on map, and also on the workflow functions we expected to execute before map was ready. This was the kind of dodge we would consider in the old Spring days - if your only tool for scheduling is expressing a data dependency, then why not introduce a piece of fake data whose only purpose is to perform the scheduling?
In practice, none of these solutions are terribly desirable since they don't get at the heart of the issue - that the name (address) they want to use to identify map is exactly the same as the name that the outer consumers do. The designer of fitBounds wants to use exactly the same name to address map as the upstream consumers (the renderer, etc.) since anything else represents a nasty loss of design cohesion - we don't expect every author to have to read the design in detail to determine that initialisedMap represents exactly the same element as "map" only in a different workflow condition.
In the end we went with a stopgap solution, based on observing that no consumers of map expected to consume it before the component's model was initialised. Rather than a "pseudoevent" as operated by fireTransformEvent, we simply attached the workflow functions to a perfectly ordinary event which was, indeed, attached to a piece of pseudostate in the component's model, but at least one which consuming authors would not need to know the name of.
// Adopted, stopgap solution - undesirable with two additional "shadow design" elements,
// an event "buildMap" and some pseudostate "mapInitialised"
fluid.defaults("hortis.leafletMap", {
gradeNames: "fluid.viewComponent",
members: {
map: "@expand:L.map({that}.dom.map.0, {that}.options.mapOptions)"
},
model: {
// abuse of expander for scheduling, actually returns "undefined"
mapInitialised: "@expand:{that}.events.buildMap.fire()",
...
},
events: {
buildMap: null
},
listeners: {
"buildMap.fitBounds": "hortis.leafletMap.fitBounds({that}.map, {that}.options.fitBounds)",
"buildMap.createTooltip": "hortis.leafletMap.createTooltip({that}, {that}.options.markup)"
},
...
}
In the pre-FLUID-6148 framework this would have been unjustifiable abuse of the framework since it was advertised that users should not attempt to fire events before a component's onCreate (although this probably worked in practice under many conditions). After FLUID-6148, the much stronger emphasis on construct-time coding has implied that the use of construct-time events is a blessed technique. However, this is still fundamentally abuse of the intention of the framework and the meaning of its primitives - we have a piece of model state which only ever holds the value undefined whose only purpose is to schedule some side-effects before other values which are known to be in the model are consumed. It's a deeply unsatisfactory compromise.
A few phenomena surround this particular situation - firstly, one might ask, why didn't we simply put the map itself into the model? But in practice, the map itself is quite un-model-like material, it actually represents a "view" element from another framework, and this would have made little improvement on our situation, although we could perhaps instead got the advantage of abusing some other framework primitive such as a modelListener.
A better answer to this question might have stepped back and considered - at what level does Infusion actually solve problems like this at all? And whilst there is some small assistance for dealing with these in-place issues where they occur in fine-grained elements such as free-form component options or model material, these problems are in practice only solved at the level of entire components. This implies that a more appropriate solution would be to upgrade the leaflet "Map" into a "mapHolder" component of its own, whose only purpose was to bear constructional responsibility for it. Which of course is what hortis.leafletMap originally was, before it got larded with additional design responsibility. We could then have shifted the workflow into the onCreate of that component.
This still raises issues as before - we are forced to create further, and more fine-grained design elements than was our original design taste, when these so-called "unforeseen" refactoring challenges arise. A lot of our taste for component granularity is based on efficiency considerations - until we have the FLUID-5304 "Infusion compiler" the run-time costs of extra components are really egregious, but also just at the design level incurring 3 extra levels of nesting in the JSON structure represents an annoying readability constraint.
But in practice this "mapHolder" doesn't even clearly solve our original problem. These workflow functions in practice interact with the map at the "view" level of our own design, and it still becomes no clearer that we should expect the "map" member of the held component to not be effectively addressable before "onCreate" of the child component has finished. We could perhaps arrange this to be so by ensuring that the mapHolder was not a modelComponent but again this represents expecting a totally unrealistic insight into the framework's scheduling primitives on the part of the designer.
The Wider Situation
This problem reappears in some guise at every level of Infusion's design - as we mentioned at the outset, from some views it is its key authorial challenge but in practice one that it fails to meet, and offers a splintered array of somewhat mismatched solutions to.
At the level of individual component options, Infusion's original design attempted to deal with this via mergePolicies. These are essentially "reduce" functions, and, being directly inherited from the corresponding notion from FP, don't do anything to address the fundamental issue - as the intermediate arguments pass through the hands of the reducers, they are unnamed in the wider design. They are also not terribly amenable to interception, although with modern namespace+priority options distributions it is at least possible to arbitrarily interpose intermediate values in the reduction chain.
At the finer-grained variety of this kind of issue, FLUID-4930 is an umbrella JIRA for a variety of "retrunking" bugs. "retrunking" is a fairly imprecise name given to patterns of options consumption where a site at a leaf attempts to consume a value of a close sibling or, still more abstractly, a direct parent in the same component. Elementary varieties of this problem look as follows:
fluid.defaults("fluid.tests.retrunking", {
gradeNames: "fluid.component",
arrowGeometry: {
length: "{that}.options.polar.length",
width: 10,
headWidth: 20,
headHeight: 20,
angle: "{that}.options.polar.angle",
start: [100, 100],
end: [100, 200]
},
polar: "@expand:fluid.tests.vectorToPolar({that}.options.arrowGeometry.start, {that}.options.arrowGeometry.end)"
});
Here we have a nested value, "arrowGeometry.length" whose value depends on "polar.length" - but in order to consume the latter, we first need to consume "arrowGeometry.start" - which necessarily restarts the process of evaluating "arrowGeometry" at a trunk node in the options, hence the name "retrunking". This involves a version of our "in-place" conundrum. If "arrowGeometry" were a classic, functional value, until we had computed all of its children, it would have to be said to be "in evaluation" - and so in this geometry it is impossible that there could be a classical function computing its value and then returning it.
The framework has been resistant to elementary versions of retrunking bugs like the above for many years (although we should return to how it manages this in detail shortly), but a more elaborate version of the bug surfaced recently after the FLUID-6145 refactoring.
fluid.defaults("fluid.tests.retrunkingII", {
gradeNames: "fluid.component",
dom: "@expand:fluid.identity({that}.options.selectors)",
selectors: {
svg: ".flc-bagatelle-svg",
taxonDisplay: ".fld-bagatelle-taxonDisplay",
autocomplete: ".fld-bagatelle-autocomplete",
segment: ".fld-bagatelle-segment",
phyloPic: ".fld-bagatelle-phyloPic",
mousable: "@expand:fluid.tests.FLUID4930combine({that}.options.selectors.segment, {that}.options.selectors.phyloPic)"
}
});
The setup is extremely similar to the above, but we have left a placeholder for the evaluation of a member "dom" whose dataflow mirrors that of a standard fluid.viewComponent. This sets in motion the process of consuming the entire options tree at "selectors", which amongst itself contains a cyclic retrunking, that is, a member who consumes a sibling via an expander. This ends up confusing the framework's workflow, since the first data demand for the trunk value "selectors" marks the entire tree as "in evaluation", and any further attempts to revisit it encounter this marker rather than the currently populating trunk value.
All this raises rather deep philosophical issues about what it means for a value to be "fully evaluated", especially if that value classically appears to be a container for other values. This clearly has a different meaning depending on the nature of the consumer. If this consumer is a classical function, it naturally expects it to be in the classical sense, and indeed we have a JIRA about this too - FLUID-5981. But if the consumer of the value is the framework's own machinery, the answer is clearly different, otherwise no variety of "retrunking" could be supported. This directly feeds into the "containment without dependency" [footnote - first cited in PPIG 2015 paper at https://github.com/amb26/papers/tree/master/ppig-2015, this is the central insight of the 2010 Pupusa Queue Conversation] notion that founded the entire framework. From an OO or FP view of a structure, the structure "depends on" its members, and from our point of view it doesn't.
So - are we "changing" a trunk value (object or hash) by "discovering" further members nested within it during our evaluation process? In our particular riff on the notion of "referential transparency", we are not - since to us, a reference is an address. Having noted that we know what the type of the "value" is, and what its address is, we have noted two unchangeable facts about it - these do not alter during the evaluation process for the "value", nor for any other values that are found to be "nested" within it.
But to return to the theme at the head of the page, implementing such a process of "discovery" in a conventional programming language leads our "functions" to have a very peculiar appearance, and also a rather stereotypical one - they necessarily must accept some kind of more or less massive "environment" as an argument, in order that they may deliver a freshly discovered value "in-place" into its appropriate spatial position at the very earliest possible moment. In terms of the base programming language, this is a huge weakness - the function as well as being massively impure, has a massive environmental dependency that greatly restricts its reuse. It is for this reason that taking the hit of "being an Infusion" is something that we want to incur as rarely as possible, despite our commitment to pluralities in Infusions.
Let's now survey what these sites actually look like within Infusion, starting at the small scale where individual options values are "delivered":