Notes on Expressionism in Model Relay

This new work on transformations together with model relay is described by JIRAS http://issues.fluidproject.org/browse/FLUID-3674http://issues.fluidproject.org/browse/FLUID-5045 and http://issues.fluidproject.org/browse/FLUID-5024. The current discussion (mostly arising on a call with Colin on Thursday 28th) relates to particular strategies to be used for writing Model Transformations "transform" elements that are used in linking together models - with the particular example taken from Infusion's Pager component which has motivated ChangeApplier work for a number of years. The 2010 ChangeApplier system of "guards" was largely designed with this use case in mind, but when the rewrite updating the Pager in Spring 2013 finally arrived, it was discovered that the system wasn't adequate. A brief sketch of the Pager's model:

 

FieldPurposeConstraint or data source
pageIndexThe currently displayed page index in the Pager's UIConstrained to lie between 0 and totalRange - 1
pageCountThe total number of pages availableRoughly equal to totalRange dividided by pageSize, but some wrinkles in the rounding formula
pageSizeThe number of items currently displayed on a pageUsually selected in the UI - not directly constrained
totalRangeThe total number of items availableUsually arising from the model structure of the table or DOM markup for the pager - not directly constrained

 

A major goal of the model (+ model transformation) system is to be able to keep the pager's model valid without irritating state-dependent code being written by the user. The pageIndex and pageCount "guards" for the 2010 pager were written using the "validation" model familiar from many web frameworks (e.g. that abstracted by Spring Validation, etc.) but still need to be manually "scheduled" by the user in order to account for the fact that they are constraints which become invalidated by the values of other model fields.

The FLUID-5045 work promised to supply a new and much clearer scheme for expressing these kinds of constraints, based on the "integral tendency" (see New Notes on the ChangeApplier - in this model, one doesn't supply dedicated rules for updates to model state which either accept or reject them (guards/validators) but instead only provides assertions (lenses) about the overall state of the model and its internal relationships, and expects the framework to take care of scheduling and operation of the rules. This is consistent with the spreadsheet-like "end user idiom" which is promoted e.g. by the IoC framework itself).

There remain a number of choices as to how to best express the model transformation rules, for which looking at the Pager case continues to be instructive. Here are some of the implementations, both old and new, for reference:

 

Old Implementation for Pager Guards
    // 10 -> 1, 11 -> 2
    fluid.pager.computePageCount = function (model) {
        return Math.max(1, Math.floor((model.totalRange - 1) / model.pageSize) + 1);
    };
    
    fluid.pager.computePageLimit = function (model) {
        return Math.min(model.totalRange, (model.pageIndex + 1) * model.pageSize);
    };
 
    // TODO: Remove this and replace with FLUID-4258 scheme
    fluid.pager.preInit = function (that) {
        var applier = fluid.getForComponent(that, "applier");
        // Explicit namespace to evade FLUID-5077
        applier.postGuards.addListener({path: "pageSize", transactional: true}, 
                fluid.pager.pageCountGuard, "pageSizeGuard");
        applier.postGuards.addListener({path: "totalRange", transactional: true}, 
                fluid.pager.pageCountGuard, "totalRangeGuard");
                
        applier.guards.addListener({path: "pageIndex", transactional: true}, 
                fluid.pager.pageIndexGuard);
        
 
    // guards pageIndex, pageSize - transactional
    fluid.pager.pageCountGuard = function (newModel, changeRequest, iApplier) {
        var newPageCount = fluid.pager.computePageCount(newModel);
        iApplier.requestChange("pageCount", newPageCount);
        // TODO: this unnatural action is required to ensure shortened pageCount properly invalidates pageIndex - failure in transactional ChangeApplier design
        iApplier.fireChangeRequest({
            type: "ADD",
            path: "pageIndex",
            value: newModel.pageIndex,
            forceChange: true
        });
    };
    // guards pageIndex - may produce culled change - nontransactional?
    fluid.pager.pageIndexGuard = function (newModel, changeRequest) {
        var newPageIndex = changeRequest.value;
        if (changeRequest.value >= newModel.pageCount) {
            changeRequest.value = newModel.pageCount - 1;
        }
        if (changeRequest.value < 0) {
            changeRequest.value = 0;
        }
    };

 

Motivating the new implementation was the following discussion, and triage of possible transformation styles as being expressed using strategy a), b) or c) :

 

Chat Transcript
(21:32:20) : The model relay stuff is a bit peculiar
(21:32:37) : Since I am discovering that the most frequently used transform might well be one that we didn't write yet
(21:32:45) : I am planning to call it "fluid.transforms.free"
(21:32:51) : Which just invokes any function with any old arguments
(21:33:24) : I wanted to show you the current state of it but it's currently hidden inside a git stash.............
(21:33:51) : But it raises this very fundamental issue of "identifiability vs concision"
(21:34:05) : Or "discovery of common intent", or whatever we want to tall it
(21:34:18) : It seems that every really useful transform has some odd "wrinkle" to it
(21:34:46) : For example, the pager you could say has two very standard transforms in it - i) division, and ii) limiting to a range
(21:35:33) : But each one has some odd wrinkle with it
(21:35:47) : So for example the division must be i) limited to integers, and ii) rounded up rather than down
(21:36:02) : And the limiting to a range is offset by one, so it limits to the range [0... n - 1]
(21:39:32) : There are three things you could do about this
(21:40:29) : a) let people pile up huge trees of elementary JSON operators that essentially map out all the "expression wrinkles" as their own nodes - b) allow a bunch of "tweaks" or "decorators" on the basic operators, allowing for say, conversions or offsets, or c) just let people right very small functions that have all of the expressions as standard code
(21:40:34) : I am generally inclining towards c)
(21:40:53) : But this fights against one of our main religions, that "the same implementation has the same name", and "people who want the same implementation should be able to find each other"
(21:41:08) : But perhaps for things as tiny a little one-line transform functions, this "findability" is not really all that valuable

Leading to the following initial implementation, which in terms of the previous discussion may be considered to be written in FORM a) :

 

Draft implementation of Pager Model using Transformations
   fluid.defaults("fluid.tests.fluid5151root", {
        gradeNames: ["fluid.standardRelayComponent", "autoInit"],
        model: {
            pageIndex: 0,
            pageSize: 10,
            totalRange: 75
        },
        modelRelay: [{
            target: "pageCount",
            singleTransform: {
                type: "fluid.transform.free",
                args: {
                    "totalRange": "{that}.model.totalRange",
                    "pageSize": "{that}.model.pageSize"
                },
                func: "fluid.tests.computePageCount"
            }
        }, {
            target: "pageIndex",
            singleTransform: {
                type: "fluid.transform.rangeLimiter",
                input: "{that}.model.pageIndex",
                min: 0,
                max: {
                    transform: {
                        type: "fluid.transforms.binaryOp",
                        operator: "-",
                        left: "{that}.model.pageCount",
                        right: 1
                    }
                }
            }
        }
        ]  
    });

As can be seen, this is unreasonably verbose for a textual form, and there was a further discussion on how to relate together the use of the various styles that would arise, possibly as transformations were authored in different modalities (visual/non-visual/textual)

Three approaches for defining transformations

Summarising and expanding the previous discussion, we describe here the three approaches for writing transformer rules:

    a) Expand the transformer fully in terms of elementary transforms - with one JSON node per expression node. This is extremely verbose and not suited to the needs of those using textual representations. It may be reasonable for those using visual or non-visual tools. The transform in the listing immediately above lines 29-31 (target: "pageIndex") is in this form.

    b) Adjust the definition of the base transformer with "decorators" or further options which try to customise its behaviour for more situations. This may result in a compact definition but creates a "findability" burden - the additional options need to be document and users must be able to find them - and somehow enough value added to this process so that users are recompensed for the cost of searching for the transform and its options.

    c) Make as much use as possible of "ad hoc" short functions written in conventional JavaScript operating the transform. This might be the most compact option and eliminates any "findability" burden - however, it inhibits reuse and code sharing by users of the system. It is also not transparent to tools, cutting non-textual users out of the system. The transform in lines 10-17 (target: "pageCount") is in this form.

Given the general ridiculousness of the "pageIndex" transform in form a) above, we might perhaps convert it into form c) as follows:

Form c) for pageIndex guard
{
target: "pageIndex",
singleTransform: {
    type: "fluid.transform.free",
    func: "fluid.tests.limitPageIndex",
    args: [
        "{that}.model.pageIndex",
        0,
        "{that}.model.pageCount"]
    }
}
 
fluid.tests.limitPageIndex = function (input, min, max) {
    if (input < min) {
        input = min;
    } else if (input > max - 1) {
        input = max - 1;
    }
    return input;
    } 

This would be effective, and still an improvement on the "pageIndexGuard" from the old implementation (top listing) - however this may still not be best for this case - which is indeed so ubiquitous that it might better be handled by approach b) - see below.

Harmonising the extremes - trajectory for authoring tools for transformations

The stark difference between approaches a) and c) and the differing interests of the communities which might author them cut to the heart of the issues we hope to address when beginning our authoring infrastructure next year (2014). During Thursday's (28/12/13) conversation with Colin (sadly this can't be abscribed a location such as "pupusa conversation") ideas for a tooling approach emerged which might help to get alternatives a) and c) in better contact. For those using approaches c) we might draw up a limited subset of simple JS functions (perhaps, those containing no control structures, references to other functions or use of higher-order functions) for which we supply special interconversions support. This is reminiscent of other "subsetting" approaches such as Mozilla's asm.js or ECMAScript 5's "strict mode" etc. This support could be operated by the well-known Esprima parser or otherwise. For a suitably simple vocabulary of short and simple functions, we could guarantee bidirectional conversion between form a) as produced by authors using visual or non-visual tools, and form c) as produced by coders - a form c) equivalent would also improve performance considerably.

Another important reminder from this conversation was of the visual form of the authoring process - which should always "carry the test cases along with the code". That is, that the initial authoring experience would consist of supplying some paradigmatic test cases, which would then permanently accompany the implementation, acting as both documentation and test cases. The fact that this procedure is really what is followed by real developers in any case can be seen by the cryptic comment // 10 -> 1, 11 -> 2 preceding the implementation of the function fluid.pager.computePageCount. The form of its body is already relatively obscure, although the expression return Math.max(1, Math.floor((model.totalRange - 1) / model.pageSize) + 1) is relatively readable compared to some "expressions of the art" which can be seen in the more complex regular expressions.

Form b) after all for range limitation

Discussion on authoring tools notwithstanding, it was considered desirable to provide a "form b)" for the range limiter after all. After all, this is the paradigmatic "validation function" (insisting that a value lies within a particular range) and if it isn't a candidate for being part of a "standard library", then nothing is. The oddity with the handling of the upper range could be handled by supplying three extra properties to the transformation function which operates it:

(documentation for standard function to be commissioned, fluid.transform.limitRange)

argumenttypedefaultpurpose
inputNumbernone: requiredThe value to be transformed
minNumber-InfinityThe minimum of the range to which the value is to be constrained
maxNumber+InfinityThe maximum of the range to which the value is to be constrained
minExclusiveBooleanfalseIf min is specified, minExclusive may be specified to indicate that the actual minimum value itself is not a valid part of the range. A value which is within granularity (defaults to 1) of the minimum value will be mapped to min + granularity
maxExclusiveBooleanfalseIf max is specified, maxExclusive may be specified to indicate that the actual minimum value itself is not a valid part of the range. A value which is within granularity (defaults to 1) of the maximum value will be mapped to max - granularity
granularityNumber1The "buffer range" which is operated by the properties minExclusive and maxExclusive. They will in addition to excluding values beyond min and max themselves, if enabled, also exclude values which lie within granularity of the limit.