The State of Options Merging 25-4-16

Several suspicious issues have emerged over time with the way we operate the options merging and expansion process in the framework, which we now collect here, as things are coming to a head prior to the significant framework rewrite hopefully expected later in the year. It seems that various of the goals and design ideals of the options merging system are conflicting, and we will need to make some hard decisions and significant changes in order to establish a coherent model.

The JIRA FLUID-5800 contains a lot of discussion about the options merging algorithm itself, and records some vacillations over whether we felt the need to move over to a more well-attested grade linearization algorithm such as C3 (described in the classic Dylan paper). As it seems to be turning out, whilst our linearization algorithm, which could be described as a "depth-first painter's algorithm", can lead to some peculiar results, it might well be defensible in the light of our overall approach to graphs of authors described in our Onward paper as well as not being directly implicated in our most serious problems. The JIRA most clearly giving the flavour of our most serious problems is FLUID-5668 "corruption when material written for members requires merging". An issue very similar to this came to light in work for the Onward paper itself.

Problems exposed by mergePolicies

The problem is most clearly exposed when trying to work with "mergePolicies" which are now by far the most immature and undeveloped part of the options merging pipeline, essentially untouched since the days of the ancient framework of 2009. They don't even feature support for global name resolution, implying that any grade definition using them can't even be represented in a JSON store. They haven't been updated primarily since it hasn't been very clear what to do with them, since almost any use of them seems to lead to difficulties. The framework makes use of them in many cases, for example where listeners, modelListeners, modelRelay, model, etc. require something other than the simple-minded "painter's algorithm" combination of options by merg