fluid-tech IRC Logs-2012-11-14

[09:37:49 CST(-0600)] <kasper> Mr. Yura2 Zenevich - you should never have gone on IRC today - I'm planning to harass you from now on till you're off work with stupid questions about model transformations

[09:43:27 CST(-0600)] <kasper> ... ignoring me, eh? Clever move

[09:51:34 CST(-0600)] <yura2> kasper: basically

[09:51:40 CST(-0600)] <yura2> sorry was distracted

[10:05:31 CST(-0600)] <sgithens> yura2 kasper : Are we at noon or 1pm eastern time?

[10:05:54 CST(-0600)] <kasper> sgithens: noon

[10:06:02 CST(-0600)] <sgithens> cool

[10:06:43 CST(-0600)] <kasper> yura2: sorry back - just got sucked into a meeting so disturbed you for no good reason

[10:15:46 CST(-0600)] <kasper> ok, before I start the spamming, a little background:

[10:17:09 CST(-0600)] <kasper> I'm working on the XML settings handler, which currently in its XML->JSON translation, translates certain XML structures into arrays

[10:19:04 CST(-0600)]

[10:19:26 CST(-0600)]

[10:21:02 CST(-0600)] <kasper> So I got this working more or less, and have a first draft of inverted rules and the inverse function

[10:21:19 CST(-0600)] <kasper> but have bumped into some issues I'm not quite sure of

[10:21:25 CST(-0600)] <kasper> yura2: ^

[10:21:37 CST(-0600)] <kasper> code is here: https://github.com/kaspermarkus/universal/blob/GPII-32-2/gpii/node_modules/settingsHandlers/src/js/XMLSettingsHandler.js#L53-118

[10:22:04 CST(-0600)] <kasper> tests are: https://github.com/kaspermarkus/universal/blob/GPII-32-2/gpii/node_modules/settingsHandlers/test/XMLSettingsHandlerTests.js#L27-141

[10:22:26 CST(-0600)] <kasper> all the first tests (l. 27-79) are fine and dandy

[10:23:00 CST(-0600)] <kasper> but I'm experiencing issues with the second batch of tests (l. 81-140)

[10:24:28 CST(-0600)] <kasper> so the questions: part of my original (complexTest.raw) json structure disappears after/during transformation ... namely the whole 'b' key/object is not in the result

[10:24:58 CST(-0600)] <kasper> it will appear if I add a rule like: "b": "b".. but would prefer this just got copied by default

[10:25:16 CST(-0600)] <kasper> ... sorry about writing a full novel yura2

[11:05:45 CST(-0600)] <Bosmon> KASPAR

[11:05:46 CST(-0600)] <Bosmon> kasper

[11:05:58 CST(-0600)] <kasper> The Bass Man himself

[11:06:01 CST(-0600)] <Bosmon> The transform is "working as designed"

[11:06:12 CST(-0600)] <Bosmon> Unless you explicitly write a rule to transfer something, it will be lost

[11:06:13 CST(-0600)] <kasper> .. as you an see I've found a new victim for my questions

[11:06:27 CST(-0600)] <Bosmon> Where is that slacker yura2 : P

[11:06:28 CST(-0600)] <kasper> Bosmon: stupid transformation system

[11:06:35 CST(-0600)] <kasper> he's in the architecture meeting

[11:06:37 CST(-0600)] <kasper>

[11:06:39 CST(-0600)] <Bosmon> In fact, where is anyone....

[11:06:43 CST(-0600)] <Bosmon> Where is it!

[11:06:46 CST(-0600)] <kasper> which, funny enough is missing you

[11:06:49 CST(-0600)] <kasper> gotomeeting

[11:06:50 CST(-0600)] <kasper> hang on

[11:06:59 CST(-0600)] <kasper> https://www3.gotomeeting.com/join/705233502

[11:07:12 CST(-0600)] <Bosmon> GOU TOUW MEETING.....

[12:07:50 CST(-0600)] <kasper> Bosmon, yura2: for my rules inversion function, the result involves a different expand type.. Does this need to be graded in some way, or..? (it's working without it atm, but I might be missing some bigger picture thing)

[12:08:20 CST(-0600)] <Bosmon> kasper - what do you mean by this:

[12:08:26 CST(-0600)] <Bosmon> " the result involves a different expand type"

[12:09:17 CST(-0600)] <kasper> in the returned rule object, I have an expander with a type: "gpii.settingsHandlers.XMLHandler.hashToArray" (as opposed to arrayToHash)

[12:09:25 CST(-0600)] <kasper> https://github.com/kaspermarkus/universal/blob/GPII-32-2/gpii/node_modules/settingsHandlers/src/js/XMLSettingsHandler.js#L53-59

[12:09:39 CST(-0600)] <kasper> but this is nowhere graded as a transformation function

[12:13:33 CST(-0600)] <kasper> Bosmon: ^ .. not sure if that makes better sense

[12:15:29 CST(-0600)] <Bosmon> If you don't grade it, it gets assumed to be a standardTransformFunction

[12:15:32 CST(-0600)] <Bosmon> But it's best to grade it

[12:16:27 CST(-0600)] <kasper> Bosmon: ok, will do! Thanks

[13:24:32 CST(-0600)] <kasper> Bosmon: I'm in trouble again

[13:24:39 CST(-0600)] <kasper> this time with nested array structures

[13:24:41 CST(-0600)] <kasper> http://pastie.org/5378598

[13:24:58 CST(-0600)] <Bosmon> Darn it KASPAR : P

[13:25:01 CST(-0600)] <Bosmon> Can't you stay out of trouble? : P

[13:25:58 CST(-0600)] <Bosmon> Where does the path "outerkey2" come from?

[13:26:02 CST(-0600)] <Bosmon> I don't see it in your material

[13:26:23 CST(-0600)] <kasper> well.. it's there in the transformed payload

[13:26:43 CST(-0600)] <Bosmon> Your 2nd block of rules doesn't make any sense

[13:26:45 CST(-0600)] <Bosmon> Can you explain it? : P

[13:26:53 CST(-0600)] <kasper> haha sure

[13:27:12 CST(-0600)] <Bosmon> Stuff actually needs to be there in the input, otherwise you can't transform it!

[13:27:24 CST(-0600)] <kasper> that's my problem

[13:27:25 CST(-0600)] <Bosmon> It's unclear what you're trying to do here

[13:27:55 CST(-0600)] <Bosmon> Well, you certainly have a problem if you're trying to transform something which isn't in the input..... but can you explain WHY!

[13:28:07 CST(-0600)] <kasper> ok, so im trying to convert both the arrays defined in the "raw" object (outer and outervar arrays)

[13:28:33 CST(-0600)] <kasper> the problem is, to be able to refer to the inner array, I first need to transform the outer array

[13:28:49 CST(-0600)] <Bosmon> That's actually backwards

[13:29:04 CST(-0600)] <kasper> depends on how you see it

[13:29:06 CST(-0600)] <Bosmon> In order to be able to get at the inner array in its transformed form, you need to transform it FIRST!

[13:29:20 CST(-0600)] <Bosmon> I see it in the order in which it can be actually implemented : P

[13:29:26 CST(-0600)] <Bosmon> You are free to see it in some other order : P

[13:30:08 CST(-0600)] <kasper> the problem is I cannot refer to the inner array, before the outer array is transformed though?

[13:30:24 CST(-0600)] <Bosmon> You have to START OFF by referring to everything in place....

[13:30:39 CST(-0600)] <Bosmon> Clearly you can already FIND the inner array at its existing path, even if it isn't in the form you want

[13:31:22 CST(-0600)] <Bosmon> It exists at the path raw.outer.1.outervar

[13:31:43 CST(-0600)] <Bosmon> Although I'm still not quite sure the final effect you are aiming at

[13:31:49 CST(-0600)] <Bosmon> What is the final transformed output that you want?

[13:32:22 CST(-0600)] <kasper> http://pastie.org/5378645

[13:32:56 CST(-0600)] <kasper> the problem is that I dont think we can rely on the order of the XML arrays

[13:33:16 CST(-0600)] <kasper> hence, it Could be at raw.outer.0.outervar or raw.outer.1.outervar

[13:33:25 CST(-0600)] <kasper> that's the whole point of these transformations

[13:33:28 CST(-0600)] <Bosmon> If you can't rely on them, then surely the whole format is bunk in any case : P

[13:33:49 CST(-0600)] <kasper> exactly - that's why were transforming them to something which you CAN rely on

[13:34:15 CST(-0600)] <Bosmon> So the XML format really has a schema where outervar may hold EITHER a string, or some huge collection of junk?

[13:34:33 CST(-0600)] <Bosmon> What does it really look like, in practice...

[13:35:35 CST(-0600)] <kasper> I dont have any example settings files doing this atm

[13:35:35 CST(-0600)] <Bosmon> If this is really the case, then it seems you will need to do TWO things:

[13:36:07 CST(-0600)] <Bosmon> Firstly, write a slightly more elaborate expander, based on your existing one, that leaves the value unchanged if it unexpectedly consists of a string

[13:36:07 CST(-0600)] <kasper> but it seems reasonable that there might be nested arrays in some XML settings files, doesn't it?

[13:36:25 CST(-0600)] <Bosmon> Secondly, write a transformation block which composes this modified expander together with the original one

[13:36:44 CST(-0600)] <Bosmon> Nested arrays are reasonable, extremely inhomogeneous ones like this are less reasonable

[13:36:50 CST(-0600)] <Bosmon> Do they even have a schema for their XML file at all!

[13:36:57 CST(-0600)] <kasper> nope

[13:37:04 CST(-0600)] <Bosmon> Stupid tossers....

[13:37:08 CST(-0600)] <kasper> haha

[13:37:09 CST(-0600)] <Bosmon> Do you have a sample one?

[13:40:14 CST(-0600)] <kasper> you do have a point though, that inconsistent structures like these are unlikely

[13:40:34 CST(-0600)] <Bosmon> Ok, so if you axe the "outerkey1" part, your task becomes easier

[13:40:45 CST(-0600)] <Bosmon> That is, if you replace it with another thing with the same structure

[13:40:52 CST(-0600)] <kasper> and considering how (parts of) this XML settingshandler might be rewritten anyway at a later point, it's probably not worth spending time on

[13:41:06 CST(-0600)] <Bosmon> You "just" need to write two nested instances of your existing expander

[13:43:36 CST(-0600)] <kasper> why two nested instances?

[13:44:03 CST(-0600)] <Bosmon> ?

[13:44:14 CST(-0600)] <Bosmon> You need one expander to invert the inner structure, and one to invert the outer structure

[13:44:20 CST(-0600)] <Bosmon> You've written them both already....

[13:44:24 CST(-0600)] <Bosmon> YOu just need to now nest them

[13:44:37 CST(-0600)] <Bosmon> Tkae a look at the ModelTransformationsTests.js file, around line 876

[13:44:48 CST(-0600)] <Bosmon> "fluid.model.transform() with idempotent rules"

[13:44:55 CST(-0600)] <Bosmon> It shows how this kind of thing is meant to work

[13:45:36 CST(-0600)] <kasper> awesome!!! Thanks a lot again again (again) Bassman!

[13:46:27 CST(-0600)] <kasper> Bosmon: btw.. I noticed a lot of 'cat's, 'CATT's and 'CATTOO's in your examples ;D

[13:46:38 CST(-0600)] <Bosmon> Is doing this transformation really a blocker for the release though?

[13:46:52 CST(-0600)] <kasper> the base case is

[13:47:29 CST(-0600)] <kasper> not sure about the nested transformations .. though I guess RWG does actually need it to edit their full settings file

[13:47:49 CST(-0600)] <Bosmon> Well, they don't really

[13:47:54 CST(-0600)] <Bosmon> All the transformations do is add convenience

[13:48:02 CST(-0600)] <Bosmon> They will get SOME invertible JSON form of their file

[13:48:09 CST(-0600)] <Bosmon> It doesn't necessarily matter if it isn't the most helpful one

[13:49:01 CST(-0600)] <kasper> it depends on whether they always have ALL the settings in their settings file, or only those differing from the defaults

[13:49:26 CST(-0600)] <kasper> if the latter, then they would need nested transformations, as they cannot rely on the indexes of the JSON arrays

[13:49:45 CST(-0600)] <Bosmon> Well, they can still modify the JSON arrays, even without relying on the indexes

[13:49:57 CST(-0600)] <kasper> how?

[13:50:03 CST(-0600)] <Bosmon> They just need to run some searches over them

[13:50:15 CST(-0600)] <Bosmon> We have some stuff to help with this, in the core framework

[13:50:25 CST(-0600)] <Bosmon> "resolvers"

[13:50:38 CST(-0600)] <kasper> oh, I see

[13:50:49 CST(-0600)] <kasper> and they can be used declaratively?

[13:50:59 CST(-0600)] <Bosmon> yes

[13:51:00 CST(-0600)] <kasper> (ie. via the solutions-registry entry

[13:51:03 CST(-0600)] <kasper> ok

[13:51:04 CST(-0600)] <Bosmon> No

[13:51:21 CST(-0600)] <kasper> haha, you cant do that - first do a yes, then a no

[13:51:29 CST(-0600)] <Bosmon> Well, they can be used declaratively

[13:51:34 CST(-0600)] <Bosmon> But not via the solutions registery entry!

[13:51:42 CST(-0600)] <Bosmon> We have no provision for that

[13:51:49 CST(-0600)] <kasper> stupid solutions registry

[13:52:00 CST(-0600)] <Bosmon> But it's unclear what exactly we want people to have the facility to do?

[13:52:21 CST(-0600)] <Bosmon> I mean, we don't have the facility to run inverse transformations via the solutions registry entry either

[13:52:48 CST(-0600)] <Bosmon> I guess you're imagining the settingsHandler will do this by itself

[13:52:55 CST(-0600)] <kasper> yes

[13:52:57 CST(-0600)] <Bosmon> ok

[13:53:22 CST(-0600)] <Bosmon> I guess the nested transformations remains the most reasonable way to do this, and you should push on with it

[13:54:04 CST(-0600)] <Bosmon> Using "resolvers" in theory makes use of more existing parts of the framework, but would in the end produce a solution which is quite cumbersome to use

[13:54:37 CST(-0600)] <Bosmon> We really do want to lens, as far as possible, the ENTIRE file if we can, and it looks like we can

[13:55:39 CST(-0600)] <kasper> hehe, I'm still a bit confused as to what constitutes a lens - but I agree

[13:55:54 CST(-0600)] <Bosmon> "lens" is just another name for a model transformation which has an inverse

[13:56:00 CST(-0600)] <kasper> ah, cool

[13:56:21 CST(-0600)] <Bosmon> That is, something which creates an editable view of a document which can then be written back

[13:56:23 CST(-0600)] <kasper> so I should grade my function as a lens as well?

[13:56:36 CST(-0600)] <kasper> that sounds exactly like what we want

[13:56:44 CST(-0600)] <Bosmon> I don't think we actually have a working "lens" grade in the current core framework

[13:57:00 CST(-0600)] <kasper> I thought I saw something graded as lens

[13:57:16 CST(-0600)] <Bosmon> You're right, we do

[13:57:33 CST(-0600)] <kasper> awesome

[13:57:53 CST(-0600)] <Bosmon> Looks like I failed to write it in for the base transformer "fluid.model.transform.value"

[13:58:09 CST(-0600)] <Bosmon> Probably a bug, but nothing uses it right now

[13:58:28 CST(-0600)] <Bosmon> All the framework does is look for the member named "invertConfiguration"

[13:58:38 CST(-0600)] <Bosmon> But you could say that for now that de facto makes the thing into a lens

[13:59:08 CST(-0600)] <kasper> ok cool!!

[13:59:44 CST(-0600)] <kasper> well, I'll see if I can get the nesting to work then.. Once that's done, I'm not too far from a pull request

[14:00:17 CST(-0600)] <kasper> (or.. actually, I need to use all this stuff in the "XMLHandler.set" function ... but.. )

[14:00:45 CST(-0600)] <kasper> thanks again, Bassman

[14:43:05 CST(-0600)] <yura1> https://github.com/collectionspace/ui/blob/master/src/test/js/RelatedRecordsTabTests.js

[14:43:37 CST(-0600)] <michelled> the ether pad we are working off: http://openetherpad.org/GTA1XnSt1g

[14:48:21 CST(-0600)] <yura1> https://github.com/collectionspace/ui/blob/master/src/test/js/AdminUsersTest.js#L448-505

[14:51:44 CST(-0600)] <Bosmon> https://github.com/collectionspace/ui/blob/master/src/test/js/AdminUsersTest.js#L586-594

[14:51:59 CST(-0600)] <Bosmon> So in this case we have multiple events on different components which need to be sequenced