fluid-work IRC Logs-2013-10-16

Owned by Fluid IRC Log Bot
Last updated: Oct 16, 2013
10 min read

[00:02:13 CDT(-0500)] <colinclark> is it possible that npm refrains from the kind of "cleverness" described above in cases where the dependency is expressed as a git url?

[00:03:01 CDT(-0500)] <colinclark> AHA

[00:03:20 CDT(-0500)] <colinclark> no

[00:03:23 CDT(-0500)] <colinclark> (smile)

[00:04:39 CDT(-0500)] <Bosmon> colinclark - what have you discovered?

[00:04:57 CDT(-0500)] <colinclark> well, I seem to have moved on to a more interesting error

[00:05:15 CDT(-0500)] <Bosmon> I think it is unavoidable that we would have run into this issue with kettle/GPII had it not been the case that npm actually somehow believes in the SHA at the end of a git url

[00:05:28 CDT(-0500)] <colinclark> though fluid.require("flock") and fluid.registerNamespace("flock") are still claiming to be different objects

[00:05:49 CDT(-0500)] <Bosmon> Since all the clients of universal make use of universal's infusion

[00:05:58 CDT(-0500)] <Bosmon> Whereas all the configuration is loaded via kettle's infusion

[00:06:32 CDT(-0500)] <colinclark> interesting

[00:06:50 CDT(-0500)] <Bosmon> As far as I can see, it would be a recipe for "instant mayhem"

[00:07:25 CDT(-0500)] <colinclark> ok, it's making sound!

[00:07:33 CDT(-0500)] <colinclark> slightly jankily

[00:07:35 CDT(-0500)] <Bosmon> wow!

[00:07:37 CDT(-0500)] <Bosmon> What did you do?

[00:07:52 CDT(-0500)] <colinclark> I hosed the inner copy of Infusion

[00:07:59 CDT(-0500)] <colinclark> which doesn't really help my users

[00:08:07 CDT(-0500)] <Bosmon> cool

[00:08:13 CDT(-0500)] <colinclark> I guess we have to figure out how npm allowed it to be there

[00:08:20 CDT(-0500)] <Bosmon> So, yzen - I believe we can put a "guard" into the implementation of fluid.require

[00:08:24 CDT(-0500)] <colinclark> I'm also still worried about this...

[00:08:25 CDT(-0500)] <colinclark> console.log("app: ", flock === fluid.getGlobalValue("flock"));

[00:08:29 CDT(-0500)] <colinclark> still returns false

[00:08:47 CDT(-0500)] <colinclark> where flock is defined by:

[00:08:48 CDT(-0500)] <colinclark> flock = fluid.require("flocking"),

[00:08:59 CDT(-0500)] <colinclark> can anyone explain that?

[00:09:11 CDT(-0500)] <yzen> have you tried passing local require as the second arg?

[00:09:15 CDT(-0500)] <Bosmon> No, I can't explain it

[00:09:32 CDT(-0500)] <colinclark> yzen: No, I haven't

[00:09:40 CDT(-0500)] <yzen> flock = fluid.require("flocking", require)

[00:09:46 CDT(-0500)] <colinclark> I'm not sure I understand when I should do that

[00:09:48 CDT(-0500)] <colinclark> can you elaborate?

[00:09:57 CDT(-0500)] <Bosmon> It shouldn't make any difference in this case

[00:10:11 CDT(-0500)] <Bosmon> Either flocking is resolvable or it isn't.... I imagine it's impossible for you to have two copies of THAT

[00:10:27 CDT(-0500)] <colinclark> passing the second argument doesn't help, no

[00:10:32 CDT(-0500)] <colinclark> hmm

[00:10:38 CDT(-0500)] <colinclark> I would hope there's only one, yes

[00:11:07 CDT(-0500)] <Bosmon> You pass the 2nd argument when the module is only visible from your own directory level and not from infusion's

[00:11:21 CDT(-0500)] <Bosmon> Since you now have infusion at the outer level, it should make no difference

[00:12:12 CDT(-0500)] <Bosmon> So, colinclark, "how different" are the two versions of your "flock" object? : P

[00:12:29 CDT(-0500)] <colinclark> I will check momentarily

[00:12:39 CDT(-0500)] <Bosmon> There may be some subtleties about what module.exports actually does with an object reference it is sent

[00:12:51 CDT(-0500)] <colinclark> you're wondering if they both resemble a full-featured instance of Flocking, I assume?

[00:12:55 CDT(-0500)] <Bosmon> Yes, for example

[00:13:08 CDT(-0500)] <colinclark> terrifying

[00:13:11 CDT(-0500)] <Bosmon> And also, whether assigning a new property to one ends up surfacing the value in the other one

[00:13:22 CDT(-0500)] <colinclark> ok

[00:13:28 CDT(-0500)] <colinclark> the pi just rebooted

[00:13:29 CDT(-0500)] <colinclark> let's see

[00:13:48 CDT(-0500)] <Bosmon> Rather than sending around raw object references, node may arrange for one of them to be a proxy for the other in some kind of undercover way.....

[00:14:03 CDT(-0500)] <Bosmon> Which might explain the inconsistent results you saw when checking for raw reference equality

[00:14:43 CDT(-0500)] <Bosmon> Often in Firebug I use a system of "ticks", and constantly update a property in what should be one alias of an object with increasing "version numbers" until I either start or stop seeing the changes showing up in the other .......

[00:33:47 CDT(-0500)] <colinclark> So I'm embarrassed to say that the reason my two flocks weren't equal was because I had left my module.exports = flock line commented out (tongue)

[00:34:02 CDT(-0500)] <colinclark> they are now happily equal

[00:34:03 CDT(-0500)] <colinclark> which I guess leaves the main two mysteries as:

[00:34:36 CDT(-0500)] <colinclark> 1. Under what circumstances does npm's aforementioned cleverness not kick in?

[00:35:08 CDT(-0500)] <colinclark> 2. How should users of a library like Flocking (or Universal, for that matter) handle this multi-tied dependency on Infusion

[00:35:27 CDT(-0500)] <colinclark> aside from just manually deleting dependencies when npm has failed to do its job

[00:36:00 CDT(-0500)] <Bosmon> I suggested our first line of defence would be to put a "guard" into our implementation of fluid.require

[00:36:10 CDT(-0500)] <colinclark> yes

[00:36:14 CDT(-0500)] <colinclark> that makes a lot of sense

[00:36:21 CDT(-0500)] <Bosmon> Although this would not protect anyone who had not supplied the 2nd argument

[00:36:34 CDT(-0500)] <Bosmon> It would at least protect current users of kettle

[00:37:41 CDT(-0500)] <colinclark> I need to provision a fresh Pi now

[00:37:52 CDT(-0500)] <colinclark> so we'll see if npm does the same thing in this case

[00:38:55 CDT(-0500)] <Bosmon> Perhaps it can ONLY detect equal versions if the git SHA is supplied explicitly

[00:39:06 CDT(-0500)] <Bosmon> Given, as we saw, it strips out all "gitliness" from the files it checks out

[00:39:38 CDT(-0500)] <Bosmon> After all, if the files in the FS are all it has to go on, there can't be any "hidden magic"

[00:39:56 CDT(-0500)] <Bosmon> So in fact anyone who points at "master" of a project could not be proved ever to have the same checkout as anyone else

[00:42:42 CDT(-0500)] <Bosmon> yzen - what became of the "kettleModuleLoader.js" concept?

[00:44:01 CDT(-0500)] <yzen> Bosmon: in what sense, it's in master for a while now

[00:44:15 CDT(-0500)] <Bosmon> yzen - yes, but why can't I see any such files in our projects?

[00:44:39 CDT(-0500)] <yzen> https://github.com/GPII/universal/blob/master/gpii/node_modules/deviceReporter/configs/kettleModuleLoader.js

[00:44:46 CDT(-0500)] <yzen> for example ?

[00:45:18 CDT(-0500)] <Bosmon> Ah of course, they go with the configs

[00:45:19 CDT(-0500)] <Bosmon> Thanks

[00:45:38 CDT(-0500)] <Bosmon> Ok, so we can't really rely on this for general 3rd-party protection ......

[00:46:27 CDT(-0500)] <Bosmon> We really don't have any alternative but to construct our own module loader

[00:46:33 CDT(-0500)] <Bosmon> We were planning this at Linz anyway

[00:48:07 CDT(-0500)] <Bosmon> It's very unfortunate, since it's a big undertaking

[00:48:32 CDT(-0500)] <Bosmon> We can at least protect all kettle users since we have control over its codebase

[00:48:50 CDT(-0500)] <Bosmon> And, for example, can prove that it always uses the 2-arg fluid.require, etc.

[01:00:45 CDT(-0500)] <colinclark> sounds like fun, yzen

[01:00:52 CDT(-0500)] <colinclark> your own module loader

[01:25:36 CDT(-0500)] <colinclark> for whatever reason, a fresh install of my app did not include a nested copy of Infusion

[01:25:48 CDT(-0500)] <colinclark> perhaps I just did something dumb during the installation process

[01:30:11 CDT(-0500)] <Bosmon> colinclark - don't both your app and flocking have an expressed npm dependency on infusion?

[01:30:24 CDT(-0500)] <colinclark> yes, they both do

[01:30:39 CDT(-0500)] <colinclark> this time, it seemed to be clever enough to not install the nested copy

[01:30:41 CDT(-0500)] <Bosmon> So isn't it then proper for both of them to install infusion as a submodule when you run "npm install" on them?

[01:31:09 CDT(-0500)] <colinclark> I ran an "npm install" on the app

[01:31:38 CDT(-0500)] <colinclark> So I'm assuming this did indeed occur:

[01:31:39 CDT(-0500)] <colinclark> "npm will be clever and install the module only once at the main project's level, allowing the require statements in one/index.js and two/index.js to search upwards in the tree until they find the same file and return"

[01:31:46 CDT(-0500)] <Bosmon> ah

[01:31:51 CDT(-0500)] <Bosmon> Now we can understand this mystical text!

[01:32:02 CDT(-0500)] <colinclark> yes

[01:32:04 CDT(-0500)] <Bosmon> Will be clever and *INSTALL* the module only once, it says!

[01:32:07 CDT(-0500)] <Bosmon> INSTALL!

[01:32:24 CDT(-0500)] <colinclark> Were any changes pushed to Infusion in the past 24 hours or so?

[01:32:29 CDT(-0500)] <Bosmon> So in fact having two physical copies of the same module ALWAYS spells certain doom

[01:32:56 CDT(-0500)] <Bosmon> I'm not aware of any

[01:33:03 CDT(-0500)] <Bosmon> github shows no changes for 5 days

[01:33:27 CDT(-0500)] <Bosmon> The initial version of the module loader need not be very sophisticated.... it could just run a directory scan and then issue full relative file paths to the standard "require"

[01:33:36 CDT(-0500)] <Bosmon> But I think we should patch fluid.require to do this

[01:33:46 CDT(-0500)] <Bosmon> As well as report on any duplicates of infusion found anywhere in the tree

[01:35:00 CDT(-0500)] <Bosmon> But this will involve fluid.require understanding parts of the npm packaging structure

[01:36:39 CDT(-0500)] <colinclark> yes

[01:38:16 CDT(-0500)] <Bosmon> So this reading of the magic text means that we no longer have a good explanation of why yzen is able to get away with the two copies of infusion we have in gpii/kettle

[01:38:29 CDT(-0500)] <Bosmon> Since it implies that the level of protection occurs during installation, rather than during resolution

[01:39:31 CDT(-0500)] <Bosmon> slacker : P

[01:39:43 CDT(-0500)] <Bosmon> Why are you not working at 2.39am ....

[01:41:39 CDT(-0500)] <colinclark> lol

[01:41:40 CDT(-0500)] <colinclark> yes

[01:42:06 CDT(-0500)] <colinclark> I need to pass out myself

[01:42:15 CDT(-0500)] <Bosmon> Although one slacker we SHOULD expect to see at this time is that damnable **KASPPARNETT**

[01:42:19 CDT(-0500)] <colinclark> I think I'm getting a Node Buffer object chewed up by the expansion system

[01:42:39 CDT(-0500)] <Bosmon> Oh dear.... have you tried the fluid.isPlainObject test on it to see what it says?

[01:42:52 CDT(-0500)] <colinclark> just testing that now

[01:42:58 CDT(-0500)] <Bosmon> If it is wrong, at least you now have an easier place to monkey-patch (smile)

[01:43:40 CDT(-0500)] <colinclark> isPlainObject reports true for it, yes

[01:43:47 CDT(-0500)] <Bosmon> Ouch..... SIR

[01:44:03 CDT(-0500)] <colinclark> easy fix as you say

[01:44:06 CDT(-0500)] <colinclark> for the morning!

[01:44:14 CDT(-0500)] <Bosmon> After we release 1.5 we can ditch support for IE8 I believe

[01:44:22 CDT(-0500)] <Bosmon> Which means that we can go ahead with a more stringent test

[01:44:26 CDT(-0500)] <colinclark> that would be awfully nice

[01:44:30 CDT(-0500)] <colinclark> though we still have the XP issue

[01:44:34 CDT(-0500)] <Bosmon> Yes

[01:44:42 CDT(-0500)] <Bosmon> It seems Microsoft themselves will ditch that lot in April

[01:44:51 CDT(-0500)] <colinclark> then perhaps it's time

[01:44:53 CDT(-0500)] <Bosmon> Even though I am still quite fond of XP myself.....

[01:45:02 CDT(-0500)] <colinclark> aren't we all, yes

[01:45:25 CDT(-0500)] <Bosmon> Millions of our users may be too

[01:45:37 CDT(-0500)] <Bosmon> But I guess none of them would expect any a11y features on using IE8 anyway

[01:46:56 CDT(-0500)] <colinclark> ARIA support wasn't great

[01:47:11 CDT(-0500)] <colinclark> though there are still a lot of users

[01:47:26 CDT(-0500)] <colinclark> since upgrades tend to be slower when AT is involved in the mix

[01:48:13 CDT(-0500)] <Bosmon> If push comes to shove, we could always have the framework monkey-patch isPlainObject by itself, using a browser detect

[01:48:31 CDT(-0500)] <colinclark> ok, my two udp-chattering robot buddies are shut down

[01:48:37 CDT(-0500)] <Bosmon> The awful issue is that IE8 and below will throw an exception when accessing the prototype or constructor properties of a "host object"

[01:48:44 CDT(-0500)] <colinclark> insane

[01:49:02 CDT(-0500)] <Bosmon> And, despite a JSfiddle to the contrary, having a try/catch block in this very innermost part of the framework would be completely unacceptable performance-wise

[01:49:21 CDT(-0500)] <Bosmon> Although I guess we could try getting the JSfiddle working properly, I didn't spend too much time with it

[01:50:06 CDT(-0500)] <colinclark> remind me what the nature of the performance impact with try/catch blocks is?

[01:50:17 CDT(-0500)] <Bosmon> It causes the VM to "deoptimise" the function containing them

[01:50:21 CDT(-0500)] <colinclark> ouch

[01:50:34 CDT(-0500)] <colinclark> both *Monkey and V8?

[01:51:07 CDT(-0500)] <Bosmon> Unknown.... we would have to "fiddle" that too : P Although testing performance on these micro-operations is always extremely fraught

[01:51:26 CDT(-0500)] <Bosmon> We really need a performance benchmark which assesses the performance of the framework as a whole on a sizeable task

[01:51:40 CDT(-0500)] <Bosmon> But that would reveal a number of other more serious problems

[01:51:54 CDT(-0500)] <Bosmon> I'm not sure that actually a "constructor" or "prototype"-based detection can be performant either

[01:52:09 CDT(-0500)] <Bosmon> Since if we require cross-frame resistance, it means calling "toString" on them

[01:52:14 CDT(-0500)] <Bosmon> Which can result in truly huge representations

[01:52:44 CDT(-0500)] <Bosmon> The great thing about the Object.prototype.toString test was that it only ever produces short and standard strings like [object Object] which can certainly be generated performantly

[01:55:52 CDT(-0500)] <Bosmon> https://github.com/jashkenas/underscore/pull/102

[01:55:57 CDT(-0500)] <Bosmon> Endless discussion on _ here

[09:46:18 CDT(-0500)] <michelled> Justin_o, cindyli, yzen: do you think it's possible to arrange for a pull request of the most recent prefs framework into the gpii repo?

[09:46:37 CDT(-0500)] <michelled> sorry, I mean the exploration tool

[09:46:47 CDT(-0500)] <michelled> the nightly for gpii is still showing the discovery tool

[09:46:59 CDT(-0500)] <michelled> and I'd rather not send links around that will be obsolete very soon

[10:11:44 CDT(-0500)] <Justin_o> michelled: sure... we could do that

[10:12:25 CDT(-0500)] <michelled> thanks Justin_o

[10:13:42 CDT(-0500)] <Justin_o> michelled: https://github.com/GPII/prefsEditors/pull/8

[10:13:57 CDT(-0500)] <michelled> cool - yzen can you review it?

[10:14:18 CDT(-0500)] <yzen> michelled: sure ill try to get to it today

[10:14:46 CDT(-0500)] <michelled> great!

[13:25:14 CDT(-0500)] <Justin_o> fluid-everyone: would anyone like to join the community meeting remotely. Today we will be doing another doc sprint.

[13:25:30 CDT(-0500)] <Bosmon> Sure, Justin_o - I will need a few minutes first

[13:25:43 CDT(-0500)] <anastasiac> Justin_o, yura just left, but he'll join once he's available again

[13:25:54 CDT(-0500)] <Justin_o> Bosmon, anastasiac okay.. thanks

[13:26:18 CDT(-0500)] <Justin_o> Bosmon: by the way.. i ran into an issue with the message resolver stuff.. if you have a couple minutes to chat with me about that, it would be helpful

[13:26:34 CDT(-0500)] <Justin_o> Bosmon: i left a comment on the jira about it http://issues.fluidproject.org/browse/FLUID-5180

[13:42:19 CDT(-0500)] <anastasiac> fluid-everyone: Planning page for today's Docs Sprint: http://wiki.fluidproject.org/display/fluid/Docs+Sprint+Planning

[13:43:27 CDT(-0500)] <anastasiac> jessm, vjoanna, clown: we have some apple cider donuts and girl guide cookies here in the collar room; feel free to come help yourself (we have too many!)

[13:47:18 CDT(-0500)] <Justin_o> anastasiac: http://wiki.gpii.net/index.php/Architecture_-_Available_transformation_functions

[14:04:23 CDT(-0500)] <Bosmon> Justin_o - is the meeting running now?

[14:08:46 CDT(-0500)] <Bosmon> Hi anastasiac

[14:09:00 CDT(-0500)] <anastasiac> hi, Bosmon, are you available for the docs sprint now?

[14:09:06 CDT(-0500)] <Bosmon> yes

[14:28:48 CDT(-0500)] <anastasiac> http://wiki.fluidproject.org/display/fluid/Docs+Sprint+Planning

[15:05:11 CDT(-0500)] <Justin_o> Bosmon: did you have a chance to look at the issue i'm having with FLUID-5180

[15:05:14 CDT(-0500)] <Justin_o> ?

[15:07:14 CDT(-0500)] <Bosmon> Justin_o - yes, I have commented on the JIRA

[15:08:28 CDT(-0500)] <Justin_o> great thanks.