fluid-work IRC Logs-2010-02-03
[05:14:27 EST(-0500)] * sveto (~sveto@62.44.108.2) has joined #fluid-work <yura> is it supposed to be [ ] ?
[07:04:31 EST(-0500)] * sveto (~sveto@62.44.108.2) has joined #fluid-work
[08:04:50 EST(-0500)] * michelled (~michelled@CPE001310472ade-CM0011aefd3ca8.cpe.net.cable.rogers.com) has joined #fluid-work
[08:24:15 EST(-0500)] * justin_o (~jmo@142.150.154.101) has joined #fluid-work
[09:34:51 EST(-0500)] * yura (~yura@142.150.154.101) has joined #fluid-work
[09:46:27 EST(-0500)] * colinclark (~colin@2002:8e01:9465:b:223:12ff:fe55:316d) has joined #fluid-work
[10:11:51 EST(-0500)] * michelled (~michelled@142.150.154.101) has joined #fluid-work
[10:19:42 EST(-0500)] * justin_o (~jmo@142.150.154.101) has joined #fluid-work
[10:21:33 EST(-0500)] * EricDalquist (~dalquist@2607:f388:e:0:221:9bff:fe37:e768) has joined #fluid-work
[10:25:48 EST(-0500)] * clown (~clown@142.150.154.101) has joined #fluid-work
[10:30:12 EST(-0500)] * anastasiac (~team@142.150.154.193) has joined #fluid-work
[10:45:16 EST(-0500)] * colinclark (~colin@142.150.154.101) has joined #fluid-work
[10:51:55 EST(-0500)] * EverettZ (~chatzilla@bas4-toronto06-1279277146.dsl.bell.ca) has joined #fluid-work
[10:57:22 EST(-0500)] * sveto (~sveto@62.44.108.2) has left #fluid-work
[10:59:57 EST(-0500)] * sveto (~sveto@62.44.108.2) has joined #fluid-work
[11:15:51 EST(-0500)] * boyan (~boyan@62.44.108.2) has left #fluid-work
[11:35:16 EST(-0500)] * Bosmon2 (~amb26free@174-29-5-229.hlrn.qwest.net) has joined #fluid-work
[11:51:59 EST(-0500)] * elicochran (~elicochra@dhcp-169-229-212-4.LIPS.Berkeley.EDU) has joined #fluid-work
[12:22:16 EST(-0500)] * elicochran (~elicochra@dhcp-169-229-212-4.LIPS.Berkeley.EDU) has joined #fluid-work
[12:22:16 EST(-0500)] * Bosmon2 (~amb26free@174-29-5-229.hlrn.qwest.net) has joined #fluid-work
[12:22:16 EST(-0500)] * sveto (~sveto@62.44.108.2) has joined #fluid-work
[12:22:16 EST(-0500)] * colinclark (~colin@142.150.154.101) has joined #fluid-work
[12:22:16 EST(-0500)] * anastasiac (~team@142.150.154.193) has joined #fluid-work
[12:22:16 EST(-0500)] * clown (~clown@142.150.154.101) has joined #fluid-work
[12:22:16 EST(-0500)] * EricDalquist (~dalquist@2607:f388:e:0:221:9bff:fe37:e768) has joined #fluid-work
[12:22:16 EST(-0500)] * justin_o (~jmo@142.150.154.101) has joined #fluid-work
[12:22:16 EST(-0500)] * michelled (~michelled@142.150.154.101) has joined #fluid-work
[12:22:16 EST(-0500)] * yura (~yura@142.150.154.101) has joined #fluid-work
[12:22:16 EST(-0500)] * jameswy (~jameswy@142.150.154.196) has joined #fluid-work
[12:22:16 EST(-0500)] * jamon (jamon@mantis.openject.com) has joined #fluid-work
[12:44:35 EST(-0500)] * sveto (~sveto@62.44.108.2) has left #fluid-work
[12:51:51 EST(-0500)] * justin_o (~jmo@142.150.154.101) has joined #fluid-work
[13:06:58 EST(-0500)] <Bosmon> Am i in the channel?
[13:07:15 EST(-0500)] <michelled> yep
[13:07:19 EST(-0500)] <yura> hi Bosmon, so we broke the dataspout for exhibitionbrowse
[13:07:31 EST(-0500)] <yura> since we do not pass the proper args
[13:07:37 EST(-0500)] <Bosmon> We are just having a private chat about how it represents a logical design error to have a localisation argument to a data feed
[13:07:49 EST(-0500)] <Bosmon> Even a correct one 
[13:08:18 EST(-0500)]
[13:08:30 EST(-0500)] <Bosmon> Since a feed represents an element of the model layer of an application, to have it "localised" is not coherent
[13:08:37 EST(-0500)] <Bosmon> Well, I assume it might be supposed to be that
[13:08:52 EST(-0500)] <Bosmon> But here we have the problem that our "data feed" actually contains URLs which lead to the view layer of the application
[13:09:28 EST(-0500)] <Bosmon> We can't really fix this very soon.... but there it is
[13:10:16 EST(-0500)] <Bosmon> Actually our "first line of defence" for information such as localisation should be cookies in any case... with a fallback to explicit URL information if these fail
[13:19:09 EST(-0500)] * elicochran (~elicochra@dhcp-169-229-212-4.LIPS.Berkeley.EDU) has joined #fluid-work
[14:15:48 EST(-0500)] * michelled (~michelled@142.150.154.101) has joined #fluid-work
[14:15:59 EST(-0500)] * colinclark (~colin@142.150.154.101) has joined #fluid-work
[14:44:57 EST(-0500)] <colinclark> jameswy: Moderation and reporting abuse of comments in Engage...
[14:45:09 EST(-0500)] <colinclark> We agreed that this was off the table for 0.3, right?
[14:45:23 EST(-0500)] <jameswy> colinclark: Correct. Comments entirely, really.
[14:45:35 EST(-0500)] <jameswy> Just a dummy filler.
[14:45:40 EST(-0500)] <colinclark> jameswy: So remind me what Hugues actually said about that...
[14:45:43 EST(-0500)] <colinclark> Does it have to work at all
[14:46:18 EST(-0500)] <jameswy> We never discussed what to do with moderation/reporting abuse explicitly for 0.3. I assume it doesn't have to work at all.
[14:46:46 EST(-0500)] <colinclark> jameswy: Do you have a sec to clarify with us on skype?
[14:46:57 EST(-0500)] <colinclark> I'm chatting with justin_o and Bosmon about comments
[14:47:00 EST(-0500)] <jameswy> colinclark: Throw me in.
[16:33:00 EST(-0500)] * EverettZ (~chatzilla@bas4-toronto06-1279277146.dsl.bell.ca) has joined #fluid-work
[16:33:23 EST(-0500)] <EverettZ> colinclark: Is there a way for me to test the Engage app on my iPhone? I am guessing not...
[16:33:51 EST(-0500)] <colinclark> EverettZ: By "app," do you mean the home screen icon version?
[16:34:01 EST(-0500)] <colinclark> EverettZ: The chrome-less version?
[16:34:51 EST(-0500)] * Bojhan (~kvirc@213-84-190-103.adsl.xs4all.nl) has joined #fluid-work
[16:38:52 EST(-0500)] <EverettZ> colinclark: yes, that is what I mean.
[16:40:53 EST(-0500)] <colinclark> EverettZ: Yep, you totally can test it. Just make the main Engage page a home screen icon. In the bottom navigation bar of Safari, there's a button that has a "plus" icon. According to the accessibility inspector, it's the "Utilities" button. Tap on that, then choose "Add to Home Screen"
[16:41:46 EST(-0500)] <EverettZ> colinclark: thanks, I will test this out.
[16:41:54 EST(-0500)] <colinclark> EverettZ: Can't wait to hear what you think.
[17:17:26 EST(-0500)] <colinclark> Bosmon / Bosmon2: Got time for a quick question?
[17:19:07 EST(-0500)] * Bosmo1 (~Bosmon@eccr224-158-dhcp.colorado.edu) has joined #fluid-work
[17:19:13 EST(-0500)] <colinclark> Hey Bosmo1
[17:19:33 EST(-0500)] <colinclark> I'm just getting up to speed on yura's character encoding issues in the import.
[17:19:41 EST(-0500)] <Bosmo1> ok
[17:19:57 EST(-0500)] <colinclark> Wow, has it been a long time since I looked at Java code... I guess since last year when we built Flutter at 2 am in Dallas 
[17:20:13 EST(-0500)] <Bosmo1> JAVER code is the awesomest
[17:20:21 EST(-0500)] <colinclark> It actually has tools!
[17:20:29 EST(-0500)] <Bosmo1> Well some, yes
[17:20:32 EST(-0500)] <colinclark> Eclipse has a purpose again
[17:20:33 EST(-0500)] <colinclark>
[17:20:39 EST(-0500)] <Bosmo1> I guess one day we might have to disassemble the rest of JSEclipse
[17:20:45 EST(-0500)] <colinclark> Anyway, can you fill me in on what you think the cause of the issue is?
[17:20:48 EST(-0500)] <Bosmo1> Yes
[17:20:56 EST(-0500)] <colinclark> I saw his code was creating two different Readers
[17:20:59 EST(-0500)] <colinclark> Which seemed odd
[17:21:10 EST(-0500)] <Bosmo1> http://wiki.fluidproject.org/display/fluid/fluid-work+IRC+Logs-2010-02-02
[17:21:15 EST(-0500)] <Bosmo1> Here are the logs from yesterday
[17:21:26 EST(-0500)] <Bosmo1> What I observed then was this:
[17:21:27 EST(-0500)] <Bosmo1> [15:10:56 EST(-0500)] <Bosmon> Your problem is here on this line
[17:21:27 EST(-0500)] <Bosmo1> [15:10:57 EST(-0500)] <Bosmon> + BufferedReader in =
[17:21:27 EST(-0500)] <Bosmo1> [15:10:58 EST(-0500)] <Bosmon> + new BufferedReader(new InputStreamReader(url.openStream()));
[17:21:27 EST(-0500)] <Bosmo1> [15:11:02 EST(-0500)] <Bosmon> You have already corrupted the stream at this point
[17:21:28 EST(-0500)] <Bosmo1> 
[17:22:27 EST(-0500)] <colinclark> What I'm a bit fuzzy on here is the why we need to copy bytes around at all.
[17:22:56 EST(-0500)] <colinclark> The XPP generator ultimately takes an InputStream and appears to use UTF-8 encoding throughout
[17:23:15 EST(-0500)] <Bosmo1> You only need to copy streams because the streams are "back-ended"
[17:23:23 EST(-0500)] <colinclark> Can you elaborate?
[17:23:30 EST(-0500)] <Bosmo1> No
[17:23:36 EST(-0500)] <Bosmo1> I've just realised we don't even need to copy the bytes
[17:24:37 EST(-0500)] <colinclark> I actually modified the code to get rid of the whole readResource() method, and to simply pass the URL's connection input stream to XPPJSONGenerator's parseStream() method
[17:24:53 EST(-0500)] <colinclark> Which, I am perhaps naively assuming is all we need to do in the simplest case
[17:25:00 EST(-0500)] <Bosmo1> It is all we need to do ever
[17:25:05 EST(-0500)] <colinclark>
[17:25:19 EST(-0500)] <colinclark> We still, however appear to have the issue
[17:25:25 EST(-0500)] <Bosmo1> ...
[17:25:27 EST(-0500)] <colinclark> Again, I haven't been in the loop, so I'm missing pieces of the puzzle
[17:25:31 EST(-0500)] <Bosmo1> JT JS NOT POSSJBRE!!!
[17:25:44 EST(-0500)] <Bosmo1> Let me think if I can see any conceivable way we could have the issue...
[17:25:51 EST(-0500)] <colinclark> But by the time we look in Couch, we've got escaped unicode characters in the document
[17:26:11 EST(-0500)] <Bosmo1> Ah
[17:26:16 EST(-0500)] <Bosmo1> Well, I guess Couch is doing it "by the book"
[17:26:24 EST(-0500)] <Bosmo1> But which escaped characters do you see, exactly?
[17:26:30 EST(-0500)] <Bosmo1> Can you show me a sample?
[17:26:43 EST(-0500)] <colinclark> yura's gonna grab an example
[17:26:53 EST(-0500)] <colinclark> They seem likely to be the correct \uxxxx codes
[17:27:06 EST(-0500)] <Bosmo1> Well, ok, but you say the document is corrupt
[17:27:12 EST(-0500)] <Bosmo1> So presumably they are not actually the correct ones
[17:27:17 EST(-0500)] <colinclark> Well, I'm not even convinced the document is corrupt
[17:27:24 EST(-0500)] <Bosmo1> In particular, do you see a group of one such character where there should be an e acute, or 3?
[17:27:30 EST(-0500)] <colinclark> This is what yura just mentioned, but I'm trying to understand what happens downstream
[17:27:42 EST(-0500)] <colinclark> Let's see
[17:27:51 EST(-0500)] * anastasiac (~team@142.150.154.193) has left #fluid-work
[17:28:19 EST(-0500)] <yura> http://142.150.154.59:5984/_utils/document.html?mccord_exhibitions/2319141f3f478c56ea7d7bd3beca292c
[17:28:30 EST(-0500)] <colinclark> AH
[17:28:32 EST(-0500)] <colinclark> Look
[17:28:37 EST(-0500)] <colinclark> No unicode
[17:29:25 EST(-0500)] <colinclark> ah, see no
[17:29:32 EST(-0500)] <colinclark> bloody hell, i'm confusing myself
[17:30:07 EST(-0500)] <colinclark> Okay, so it looks like we've got a case where a single character is getting interpreted as two...something is not handling multibyte characters correctly, right?
[17:30:23 EST(-0500)] <Bosmo1> hmm
[17:30:25 EST(-0500)] <Bosmo1>
[17:30:41 EST(-0500)] <Bosmo1> Perhaps this is another of the perilous results of code non-reuse....
[17:31:02 EST(-0500)] <Bosmo1> It is possible that Couch needs to be informed of the encoding type in the originating request header
[17:31:23 EST(-0500)] <Bosmo1> Trying using the header which is defined in kettle "contentTypes.js"
[17:31:35 EST(-0500)] <colinclark> aha!
[17:31:50 EST(-0500)] <colinclark> let's take a look
[17:32:14 EST(-0500)] <colinclark> Bosmo1: You rule... we're getting somewhere now
[17:32:15 EST(-0500)] <colinclark>
[17:32:46 EST(-0500)] <Bosmo1> What's interesting is that the effects of the two sources of corruption are identical in terms of their effect on the stored data
[17:33:10 EST(-0500)] <Bosmo1> It is a kind of "idempotent corruption".... once the document has been "degraded" down to ISO-8859, it cannot be degraded any further
[17:33:18 EST(-0500)] <colinclark>
[17:33:40 EST(-0500)] <Bosmo1> fluid.kettle.headerFromEntry(fluid.kettle.contentTypeRegistry.JSON)
[17:33:49 EST(-0500)] <Bosmo1> In particular you can write this, to manufacture the correct header block
[17:33:57 EST(-0500)] <Bosmo1> Of course it could be argued that this is a lot harder than simply writing it out
[17:34:02 EST(-0500)] <Bosmo1> But old JAVER habits die hard...
[17:34:17 EST(-0500)] <colinclark> So, again, I'm new here...
[17:34:25 EST(-0500)] <colinclark> What code is POSTing the documents to Couch here?
[17:34:30 EST(-0500)] <Bosmo1> Yura's JAVER code
[17:34:32 EST(-0500)] <colinclark> I guess I will have to actually look
[17:34:33 EST(-0500)] <Bosmo1> Ah, funny
[17:34:33 EST(-0500)] <colinclark>
[17:34:38 EST(-0500)] <colinclark> ThinkPad away
[17:34:52 EST(-0500)] <colinclark> Wait, the code is not on the ThinkPad
[17:34:56 EST(-0500)] <colinclark> It's on Windows 7
[17:35:00 EST(-0500)] <Bosmo1> !!!
[17:35:01 EST(-0500)] <Bosmo1>
[17:35:07 EST(-0500)] <colinclark> We have three computers here
[17:35:08 EST(-0500)] * clown (~clown@142.150.154.101) has left #fluid-work
[17:35:10 EST(-0500)] <colinclark> Now I am confused
[17:35:22 EST(-0500)] <Bosmo1> The only code I have ever seen is on the patch
[17:35:25 EST(-0500)] <Bosmo1> So I have no idea where it is
[17:35:35 EST(-0500)] <colinclark> Yura is using a third-party library
[17:35:40 EST(-0500)] <colinclark> Which he was theorizing is at fault
[17:37:27 EST(-0500)] <Bosmo1> A Couch Communications Library?
[17:38:19 EST(-0500)] <colinclark> Yeah, for Java
[17:38:24 EST(-0500)] <colinclark> I haven't yet figured out what it is called
[17:38:32 EST(-0500)] <colinclark> but it offers a class entitled Database
[17:38:41 EST(-0500)] <colinclark> jCouchDB, apparently
[17:39:05 EST(-0500)] <Bosmo1> Well, I notice you already have commons-httpclient in your list of dependencies
[17:39:11 EST(-0500)] <Bosmo1> Which is more than capable of making the call correctly
[17:39:15 EST(-0500)] <Bosmo1> I suggest you just fall back to using that
[17:39:33 EST(-0500)] <Bosmo1> After all, you just have a block of text which you want to be POSTed to a particular URL with particular headers
[17:40:03 EST(-0500)] <colinclark> I guess we are using Couch's bulk import features
[17:40:30 EST(-0500)] <Bosmo1> It will be massively quicker as wel
[17:43:37 EST(-0500)] * Bosmo1 (~Bosmon@eccr224-158-dhcp.colorado.edu) has joined #fluid-work
[17:53:03 EST(-0500)] <colinclark> Bosmo1:
[17:53:11 EST(-0500)] <colinclark> Okay, we have successfully traced the Evil
[17:53:43 EST(-0500)] <colinclark> This jCouch library is thrashing unicode characters when it generates JSON... we'll toss it
[17:57:23 EST(-0500)] <Bosmo1> It is also wasting huge amounts of time, reencoding something that is already properly encoded
[17:57:28 EST(-0500)] <Bosmo1> http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
[17:57:31 EST(-0500)] <Bosmo1> Here is the bulk document API
[17:57:33 EST(-0500)] <colinclark> That's exactly what I'm trying to get at here
[17:57:54 EST(-0500)] <Bosmo1> All that seems to be required is to put a simple header, some commas between the documents, and a closing ]}
[17:58:33 EST(-0500)] <Bosmo1> http://hc.apache.org/httpclient-3.x/methods/post.html
[17:58:38 EST(-0500)] <Bosmo1> Here is a guide to the HttpClient API...
[17:58:49 EST(-0500)] <Bosmo1> Showing how to set up a POST request
[17:58:59 EST(-0500)] <Bosmo1> http://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/methods/PostMethod.html
[17:59:00 EST(-0500)] <Bosmo1> Or docs here
[18:01:05 EST(-0500)] <Bosmo1> Amuzingly the name of the 4th HttpClient author has been badly corrupted by a bad document encoding
[18:05:54 EST(-0500)] <colinclark> lol
[18:06:50 EST(-0500)] <colinclark> Bosmo1: Ok, so here's where we're at
[18:06:59 EST(-0500)] <colinclark> I'm going to learn about the Fuel System
[18:07:27 EST(-0500)] <colinclark> yura is going to build up a bulk document request himself and send it off without using this jCouchDB library
[18:07:39 EST(-0500)] <colinclark> He seems clear on how he'll do it, right yura?
[18:08:52 EST(-0500)] <yura> colinclark: yes, I since the JSONProvider stringifies the parsed documents it shouldn't be too difficult
[18:08:56 EST(-0500)] <colinclark> awesome
[18:08:59 EST(-0500)] <colinclark> so we're all cool?
[18:09:03 EST(-0500)] <yura> colinclark: yep
[18:27:14 EST(-0500)] <Bosmo1> yura - where is the code for the views we have written in Couch kept?
[18:32:39 EST(-0500)] <yura> oh yes, it is in scratchpad
[18:32:43 EST(-0500)] <yura> couch-views
[18:32:48 EST(-0500)] <yura> Bosmo1: ^
[18:41:38 EST(-0500)] * michelled (~michelled@CPE001310472ade-CM0011aefd3ca8.cpe.net.cable.rogers.com) has joined #fluid-work
[18:48:19 EST(-0500)] <Bosmo1> Thanks
[19:08:13 EST(-0500)] * michelled (~michelled@CPE001310472ade-CM0011aefd3ca8.cpe.net.cable.rogers.com) has joined #fluid-work
[21:03:36 EST(-0500)] * yura (~yura@bas3-toronto06-2925097429.dsl.bell.ca) has joined #fluid-work
[21:19:24 EST(-0500)] * yura (~yura@bas3-toronto06-2925097429.dsl.bell.ca) has joined #fluid-work
[21:45:51 EST(-0500)] * michelled (~michelled@CPE001310472ade-CM0011aefd3ca8.cpe.net.cable.rogers.com) has joined #fluid-work