/
Phase 2 - Governance

Phase 2 - Governance

Beginning January 2009, work will be progressing on implementing basic features mentioned in this document. For more information on this plan, please see "OSDPL roadmap (Jan. - Mar. '09), planned pattern workflow, and features".

 

Adding Patterns

Roles:

  • Author: Creates the initial draft. Controls the revision process if preferred.
  • Editor: A Reviewer who facilitates the author's revision process.
  • Other Reviewers: Can collaboratively contribute to an author's design pattern if requested, otherwise provides occasional advice.

Author's process

  1. creates first draft.
  2. indicates that their draft is ready for review.
    • given the option to make their draft viewable by the public. [AB: It will be important to put "in progress" drafts in a separate section of the library, or mark them clearly as drafts and easily allow users to filter them out if necessary. It may also be helpful to indicate the author's community association(s) (e.g. Fluid, uPortal) in order for users of the patterns to understand their perspective.]
  3. contacted by an Editor.
  4. together with Editor (and other individuals if necessary), revisions are made on draft.
    • feedback suggested by other users and incorporated if appropriate (this applies only if draft is viewable by the public)
  5. both author and Editor are satisfied with draft, final review is done by a Reviewer.
    • If Reviewer is satisfied, go to Step 6. [AB: We should say that there will be guidelines in How to Write a Good Design Pattern which will help them decide whether the pattern is "ready to go."]
    • Otherwise, Reviewer gives feedback. Go back to step 4.
  6. content is marked as "complete" and publicly viewable.

Reviewers' and Editor's process

  1. Reviewers are notified that a new draft is available.
  2. Reviewers examine the draft and one Reviewer takes the initiative to contact the author if revisions are necessary (this reviewer is now called the "Editor").
  3. By contacting the author, the Editor determines how the author would like the review process to proceed:
    • If the the Editor feels the Author would prefer a one-on-one approach, then interaction takes place primarily between the Editor and Author.
    • If the Editor feels the Author is open for a more peer process, the other Reviewers are invited to make their contributions.
  4. together with author (and other individuals if necessary), revisions are made on draft. [AB: We should say that the be guidelines in How to Write a Good Design Pattern will provide guidance on editing the draft.]
  5. both author and Editor are satisfied with draft, final review is done by a Reviewer.
    • If Reviewer is satisfied, go to Step 6.
    • Otherwise, Reviewer gives feedback. Go to Step 4.
  6. content is marked as "complete" and publicly viewable.

Communication:

  • The Comment system may be a good way to carry out much of the discussion between Author, Editor, and Reviewer.
  • Comment ranking system can help indicate which comments are worth reading. comments."[AB: I'm a little skeptical that folks will actually make the effort to rank comments in the (probably relatively short) editing phase unless we solicit them in some way. Perhaps a notice on the front page of the site saying something like, "the x design pattern is in review (or has just been published)--please let us know if we can improve the pattern by leaving your comments, using "How to Create a Good Design Pattern" as a guide, and rating any existing]
  • Otherwise email, comments within the document, mailing list, or a forum are good systems.
  • DO - I find most comment systems cumbersome unless the comments can display inline.  For instance, I'm not using comments here because for each comment I'd first have to describe which text I'm commenting on and then the reader has to scroll between the original text and the comment.  I don't know  anything about the comment ranking system but is sounds a little strange -- who gets to say which comments are worth are reading?  If I'm writing them hopefully I feel like they are worth reading.  If I'm the reader than who is the other person to say something I wrote is not worthy?
  • JH - Inline commenting or a better UI for comments is a good idea. It is tedious to scroll to the bottom to get to the contents, but perhaps it can be alleviated by using a "skip to comments" link. Inline comments make a lot of sense... that one deserves a pattern in itself!
  • AB - It may be a bit distracting to have inline comments in a published pattern. We may want to consider handling comments differently for published vs. draft patterns. It would be ideal if there were a way to link to the pertinent place in the pattern from the comments, but have it be invisible in the pattern if someone wasn't viewing comments. Alternatively, we could advise folks to quote the relevant portion of the pattern in their comment (though of course we know people rarely read directions (smile) ).
  • AB - The ranking system for comments I don't think will be indicating that a comment is not "worthy"...what it will really be like is a feature request system in JIRA where people +1 a new feature, but in this case it will be +1'ing a change or improvement to a pattern. Perhaps we should make it really clear that folks should only suggest one change per comment so it's easy for people to +1 a particular suggestion in a comment without making it appear that they are +1'ing others.

An example of ranked comments is [digg.com|http://www.digg.com] or [Yahoo answers|http://answers.yahoo.com/]. The community determines what is worth reading and for the most part it works. Most comments will have a neutral rank of 0, whereas a rare good comment will have +1 or higher. Really bad comments (like spam, jokes, stuff totally off topic) get negative rankings. It does sound a bit odd but it's really helpful in appropriate contexts.

That said, it's unlikely ranked comments will be a critical feature to implement right away. It's a feature that works better in large communities, and the OSDPL is not at that point right now.

Co-Authoring:
At some point, another individual may want to co-author a design pattern.

  • How do we allow particular users to be able to edit a particular pattern?
  • DO:  Because of the way our community works in pairs, patterns coming from our internal group will most likely always be co-authored.  Can authors have equal weight?
  • JH - We would want only a subset/pair of users to be able to collaborate on a particular pattern and we don't necessarily want a wiki style free-for-all where every user can edit anyone else's content. We would want to restrict which users get to modify which content.
  • AB - The way things are set up now, everyone is listed as an equal "Contributor." In terms of giving permissions to edit a particular pattern, I'm sure there is a Drupal module that does this--I don't think we'd need anything as heavy-duty as organic groups. I am pretty familiar with these types of modules and can do some research if need be.

I'm trying not to get into a technical discussion since it's premature, but without any research, this sounds like a technical challenge (Drupal has "Organic groups" that can do something similar, but we'll need to research it). Also determining a easy process for allowing users to become collaborators needs to be examined.

Q1. Who is a "Reviewer"? What role do they play?

A reviewer is a role assigned to an OSDPL user which allows them to view and edit all design patterns. They also approve design patterns to be published. Reviewers also play a role in approving design patterns to be published.

Q2. Are there requirements or a typical background, knowledge set, etc. we see in Reviewers? How do you become a Reviewer? Are there many Reviewers or just one?

Reviewers are individuals who have proven knowledge of writing or designing design patterns. A normal user can become a Reviewer by request, or promoted to Reviewer status by other Reviewers who have observed the individual's activities on the site. This [AB: What will be disclosed here? Just who has become a reviewer, or what they did to become a reviewer? I'd vote for the former plus a "Guidelines for becoming a Reviewer" page.] will be disclosed to users so they can see how they can become a Reviewer too.

Though there is more than one Reviewer on the OSDPL, one Reviewer assumes the role of "editor" for a particular pattern. Hopefully the number of Reviewers will will grow as the community grows.

Q3. Should design patterns in draft stage be viewable by everyone?

Yes. Transparency is created if drafts are viewable by everyone. Helps create a sense of community, create some interesting interactions between users, and provide educational value to others.

  • User will be able to specify if they want their draft public or not.
  • Draft patterns will be segregated from polished patterns in some way (either by category or some other method).
  • These patterns will be clearly marked as "In progress" [AB: I'd prefer the word "Draft" here. May be something to user test.]

Q4. What constitutes a design pattern that is complete?

A design pattern is complete when all the required items are there with sufficient and insightful information. Completeness is judged by the pattern author, the Editor, and the Reviewer - all three need consensus before a pattern is marked as "complete". A guide will be written to assist in this. [AB: Assuming this would be How to Write a Good Design Pattern ?]

Q5. What status flags are there for a given design pattern?
Complete / In progress / In review (question) [AB: I think we also need a "Draft" state to show that the pattern is just being started. I'd probably vote for the wording of "Draft" vs. "In Progress" as I think that would make it clearer to users of the pattern library that it is not even close to a "vetted" pattern.]
Private / Public

Q6. What is the relationship between the author and the reviewer?

Ideally, a reviewer will work with a pattern author, facilitating and enabling the process. In some cases the reviewer can act as a mentor to the author, as well as serving as an editor - much as a book or magazine editor assists a writer. Experienced authors may not require such assistance, in which case the role of the reviewer is to provide a constructive critique of the candidate pattern, making suggestions for improvement or identifying deficiencies. [AB: How will authors be paired with reviewers? Perhaps we can use the Actions module to inform someone monitoring the library for additions that a new draft pattern has been added.]

Q7. Can an author withdraw a pattern from the library?

Once a pattern has been published it is freely available under the Creative Commons license, and cannot be withdrawn. Prior to publication, the author can withdraw any work in progress. [AB: We'll need to make the different workflow states REALLY clear to pattern authors to ensure they understand this.]

Q8. What are the rewards of pattern authorship?

As it is with most Open Source material, authors of patterns earn the respect of their colleagues and co-professionals. The respect of peers and the wide publication of original work is an important cultural value in open source movements. [AB: I think we've also talked about them gaining status on the pattern library by being rated as individuals for their contributions to the pattern library.]

Updating patterns [AB: I think it would be clearer if we called this "Editing a Published Design Pattern"]

  • All Reviewers will be able to update patterns. [AB: Does this mean that reviewers only edit patterns if requested by the author? As specified above: "Can collaboratively contribute to an author's design pattern if requested, otherwise provides occasional advice." Or can any reviewer edit any design pattern anytime? We should define the situations in which that would be appropriate.]
  • Pattern authors can update their own patterns. [AB: Should pattern authors be encouraged to regularly review their patterns?]
  • All regular users will need to "submit" edits using the comment system and edits are ranked by peers
  • Comments can be ranked to make good input easier to spot.
  • DO: Do we need to say anything about the collaboration that needs to happen here or is it implicit? Seems like there is lots of room for confusion here is communication isn't optimal. Does the system allow for anything like tracking changes? [AB: The system does track revisions, but not diffs. So you can view or roll back to all previous revisions, but you'd have to pull the text out of both revisions and diff it in a text editor to see exactly what the changes were between revisions.]
  • JH: It's my hope that it becomes implicit. Using good help text and obvious points of interaction should help. Text such as "Use comments to give feedback on this pattern" may help.

If edit is suggested by a user, this is what a reviewer or author does: [AB: When would a reviewer do this? If they'd been requested to by an author?]

  1. Read the suggestion and evaluate merits
    1. If a worthy update, click the Edit button and add it to the DP. Save.
    2. If reviewer is unsure or needs second opinion, discussion should be carried out in comments with other reviewers. Accept/reject modification. [AB: This might make the comments rather cluttered. What about having this discussion in the forum instead? Or perhaps the comments should be removed after a decision is made? The downside to this is that there wouldn't be a historical record of the discussion.]
  • To ensure that the same suggestion doesn't get added twice, reviewer can indicate in the comments which updates have been applied.
  • DO:  Is there a reviewer for each pattern or is it whoever gets to it first?  Seems like there might be some folks more familiar with a particular DP and thus a better candidate for judging than someone coming fresh at it.  This is different than code review since you don't have the back up of 'it works or not'.  There are lots of right answers in design so we may need some additional checks and balances to protect the original intent?  Do the reviewer and author need to agree on this change?
  • JH - See #2 in "Reviewers' and Editor's process" near top. There is at least 1 Reviewer (the Editor) and he/she is a volunteer and presumably capable of editing patterns. We should not be afraid of inexperience because there are other knowledgeable Reviewer peers who would help keep things in check. [AB: I think there could be some issues if a new Reviewer and a new Author do a lot of work together on a pattern, as it may make other reviewers may be reluctant to ask them to redo it all if they've gone down the wrong path. I think it would be best to pair new people with experienced people, or if we must pair a new reviewer with a new  author, we should have another reviewer check in with them periodically so they can provide feedback along the way, before the pattern is made ready for the Reviewer Review.]

"Do the reviewer and author need to agree on this change?"
Yes and no. It really depends on the situation and the individuals involved. Some pattern authors are more open to collaborative changes, whereas some aren't. This is why the Editor is a critical person in the process as they will help each individual pattern author in a way that is appropriate for the situation.

Q1. How do we monitor / moderate updates? How to prevent abuse while not making the process of updating tedious?
As long as modifications are tracked and data can't be lost, we don't need to worry about it now.

DO:  I agree that we don't want to make this tedious and that tracked changes are a good protection.  Is it up to author to keep track of modifications?  I assume their name stays on it as the author which is a little different than something like wikipedia.

JH - Both author and Reviewers will need to keep on top of things. This would mean that we should probably have a decent notification system where Reviewers and authors can get notified when changes have been made. [AB: Sounds great--I think the actions module may work for this. I do think we need to clarify a bit more whether authors must always be involved in edits.]

Moderation

The OSDPL will use a combination of both human and machine moderation.

1. Filtering by ranking

  • Users can rank comments, design patterns, and even users.
  • Items ranked below a threshold will not be displayed (collapsed, but can be revealed on demand)
  • Items ranked above threshold will be displayed
  • This is not destructive moderation, rather it's a method of content filtering.

2. Moderation by humans

  • Privileged users can delete / edit content as necessary.
    • Reviewers will not be able to moderate non DP content (i.e. can't change announcements, site news, user accounts etc.)
  • Ranked content (see #1 above) makes it easier to spot bad comments / content more easily and act upon them.
  • Guests/anonymous users can not rank content to discourage artificial ranking.

3. There should be a software spam filter in place for all content.

DO:  Sounds great!

Ownership

Since a design can evolve over time, the design pattern may become a collection of contributions from multiple people and sources. Therefore "ownership" in a strict sense may not apply to the OSDPL.

However, if a pattern is derived from another pattern, proper attribution should be given. The original author should also be contacted to ensure their intentions of the pattern are maintained.

If participation from the original author can not be found, then a "fork" (i.e. derivative work) should be done and indicated as such.

DO:  I think these 2 comments are really important to encourage authorship.

The OSDPL will maintain a list of editors / contributors and the person who originally submitted the DP. This way contributors get the recognition they deserve while spreading ownership across the community. [AB: Nice!]

This kind of ownership continues in the spirit of the Fluid Project where community and sharing is encouraged.

DO:  Yes, this is key for our community - collaboration and kudos!






Guidelines for design patterns