Phase 2 - Governance

Adding Patterns

Roles:

• Author: Creates the initial draft. Controls the revision process if preferred.
• Editor: A Reviewer who facilitates the author's revision process.
• Other Reviewers: Can collaboratively contribute to an author's design pattern if requested, otherwise provides occasional advice.

Author's process

1. creates first draft.
2. indicates that their draft is ready for review.
   • given the option to make their draft viewable by the public. [AB: It will be important to put "in progress" drafts in a separate section of the library, or mark them clearly as drafts and easily allow users to filter them out if necessary. It may also be helpful to indicate the author's community association(s) (e.g. Fluid, uPortal) in order for users of the patterns to understand their perspective.]
3. contacted by an Editor.
4. together with Editor (and other individuals if necessary), revisions are made on draft.
   • feedback suggested by other users and incorporated if appropriate (this applies only if draft is viewable by the public)
5. both author and Editor are satisfied with draft, final review is done by a Reviewer.
   • If Reviewer is satisfied, go to Step 6. [AB: We should say that there will be guidelines in How to Write a Good Design Pattern which will help them decide whether the pattern is "ready to go."]
   • Otherwise, Reviewer gives feedback. Go back to step 4.
6. content is marked as "complete" and publicly viewable.

Reviewers' and Editor's process

1. Reviewers are notified that a new draft is available.
2. Reviewers examine the draft and one Reviewer takes the initiative to contact the author if revisions are necessary (this reviewer is now called the "Editor").
3. By contacting the author, the Editor determines how the author would like the review process to proceed:
   • If the the Editor feels the Author would prefer a one-on-one approach, then interaction takes place primarily between the Editor and Author.
   • If the Editor feels the Author is open for a more peer process, the other Reviewers are invited to make their contributions.
4. together with author (and other individuals if necessary), revisions are made on draft. [AB: We should say that the be guidelines in How to Write a Good Design Pattern will provide guidance on editing the draft.]
5. both author and Editor are satisfied with draft, final review is done by a Reviewer.
   • If Reviewer is satisfied, go to Step 6.
   • Otherwise, Reviewer gives feedback. Go to Step 4.
6. content is marked as "complete" and publicly viewable.

Communication:

• The Comment system may be a good way to carry out much of the discussion between Author, Editor, and Reviewer.
• Comment ranking system can help indicate which comments are worth reading. comments."[AB: I'm a little skeptical that folks will actually make the effort to rank comments in the (probably relatively short) editing phase unless we solicit them in some way. Perhaps a notice on the front page of the site saying something like, "the x design pattern is in review (or has just been published)--please let us know if we can improve the pattern by leaving your comments, using "How to Create a Good Design Pattern" as a guide, and rating any existing]
• Otherwise email, comments within the document, mailing list, or a forum are good systems.
• DO - I find most comment systems cumbersome unless the comments can display inline.  For instance, I'm not using comments here because for each comment I'd first have to describe which text I'm commenting on and then the reader has to scroll between the original text and the comment.  I don't know  anything about the comment ranking system but is sounds a little strange -- who gets to say which comments are worth are reading?  If I'm writing them hopefully I feel like they are worth reading.  If I'm the reader than who is the other person to say something I wrote is not worthy?
• JH - Inline commenting or a better UI for comments is a good idea. It is tedious to scroll to the bottom to get to the contents, but perhaps it can be alleviated by using a "skip to comments" link. Inline comments make a lot of sense... that one deserves a pattern in itself!
• AB - It may be a bit distracting to have inline comments in a published pattern. We may want to consider handling comments differently for published vs. draft patterns. It would be ideal if there were a way to link to the pertinent place in the pattern from the comments, but have it be invisible in the pattern if someone wasn't viewing comments. Alternatively, we could advise folks to quote the relevant portion of the pattern in their comment (though of course we know people rarely read directions ).
• AB - The ranking system for comments I don't think will be indicating that a comment is not "worthy"...what it will really be like is a feature request system in JIRA where people +1 a new feature, but in this case it will be +1'ing a change or improvement to a pattern. Perhaps we should make it really clear that folks should only suggest one change per comment so it's easy for people to +1 a particular suggestion in a comment without making it appear that they are +1'ing others.

An example of ranked comments is [digg.com|http://www.digg.com] or [Yahoo answers|http://answers.yahoo.com/]. The community determines what is worth reading and for the most part it works. Most comments will have a neutral rank of 0, whereas a rare good comment will have +1 or higher. Really bad comments (like spam, jokes, stuff totally off topic) get negative rankings. It does sound a bit odd but it's really helpful in appropriate contexts.

That said, it's unlikely ranked comments will be a critical feature to implement right away. It's a feature that works better in large communities, and the OSDPL is not at that point right now.

Co-Authoring:
At some point, another individual may want to co-author a design pattern.

• How do we allow particular users to be able to edit a particular pattern?
• DO:  Because of the way our community works in pairs, patterns coming from our internal group will most likely always be co-authored.  Can authors have equal weight?
• JH - We would want only a subset/pair of users to be able to collaborate on a particular pattern and we don't necessarily want a wiki style free-for-all where every user can edit anyone else's content. We would want to restrict which users get to modify which content.
• AB - The way things are set up now, everyone is listed as an equal "Contributor." In terms of giving permissions to edit a particular pattern, I'm sure there is a Drupal module that does this--I don't think we'd need anything as heavy-duty as organic groups. I am pretty familiar with these types of modules and can do some research if need be.

I'm trying not to get into a technical discussion since it's premature, but without any research, this sounds like a technical challenge (Drupal has "Organic groups" that can do something similar, but we'll need to research it). Also determining a easy process for allowing users to become collaborators needs to be examined.

Q1. Who is a "Reviewer"? What role do they play?

A reviewer is a role assigned to an OSDPL user which allows them to view and edit all design patterns. They also approve design patterns to be published. Reviewers also play a role in approving design patterns to be published.

Q2. Are there requirements or a typical background, knowledge set, etc. we see in Reviewers? How do you become a Reviewer? Are there many Reviewers or just one?

Reviewers are individuals who have proven knowledge of writing or designing design patterns. A normal user can become a Reviewer by request, or promoted to Reviewer status by other Reviewers who have observed the individual's activities on the site. This [AB: What will be disclosed here? Just who has become a reviewer, or what they did to become a reviewer? I'd vote for the former plus a "Guidelines for becoming a Reviewer" page.] will be disclosed to users so they can see how they can become a Reviewer too.

Though there is more than one Reviewer on the OSDPL, one Reviewer assumes the role of "editor" for a particular pattern. Hopefully the number of Reviewers will will grow as the community grows.

Q3. Should design patterns in draft stage be viewable by everyone?

Yes. Transparency is created if drafts are viewable by everyone. Helps create a sense of community, create some interesting interactions between users, and provide educational value to others.

• User will be able to specify if they want their draft public or not.
• Draft patterns will be segregated from polished patterns in some way (either by category or some other method).
• These patterns will be clearly marked as "In progress" [AB: I'd prefer the word "Draft" here. May be something to user test.]

Q4. What constitutes a design pattern that is complete?

A design pattern is complete when all the required items are there with sufficient and insightful information. Completeness is judged by the pattern author, the Editor, and the Reviewer - all three need consensus before a pattern is marked as "complete". A guide will be written to assist in this. [AB: Assuming this would be How to Write a Good Design Pattern ?]

Q5. What status flags are there for a given design pattern?
Complete / In progress / In review [AB: I think we also need a "Draft" state to show that the pattern is just being started. I'd probably vote for the wording of "Draft" vs. "In Progress" as I think that would make it clearer to users of the pattern library that it is not even close to a "vetted" pattern.]
Private / Public

Q6. What is the relationship between the author and the reviewer?

Ideally, a reviewer will work with a pattern author, facilitating and enabling the process. In some cases the reviewer can act as a mentor to the author, as well as serving as an editor - much as a book or magazine editor assists a writer. Experienced authors may not require such assistance, in which case the role of the reviewer is to provide a constructive critique of the candidate pattern, making suggestions for improvement or identifying deficiencies. [AB: How will authors be paired with reviewers? Perhaps we can use the Actions module to inform someone monitoring the library for additions that a new draft pattern has been added.]

Q7. Can an author withdraw a pattern from the library?

Once a pattern has been published it is freely available under the Creative Commons license, and cannot be withdrawn. Prior to publication, the author can withdraw any work in progress. [AB: We'll need to make the different workflow states REALLY clear to pattern authors to ensure they understand this.]

Q8. What are the rewards of pattern authorship?

As it is with most Open Source material, authors of patterns earn the respect of their colleagues and co-professionals. The respect of peers and the wide publication of original work is an important cultural value in open source movements. [AB: I think we've also talked about them gaining status on the pattern library by being rated as individuals for their contributions to the pattern library.]

Updating patterns [AB: I think it would be clearer if we called this "Editing a Published Design Pattern"]

• All Reviewers will be able to update patterns. [AB: Does this mean that reviewers only edit patterns if requested by the author? As specified above: "Can collaboratively contribute to an author's design pattern if requested, otherwise provides occasional advice." Or can any reviewer edit any design pattern anytime? We should define the situations in which that would be appropriate.]
• Pattern authors can update their own patterns. [AB: Should pattern authors be encouraged to regularly review their patterns?]
• All regular users will need to "submit" edits using the comment system and edits are ranked by peers
• Comments can be ranked to make good input easier to spot.
• DO: Do we need to say anything about the collaboration that needs to happen here or is it implicit? Seems like there is lots of room for confusion here is communication isn't optimal. Does the system allow for anything like tracking changes? [AB: The system does track revisions, but not diffs. So you can view or roll back to all previous revisions, but you'd have to pull the text out of both revisions and diff it in a text editor to see exactly what the changes were between revisions.]
• JH: It's my hope that it becomes implicit. Using good help text and obvious points of interaction should help. Text such as "Use comments to give feedback on this pattern" may help.

If edit is suggested by a user, this is what a reviewer or author does: [AB: When would a reviewer do this? If they'd been requested to by an author?]

1. Read the suggestion and evaluate merits
   1. If a worthy update, click the Edit button and add it to the DP. Save.
   2. If reviewer is unsure or needs second opinion, discussion should be carried out in comments with other reviewers. Accept/reject modification. [AB: This might make the comments rather cluttered. What about having this discussion in the forum instead? Or perhaps the comments should be removed after a decision is made? The downside to this is that there wouldn't be a historical record of the discussion.]

• To ensure that the same suggestion doesn't get added twice, reviewer can indicate in the comments which updates have been applied.
• DO:  Is there a reviewer for each pattern or is it whoever gets to it first?  Seems like there might be some folks more familiar with a particular DP and thus a better candidate for judging than someone coming fresh at it.  This is different than code review since you don't have the back up of 'it works or not'.  There are lots of right answers in design so we may need some additional checks and balances to protect the original intent?  Do the reviewer and author need to agree on this change?

• JH - See #2 in "Reviewers' and Editor's process" near top. There is at least 1 Reviewer (the Editor) and he/she is a volunteer and presumably capable of editing patterns. We should not be afraid of inexperience because there are other knowledgeable Reviewer peers who would help keep things in check. [AB: I think there could be some issues if a new Reviewer and a new Author do a lot of work together on a pattern, as it may make other reviewers may be reluctant to ask them to redo it all if they've gone down the wrong path. I think it would be best to pair new people with experienced people, or if we must pair a new reviewer with a new  author, we should have another reviewer check in with them periodically so they can provide feedback along the way, before the pattern is made ready for the Reviewer Review.]

"Do the reviewer and author need to agree on this change?"
Yes and no. It really depends on the situation and the individuals involved. Some pattern authors are more open to collaborative changes, whereas some aren't. This is why the Editor is a critical person in the process as they will help each individual pattern author in a way that is appropriate for the situation.

Q1. How do we monitor / moderate updates? How to prevent abuse while not making the process of updating tedious?
As long as modifications are tracked and data can't be lost, we don't need to worry about it now.

DO:  I agree that we don't want to make this tedious and that tracked changes are a good protection.  Is it up to author to keep track of modifications?  I assume their name stays on it as the author which is a little different than something like wikipedia.

JH - Both author and Reviewers will need to keep on top of things. This would mean that we should probably have a decent notification system where Reviewers and authors can get notified when changes have been made. [AB: Sounds great--I think the actions module may work for this. I do think we need to clarify a bit more whether authors must always be involved in edits.]

Moderation

The OSDPL will use a combination of both human and machine moderation.

1. Filtering by ranking

• Users can rank comments, design patterns, and even users.
• Items ranked below a threshold will not be displayed (collapsed, but can be revealed on demand)
• Items ranked above threshold will be displayed
• This is not destructive moderation, rather it's a method of content filtering.

2. Moderation by humans

• Privileged users can delete / edit content as necessary.
  • Reviewers will not be able to moderate non DP content (i.e. can't change announcements, site news, user accounts etc.)

• Ranked content (see #1 above) makes it easier to spot bad comments / content more easily and act upon them.
• Guests/anonymous users can not rank content to discourage artificial ranking.

3. There should be a software spam filter in place for all content.

DO:  Sounds great!

Ownership

Since a design can evolve over time, the design pattern may become a collection of contributions from multiple people and sources. Therefore "ownership" in a strict sense may not apply to the OSDPL.

However, if a pattern is derived from another pattern, proper attribution should be given. The original author should also be contacted to ensure their intentions of the pattern are maintained.

If participation from the original author can not be found, then a "fork" (i.e. derivative work) should be done and indicated as such.

DO:  I think these 2 comments are really important to encourage authorship.

The OSDPL will maintain a list of editors / contributors and the person who originally submitted the DP. This way contributors get the recognition they deserve while spreading ownership across the community. [AB: Nice!]

DO:  Yes, this is key for our community - collaboration and kudos!

Guidelines for design patterns

A guide being written to attempt to define what a good design pattern is for the OSDPL.

• challenge is to define a guideline that is understandable by both authors and editors of various experience levels.
• Intentionally restricting ourselves to the domain of the OSDPL to focus our scope and understanding.
• It will be important to include non-designers / developers in the discussions since they can help with the formulation.

Remarks from Aug 20 meeting

• answer will evolve (put this in the charter?) [AB: I think it might be a good idea to say that we periodically review and add to the guidelines as a sort of "lessons learned" exercise.]
• granularity: do we link or include basic DPs?
  • what about sub-patterns within DPs (i.e. progress bar in the File Uploader).
• Is content on the OSDPL design or a design pattern? We'd like to be a design pattern, so be wary of being too prescriptive.
• design patterns are informed by best practices
• not too prescriptive - general and not too component specific
• But can be specific - useful as examples and illustrations
• communities contributing their own patterns that may not be general. [AB: I think we may want to ensure that the group of reviewers for patterns for a particular pattern includes folks from multiple communities.]
  • "What makes a good design pattern" guideline may help.
  • What do you do with this community specific content that makes its way to the OSDPL?

Q1. What qualities would we like to see in a design pattern on the OSDPL?

1. Brief and concise
2. Practical and applicable
3. Educational (read: theoretical / broad) [AB: Is this at odds with #1? Or will we have two views of the patterns, one with more and one with less detail?]
4. Useful to 2 primary audiences: novice designers, and developers.

Q2. What is a good design pattern for a developer?

• easy to follow step-by-step structure (like making a cake... layers, icing, and all the good stuff)
• simple explanations for subtle design decisions
• indication where implementation may differ from the pattern [AB: As there can be multiple implementations of each pattern, I don't think the pattern itself has a single implementation. So implementation will almost always be a bit different from the pattern. Tidwell says, "They aren't off-the-shelf components; each implementation of a pattern differs a little from every other."]
• glossary of design terms and explanation / justification of UX designs

From Erin:

"A pattern where the design rationale is clearly stated to help the developer understand the reasoning behind the design and to empower them to modify the design as needed to accommodate the particular context they have." [AB: I agree. However, we need to be careful to ensure that design patterns don't simply explain Fluid component designs, but express wide-reaching patterns that others in the UX/UI design community would be interested in using.]

From Paul:

"Ideally, a good design pattern for a developer will answer questions about the user's expectations and reactions, but not make decisions that should be left up to the developer. In other words, the pattern author should not try to do the developer's job by over-specifying implementation details. Having said that, I realize that it is somewhat at odds of our intention to supplement our patterns with a lot of examples and implementation advice. I think this can be dealt with through refinement of the structure of the pattern description - see my response to Q3."

Q3. What is a good design pattern for a novice designer?

• good sound design advice

DO:

• clear rationale so they can understand why design decisions were made
• reference to design principles used in pattern creation (i.e. users should always have an escape path)

[AB:

• Daphne's two points above are some of the most important considerations.
• A pattern represents a proven solution to a common problem in a specified context.
• Tidwell says, "Part of their value lies in the way they resolve tensions in various design contexts." Though there may be multiple solutions to a particular problem, patterns help you choose the best solution for your context. ]

From Erin:

"Provides many examples of good design practice for them to learn and be inspired from.
Clearly identifies the context where the pattern could be most suitable in order to help the designer determine which pattern to use where. "

From Paul:

"A novice designer will benefit most from a design pattern that makes clear the principle behind each design choice and why this is of benefit to the user. A seasoned designer will already understand the principles and won't require as much explication."

Q4. Can a single design pattern satisfy both Developer and Designers without being verbose and heavy?

• Should there be two variations of the same pattern?
• If using a single pattern for both audiences, are there graphic design techniques we can leverage to make the content easier to process?

From Erin:

"Personally, I don't think there should be two variations. What the designers and developers are looking for in a "UI Design" pattern wouldn't be so different in my opinion. If there were two variations, I'm not sure what each of those would include. Could you explain what you mean?

I'm sure we can break down the larger sections (e.g. How) into smaller sub-sections with headers to make information easier to process. Bolding critical information, boarder around the problem section, etc. are all helpful ideas."

From Paul:

"I am of the opinion that we should try to satisfy all users with a single instance of a pattern, but it should be partitioned so that developer-specific issues (for example) are dealt with in a section that other readers can skip over or ignore. I'm not quite sure how the partitioning should be structured - I think we'll have to experiment a bit with this - but I expect we can come up with information categories that address particular audiences and work from there. "

DO:  I agree and the same goes for rationale that might be more interesting to designers but developers just want to know how to implement it.

AB: I'm a proponent of the idea that you set up a design pattern view (e.g. designer or developer) in a profile which customizes the way you see patterns when you log in. I think the only difference would be that some sections would be opened or closed by default ("Extras on Demand"). Even better, it would be great for users to be able to customize for themselves which sections they'd like to be open or closed by default. 

Roles

Having roles will be necessary to help moderate content, workflow, and ensure proper security and abstraction of technical details.

Formal Roles

Unregistered user (Content consumer)

• can view public content on the site including announcements, design patterns, and comments
• can comment on other people's patterns (proper spam deterrent/filtering should be in place)
• can not create content design patterns

Registered user (Content author)

• The basic user account allowing a person to create design patterns, rank users and content.
• anyone can become a registered user and begin writing patterns

Reviewer (Content editor and moderator)

• Someone who has demonstrated an understanding of design patterns and can help edit and polish draft patterns.
• A reviewer can also moderate comments and facilitate discussion.

Patterns Administrator (Content Administrator)

• responsible for administering content, pattern users, and keeping an eye on activity
• can create, edit, and delete design patterns, comments, announcements, users

Site Administrator (Technical Administrator)

• Same as the patterns administrator, but with full site access.
• responsible for the operation of the OSDPL site
• updating modules, data backup, security, etc.

For now, many of these roles will be blurry as many users will play different roles. But as more users register, the above role segregation is something worth aiming for.

DO:  Generally in design we find a user plays many roles -- that's just the way the way the world works.  The roles are good but I'm not sure we should assume we'll be able to put humans in one bucket.

AB: I think the unspoken assumption that is here is that the roles progressively include the permissions (but of course not the restrictions) of the roles before it. So a Reviewer definitely has all the permissions as a Registered User or Unregistered User. I'm going to guess that the Pattern Administrator also has all the permissions of the roles before it, but not necessarily the Site Administrator. Drupal's permissions are additive, meaning that users can be in multiple roles and they'll get the most permissions that any of their roles allows for. 

User Promotion

Once a user has submitted enough insightful comments, they are promoted to Reviewer status by peers. AB: I do think we need more discussion about the qualifications necessary for the initial set of reviewers. So is someone promoted to Reviewer by the other Reviewers (only)? That I think would be best. Should it be a similar procedure to nominating someone to be a committer? That seems to make sense to me.

• If promotion is based on a user's insightful comments, how do we keep track of this?
  • Should we have a user "scoring" / ranking. Once they reach a certain level they are automatically promoted by the system? AB: I believe we could use Fivestar for user rankings. I think it would be better to rank users by ranking their comments and aggregating that into a user rating, however.
• Who gets to rank users? Everyone? AB: I'd limit this to logged in users, limit the # of rankings each user can give to 1 per comment, and rank comments instead of users.
• Are user ranks visible to everyone? look for user ranking moduleDO:  I wonder how this works for Wikipedia? AB:I think it should definitely be visible to logged in users. This will be the way we "reward" contributors for their helpful contributions. It may be that it would help this effort to even display ratings to non-logged in users.

Informal Roles

Author

• Someone who creates a design pattern
• Could be an experienced design pattern author, or someone completely new to them.

Editor

• A Reviewer who helps an Author bring their draft design pattern to a published state.
• They serve as a "mentor", facilitator to an Author if needed.

######################################################

Resources to check out

• Simple review process for Drupal: http://drupal.org/node/156317
• Automatically track contributors? Currently you have to manually select contributors. AB: I think this would be an excellent feature.
• user ranking
• comment ranking
• content ranking
• is there a way to trigger publishing of a document based on votes? (i.e. 2 Reviewers need to approve before a document is published). AB: You may be able to do this with the Actions module.
• design pattern status (complete / incomplete, draft / published) AB:The Workflow module can handle this, and some of it may even be in Drupal core.
• threshold display module (if rank falls below a level, it disappears or collapses). Is it user configurable?
• Organic Groups may be a good way to specify users who can edit small groups of content. AB: Michelle at UC Berkeley is using Organic Groups very successfully, so let me know if you'd like some advice from her.
• Views can also filter content by status... combine this with Reviewers to get "Drafts" and "Incomplete" sections to the site? AB: Using arguments is key for filtering content based on parameters. Check out some of our current views (and/or ask me) to see how they are used.
• how can Reviewers be notified of changes? AB: I think the Actions module could do this.

• How can co-authoring be accomplished? 
  • try Organic groups?
  • How are co-author requests handled? Can it be automated? Should requests to the a Reviewer or an Admin?