Assignments Problem Space

Discussion about Assignments workflows and what components might emerge.

Assignments Definition

The pivot of assignments activities is the submission-feedback workflow, and it is a broad, a broad, ubiquitous process for academic settings. While its particular tasks vary greatly by discipline, context, etc., the basic mechanism remains the same: someone posts a task/assignment description (a bundle which may include instructions, attached files, evaluation criteria, etc.) which becomes the basis for others to make a submission (typically by some due date), whereupon those submissions are reviewed and then returned with some form of feedback. Many assignments involve multiple such submissions (e.g. working through several drafts of a paper).

So framed, the submission/review workflow can even be broader than the classroom: the functionality might easily be repurposed for other academic activities such as submitting research papers or working through the stages of thesis production.

Assignment Content

Two distinct types of content objects are primarily involved: the assignment description bundle itself, and the assignment submission. Assignment descriptions are typically authored by an instructor, and available to that instructor for future courses or other contexts; they may also be linked to particular curricular goals after the manner of portfolios. Assignment submissions, on the other hand, are typically authored by a learner, and they may be the basis for assessment or stored as a sample of the learner's work. The ambiguity in the English term 'assignment' is not helpful here, since it may be applied to both descriptions ("The assignments are in the syllabus") and submissions ("Some of the assignments were turned in late"), so the term assignment will be reserved for the posed description, and submission will be used to disambiguate the content submitted to satisfy an assignment.

A secondary type of object is the rubric for evaluation.  In the simplest case a sweeping point system is used for all assignments, but in principle each assignment may be weighed by its own criteria, and so associating a rubric with (or creating a rubric for) an assignment is a basic (though optional) part of description authoring.

Assignment User Roles

There are three roles which have to deal with the various content objects:

1. The assignment author
2. The submitter
3. The reviewer (i.e. that person who reviews the submission, returning it to the submitter with some sort of evaluation or feedback)

In the simplest case the author and reviewer are the same person, and each submitter is an individual. More complex cases can be found in such things as group assignments, peer review, etc., while assignments might be authored by external users (e.g. publishers) or by a coordinating teacher who does not grade or assess individual assignments (e.g. a teaching assistant might do this instead).

In the context of a particular assignment a user will typically (but not necessarily - e.g. peer reviewing another team's submission while they do the same for you) be either a submitter or reviewer, but not both.  When taken across all assignments they occasionally serve in both roles.  Taken across all collaboration spaces on the system - whether course sites or not - the most typical user may frequently serve in both roles, and this presents its own design challenges for a synoptic or dashboard view.

Workflow stages

With the exception of the initial authoring of an assignment, the submission workflow is an exchange between the submitter and reviewer. From the varieties of assignment object and user interaction arise four main workflow stages, with perspectives varying by user:

Far more complicated workflows can be derived by mixing in more complex arrangements of reviewers and submitters, but such cases can still be understood as more elaborate arrangements of these same basic stages.

Gradebook Integration

Charting assessment across assignments can be problematic by virtue of the breadth of "feedback" that is possible. In some contexts a rubric is not at issue: some freeform, written feedback is what's called for, and there may be no grades at all. In another simple situation each assignment receives some quantitative assessment that plugs neatly into a gradebook chart for calculation. More often the situation is more complex than either of these cases, with a mixture of numerical scores for some assignments and more qualitative measures for others.

Adding to the design and technical challenges is the fact that the recording of a score (or some other measurement) is a basic part of most assignment workflows, but these tasks of recording and tabulating scores represent a functionally complex space in and of itself. So much so, in fact, that most LMSes have entirely separate tools for assignments and gradebook work, respectively. 

So while assignments functionality should not strictly require grading and gradebook services, it must nevertheless be deeply integrated with them when they are involved. 

Problem/Pain

The submission/feedback workflow needs to be extraordinarily clear: users are quite rightly intolerant of any imprecision or error about the status of submissions at any given moment. Currently in Sakai submission status is represented by keywords which do not meet the goal (e.g. "open," "returned," "released"): these terms carry an unfamiliar weight or they are ambiguous, and are buried in a table cluttered with other information.  As such they offer little sense of where things really stand, who did what when, and what can be expected next. 

The experience of assignments differs greatly for submitters and reviewers, both in terms of roles and workflows, and yet their respective representations in the software are often largely the same - presumably for the sake of technical simplicity. But this technical simplicity puts too much cognitive burden on the user. In the face of such differing user perspectives on a common set of objects, a common interface often simply means that one group of users or other is being done a disservice.

Even minor technical issues and errors are a significant pain point for assignments functionality: the mere appearance of unreliability can be devastating to the user experience.  Submissions need to be guaranteed either delivery or very clear communication about the failure and what can be done. Although an ideal system would work flawlessly, even a flawless system would not prevent disputes and complaints about what happened and when: dogs were eating homework long before the Web. Instructors and teaching assistants require better diagnostic information in order to resolve disputes, while learners should be presented with enough information that room for confusion and complaint is minimized.

Assignments (and assignment-like activities) will often be spread across many sites, such that site-based views are not particularly helpful for staying on top of work that needs to be done.  Users do not typically organize themselves one site at a time, they want to be able to organize themselves for, say, the entire week ahead.  A comprehensive "To Do List" of some sort - for both submitters and reviewers (and typically a combination of the two) - is an important need to be satisfied.

Good performance is a critical area of the user experience here, as assignment activity sees very sharp spikes near due dates, and anxieties are already fairly high without introducing slowdowns.  Performance is also a concern for the instructor's handling of large classes, where assignment submissions can number in the thousands. 

Beyond performance, the processes for reviewer handling of assignments need to be greatly streamlined and flexible in their management.

The Dropbox tool currently exists in Sakai (as well as other LMSes) as a tool distinct from assignments.  This separation seems unhelpful. 

Scenarios

Sarah Windsor - Primary Persona

• Sarah wants to create a list of weekly assignments for a semester-long course, and have each successive one become available for submissions at the beginning of each week.
• Along with a brief assignment description, Sarah wants to include a recommended reading list and some guidelines on how to cite references.
• Sarah has a standing assignment which requires first the submission of a thesis statement, then a first draft, and then a final draft. The thesis statement is just for approval and ungraded, both the drafts are graded, and she wants to keep each submission clumped together as a composite submission relevant to a particular assignment.
• Sarah wants the option to be notified when new submissions come in (but probably not for her biggest class), and when she visits the course site she wants to be able to quickly see if there are submissions waiting for her attention.
• Sarah regularly gets complaints from a fraction of her students that they submitted the assignment, even though she can't find it. She often feels dragged into trying to troubleshoot what the student did or didn't click on, and sometimes even feels compelled to send out instructions to make sure each student is using the software correctly. She's certain that some of her students are simply lying, and using uncertainties about the software as an excuse, but she can't really prove this and most often it's not worth her time to try.
• Sarah also coordinates a large multi-section course, each of which has its own instructors or teaching assistants. She needs to have the handling of assignment submissions broken down into an appropriate division of labor.
• Every assignment seems to involve special circumstances for someone or other, and Sarah wants to be able to easily extend deadlines for particular individuals, where appropriate.
• Sarah wants to be able to add links to her assignments to the syllabus and to discussion threads.
• Sarah wants to assign certain tasks to an entire group of students, and provide a grade both for the shared work product and the individual's group participation.
• Sarah wants to be able to showcase an exemplary submission, and to do it anonymously. Next semester she thinks she might just include this submission in the assignment itself as a sample of expectations.
• Sarah wants to connect assignments to particular expected outcomes for her course.
• Sarah is teaching a course for the first time, and wants to see the assignments used by instructors of the course in past terms, most of which she can probably simply reuse with minor adjustments.
• Sarah is happy managing the submission workflow electronically, but when it comes to grading she'd really rather just print them all out and go down to the coffee shop with a red pen.
• For an introductory course Sarah finds that most of her students make similar mistakes, and her corrections and feedback comments are repeated ad nauseum. It would be nice to simply have these stock terms or phrases of feedback available to easily inject where appropriate.

Ed McClellan, Undergraduate

• Ed mainly wants to know what he has to do this week, and when it's due.
• Ed is taking several courses, and he doesn't plan his week one course at a time. He needs to be able to quickly determine which assignment in which class is due first, so that he can step through them. He needs a kind of To-Do list for all his courses.
• When Ed submits an assignment he wants some kind of confirmation that it went through. He's submitted things before when the instructor told him they never got it, and he thinks it was just because they didn't know what they were doing with the software, or the software itself is buggy, and he needs some proof.
• Ed wants to be notified when a submission has been returned with a grade.
• Ed wants to see how well he did compared with the rest of the class on the assignment.
• Ed's particularly proud of a few paper submissions, and wants to store copies of them for himself.

Andrew Devall, Principle Investigator Research Project

• Andrew likes to circulate drafts of papers to certain trusted colleagues in advance of publication and get their feedback.
• The associate dean for Andrew's college is asking for grant proposals, and Andrew wants to submit one for review.
• Andrew wants to post a public research challenge, for which anyone in the world might submit a solution.

Assignments Component Ideas