Simple Text Inline Edit User Testing - Round 1 Results

Summary

8 of 8 users were able to easily edit their text once they discovered the functionality.
7 of 8 users had trouble discovering the inline-edit ability. They eventually found it but after some considerable looking and frustration. At least 6 of 8 indicated they wanted a clearer indication of text editability - either an edit button or a text state that indicated editability (underlining or colour).
At least 6 of 8 users never saw the hover text to tell them to "Click the section name to edit". One user even read it out loud while looking at the interface, subsequently clicked on it, made her edit and then told us she never saw the hover message during post-test discussion.
7 of 8 users said that the edited text going back to display and (for 3 of these users) the Undo link appearing, were indicators that their changes had been saved.
The undo link as an indication that changes were saved is a surprising benefit of that interaction.
3 of 8 users weren't sure their changes were saved.
3 of 8 users mentioned they expected to see all the fields editable on the page and when some weren't it further threw them off from inline editability.

Demographics

User Number	Location	Gender	Age	Role	Tech-savvy	Do you own a personal computer?
User 1	Toronto	Female	19-24	Undergrad student	Medium	Yes, a desktop PC
User 2	Toronto	Male	25-30	Grad student	High	Yes, a laptop PC
User 3 (ER)	Berkeley	Female	over 60	Staff		Yes, Imac
User 4	YorkU	Male	25-30	Grad student		Yes, a laptop PC
User 5	YorkU	Female	19-24	Grad student		Yes, a laptop PC
User 6	YorkU	Female	31-35	Grad student		Yes, a laptop PC
User 7	YorkU	Female	41-50	Grad student		Yes, a desktop PC and a laptop PC
User 8	YorkU	Male	41-50	Grad student		Yes, a desktop PC, laptop Linux
User 9 (DS)	Berkeley	Male	35-40	Faculty		Yes, Dell laptop for work and Dell desktop for home
User 10 (KB)	Berkeley	Female	41-50 years	Staff (Admin Asst)		Yes, Dell

Do you do any of the following and if so how often?

User	Checking email	Instant message	Shop online	Online banking	Internet research	Take class online	Social networking
User 1	All the time	A few times a week	Hardly ever	A few times a month	All the time	Never	All the time
User 2	All the time	All the time	Never	A few times a week	All the time	Hardly ever	A few times a week
User 3	All the time	Hardly ever	A few times a week	A few times a month	A few times a month	Never	A few times a week
User 4	All the time	A few times a week	A few times a month	All the time	All the time	A few times a week	A few times a month
User 5	All the time	All the time	Hardly ever	A few times a month	All the time	A few times a month	All the time
User 6	All the time	All the time	A few times a month	A few times a week	All the time	Never	All the time
User 7	All the time	Never	Hardly ever	A few times a month	All the time	A few times a month	A few times a month
User 8	All the time	Hardly ever	Hardly ever	All the time	All the time	A few times a month	A few times a month
User 9 (DS)	All the time	Never	A few times a month	Never	A few times a week	Hardly Ever	A few times a week
User 10 (KB)	All the time	All the time	A few times a month	A few times a month	A few times a month	Never	Hardly ever

Do you upload files on the web and if so how often?

User	Pictures	Media	Docs to course site	Docs to soc. ntwkg	File to email
User 1
User 2
User 3	A few times a month	A few times a month	Never	Hardly ever	A few times a week
User 4	All the time	A few times a week	A few times a week	A few times a month	All the time
User 5	All the time	All the time	All the time	A few times a week	All the time
User 6	All the time	A few times a month	Never	Hardly ever	All the time
User 7	A few times a month	A few times a month	A few times a week	Hardly ever	All the time
User 8	Hardly ever	All the time	All the time	Hardly ever	All the time
User 9 (DS)	A few times a week	A few times a month	A few times a month	A few times a month	A few times a week
User 10 (KB)	Never	Hardly ever	Hardly ever	Never	All the time

Interaction Notes

User Number	Task 1 - Remove section	Task 2 - Edit from LEC to DISC	Task 3 - Edit from L to P	Task 4 - Edit from 104 to 105	Task 5 - Undo	Task 6 - Redo	General comments
User 1		First reaction was "hmm, I don't know" but upon seeing the yellow highlight, user seemed to have felt intrigued to click on the lecture title. Once it turned to a textbox, the user quite easily highlighted the text "LEC" and changed it to "DIS".	Knowing that inline edit is possible, the user changed the text without hesitation.		The undo link appeared as soon as a change was made. It was very clear and obvious to click on this link to undo.	Same as Undo.	Overall, user found it very straight-forward to make changs inline. Discoverability needs improvement.
User 2		"Do I have access to this site? What do you mean... I can administer the website? I would stop the web server, and update the HTML page, then restart the web server. " User was then told he can make the edits from this page. User examined the page and clicked on a title, then tried to right click. User got out of the edit mode, then clicked on the title again to go back into edit mode. "Oh, I can edit right here. I didn't know I can do that."	"Well, that's easy. Just do the same thing."		Undo	Redo	User liked the experience of inline editing, but said "the question is when do I save? How do I commit the changes. What happens if I navigate out? Are the changes committed?"
User 3		User had never used section info and seemed confused about "where" she was. Starts by saying "I have no idea how to do that." She mouses over the field and the yellow highlight and pop-up show up. She looks at all the navigational choices in Course Tools. She tries to click on Section Info in this section, but it doesn't work. She mouses over the field again and the pop-up appears again, but she doesn't see it. She goes to Help and Daphne tells her that won't have anything to help here with this. She says, "I don't see anything here that's obviously what I want to go to. I want to make a correction." Then she clicks to edit and discovers it. While editing, she accidentally erases everything and uses "CTRL-Z" to undo her mistake. She says she finally figured it out through exploration, thinking, "if it's something I want to do something to, maybe I should go to it."	User easily and quickly accomplishes this task.		For this user the Undo link was in the wrong place, underneath the field (probably due to low screen resolution). She doesn't find Undo herself (even when Daphne asks her if there is anything on the page she could use if she forgot what she'd done), but seems to use it easily after Daphne tells her about it and later even says "I should have seen it there (underneath the field)."	User able to quickly complete task; she played with Undo & Redo in the previous task. She asks, "Do these (undo & redo) all stay until you get out of there (the current screen)?"	"It would have been nice to have the (editable fields) be underlined in blue (like the links)."
User 4	A little time needed to find the section to delete, but once found immediately used delete button - its function was obvious.	He did not know how to edit section names at first; moved mouse over various areas, but when the section name highlighted on rollover and the popup help appeared he immediately clicked and edited. "I would like to see an edit button, it's not clear how to edit the text."	Had no difficulty in quickly completing task. He knew immediately how to edit name field.	An easy and quick edit.	He used undo button with no hesitation.	He immediately used redo button with no issues.	His only significant obstacle was the first location of an editable text field - after that everything was effortless.
User 5	Easily identified the "red cross" button to delete a section.	Did not notice the rollover highlight while searching for "edit" button. When she noticed the highlighted section, she clicked on it and made the necessary changes.	Completed the task by applying the same procedure.	Completed the task by applying the same procedure.	Changed the lecture section title back by clicking the highlighted area. But then she saw the "undo."	Used the "redo" icon.
User 6	Clicked the "delete" icon.	Did not notice the rollover highlight. She was searching for how to edit by moving a cursor around. Then she identified the highlighted box, clicked on it, and made changes. She saw that change was made by clicking outside.	Completed the task by applying the same procedure.	Completed the task by applying the same procedure.	Used the "undo" icon.	Used the "redo" icon.
User 7	Didn't notice the "delete" icon at once.	Had trouble discovering an editable field. She searched the screen by clicking Options. She'd like to have a visual cue to see that the box is editable.	Completed the task by applying the same procedure.	Completed the task by applying the same procedure.	Used the "undo" icon.	Used the "redo" icon.
User 8	Looked at "remove" icon, clicked it. He'd like to see a conformation that the task was successfully deleted.	He didn't see the pop-up when it appeared. He didn't realize at first that the area is clickable. It would be easy to make changes if the "edit" icon was visible.	Completed the task by applying the same procedure.	Completed the task by applying the same procedure.	Used the "undo" button. He was curious about how many times the change can be undone.	Used the "redo" icon.
User 9 (DS)	Completed this task easily.	It took him awhile to discover the inline edit. He was looking for an edit link and mentioned that it would be nice if there was one -- or perhaps instructions that are there all the time. He said he'd normally go to Site Info to make these kinds of changes. He finally noticed the hover text and and easily made the change. He clicked in the box where he wanted to make the change. Since we auto-select the text he couldn't just start typing.	Completed the task with ease.	Completed the task with ease.	Didn't use Undo. He later said he never noticed it. He easily made the edit manually.	Made this change manually. He never had a Redo icon since he didn't use Undo.
User 10 (KB)	Completed this task but suggested there be a confirmation message before removing the section.	She completed this task with ease. She noticed the rollover text quickly. She also noticed the undo pop quickly and played around with undo and redo. She wondered if she could edit everything on the page and checked out other fields on the page.	Quick change although she was a bit apprehensive because she knew this to be "official information" (see comment).	Completed task with ease.	Quickly used undo	Quickly used redo	She suggested column headers give information about the edit availability. After edit, she was clicking way away from the edit box like she was looking for a "safe zone".

Post-test Questionnaire Responses

Question	User 1	User 3	User 4	User 5	User 6	User 7	User 8	User 9 (DS)	User 10 (KB)
How easy or difficult was it for you to realize you could edit directly on the page (inline)?	Neutral	Difficult	Neutral	Very easy	Neutral	Difficult	Neutral	easy; it was accedental, nice if edit option here or instructions that were there all the time	Easy
How easy or difficult was it for you to edit text inline?	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy	Easy	very easy	Easy
How easy or difficult was it for you save your edits?	Very easy	N/A (didn't know if it was saved)	Very easy	Very easy	Very easy	Very easy	Easy	very easy	Easy
How easy or difficult was it to know your edit was successful?	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy	very easy	Easy
How confident were you that your inline edit had been saved?	Very confident	Very difficult	Very confident	Very confident	Very confident	Very confident	Confident	very confident	Easy
How easy or difficult was it to undo your changes (edit)?	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy	Easy	very easy	Very Easy
How easy or difficult was it to get your changes back (edit)?	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy	Neutral	very easy	Very Easy

Post-test Question Responses (verbally asked)

Question	User 1	User 2	User 3	User 4	User 5	User 6	User 7	User 8	User 9 (DS)	User 10 (KB)
1) Did you see anything that told you you could edit the text directly on this page? How helpful was it to you?	(Question wasn't clear) I think you should've had an underline or make it look like you can modify it so you can see at a first glance you can modify it. It just looked like a fixed text. I didn't know until I moused-over it. I would say it's medium. Should have "edit" link in place of undo/redo link before any changes are made.	(Question wasn't clear) No, I did not. No Visual cue. I did not know that until you told me.	"I felt like I was staring open-mouthed, stupidly." A really slow process of elimination. Didn't see a visual indication. Then Daphne pointed out the yellow highlight and user confirms that is what told her she could edit. Though the pop-up came up several times, even when she was looking directly at it she said she didn't see it.	I saw the popup - it was very helpful, told me what I needed to do to edit.	No, I didn't. I thought it had to be somewhere. So, I started looking for editing option by clicking on the title.	I didn't see it. I started clicking everywhere on the screen until I saw the highlighted area.	No. It was obvious.	No.	Rollover text instruction box. After we asked he noticed this question, he noticed the Undo link and asked if that was there originally.	"Once highlighted you realize you can change" [fluid:Although it took her a few times to see the highlight]. The mouse indicator turning to hand also told her she could do something. And if you got the tooltip that helps too.
2) Did you expect that clicking the highlighted area would edit the text directly on this page? If not, what did you expect?	I would expect it would take me to a link that's associated with it. I wouldn't click on it if I wanted to edit.	I would expect it would just highlight the text with no inline editing. But inline editing is cool. I would probably know if I played around with it a little more.	Once she saw it. (She rolled over it 3 times before seeing the highlight.)	Yes, but only after reading the pop-up help.	Yes, I did. However, I didn't notice the pop-up.	Yes. I did not see the pop-up. Actually, I usually don't read pop-ups.	The color of the text box suggested that the area is editable.	I didn't notice the pop-up, but the highlighted area was very helpful.	Didn't expect it but not too surprised. It's like renaming files.	She didn't expect to be able to edit the section names because she works with DB2 and knows you can't play with section names. So, it was the context that confused her. She had those expectations because of her scheduling experience but though faculty might not.
3) Did you notice any indication that your edit was successful?	I know it's successful, because it shows what I edited. But I don't think there was any specific cues that showed it was successful.	No. Doesn't say anything. It doesn't say "Yes, you have saved it"	She comments right after the test, "I still don't know whether my edits are saved, and seems to differentiate a "successful" edit from a saved edit. She said she new her edit was successful, "When "Undo" appeared and "LEC" changed to "DISC." Then, "it wasn't clear that it was saved." Doesn't say to me that each time you do it (edit) it is saved. (Note: this user had a smaller screen and the "Undo edit" link appeared below the edited field, so it was a bit hard to see. Because of this, Daphne showed it to her at one point. When Daphne asked if she would have seen it if it was in the correct place, user even said she should have noticed it underneath.)	When he clicked on the area outside the text box and the box went away he knew the text was saved. [fluid:He had some trouble describing explicitly the change in state that occured on clicking outside the box that cued his awareness of the editing change - he did not at first realize that is was the text box frame dissapearing.]	I saw that the change was saved when I clicked the outside area.	I saw that my change was saved when I clicked outside.	Yes, when I clicked outside.	By clicking outside of the highlighted box did not completely confirmed that my changes were saved.	When I clicked outside, it went back to display text	The undo / redo icons.
4) Did you think there was a way to "undo" your edit? Did you think there was a way to "redo" an edit that you'd undone?	Yes. It was obvious to me.	Yes.	Yeah, it said "Undo."	Yes because you can see the undo and redo buttons.	To undo the change, I first clicked the highlighted area to edit. But, then, I realized that I could use the "undo."	Not at first, but then I saw the "undo."	Yes, because it says "undo."	Yes.	He didn't see the undo until after he completed the tasks. He suggests that it start as an "Edit" link that changes to the appropriate action (undo / redo)
5) Did you notice an "undo" or "redo" link?	Yes. DUh.	Yes. Bien sur.	It seemed obvious that you could go back and forth.	Yes.	Yes.	Yes.	Yes, it was very obvious. It's a great feature.	Yes.	The redo was red when he saw it (because of a visited link bug). He said he liked it being red because it jumped out at him. The undo link looks like all the other links on the page.
Did you enjoy using inline edit? Is there anything you would improve?	Yes, I would like to see the edit sign.	I loved it. I would improve... I didn't even know if you could do it until I hovered around it. I would like to change the font to something else. Rich text would be nice.	It was easy once I realized (how it worked). If i was a harried instructor (this type of quick editing functionality) would be good. I guess so. Would like to clearly indicate that the inline editable fields are "live," just like the "Assign GSIs" & other links. Even though we explained that those links go other places, she thought it would be helpful even if it was just blue. Comments: "It just looks dead while the others (links) look live." It would be nice if there was a page for instructors that explains this (along with other features that are available, e.g. a help page). She wondered if any of the other fields on the page (e.g. location) were editable and Daphne explained that ideally they would all be. She thought it strange that these fields looked the same as the inline editable fields, saying they are all "dead as a doornail." She thought that instructors would expect to be able to make changes to all the fields on the page.	It was new to me, but easy to use, change is immediately apparent. There should be an edit button that when clicked on opens the editable field for editing so that what is editable and how to do so is obvious.	It was pretty user-friendly. I would provide some indicator to make sure that the area is editable.	It is a nice feature and it was easy-to-use. I would add something that would suggest that the area is editable, like "edit" button. I am also wondering if it's possible to restore the information I deleted when completing the first task.	Yes. It was easy to figure out how to edit.	It's a good tool. I would suggest to improve functionality of the editable filed, like an underlined text that would suggest it's an editable field, or like an "edit" box.	joy in ease of use	"I did in the clerical high world "that was great"". She suggested an open dialog box saying you can edit names by pointing and clicking.

Potential Design Improvements (based on testing)

Discoverability of the Pop-up: Better distinction of the popup. Brighter color, border
Discoverability of Inline Edit: Showing a notice at the top to let the user know that Inline Edit is available with [fluid: ] don't show me this message again; or, if only some fields are editable: a small edit button beside each editable field or a unique text state (color, underline) for editable fields (Note: a disadvantage to the Edit button is unless they want to edit where we've already put the cursor, users may need to perform two clicks to put it where they want it--once on the edit button and once to place the cursor in the proper location between the letters).
When the inline editable text is moused-over, an icon appears beside the field to let the user know they can act on that item
Confirmation about Saving: A transient "Saved" message - make it configurable whether or not this message appears
Make the roll-over highlight look like an editable textbox (i.e. Google Calendar) vs. file system metaphor - Daphne to think about it more
Make sure the [fluid:undo] [fluid:redo] links not wrap to the next line