Date Time Picker Paper Prototype User Testing - Round 2 Results

Summary

4 of 5 realized they could type or use the mouse to pick a date and time. One user didn't think to type at all, since the calendar popped up right away.
5 of 5 knew they could just click outside the calendar or time picker to close it
4 of 5 of the users commented that the time picker was intuitive or very straight-forward or (even) pleasing
2 of 5 users wanted to be able to enter fewer things in the time picker when it had no defaults (e.g. it should assume :00 or PM)
3 of 5 users thought Accept Until Date should change to at least match Due Date
3 of 5 users clicked on the calendar icon when it was available. They didn't feel the icon was necessary, as long as the calendar pops up when they click inside the field.
2 of 5 users preferred the combined date and time picker to the separated picker.

Demographics

User Number	Location	Gender	Age	Role	Tech-savvy	Do you own a personal computer?
User 1 (EK)	Berkeley	Male	41-50	Staff	Medium-high	Yes, Mac & PC
User 2 (BW)	Berkeley	Male	35-40	Student	Medium	Yes, MacBook Pro
User 3 (TA)	Toronto	Male	19-24	Student	Medium	Yes, Windows Laptop
User 4 (JL)	Toronto	Male	31-35	Staff (researcher)	Medium	Yes, Windows Laptop
User 5 (LH)	Berkeley	Female	51-60	Faculty	Medium-Low	Yes, Mac Power PC Laptop

Do you do any of the following and if so how often?

User	Checking email	Instant message	Shop online	Online banking	Internet research	Take class online	Social networking
User 1	All the time	Hardly Ever	All the time	A few times a week	Never	A few times a month	A few times a month
User 2	All the time	A few times a week	A few times a week	All the time	All the time	Hardly Ever	A few times a week
User 3	All the time	Hardly Ever	Hardly Ever	A few times a week	All the time	Never	A few times a week
User 4	All the time	All the time	A few times a month	A few times a week	A few times a week	A few times a month	A few times a week
User 5	All the time	Never	A few times a week	A few times a month	A few tunes a week	Never	All the time

Do you upload files on the web and if so how often?

User	Pictures	Media	Docs to course site	Docs to soc. ntwkg	File to email
User 1	A few times a month	A few times a week	A few times a week	Hardly Ever	All the time
User 2	A few times a week	Hardly Ever	A few times a month	A few times a week	A few times a week
User 3	n/a	n/a	n/a	n/a	n/a
User 4	n/a	n/a	n/a	n/a	n/a
User 5	Hardly Ever	Hardly Ever	A few times a week	Never	A few times a week

Interaction Notes

User	Scenario 1 - Enter an Assignment into Learning Management system. (separate date & time picker with default date and time)	Scenario 2 - Enter Course Modules into Learning Management System (combined date & time picker with no default date or time)	Post-test questions (if recorded separately)
User 1	Note: this user had an early version of the prototype with no default date and time he tabbed between fields, but always clicked on the date & time pickers except when entering the '59' he had no trouble with time picker said he would expect the minutes to default to :00 would like a default time in open on if he didn't care when it opened and would just like it to open NOW would expect to see different dated for open and due though he understood the colors used in the calendar realized if he clicked outside the calendar it would disappear he didn't use the drop-down to select the next month, used >> said he'd expect accept until to change (to at least match due date) if he changed due date he said he'd never seen a drop-down for years, and didn't think he'd ever use it as he doesn't imagine entering information that would cross years he said he'd like it to default to :00 and he'd know to click to change them if he wanted to Suggested the use of (noon) and (midnight) in parentheses next to 12am/pm said he would click 55 or type 59 since 59 isn't on the list	realized he could click date tab to go back to the date "I'm more of a typer than a clicker." clicked outside the time picker to close it. likes the fact that in this screen he can set the dates and times of all modules at once -- doesn't have to go into individual screens -- would like this type of list of assignments in Gradebook didn't seem at first to notice the change in highlight in the time picker thought the time picker was intuitive and that the shading of the different time sections made them obvious. "The blocks and shading change makes it very straight-forward." said date picker "seems pretty obvious to me. it's easy to move in the month(s) and select the date." wondered if he just clicked in the field if the calendar would come up (we said yes) might just type in "1/19/09 6am" if he found that that worked while in the calender, said he might try to change the month by using the right or left arrows (we explained that wouldn't work in the current design) thought the today's date was sort of redundant, at least if you're in the current month "In some ways, I like the fact that date and time are in the same field." thought it was always important to have the student view would really like Sakai to go faster > biggest pain point - makes it hard to use Gradebook Gradebook feedback: wishes GSIs could import scores into Gradebook -- can't do it now unless they are made head GSIs (and they don't want to do that with everyone) would like the class average in the spreadsheets to go to several decimal points, not just 1. is really annoyed at applications that just put in the current time (e.g. 4:12) as the default; rounding to the hour or half hour would be better said he would never type "next thursday" unless something told him it was possible thought a space should automatically be put after the date if you chose a date (I agree) suggests auto-population of Accept Until date for a day later than Due Date, or setting defaults for this kind of thing in preferences.
User 2	Note: this prototype had inconsistent help text, displayed only for the date, which changed to an alternate presentation of the date only after a non-default was selected (it should have always had that alternate date presentation). Clicks always= instead of types (including moving from date to time field) Re: the time picker "I like this idea. It's surprising and pleasing. It's intuitive because all the numbers are there. There's no scrolling needed, as you have to do with Google calendar. Everything is just in a small box you can tap, tap." Skips over the "PM", says he hopes it will fill in :00 Asks if Accept Until date changes to match Due On, saying it should at least be the same as, if not after, Due On. Suggests surveying professors to determine how far apart the dates should be. Assumes he could just type in the text for 11:59 since the options are only 11:55 and 12 Wants to enter "1159" with no colon, as he often uses his 10 key pad to type numbers (though now he's working with a MacBook Pro that has no 10 key pad) When asked, explains the meaning of all colors except the dark blue background (currently selected date) and gray background (date selected in another field), which he asks the meanings of. He mentions that there are no color keys so he's making a leap about the meanings. When asked, says something that would show that there were two fields set to the same date (e.g. a border) would be helpful. When asked how he'd close the calendar if needed, he initially says, "I have no idea" but one second later says he'd click outside the box. To change the month, uses >> but says he'd use the drop-down if his target was more than one month away. When he changed from 2007 to 2008, says he initially thought the gray box indicating a selected date in Aug 2007 should be in the same place on Aug 2008 calendar.	Says, "In Google Calendar, I'd write 19 Jan 09 6pm (or 18:00)." Facilitator explains that if that wasn't acceptable, you would get a message with an acceptable format. User says he may also try the slash, then would follow the format in the example. Also, that if he was encountering lots of issues with format, he'd go back go clicking on the calendar. Liked the fact that there was a time tab, but thought it would be even better if he didn't have to use it to move to time, that it happened automatically (we explained that's how it would work).	About time picker, "I'm so pleased to see the box in that format. It's easier than Google. How often does someone get compared positively to Google?" 1) I didn't see the Close X right away but would have clicked outside the calendar (to see something underneath it) 3) (Today's date) Would go to the bottom and click on today's date. Excellent! 4) (Time picker) "I'm very pleased that all the info is right there without having to scroll up or down." "These days it's natural to see it delimited by 5s, but you can still type." He can only pick increments of 5 and not type in his Palm 650. He knew he could move from date to time via tabs. Says he likes this time picker "even better than the scroll wheels on the iPhone and iPod Touch."
User 3	Note: There were no close "X"s in this prototype, as we were testing whether they were necessary Calendar appeared when the user clicked on the field. User commented that both of the designs should have the calendar icons or both not have them. He would be okay without the calendar icons, if the calendar just popped up as soon as he clicked on the field. "The only reason I clicked on it was because it was there." Preferred the date and time pickers combined in one field. With the separate fields, he assumed it would "take me to the time thing" when he finished picking the date. When he saw that the calendar just closed and did not take him to the time field, he was disappointed. "I guess it's not automated" When asked to pick 11:59, he commented "there is no option for 59. I would probably pick 55." He thought 59 was "too specific" He prefers to use the mouse than to type. "If it gives me [fluid:the date and time picker] as soon as I click, I'm more inclined to pick than to type."	Would click on the calendar icon There should be more an option in the "time slot" where you can pick either 24 or 12 hr clock Recognized the different treatments for today's date and previously selected date When presented with date/time separate (without calendar icon), the user was confused and didn't know how to open the calendar
User 4	Note: There were no close "X"s in this prototype, as we were testing whether they were necessary User is told the calendar pops up right when he clicks inside the field. He would click the date instead of typing, since it's already presented to him. Would then click on the time field, and pick the hour, minute (00), and AM. Would do the same (picking using mouse) for the rest. Would pick 55 for 11:59PM.	Would type inside the date field. Would be "super frustrated" without the example. Would type it in the format suggested by the example. Would've expected to type in digits for the months.
User 5	Note: There were no close "X"s in this prototype, as we were testing whether they were necessary User starts by doing a lot of clicking, then as test progresses says she'd do more typing. Later explains she prefers not to click as she has carpal tunnel in her thumb. #1: She asks "What is Open On"? (We explain it means 'available'.) #2: When entering the time, she clicks 5, :00 and then wants to skip everything else (because it's already filled in with a default time), looks for save, then says she would just click someplace else. When asked, she said having to do that "wouldn't confuse me at all." #3: Would click on 55, then put cursor after the second 5, would backspace and type the 9. Mentioned she had been "trained" to click off things to close them. #4: Eventually opens one of the calendars, but before she does, says, "They are right here. I would never do that. I might look to see if there were two weeks in between." #5: Says she would select the 8 in the text field and change it to a 9. #6 & #7: Would click to the right of the character she wants to change, then backspace.	Was confused with the scenario at the start: "For some reason you want modules available at a given time?" Commented that she would love a button to make them all the same (seems she really just wants them all available now) Says she'd probably type, as "I'm not really a GUI person. Once I know the format (e.g. she learns it from the GUI), I may go back and forth. It's nice to have both options available." "Aha, the calendar shows (what's been entered). That's comforting." "The calendar is handy to see how far things are apart, but I'd probably just plan in advance (rather than pick dates from the calendar). I think I could do it much more quickly." "Every time you use Travelocity you use this...it's consistent with what you use in life...it's frustrating when things are different."	#1: I would click outside the box to close it. Might want to grab and drag it out of the way too, it'd be great if you could drag it over." #3: She noticed Today's Date is at the bottom after thinking about it. Says she would probably be entering due dates in the future when using the date-time picker. Says she might want to know the relationship of today's date and what she's doing. Might page, or use Today's date when dealing with dates far in the future. #4: (time picker) Ideally the picker wouldn't obscure the other fields, but I can drag.

High-level summaries/findings by user

#1

Is a combined date-time field better? Should design pattern recommend a combined date-time field because it is more efficient for users and requires only one required field label instead of two (if date & time are both required)?
Consider using contexts people aren't familiar with for user testing of components so they don't get caught up in details about the apps we aren't trying to test.
would like the time to default to :00 - he says he'd understand he could change it
is the yellow highlight even needed? He didn't notice that the highlight changed. Is the gray background too inactive looking?
should the active tab be white instead of gray?
he's annoyed at applications that just put in the current time (e.g. 4:12) - make sure to recommend good defaults and remember that context matters in terms of how this would work
said the time-picker was very straight-forward
he said he's a typer (instead of a clicker--but he actually did all clicking except for the :59), and would tab between fields
suggested having noon and midnight in parentheses

#2

Thinks the 5 minute increments are very natural - his Palm won't even let him do anything else
Would skip entering PM because he'd hope it would be filled in (this was when there were defaults and it was already filled in)
He suggests that accept until date should be automatically set to not be sooner than due date after due date is set (because that wouldn't make sense).
He would like to use the 10 key number pad which has no colon, so he'd like to be able to type the time without a colon
He would like the system to recognize/parse any date format
he wanted styling of the dates on the pop-up calendar to tell him when there are two different events on the same day - suggested a border. When asked, he also commented that there was no color key to tell him what the colors meant.
he didn't see the close "X" but says he'd click outside the box to close it
noticed both the >> and the months drop-down, and when scrolling to the very next month he wanted to use >>
at first thought the gray box representing the selected date should remain when he switched the calendar to a new year, then changed his mind.

#3

Would click on the calendar icon
There should be more an option in the "time slot" where you can pick either 24 or 12 hr clock
Recognized the different treatments for today's date and previously selected date
When presented with date/time separate (without calendar icon), the user was confused and didn't know how to open the calendar
Calendar appeared when the user clicked on the field. User commented that both of the designs should have the calendar icons or both not have them. He would be okay without the calendar icons, if the calendar just popped up as soon as he clicked on the field. "The only reason I clicked on it was because it was there."
Preferred the date and time pickers combined in one field. With the separate fields, he assumed it would "take me to the time thing" when he finished picking the date. When he saw that the calendar just closed and did not take him to the time field, he was disappointed. "I guess it's not automated"
When asked to pick 11:59, he commented "there is no option for 59. I would probably pick 55." He thought 59 was "too specific"
He prefers to use the mouse than to type. "If it gives me [fluid:the date and time picker] as soon as I click, I'm more inclined to pick than to type."

#4

Was presented with the design with date and time in separate fields first
Would type inside the date field. Would be "super frustrated" without the example. Would type it in the format suggested by the example. Would've expected to type in digits for the months.
User is told the calendar pops up right when he clicks inside the field. He would click the date instead of typing, since it's already presented to him.
Would then click on the time field, and pick the hour, minute (00), and AM.
Would do the same (picking using mouse) for the rest.

#5

users says she would type but may type or click depending on efficiency of interaction
the date picker is consistent - it's a known metaphor that she's familiar with from sites like Travelocity
user had an injury to her thumb which made her not want to click - she now wants to type as much as she can
thinks she would be prepared with modules' dates before going to Sakai, and would just want to enter (not analyze) them. She does think she may find the ability to see visually how far apart dates are helpful though.
she liked that the calendar display would change based on what she's typing. She says,'that's comforting.'
thinks it would be very helpful to drag the calendar
one possible feature: connected date pickers remember the month you were on and use it to figure out where to start the next date picker (this helps with entering a lot of related dates - e.g. for the semester - prior to when they occur)
she was able to figure out what the aqua border on a day meant when she say the color mapping to "Today is" at the bottom of the calendar. It might have been easier for her to figure it out though if she really knew what today was (we didn't say in the scenario).

Post-test Questionnaire Responses

User	Selecting a date	Selecting a time	Recognizing option groups in time picker	Recognizing selected dates and times	Tabs	Changing previously selected date
User 1	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy
User 2	Easy	Very easy	Very easy	Easy	Easy	Easy
User 3	Very easy	Very easy	Very easy	Very easy	Very easy	Easy
User 4	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy
User 5	Very easy	Very easy	Very easy	Very easy	Very easy	Very easy

Potential Design Improvements (based on testing)

Make whether or not a calendar icon appears a config option. The default is to not have the icon. When a calendar does appear, it should be inside the text field and toggle the calendar open & closed like the one on Kayak.com.
Change prototype/storyboard so that dates which cannot be selected are gray. Use same or similar gray for numbers in the previous month, but don't show a mouseover highlight because it isn't clickable.
Add "noon" or "midnight" above the text field when it is 12am/pm.
Remove yellow highlight from time picker except when it's empty. Ask for feedback from design team whether it should be removed when it's empty as well if it makes users think they have to fill in every piece when they really don't.
Remove close X.
Allow defaulting to :00 if the user only types, e.g., "5pm" or picks 5 and PM.
Change the active tab to white instead of gray so it looks more connected to the content area. Inactive tab should be changed to a flat gray.
Ensure that date/time picker updates with as things are changed in the text field and vice versa.
Add a configuration option as to whether date picker only goes forward in years in drop down or whether it can go into past as well - erin to spec out how this would work
Date picker should be "smart" and change dates in other related fields (e.g. Due On should be = or before Accept Until).
Consider a feature where implementers can connect date pickers in an app, so they remember the month you were on last and use it to figure out where to start the next date picker.
If user testing is done in the future, test adding the border back when two fields have the same date to indicate this.
Design pattern:
- Ensure important information isn't hidden when the date picker pops up
- recommend good defaults, and remember that context matters in terms of how this would work
- recommend when to use a combined date-time field – often better because it is more efficient for users as they don't have to move from a date to a time field, it's simpler, and requires only one required field label instead of two (if date & time are both required). A reason to separate them would be if date or time is required and the other isn't.
- talk about when calendar toggle should appear in the text field
Consider using contexts people aren't familiar with for user testing of components so they don't get caught up in details about the apps we aren't trying to test.