Task Order 0003 Questions and Thoughts

General Information

Daily Stand-Up: Noon Eastern time, https://zoom.us/j/3935010675


  1. Nov. 30 - Dec. 18
  2. Dec. 21 - Jan. 14
  3. Jan. 18 - Feb. 5 (final development sprint)
  4. Feb. 8 - 26

Some Documents:

Evaluation Plans:

  • Planned: 2 days on-site (each site)
  • The evaluations will be carried out which hardware brought in by the PGA team (i.e. not installed on the site's own systems).  These will likely be desktop systems, though I don't think laptops are out of the question. I think the idea is to NOT use tablets/phones. They will do their best to mimic the systems that are in place at the site (e.g. operating system, etc).


First Discovery Tool Construction Sprint Requirements

 from: https://docs.google.com/document/d/1p88RZghb4sDjn0aUi8bRMo5booy4rC-Di11t2pwQHBs/edit

  1.  Integration with the GPII Preferences Server
    1. Requirement All-2: KeyToken & Save
      "The tool(s) must, at the start of the process, create a KeyToken (e.g., NFC, USB, etc.)". Currently, the first discovery tool stores a local version of the user's preference set in their browser (using a cookie or other pluggable data store). At the end of the process, the preference set is saved to the GPII preferences server. The system can be restarted using the ctrl-option/alt-r keyboard command.
      1. This may require a new OAuth 2 client credential grant to be added to the Authorization Server, supporting modification of preference sets that haven't yet been linked to a GPII user account (for example, because the user hasn't yet created an account, or may choose not to)—this will need to be done in coordination with the APCP development team
      2. Will this work package include the development of a new module that will allow a user to  provision a key token (such as an NFC tag) automatically?
      3. If so, how will it be implemented? What types of key tokens will be supported (e.g. NFC, USB stick, YubiKey?)

Notes from the Dec. 9, 2015 requirements meeting:

  • Discussions about the need to start auto-saving preferences at the start of the process resulted, ultimately, in the conclusion that this requirement is not a priority for these pilots, and so will (likely) not be implemented.
  • We did discuss the desire for users to engage in activities beyond just the actual FD process, requiring application of preferences to the system, requiring transport of the key token somehow. There was back-and-forth, and no decision yet, but NFC seemed to be the favoured option. Creation of "dummy" GPII accounts was also proposed. Ben will research the options, then a decision will be made.
    • It was pointed out that these considerations should be made in conjunction with considerations around evaluation; i.e. you need to think about what you want to/can evaluate, to determine what you need to implement.
  • Still not clear whether or not evaluations will include "later" adjustments of preferences using a different tool.
  • See Google doc (linked above) for more notes.

Site Survey and Evaluation Plan

  1.  When will the functional tests (QA) take place? What is needed for this? Will there be time to make fixes/changes before the usability testing starts?
  2. Is the tester from the Technical Development Team also part of the "Tool Evaluation Team"? If not, will he be available to help with the functional testing?
  3. Who will perform this "construction" goal: "Where feasible, perform any ICT and Assistive Technology enhancements necessary at the chosen sites"?
  4. We should include an evaluation of the ease of integration in the overall usability testing. Need to consider usability test questions for integrators.
  5. How are we assessing whether or not a site meets the criteria for covering the necessary range of user needs? 
    1. We should add a survey/interview question that prompts our prospective sites to tell us more about how their users use technology–what kinds of sites and content do they most often use?
    2. We should add a survey/interview question that prompts prospective end-users (or staff/assistants at potential sites) to tell us more about users' needs (there is mention of a "pre-survey" to do initial research on the test population - but it's not clear at what stage this would happen - this should take place prior to selecting a site). What barriers do users most often face when using the technology at this site? What assistance do they ask for? 
    3. This would also inform, to some extent, the question of which site-specific preferences to include in the FD tool. However, we should err on the side of including "too many" preferences, to ensure that we don't leave any out that a user may want to try. 
  6. Are there other systems that the FD Tool will need to be integrated with (such as a learning management tool)? If so, should we ask questions about how they will be integrated and what kinds of workflows will be involved?
  7. The User Experience Testing mentions both the First Discovery and a tool for refining preferences after first discovery is complete ("Explore tools"). Are you thinking of an existing tool, or have you been working on designs for something new?
  8. The User Experience Testing mentions two tests at each site and indicates that the second test (2 - 3 weeks after the first) may include changes made to the tools. Who will make these changes (e.g. integrators, dev team, evaluation team, etc.)?