Decapod Workflow

Owned by Jonathan Hung
Last updated: Jun 15, 2011Version comment

This workflow needs to be updated with respect to the v1.2 User Interaction Workflow. -JH. Feb 3, 2010.

Introduction

This document describes the user interaction with the Decapod system, and technical back-end activity that occurs in Decapod. Structuring this information as a start-to-finish workflow is a way of portraying Decapod helps serve as a work plan for both design and development, as well as provide a conceptual view into how Decapod functions as a whole.

If you are new to Decapod, this document may be heavy on details, but gives a good description of how the system will work, and where we are going in our work.

Digitization Process Diagram (Draft 2/August 6, 2009)

The following is a graphic illustrating the high-level workflow. More detailed descriptions follow further in this document.

Download PDF of digitization process diagram, Draft 2 (August 6, 2009)

Download PDF of digitization process diagram, Draft 1

Overview of Workflow for User and System

  1. Start Decapod
  2. Assemble hardware if not a fixed installation (Wireframe: Camera Setup Wizard)
  3. Calibrate (Wireframe: Camera Setup Wizard)
  4. User begins capturing (Wireframe: Detailed and Thumbnail View)
    • Capture to memory card or direct to USB
    • Download to system
    • Pre-process for user presentation:
      • image scale / downscale
      • image crop
      • normalization
    • Present pre-processed results of page spread stereo pair (Wireframe: Detailed and Thumbnail View)
    • In background, Dewarp and page splitting is being applied to unprocessed images
    • Pairs are merged and spreads are split.
  5. User manages individual pages for Exporting (Wireframe: Furture work)
  6. Indicates they want to Export (Wireframe: Menu Bar)
    • Page Segmentation and Document Analysis occurs. Can take a long time.
  7. Output generated to PDF. (Wireframe: Furture work)

Start Decapod

  • Load Decapod (doesn't matter if fixed/mobile system, or if booted by LiveCD or installed)
  • If user using Ubuntu Live CD, user should be prompted for a storage location.
  • User presented with options to start a new project or continue an old one.
  • If starting a new project, user will be prompted for Metadata*
    • who, where, when, general comment field
    • Motivation: Sometimes detailed metadata is not available (old text, obscure indexing, little/no instruction by staff), therefore simple fields that any user can answer is all we're requiring. More detailed metadata can be filled in during Remastering.
  • If required, a help option is available to show how to assemble hardware.
  • The "Project Manager" UI may appear at this step in the workflow.
  • Example of Server/Client application in Ubuntu distro: SageMath - server/client application.

Hardware Assembly

  • This step is only necessary if using a portable Decapod system
  • Mark the center using the UI for page splitting.
  • Avoid any construction
  • Gaylord book library display (http://www.gaylord.com)
  • A welcome screen to ask user where they would like to store their information.
  • Required parts: Decapod system, 2 matching Decapod supported cameras, tripod, accessory bar
  • Optional parts: cradle, foot pedal

Calibration

  • gphoto detects cameras are attached. Works well with subversion version of gphoto.
  • move a target around the view and outputs an XML file with calibration information
  • Then take photos and dewarping applied to image.
  • Can detect drifting of pages.
  • Need some way to help position cameras and book in Decapod.
  • Add colour and grey cards to calibration process
    • Cards can be glued to cradle to simplify process.
  • Produce a short user guide into how to set up and mark the book position and cameras:
  • Can use transparent tape to mark location of book edges.
  • Can also use bull-nose clips to clamp down book (but could be damaging).
  • Have the ability to identify cameras and swap them if the left camera is not appearing properly in the left image review.
  • there is no need to mark the reference plane or boundaries or use a different coloured mat since the calibration process does not require it.

The process:

  • place calibration target viewable by both cameras
  • System captures images to be used for calibration
  • Once sufficient number of images are taken (i.e. 4 to 8 images), the system will report back to the user and the user can begin their work.
  • see video of Calibration Process

Capturing

  • Future development: Quality Control plugin that does partial detection to help improve responsiveness and UX
  • Errors are generally non-descriptive because the nature of the beast has a lot of varied errors. Still needs to be communicated to the user.
  • Memory card for cameras are optional. gphoto can capture directly to server storage.
    • it used to be the case that using memory cards as intermediate storage was more reliable to write to card and then transfer to computer.
  • gstreamer - image capture library for gnome. Can do realtime capture.
  • gphoto - talks to photo cameras. Use SVN version for better camera support and direct capture to usb.
  • Real-time camera preview - not deal with preview at the moment because it's only used for positioning. Really need to position once and clamp down.
    • Also preview is not a priority because it's not reliable across all brands and models, and there may not be a high demand for such a feature.
  • Blind capture: so users can concentrate on the manual task. Therefore keep capturing UI very simple.
  • Prompt for readiness
    • visual and audio prompts. Visual prompt: green/yellow/red (consider accessibility and colour casting). Audio prompt: chimes, beeps, etc.
  • Foot pedal or alternative input to activate the camera, thus freeing the hands to manipulate the book.
  • Capturing the spine of a book - no technical implications (i.e. focusing), just like any other capture for the book.
    • Will not be explicitly written into the user's workflow because not every institution will want to capture the spine.
  • (question) Drag and drop support?
    • User drags images from their desktop and drops it into the decapod window.
    • This is a way of accomplishing splitting and merging projects together to form a logical collection of pages (i.e. a "book" or "volume")

Capture Post-Processing

  • Dewarping involves two steps: combine stereo pair and apply the 3d model to dewarp. The output is a single image with two facing pages.
  • Dewarping is costly. 15 second to few minutes depending on the number of reference points required.
    • 1 image is picked as the 'primary' (in Decapod's case this will always be the Left camera) and the second image of the pair will be warped / dewarped to match the primary.
    • This is done by overlaying the 2nd image on top of the primary and calculating the shifts.
  • Normalization will occur here as well.
  • Splitting individual pages from a spread: user will have to open the book to the middle during calibration and mark the center/split. Then after Dewarp, the pages can be split and presented as separate pages to the user.
  • If dewarp doesn't work , then recapture
    • (warning) What does it mean "doesn't work" (i.e. the dewarped output just looks bad?)
    • (warning) From a UX perspective, how does recapture fix a problem if initial capture was bad? Capturing can be separated by time, space, and configuration - so a recapture may provide different results.
  • Dewarping process runs continuously hidden from the user:
    • visually indicate to the user that some images are not dewarped-unsplit, and indicate when those pages have been