PhET Energy Skate Park Auditory Interface Sketches

Owned by Jonathan Hung
Last updated: Oct 27, 2017 by Caren Watkins
4 min read

Track Shape Sonification

Continuous Pitch

  • Move left and right and you can hear the shape of the track based on pitch.
  • Ascending pitch - ascent
  • Descending pitch - descent
  • Problem - This works well if the track is easily navigated left to right, but how do you convey a loop?

Sonar "Ticks"

  • The closer you get to the track, the more frequent the ticks.
  • The farther away, the less frequent the ticks.
  • You can effectively "paint" the scene by panning the mouse around the scene
  • Problem - this requires ability to use a gesture based input.
  • Problem - your perception of the track is relative to where you start painting with ticks. For example, two users with the same track may start at two different locations and get a different interpretation of the same track

Track Shape Sonification Sketch #1: Pitch, volume, and descriptions

Below: When moving the skater around to be positioned, sonification is used to indicate distance from the track below (represented by volume), and the elevation of the track (represented by pitch). For example: the further away the skater is from the track the quieter the volume, as the skater comes closer to the track the volume increases; the higher the skater is on a slope (elevation) the more the pitch increases, as the skater moves down the slope the pitch decreases.

 

Below: This is a special case where the skater is being positioned within a loop.

Energy Charts

There are two charts, bar and pie, which show the conservation of energy, which are key to the learning outcome. The charts show:

  • Kinetic (bar + pie)
  • Potential (bar + pie)
  • Thermal (bar + pie)
  • Total (bar only)

When the simulation is running, the bars / sections of the chart animate to show the conversion of potential to kinetic energy. The pie chart follows the skater, while the bar chart stays in a static location.

Since the two charts are showing the same information but presented in different ways, it may be possible to have a single sonification of the two charts.

The thermal energy can be "thrown out" - this resets the thermal energy to zero, and decreases the total energy by the amount thrown away. After zeroing, thermal energy will continue to accumulate if there is friction. The use case for this isn't very clear but was a user feature request.

 

Sonification challenges:

  • How do you sonify both the bar chart and the pie chart if both are visible?
  • Since they show the same information, would you sonify only one?

Possible features:

  • Ability to choose which energy types to sonify. This would allow users to isolate just the kinetic energy and hear it rise and fall.

Bar Chart Sonification Sketch #1

Use 4 distinct sounds to the different energy types and total.

Use volume to indicate quantity. Quieter = less, louder = more.

The total energy would be a constant hum in the background that increases or decreases in volume depending on the total energy of the system.

The following sound clip is a rough rendition of a sonification of the bar graph. The funky guitar is potential energy, the deeper bass is kinetic, the bell is thermal energy, and the drum playing throughout represents the total energy.