Skip to end of banner Go to start of banner

Adding Speech Output to Cubetto

Skip to end of metadata

Created by Simon Bates, last modified on Oct 15, 2019

Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Cubetto Hardware

ATSAMD21G18 microcontroller
- 32-Bit ARM Cortex M0+ at 48 MHz
- Flash Memory: 256 KB
- SRAM: 32 KB
Compatible with the Arduino Zero board and can be programmed using the Arduino IDE as an Arduino Zero
A buzzer is mounted on the Cubetto PCB and connected to Arduino pin number 17 (pin "A3" on the Arduino Zero and named "PA04" in the SAMD21 documentation)
- https://github.com/simonbates/cubetto/blob/e1a6dd2e658eb107b8e077f41daf28cacc614a1f/CubettoLibrary/CubettoLibrary.h#L24
- https://github.com/arduino/ArduinoCore-samd/blob/142814a5d2b4793a8db5a01093cea9d2a47ac3cb/variants/arduino_zero/variant.cpp#L47

Cubetto Firmware

Primo Video Guide to Updating the Cubetto Firmware Using the Arduino IDE:

Firmware Files

Primo Cubetto Firmware: https://drive.google.com/file/d/0B0CGyWe9uilVc0Nycms2QnFJOTg/view (as referenced in the video above)
GitHub repo of the firmware files: https://github.com/simonbates/cubetto

Speech Option 1: Text-To-Speech Library

Talkie
- https://github.com/going-digital/Talkie
- PWM text-to-speech on 168 or 328 based Arduinos at 16MHz
- Modifications by Paul Stoffregen
  - https://github.com/PaulStoffregen/Talkie
  - https://forum.pjrc.com/threads/33446-Talkie-speech-library
  - Supports DAC, PWM, and "propshield" outputs on Teensy LC, 3.1 & 3.2
  - https://www.pjrc.com/teensy/teensyLC.html
  - PWM on Teensy relies on the analogWriteFrequency() Teensy function
    - https://github.com/PaulStoffregen/Talkie/blob/bc609c6c16d0940f01af69366e3e87161c39cbe2/Talkie.cpp#L157
    - https://www.pjrc.com/teensy/td_pulse.html
TTS
- https://github.com/jscrane/TTS
- Does not support the SAMD21

Speech Option 2: Playback of Recorded Audio

Outline

Prepare a recording of the word that we want to have Cubetto speak
Store the audio data in our program by using 'xxd' to generate a C array notation for the audio data
Use the Arduino PROGMEM keyword to store the audio data in flash storage, rather than RAM

Preparing Audio using eSpeak NG, FFmpeg, and xxd

We can use the eSpeak NG text-to-speech system to generate speech samples. For example to make a sample file for the word "forward":

> espeak-ng -v en -w forward.wav "forward"

The audio data from eSpeak NG is 16-bit at 22050 Hz in WAV file format:

> ffprobe forward.wav
Input #0, wav, from 'forward.wav':
Duration: 00:00:00.81, bitrate: 353 kb/s
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 22050 Hz, 1 channels, s16, 352 kb/s

Use FFmpeg to convert to 8-bit and strip the file header information:

> ffmpeg -i forward.wav -f u8 forward_u8_22050.raw

Make a C array notation version of the audio data with xxd:

> xxd -i forward_u8_22050.raw > forward_u8_22050.h

Change the array type to byte and add the PROGMEM keyword:

byte forward_u8_22050_raw[] PROGMEM = {
...

See: https://github.com/simonbates/arduino-audio-examples/blob/master/samd21_dac_sampled_audio/forward_u8_22050.h

Option 2A: Playback using the SAMD21 DAC

Proof of Concept Code for MKR1000

// Based on
// https://github.com/arduino-libraries/AudioZero/blob/master/src/AudioZero.cpp

#include "forward_u8_22050.h"

byte* audioData;
uint32_t audioDataLen;
uint32_t sampleIndex;

void TC5_Handler (void) __attribute__ ((weak, alias("Play_Sample_Handler")));

bool tcIsSyncing()
{
    return TC5->COUNT16.STATUS.reg & TC_STATUS_SYNCBUSY;
}

void resetTc()
{
    TC5->COUNT16.CTRLA.reg = TC_CTRLA_SWRST;
    while (tcIsSyncing());
    while (TC5->COUNT16.CTRLA.bit.SWRST);
}

void disableTc()
{
    TC5->COUNT16.CTRLA.reg &= ~TC_CTRLA_ENABLE;
    while (tcIsSyncing());
}

void configureTc(uint32_t sampleRate)
{
    // Enable GCLK for TCC2 and TC5 (timer counter input clock)
    GCLK->CLKCTRL.reg = (uint16_t) (GCLK_CLKCTRL_CLKEN | GCLK_CLKCTRL_GEN_GCLK0 | GCLK_CLKCTRL_ID(GCM_TC4_TC5)) ;
    while (GCLK->STATUS.bit.SYNCBUSY);

    resetTc();

    // Set Timer counter Mode to 16 bits
    TC5->COUNT16.CTRLA.reg |= TC_CTRLA_MODE_COUNT16;

    // Set TC5 mode as match frequency
    TC5->COUNT16.CTRLA.reg |= TC_CTRLA_WAVEGEN_MFRQ;

    TC5->COUNT16.CTRLA.reg |= TC_CTRLA_PRESCALER_DIV1 | TC_CTRLA_ENABLE;

    TC5->COUNT16.CC[0].reg = (uint16_t) (SystemCoreClock / sampleRate - 1);
    while (tcIsSyncing());
    
    // Configure interrupt request
    NVIC_DisableIRQ(TC5_IRQn);
    NVIC_ClearPendingIRQ(TC5_IRQn);
    NVIC_SetPriority(TC5_IRQn, 0);
    NVIC_EnableIRQ(TC5_IRQn);

    // Enable the TC5 interrupt request
    TC5->COUNT16.INTENSET.bit.MC0 = 1;
    while (tcIsSyncing());
}

void playSample(byte* data, uint32_t length, uint32_t sampleRate)
{
    pinMode(PIN_A0, OUTPUT);
    analogWriteResolution(8);

    audioData = data;
    audioDataLen = length;
    sampleIndex = 0;

    configureTc(sampleRate);
}

void stopPlaying()
{
    resetTc();
    disableTc();
}

void sayForward()
{
    playSample(forward_u8_22050_raw, forward_u8_22050_raw_len, 22050);
    delay(3000);
    stopPlaying();
}

void setup() {
}

void loop() {
    sayForward();
    delay(4000);
}

#ifdef __cplusplus
extern "C" {
#endif

void Play_Sample_Handler (void)
{
    if (sampleIndex < audioDataLen - 1) {
        analogWrite(PIN_A0, pgm_read_byte_far(audioData + sampleIndex));
        ++sampleIndex;

        // Clear the interrupt
        TC5->COUNT16.INTFLAG.bit.MC0 = 1;
    } else {
        // This will loop until stopPlaying() is called
        sampleIndex = 0;
        TC5->COUNT16.INTFLAG.bit.MC0 = 1;
    }
}

#ifdef __cplusplus
}
#endif

GitHub: https://github.com/simonbates/arduino-audio-examples/blob/master/samd21_dac_sampled_audio

Option 2B: Playback using PWM

Proof of Concept Code for MKR1000

// PWM based on code from MartinL in
// https://forum.arduino.cc/index.php?topic=346731.0
// Sample playback based on 
// https://github.com/arduino-libraries/AudioZero/blob/master/src/AudioZero.cpp

#include "forward_u8_22050.h"

byte* audioData;
uint32_t audioDataLen;
uint32_t sampleIndex;

void TC5_Handler (void) __attribute__ ((weak, alias("Play_Sample_Handler")));

bool tcIsSyncing()
{
    return TC5->COUNT16.STATUS.reg & TC_STATUS_SYNCBUSY;
}

void resetTc()
{
    TC5->COUNT16.CTRLA.reg = TC_CTRLA_SWRST;
    while (tcIsSyncing());
    while (TC5->COUNT16.CTRLA.bit.SWRST);
}

void disableTc()
{
    TC5->COUNT16.CTRLA.reg &= ~TC_CTRLA_ENABLE;
    while (tcIsSyncing());
}

void configureTc(uint32_t sampleRate)
{
    // Enable GCLK for TCC2 and TC5 (timer counter input clock)
    GCLK->CLKCTRL.reg = (uint16_t) (GCLK_CLKCTRL_CLKEN | GCLK_CLKCTRL_GEN_GCLK0 | GCLK_CLKCTRL_ID(GCM_TC4_TC5)) ;
    while (GCLK->STATUS.bit.SYNCBUSY);

    resetTc();

    // Set Timer counter Mode to 16 bits
    TC5->COUNT16.CTRLA.reg |= TC_CTRLA_MODE_COUNT16;

    // Set TC5 mode as match frequency
    TC5->COUNT16.CTRLA.reg |= TC_CTRLA_WAVEGEN_MFRQ;

    TC5->COUNT16.CTRLA.reg |= TC_CTRLA_PRESCALER_DIV1 | TC_CTRLA_ENABLE;

    TC5->COUNT16.CC[0].reg = (uint16_t) (SystemCoreClock / sampleRate - 1);
    while (tcIsSyncing());
    
    // Configure interrupt request
    NVIC_DisableIRQ(TC5_IRQn);
    NVIC_ClearPendingIRQ(TC5_IRQn);
    NVIC_SetPriority(TC5_IRQn, 0);
    NVIC_EnableIRQ(TC5_IRQn);

    // Enable the TC5 interrupt request
    TC5->COUNT16.INTENSET.bit.MC0 = 1;
    while (tcIsSyncing());
}

void playSample(byte* data, uint32_t length, uint32_t sampleRate)
{
    audioData = data;
    audioDataLen = length;
    sampleIndex = 0;

    configureTc(sampleRate);
}

void stopPlaying()
{
    resetTc();
    disableTc();
}

void sayForward()
{
    playSample(forward_u8_22050_raw, forward_u8_22050_raw_len, 22050);
    delay(3000);
    stopPlaying();
}

void setup()
{
    REG_GCLK_GENDIV = GCLK_GENDIV_DIV(1) |          // Divide the 48MHz clock source by divisor 1: 48MHz/1=48MHz
                      GCLK_GENDIV_ID(4);            // Select Generic Clock (GCLK) 4
    while (GCLK->STATUS.bit.SYNCBUSY);              // Wait for synchronization

    REG_GCLK_GENCTRL = GCLK_GENCTRL_IDC |           // Set the duty cycle to 50/50 HIGH/LOW
                       GCLK_GENCTRL_GENEN |         // Enable GCLK4
                       GCLK_GENCTRL_SRC_DFLL48M |   // Set the 48MHz clock source
                       GCLK_GENCTRL_ID(4);          // Select GCLK4
    while (GCLK->STATUS.bit.SYNCBUSY);              // Wait for synchronization

    // Enable the port multiplexer for the digital pin D7
    PORT->Group[g_APinDescription[7].ulPort].PINCFG[g_APinDescription[7].ulPin].bit.PMUXEN = 1;
 
    // Connect the TCC0 timer to digital output D7 - port pins are paired odd PMUO and even PMUXE
    // F & E specify the timers: TCC0, TCC1 and TCC2
    PORT->Group[g_APinDescription[6].ulPort].PMUX[g_APinDescription[6].ulPin >> 1].reg = PORT_PMUX_PMUXO_F;

    // Feed GCLK4 to TCC0 and TCC1
    REG_GCLK_CLKCTRL = GCLK_CLKCTRL_CLKEN |         // Enable GCLK4 to TCC0 and TCC1
                       GCLK_CLKCTRL_GEN_GCLK4 |     // Select GCLK4
                       GCLK_CLKCTRL_ID_TCC0_TCC1;   // Feed GCLK4 to TCC0 and TCC1
    while (GCLK->STATUS.bit.SYNCBUSY);              // Wait for synchronization

    // Dual slope PWM operation: timers countinuously count up to PER register value then down 0
    REG_TCC0_WAVE |= TCC_WAVE_POL(0xF) |           // Reverse the output polarity on all TCC0 outputs
                     TCC_WAVE_WAVEGEN_DSBOTH;      // Setup dual slope PWM on TCC0
    while (TCC0->SYNCBUSY.bit.WAVE);               // Wait for synchronization

    // Each timer counts up to a maximum or TOP value set by the PER register,
    // this determines the frequency of the PWM operation:
    REG_TCC0_PER = 255;
    while (TCC0->SYNCBUSY.bit.PER);                // Wait for synchronization
 
    REG_TCC0_CC3 = 0;         // TCC0 CC3 - on D7
    while (TCC0->SYNCBUSY.bit.CC3);                // Wait for synchronization
 
    // Divide the 48MHz signal by 1 giving 48MHz TCC0 timer tick and enable the outputs
    REG_TCC0_CTRLA |= TCC_CTRLA_PRESCALER_DIV1 |   // Divide GCLK4 by 1
                      TCC_CTRLA_ENABLE;            // Enable the TCC0 output
    while (TCC0->SYNCBUSY.bit.ENABLE);             // Wait for synchronization
}

void loop() {
    sayForward();
    delay(4000);
}

#ifdef __cplusplus
extern "C" {
#endif

void Play_Sample_Handler (void)
{
    if (sampleIndex < audioDataLen - 1) {
        // TCC0 CC3 - on D7
        REG_TCC0_CC3 = pgm_read_byte_far(audioData + sampleIndex);
        ++sampleIndex;

        // Clear the interrupt
        TC5->COUNT16.INTFLAG.bit.MC0 = 1;
    } else {
        // This will loop until stopPlaying() is called
        sampleIndex = 0;
        TC5->COUNT16.INTFLAG.bit.MC0 = 1;
    }
}

#ifdef __cplusplus
}
#endif

GitHub: https://github.com/simonbates/arduino-audio-examples/tree/master/samd21_pwm_sampled_audio

Resources

SAMD21 Product Information and Datasheet

Arduino Memory

Audio Libraries and Code Samples

Arduino PWM tone() function for SAMD
- https://github.com/arduino/ArduinoCore-samd/blob/master/cores/arduino/Tone.cpp
Arduino PWM tone() function for AVR
- https://github.com/arduino/ArduinoCore-avr/blob/master/cores/arduino/Tone.cpp
AudioZero audio library for Arduino Zero
- https://github.com/arduino-libraries/AudioZero/blob/master/src/AudioZero.cpp
- Play sampled audio using the built-in DAC on pin A0
PWM audio for Arduino with Atmega168 at 16 MHz
- https://playground.arduino.cc/Code/PCMAudio/

Arduino/SAMD21 Timers

Forum thread: "Changing Arduino Zero PWM Frequency"
- https://forum.arduino.cc/index.php?topic=346731.0
Forum Thread: "PWM on Arduino Zero Questions"
- https://forum.arduino.cc/index.php?topic=330735.0
Forum thread: "How this timer even works ?"
- https://forum.arduino.cc/index.php?topic=396396.0
https://www.hackster.io/voske65/high-speed-pwm-on-arduino-atsamd21-859b06
http://www.ermicro.com/blog/?p=1971
- Single and dual slope PWM
Newbie's Guide to AVR PWM
- https://www.avrfreaks.net/forum/tut-c-newbies-guide-avr-pwm-incomplete?name=PNphpBB2&file=viewtopic&t=68302
Smoothly Changing a Timer’s Frequency on the Arduino Zero
https://gist.github.com/nonsintetic/ad13e70f164801325f5f552f84306d6f
https://arduino.stackexchange.com/questions/41569/arduino-zero-timer-setup
SAMD21 Timer library for the Arduino Zero
- https://github.com/avandalen/avdweb_SAMDtimer
- https://forum.arduino.cc/index.php?topic=454243.0
https://github.com/adafruit/Adafruit_ZeroTimer

No labels