Back to courses
AI Skills for Everyone beginner

Voice Ai and Assistants

Duration0 hours
Modules7
ForGeneral public, lifelong learners
Video1-2min
Get started — free →

Course outline

Module 1 Module 1: Foundations of Speech Recognition

Foundations of Speech Recognition

Introduction

Core content I

Core content II

Core content III

Summary

Further learning

Knowledge check

Q1. What is the primary goal of an automatic speech recognition (ASR) system?
  • A. To generate natural-sounding speech from text
  • B. To convert spoken audio into written text ✓
  • C. To identify the speaker's emotional state
  • D. To translate speech into another language in real time

Why: ASR systems take an audio signal containing speech and produce the corresponding transcription as text. Text-to-speech does the reverse, and emotion detection and translation are separate tasks that may build on ASR.

Q2. Which of the following is the most common first step in processing raw audio for speech recognition?
  • A. Language modeling
  • B. Speaker diarization
  • C. Feature extraction, such as computing spectrograms or MFCCs ✓
  • D. Text normalization

Why: Raw audio waveforms are high-dimensional and noisy, so systems first convert them into compact feature representations like spectrograms or Mel-frequency cepstral coefficients (MFCCs) that capture the perceptually relevant characteristics of speech.

Q3. What does the term 'phoneme' refer to in speech recognition?
  • A. A complete spoken sentence
  • B. The smallest unit of sound that distinguishes one word from another ✓
  • C. A type of acoustic model architecture
  • D. The pitch contour of a speaker's voice

Why: A phoneme is the smallest distinct sound unit in a language that can change the meaning of a word, such as the /p/ and /b/ in 'pat' and 'bat'. Many classical ASR systems model speech as sequences of phonemes.

Q4. What is the role of a language model in an ASR system?
  • A. It extracts acoustic features from the audio signal
  • B. It estimates the probability of word sequences, helping choose the most plausible transcription ✓
  • C. It synthesizes audio for the output
  • D. It removes background noise from the recording

Why: The language model captures how words typically follow one another, so when acoustically similar options exist (like 'recognize speech' vs. 'wreck a nice beach'), it helps the system pick the more likely transcription.

Q5. What is Word Error Rate (WER) used for?
  • A. Measuring the processing speed of an ASR system
  • B. Measuring the accuracy of a transcription by comparing it to a reference transcript ✓
  • C. Measuring the loudness of the input audio
  • D. Measuring the number of speakers in a recording

Why: WER is computed from substitutions, deletions, and insertions needed to turn the system's output into the reference transcript, divided by the number of words in the reference. Lower WER indicates more accurate recognition.

Module 2 Module 2: Natural Language Understanding

Natural Language Understanding

Introduction

Core content I

Core content II

Core content III

Summary

Further learning

Knowledge check

Q1. What is the primary goal of Natural Language Understanding (NLU) in a voice assistant?
  • A. Converting text into speech
  • B. Interpreting the meaning and intent behind a user's spoken or written words ✓
  • C. Compressing audio files for faster transmission
  • D. Displaying visual results on a screen

Why: NLU focuses on extracting meaning and intent from language input so the assistant can determine what the user wants. Converting text to speech is the job of text-to-speech (TTS) systems, not NLU.

Q2. In the sentence 'Book a flight to Paris tomorrow,' what is the intent?
  • A. Paris
  • B. Tomorrow
  • C. Book a flight ✓
  • D. Flight to Paris

Why: The intent is the action the user wants performed, which here is booking a flight. 'Paris' and 'tomorrow' are entities or slot values that provide details for the intent.

Q3. What is an 'entity' in NLU?
  • A. The overall goal of the user's request
  • B. A specific piece of information in an utterance, such as a date, location, or name ✓
  • C. The audio waveform of the user's voice
  • D. The response the assistant gives back to the user

Why: Entities are key pieces of information within an utterance, like dates, places, people, or quantities. They help fill in the parameters needed to fulfill an intent.

Q4. Which NLU task assigns a label such as 'LOCATION' or 'DATE' to words in a sentence?
  • A. Sentiment analysis
  • B. Speech recognition
  • C. Named entity recognition (NER) ✓
  • D. Speaker diarization

Why: Named entity recognition identifies and classifies words or phrases into predefined categories such as locations, dates, and person names. Sentiment analysis measures emotional tone, speech recognition converts audio to text, and speaker diarization identifies who is speaking when.

Q5. Why is handling ambiguity important in Natural Language Understanding for voice assistants?
  • A. Ambiguity makes audio files larger
  • B. Users often phrase requests in multiple ways, and the same words can have different meanings depending on context ✓
  • C. Voice assistants can only process one word at a time
  • D. Ambiguity is removed automatically by microphones

Why: Human language is flexible and context-dependent. Words like 'bank' or 'book' can mean different things, so NLU systems must use context to infer the correct meaning and respond appropriately.

Module 3 Module 3: Dialogue Management Basics

Dialogue Management Basics

Introduction

Core content I

Core content II

Core content III

Summary

Further learning

Knowledge check

Q1. In a voice assistant, what is the primary role of dialogue management?
  • A. Converting speech into text
  • B. Deciding what the system should do or say next based on the conversation so far ✓
  • C. Synthesizing the assistant's spoken responses
  • D. Detecting the wake word that starts an interaction

Why: Dialogue management tracks the conversation state and selects the next action or response. Speech recognition and speech synthesis are separate components that the dialogue manager works alongside.

Q2. What is a 'slot' in a task-oriented dialogue system?
  • A. A time limit for how long the assistant waits for user input
  • B. A piece of information the system must collect from the user to complete a task, such as a date or destination ✓
  • C. A pre-recorded audio clip used for confirmations
  • D. A category of intent the user can express

Why: Slots are the parameters needed to fulfill a request, like the city and date for booking a flight. Dialogue managers track which slots are filled and ask for the missing ones.

Q3. What does the term 'dialogue state' refer to?
  • A. The emotional tone detected in the user's voice
  • B. The current snapshot of the conversation, including user intents, filled slots, and history of turns ✓
  • C. The hardware status of the microphone and speaker
  • D. The language model's temperature setting

Why: Dialogue state is the representation of everything the system knows so far in the conversation. It is used to decide the next action and to maintain context across turns.

Q4. Why is a confirmation turn often used in voice dialogue systems?
  • A. To reduce the total number of words the system must generate
  • B. To verify understanding and prevent errors when speech recognition may be uncertain ✓
  • C. To switch the user to a different language automatically
  • D. To end the conversation politely

Why: Voice input is error-prone due to noise, accents, and recognition mistakes. Confirming key information, such as a booking detail, helps catch misunderstandings before the system acts on them.

Q5. Which approach to dialogue management uses fixed rules or a predefined flow to decide responses?
  • A. Rule-based (finite state or frame-based) dialogue management ✓
  • B. End-to-end neural generation without any state tracking
  • C. Random response selection
  • D. Acoustic model adaptation

Why: Rule-based systems use predefined states, transitions, or frames to control the conversation. They are predictable and easy to debug, though less flexible than learned approaches for open-ended dialogue.

Module 4 Module 4: Text-to-Speech Synthesis

Text-to-Speech Synthesis

Introduction

Core content I

Core content II

Core content III

Summary

Further learning

Knowledge check

Q1. What is the primary purpose of a text-to-speech (TTS) system?
  • A. To convert spoken language into written text
  • B. To convert written text into spoken audio ✓
  • C. To translate text from one language to another
  • D. To detect the emotional tone of a speaker's voice

Why: Text-to-speech systems take written text as input and generate spoken audio as output. Converting speech to text is the job of automatic speech recognition (ASR), not TTS.

Q2. Which stage of a typical TTS pipeline is responsible for converting abbreviations like 'Dr.' and numbers like '42' into their spoken forms?
  • A. Vocoder
  • B. Text normalization ✓
  • C. Acoustic modeling
  • D. Speaker verification

Why: Text normalization (also called text preprocessing) expands abbreviations, numbers, dates, and symbols into the words that should be spoken, such as 'Doctor' and 'forty-two'.

Q3. What does a vocoder do in a neural TTS system?
  • A. It transcribes speech into phonemes
  • B. It converts intermediate acoustic features, such as mel-spectrograms, into a raw audio waveform ✓
  • C. It assigns stress markers to words in a sentence
  • D. It measures the speaking rate of the input text

Why: A vocoder takes an intermediate acoustic representation, typically a mel-spectrogram, and synthesizes the time-domain audio waveform that can be played back. Examples include WaveNet, HiFi-GAN, and WaveGlow.

Q4. What is the main advantage of neural TTS systems over older concatenative synthesis methods?
  • A. They require no training data at all
  • B. They produce more natural and expressive speech with smoother prosody ✓
  • C. They always run faster on low-power devices
  • D. They can only generate speech in a single fixed voice

Why: Neural TTS models learn prosody, intonation, and rhythm directly from data, producing speech that sounds more natural than concatenative methods, which stitch together pre-recorded fragments and often sound choppy.

Q5. What is prosody in the context of speech synthesis?
  • A. The list of phonemes used in a language
  • B. The rhythm, stress, pitch, and intonation of speech ✓
  • C. The grammatical structure of a sentence
  • D. The file format used to store synthesized audio

Why: Prosody refers to the suprasegmental features of speech, including rhythm, stress, pitch contour, and intonation. These features make speech sound natural and convey meaning such as questions versus statements.

Module 5 Module 5: Designing Conversational Interfaces

Designing Conversational Interfaces

Introduction

Core content I

Core content II

Core content III

Summary

Further learning

Knowledge check

Q1. What is the primary goal of designing a conversational interface for a voice assistant?
  • A. To display as much information on screen as possible
  • B. To let users accomplish tasks through natural, turn-by-turn dialogue ✓
  • C. To require users to memorize a fixed set of voice commands
  • D. To replace all human-computer interaction with video

Why: Conversational interfaces aim to make interactions feel natural by letting users speak in everyday language and move through tasks in back-and-forth exchanges, rather than forcing rigid commands or visual navigation.

Q2. Which term describes the intent a user wants to accomplish, such as 'book a flight' or 'check the weather'?
  • A. Utterance
  • B. Slot
  • C. Intent ✓
  • D. Prompt template

Why: An intent represents the goal behind a user's request. Utterances are the actual words spoken, and slots are the specific pieces of information (like a date or city) needed to fulfill the intent.

Q3. What is a 'slot' in conversational design?
  • A. A time window during which the assistant listens for input
  • B. A parameter or piece of information the assistant must collect to complete an intent ✓
  • C. A pause inserted into the assistant's speech
  • D. The hardware port used for a microphone

Why: Slots (also called entities or parameters) are variables such as location, date, or quantity that the system needs to fill in order to carry out a user's intent, for example the destination in a travel booking request.

Q4. Why is a 'fallback' or 'recovery' strategy important in voice interface design?
  • A. It increases the volume of the assistant's responses
  • B. It helps the assistant gracefully handle misunderstood or unsupported requests ✓
  • C. It permanently stores all user data for advertising
  • D. It removes the need for speech recognition

Why: Speech recognition and language understanding are imperfect, so users will sometimes say things the system doesn't understand. A good fallback response acknowledges the problem, offers guidance, and keeps the conversation moving rather than leaving the user stuck.

Q5. When designing voice responses, what is a recommended practice?
  • A. Read long lists of options aloud in full, so the user hears everything at once
  • B. Keep responses concise and offer no more than a few choices at a time ✓
  • C. Use complex technical jargon to sound more authoritative
  • D. Always end responses with a question, regardless of context

Why: Spoken information is harder to retain than read text, so responses should be brief and options limited to a small number. Long lists overwhelm listeners, and unnecessary questions or jargon make the interaction harder to follow.

Module 6 Module 6: Integrating Voice Assistants with APIs

Integrating Voice Assistants with APIs

Introduction

Core content I

Core content II

Core content III

Summary

Further learning

Knowledge check

Q1. In a voice assistant architecture, what is the primary role of an API when it is integrated with the assistant?
  • A. To convert spoken audio directly into text without any server communication
  • B. To allow the assistant to request data or trigger actions from external services
  • C. To store the user's voice recordings permanently on the device
  • D. To replace the speech recognition engine entirely
Q2. Which HTTP method is most commonly used by a voice assistant to retrieve information, such as the current weather, from an external API?
  • A. DELETE
  • B. PUT
  • C. GET
  • D. PATCH
Q3. Why is it important for a voice assistant to send only the parsed intent and key slot values to an API, rather than the raw audio recording?
  • A. Raw audio cannot be transmitted over the internet under any circumstances
  • B. Sending structured data is smaller, faster, and reduces unnecessary exposure of user audio
  • C. APIs are only able to process numeric values
  • D. The voice assistant is legally required to delete all audio after transcription
Q4. What is the purpose of an API key or access token when a voice assistant calls a third-party service?
  • A. To improve the accuracy of speech recognition
  • B. To authenticate the application and control access to the service
  • C. To compress the response data before it is returned
  • D. To translate the response into the user's spoken language
Q5. If an API call made by a voice assistant takes several seconds, what is a good user experience practice?
  • A. Remain silent until the response arrives so the user is not interrupted
  • B. Repeat the original question endlessly until the API responds
  • C. Give the user brief feedback, such as a spoken acknowledgment or filler phrase, while waiting
  • D. End the session immediately and ask the user to start over
Module 7 Module 7: Evaluating and Deploying Voice AI

Evaluating and Deploying Voice AI

Introduction

Core content I

Core content II

Core content III

Summary

Further learning

Knowledge check

Q1. Which metric is most commonly used to measure the accuracy of a speech recognition (ASR) system?
  • A. Word Error Rate (WER) ✓
  • B. Mean Opinion Score (MOS)
  • C. Latency Percentile
  • D. Signal-to-Noise Ratio (SNR)

Why: Word Error Rate compares the transcribed output to a reference transcript and counts substitutions, deletions, and insertions, making it the standard measure of ASR accuracy.

Q2. What does latency refer to in the context of a voice AI assistant?
  • A. The volume level of the assistant's spoken responses
  • B. The total time between the user finishing speaking and the assistant beginning to respond ✓
  • C. The number of languages the assistant supports
  • D. The percentage of user requests that are successfully completed

Why: Latency measures the delay in the conversational loop, from end of user speech to the start of the system's response. High latency makes conversations feel unnatural.

Q3. Why is a Mean Opinion Score (MOS) used when evaluating text-to-speech output?
  • A. It measures how many words per minute the voice produces
  • B. It captures human perception of naturalness and quality through listener ratings ✓
  • C. It calculates the storage size of the generated audio files
  • D. It determines which speaker accent the model was trained on

Why: MOS is a subjective quality metric where human listeners rate audio samples, typically on a 1 to 5 scale, to capture perceived naturalness that automated metrics may miss.

Q4. Which deployment consideration is most important for a voice assistant used in a noisy environment such as a factory floor?
  • A. Using a larger font size in the companion app
  • B. Robustness to background noise through noise suppression and acoustic training data ✓
  • C. Supporting the maximum number of voice commands possible
  • D. Storing all conversation history in the cloud by default

Why: Noisy environments degrade speech recognition, so deployment should include noise suppression, microphone array processing, and training or fine-tuning on representative noisy audio.

Q5. What is the main purpose of A/B testing a voice assistant before a full deployment?
  • A. To compare two versions of the system with real users and measure which performs better on chosen metrics ✓
  • B. To test the audio hardware for two different microphone brands
  • C. To translate the assistant into two languages simultaneously
  • D. To split the assistant's responsibilities between two separate servers

Why: A/B testing exposes different system versions to comparable user groups and compares outcomes such as task completion, satisfaction, or error rates to guide the deployment decision.

Stay in the loop on this course.

We’ll email you when new modules or related material drops. No spam.

Built with Fragment Language. Sovereign architecture — no third-party trackers.
FragRun.mfgAFImemoryFGOwnership: B-Systems.ee