Natural Language Processing

• A. To translate text into another language
• B. To split text into smaller units such as words or subwords ✓
• C. To compress text files for faster storage
• D. To remove all punctuation from a document

Why: Tokenization breaks raw text into smaller units (tokens) such as words, subwords, or characters so that a computer can process and analyze the text systematically.

• A. ['The', 'cat', 'sat', '.'] ✓
• B. ['T', 'h', 'e', ' ', 'c', 'a', 't']
• C. ['The cat sat.']
• D. ['Thecatsat']

Why: Word-level tokenization splits text on word boundaries and typically separates punctuation as its own token, producing ['The', 'cat', 'sat', '.'].

• A. A single sentence used for testing a model
• B. A large, structured collection of text used for training or analysis ✓
• C. A type of neural network architecture
• D. A list of stop words to be removed

Why: A corpus is a large collection of text documents, often organized and annotated, that serves as the data source for training and evaluating NLP models.

• A. It cannot handle lowercase letters
• B. It may leave punctuation attached to words, such as 'go,' and fail to split contractions like 'can't' properly ✓
• C. It converts all words into numbers
• D. It removes all verbs from the sentence

Why: Splitting only on whitespace keeps punctuation glued to adjacent words (e.g., 'go,') and does not handle contractions like "can't" into meaningful parts, which is why more sophisticated tokenizers are used.

• A. They guarantee every token is a complete dictionary word
• B. They handle rare and unseen words by breaking them into known subword pieces ✓
• C. They eliminate the need for any vocabulary
• D. They work only on English text

Why: Subword methods like BPE learn frequent character sequences from data, so rare or unseen words can be represented as combinations of known subword units, reducing out-of-vocabulary problems.

• A. Representing words as dense numerical vectors that capture semantic meaning
• B. Assigning each word a unique integer ID with no relationship to other words
• C. Counting the frequency of each word in a document and storing the counts
• D. Converting words into their phonetic spellings for speech recognition

• A. They are orthogonal to each other
• B. They have vectors that are close together by a distance measure such as cosine similarity
• C. They always have identical vector values
• D. They are stored at the same index in the vocabulary

• A. One-hot vectors are dense and low-dimensional, while embeddings are sparse
• B. One-hot vectors are sparse and high-dimensional with no notion of similarity, while embeddings are dense and low-dimensional and capture similarity
• C. One-hot vectors can only represent numbers, while embeddings represent text
• D. There is no difference; the terms are interchangeable

• A. Embeddings can compress text to save storage space
• B. Embeddings can encode semantic relationships as consistent vector offsets
• C. Embeddings can translate words automatically between languages
• D. Embeddings always spell words correctly

• A. Predict a target word from its surrounding context words
• B. Predict the context words given a target word
• C. Classify whether a sentence is grammatically correct
• D. Count how many times each word co-occurs across the corpus

• A. To translate text from one language to another
• B. To assign a probability to a sequence of words or predict the next word given previous words ✓
• C. To classify documents into predefined categories
• D. To extract named entities from text

Why: A language model estimates the probability distribution over sequences of words, which lets it predict the next word given the context. Translation, classification, and entity extraction are separate tasks that may use language models but are not their defining purpose.

• A. Every word in a sentence depends on all previous words equally
• B. The probability of a word depends only on a fixed number of preceding words ✓
• C. Words are independent of each other regardless of context
• D. The probability of a sentence is the same regardless of word order

Why: The Markov assumption simplifies computation by assuming that the probability of the next word depends only on the last n-1 words rather than the entire history. This is what makes n-gram models tractable.

• A. P(the) + P(cat) + P(sat)
• B. P(the) × P(cat | the) × P(sat | cat) ✓
• C. P(the, cat, sat) estimated directly from a single count
• D. P(sat) × P(cat) × P(the)

Why: Using the chain rule and the bigram (first-order Markov) assumption, the sentence probability is the product of the unigram probability of the first word and each subsequent word's conditional probability given only the previous word.

• A. The model becomes too slow to train
• B. The model assigns zero probability to valid sequences, making the whole sentence probability zero ✓
• C. The model produces sentences that are too long
• D. The model cannot handle punctuation marks

Why: If an n-gram never appears in the training data, its estimated count is zero, so its probability is zero. Since sentence probability is a product, a single zero makes the entire sequence's probability zero, which is why smoothing techniques are needed.

• A. It measures how many parameters the model has
• B. It measures how well the model predicts a held-out test set, with lower values indicating better prediction ✓
• C. It measures the speed at which the model generates text
• D. It measures the grammatical correctness of generated sentences

Why: Perplexity is the exponentiated average negative log-likelihood per word on a test set. It can be interpreted as the effective number of choices the model is uncertain between, so a lower perplexity means the model predicts the test data better.

• A. It cannot process text at all
• B. It suffers from vanishing and exploding gradients, making it hard to learn long-range dependencies ✓
• C. It requires labeled data for every time step
• D. It can only output a single fixed-length vector

Why: Basic RNNs repeatedly multiply gradients through many time steps, causing them to shrink or blow up, which makes it difficult to capture dependencies between words that are far apart in a sequence.

• A. It stores the vocabulary of the model
• B. It summarizes information from previous inputs and is passed to the next time step ✓
• C. It is the final output layer that produces class probabilities
• D. It holds the training labels for each sequence

Why: The hidden state acts as the network's memory, carrying a summary of earlier tokens forward so that each new prediction can take prior context into account.

• A. Softmax gate
• B. Pooling gate
• C. Forget, input, and output gates ✓
• D. Dropout gate

Why: LSTMs use forget, input, and output gates to decide what information to discard, store, and expose, which helps preserve useful signals over long sequences.

• A. It runs faster than a unidirectional RNN
• B. It uses no hidden state
• C. It uses context from both the past and the future of each token ✓
• D. It removes the need for word embeddings

Why: A bidirectional RNN processes the sequence in both directions and combines the two representations, giving each position access to both left and right context, which is useful for tasks like named entity recognition.

• A. To generate the output sequence one token at a time
• B. To compress the input sequence into a context representation ✓
• C. To compute the loss function during training
• D. To split the text into sentences

Why: The encoder reads the entire input sequence and condenses it into a fixed or contextual representation, which the decoder then uses to generate the output sequence.

• A. To compress the entire input sequence into a single fixed-length vector
• B. To let the model weigh the relevance of different input tokens when producing each output ✓
• C. To randomly drop tokens during training to prevent overfitting
• D. To convert text into numerical token IDs before processing

Why: Attention lets the model dynamically focus on the most relevant parts of the input for each output step, instead of relying on one fixed summary vector. Option A describes the older encoder-decoder bottleneck that attention was designed to overcome.

• A. Input, output, and hidden vectors
• B. Query, key, and value vectors ✓
• C. Encoder, decoder, and context vectors
• D. Positional, segment, and token vectors

Why: Each token is projected into a query (what it is looking for), a key (what it offers for matching), and a value (the information it passes along). Attention scores come from comparing queries to keys, and the output is a weighted sum of values.

• A. To normalize the output so it sums to one across all tokens
• B. To keep dot products from growing too large, which would push softmax into regions with very small gradients ✓
• C. To ensure the query and key matrices have the same shape
• D. To remove the effect of token position in the sequence

Why: As the key dimension grows, dot products grow in magnitude, which can saturate the softmax and make gradients vanishingly small. Scaling by the square root of the dimension keeps the variance of the scores stable. Softmax, not this scaling, is what makes the weights sum to one.

• A. Self-attention is order-agnostic, so without positional information the model cannot distinguish word order ✓
• B. Positional encodings speed up training by reducing the number of parameters
• C. They are required to compute the softmax over attention scores
• D. They replace the need for word embeddings entirely

Why: Self-attention treats its input as a set, so 'dog bites man' and 'man bites dog' would look identical without positional signals. Positional encodings add information about each token's position so the model can use word order.

• A. Transformers process all tokens in parallel using self-attention, while RNNs process tokens one step at a time ✓
• B. Transformers can only handle sequences of fixed length, while RNNs handle variable length
• C. RNNs use attention while Transformers use only convolutions
• D. Transformers store no hidden state, while RNNs cannot be trained with gradient descent

Why: RNNs must process tokens sequentially because each step depends on the previous hidden state, which limits parallelism. Transformers compute attention across all positions at once, enabling much faster training on modern hardware and easier modeling of long-range dependencies.

• A. Named entity recognition
• B. Sentiment analysis ✓
• C. Machine translation
• D. Part-of-speech tagging

Why: Sentiment analysis classifies text by the opinion or emotional tone it expresses, typically as positive, negative, or neutral. Named entity recognition identifies entities like people and places, machine translation converts text between languages, and part-of-speech tagging labels grammatical roles.

• A. To translate text into another language
• B. To identify and classify entities such as persons, organizations, and locations in text ✓
• C. To generate a summary of a document
• D. To determine the grammatical structure of a sentence

Why: NER locates spans of text that refer to real-world entities and assigns each a category such as PERSON, ORGANIZATION, or LOCATION. It does not translate, summarize, or parse grammatical structure.

• A. Text summarization ✓
• B. Tokenization
• C. Language identification
• D. Stemming

Why: Text summarization condenses a longer document into a shorter form while preserving key information. Tokenization splits text into units, language identification detects which language is used, and stemming reduces words to root forms; none of these produce a condensed version of the article.

• A. Convert spoken audio into written text
• B. Automatically convert text from one natural language into another while preserving meaning ✓
• C. Correct spelling errors in a document
• D. Group similar documents into clusters

Why: Machine translation converts text from a source language to a target language while preserving its meaning. Converting speech to text is speech recognition, spelling correction fixes typos, and clustering groups similar documents.

• A. To generate the final answer word by word
• B. To find relevant passages or documents that may contain the answer ✓
• C. To label the grammatical role of each word
• D. To remove stop words from the input

Why: Many question answering systems first retrieve candidate passages or documents relevant to the question, then extract or generate an answer from them. Generating the answer, tagging grammar, and removing stop words are different steps or tasks.

• A. Perplexity
• B. BLEU ✓
• C. Accuracy
• D. Mean squared error

Why: BLEU (Bilingual Evaluation Understudy) measures n-gram precision between a machine translation and one or more human reference translations, making it the standard automatic metric for translation quality. Perplexity measures language model fit, accuracy is for classification, and MSE is for regression.

• A. The fraction of all actual positive instances that were correctly identified
• B. The fraction of predicted positive instances that were actually positive ✓
• C. The fraction of all predictions that were correct
• D. The harmonic mean of recall and accuracy

Why: Precision = true positives / (true positives + false positives), so it measures how many of the instances the model labeled positive really were positive. Recall measures the fraction of actual positives found, and overall accuracy counts all correct predictions.

• A. Increased inference latency
• B. Reproduction and amplification of social biases present in the training data ✓
• C. Excessive memory usage during tokenization
• D. Difficulty in choosing a learning rate

Why: Web-scraped text reflects existing societal stereotypes and imbalances, and models trained on it can learn and reproduce gender, racial, or cultural biases. The other options are technical engineering issues rather than ethical concerns about fairness and harm.

• A. To speed up training by reducing the dataset size
• B. To provide an unbiased estimate of performance on unseen data ✓
• C. To increase the vocabulary size of the model
• D. To automatically tune the model's hyperparameters

Why: A test set that is never used during training or tuning gives an honest estimate of how the model generalizes to new data. Evaluating on training data overstates performance because the model may have memorized it.

• A. Deploying without monitoring, since the model was already tested offline
• B. Logging inputs and outputs and monitoring for drift and failure cases over time ✓
• C. Disabling all user feedback to avoid bias
• D. Using the training set as the production benchmark

Why: Real-world language distributions shift over time and models can fail on inputs not seen during development, so ongoing monitoring of performance, drift, and problematic outputs is a core deployment practice. Offline test results alone do not guarantee good live behavior.