DEV Community

Rao Asad
Rao Asad

Posted on Edited on AI-assisted

Grading spoken answers for a U.S. citizenship mock interview

At the U.S. naturalization interview, the officer reads civics questions aloud and you answer in English, out loud. Many study apps practise this as a multiple-choice quiz, which is a different skill: recognising an answer is easier than producing one under pressure.

So Belong, my citizenship test app for Android, has a mock interview that works the way the real one does: each question is read aloud, you answer by voice or by typing, and it stops when the result is settled. The interesting part turned out to be deciding whether a spoken answer is right. (Belong isn't affiliated with USCIS; the questions come from the official USCIS study materials.)

The rules, as plain Kotlin

All of the test logic lives in a :core module with no Android dependencies, so it's fast to unit test.

There are two tests in use. On the 2025 test the officer asks up to 20 questions and you pass at 12 correct. On the 2008 test it's up to 10 questions, passing at 6. The interview doesn't run to the end: it stops as soon as the outcome is decided.

data class TestRules(val maxQuestions: Int, val passAt: Int) {
    val failAt: Int get() = maxQuestions - passAt + 1   // 2025: 9 wrong ends it
}

class MockInterview private constructor(val questions: List<Question>, val rules: TestRules) {
    private val answers = mutableListOf<AnsweredQuestion>()

    val correctCount: Int get() = answers.count { it.correct }
    val wrongCount: Int get() = answers.size - correctCount

    val isFinished: Boolean
        get() = correctCount >= rules.passAt || wrongCount >= rules.failAt || answers.size == questions.size

    val current: Question? get() = if (isFinished) null else questions[answers.size]
}
Enter fullscreen mode Exit fullscreen mode

failAt falls out of the other two numbers: once you can no longer reach the pass mark, there's no point asking more. Questions are drawn at random from the pool you study, and the drawn order is saved, so an interview in progress can be restored with the same questions.

Matching answers: lenient like the officer, strict where it matters

The speech recognizer gives you a string like "the president of the united states". USCIS lists accepted answers, sometimes several, sometimes with optional parts. A naive == fails almost everything; a naive "contains" accepts nonsense.

The rule I settled on: every word of an accepted answer must be there, extra words are fine, but certain extra words are not.

// Extra words are fine ("the President of the United States"), except these:
// each turns an accepted answer into a different one.
private val meaningChangers = setOf("vice", "not", "no", "never", "former", "deputy", "lieutenant", "assistant")

fun matchedAnswers(given: String, accepted: List<String>): List<String> {
    val words = AnswerText.words(given)
    if (words.isEmpty()) return emptyList()
    return accepted.filter { answer ->
        variants(answer).any { variant ->
            containsAll(words, variant) && words.none { it in meaningChangers && it !in variant }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Without that list, "the Vice President" would match the accepted answer "the President", because every word of "the President" is present. With it, the extra "vice" flips the match off, unless the accepted answer itself contains it.

Variants: what the officer would also accept

The official answers are written for people, not parsers:

  • Words in parentheses are optional. "(U.S.) Constitution" is also just "Constitution".
  • A number in parentheses is the same answer in digits. "One hundred (100)" is also "100".
  • "The United States" inside a longer answer is optional, because every question is about the country: "Obey the laws of the United States" is also "obey the laws". An answer that is only the country still needs it.

Each accepted answer expands into these variants, and a match against any variant counts. Some questions need more than one answer ("name two rights"), so the matcher counts distinct accepted answers found in what you said.

Absorbing recognizer slips, but not in numbers

Speech recognition mishears, and people misspell when typing. So words count as matching when they're within a small edit distance, scaled by length:

internal fun close(a: String, b: String): Boolean {
    if (a == b) return true
    if (a.any(Char::isDigit) || b.any(Char::isDigit)) return false   // numbers must match exactly
    val allowed = when (minOf(a.length, b.length)) {
        in 0..3 -> 0
        in 4..7 -> 1
        else -> 2
    }
    return allowed > 0 && Edits.distance(a, b, limit = allowed) <= allowed
}
Enter fullscreen mode Exit fullscreen mode

Short words get no slack, because "war" and "was", or "law" and "low", are one letter apart and mean different things. "constitusion" still matches "constitution". Numbers get no slack at all: 27 amendments is not 28. The Levenshtein check gives up early once it passes the limit, since it runs for every word pair.

Each accepted word also has to match a different spoken word, so one "states" can't satisfy an answer that needs it twice.

Testing it

Because it's all plain functions, the matcher, the interview rules and the review schedule are covered by ordinary JVM unit tests: a recognizer-style string in, a verdict out. Adding a tricky phrase is one more test case, with no device or emulator involved.

What I'd pass on

  • If your app grades free-form answers, write down why each answer is accepted. The edge cases come from that reasoning, not from the answer list.
  • Be lenient about form and strict about meaning, and decide explicitly which words carry meaning.
  • Keep the rules in a pure module, so every tricky phrase can become a fast unit test.

Belong is on Google Play. Every civics question, flashcards, quizzes and audio are always free, with no account, and it works offline. Your first mock interview is free, and Pro unlocks unlimited ones. If you've been through the interview, I'd like to know what would have helped you most.

Top comments (0)