An AI can’t listen to sound the way a human does. Our ears process continuous, analog sound waves, but a neural network needs numbers—discrete, orderly data. The first step in any voice AI system is to translate the rich, messy world of sound into the clean, structured language of math. This process is called digitization.

The standard method for this is Pulse-Code Modulation (PCM). Think of it like describing a curve by plotting points on a graph. You can't record every single point on the continuous line, but if you take enough snapshots, or samples, you can create a very close approximation of the original curve. PCM does exactly this for sound waves.

Two key settings control how accurately we capture the sound wave: sampling rate and bit depth.

The sampling rate is the number of snapshots we take of the sound wave every second. It’s measured in Hertz (Hz). For music, a common rate is 44,100 Hz, which captures frequencies far beyond what humans can hear. But for speech, we don't need that much detail. Most of the crucial information in a human voice lies below 8,000 Hz.

To accurately capture a frequency, you need to sample at least twice as fast—a principle known as the Nyquist-Shannon sampling theorem . This is why a sampling rate of 16,000 Hz (16 kHz) is a common standard for voice AI. It provides enough detail to understand speech clearly without generating unnecessary data, which would slow down the system.

If the sampling rate is how often we measure, bit depth is how precisely we measure. It determines the number of possible amplitude values we can assign to each sample. A higher bit depth provides a greater dynamic range—the difference between the quietest and loudest possible sounds. A common bit depth for voice is 16-bit, which offers 65,536 distinct amplitude levels. This is more than enough to capture the nuances of human speech without the massive file sizes of 24-bit or 32-bit audio used in professional recording.

An AI can't process a continuous stream of audio data indefinitely. It needs to look at the sound in small, manageable chunks. This process is called windowing, and each chunk is a frame.

Imagine reading a book by looking at it through a narrow slot that only reveals a few words at a time. That slot is the window. The AI analyzes the sound within that short frame, then the window slides forward to analyze the next overlapping segment. A typical frame size for real-time voice AI is just 20 to 30 milliseconds.

But even a 20-millisecond frame of raw PCM data isn't ideal for a neural network. Raw waveforms are complex and hard to interpret. To make the data more useful, each frame is converted into a spectrogram .

A spectrogram is a visual representation of sound. It plots frequency on the y-axis, time on the x-axis, and uses color or intensity to represent the amplitude of different frequencies. It basically shows 'what notes were played, when, and how loudly.' This format is much easier for an AI to analyze because it turns a sound problem into an image recognition problem, which neural networks are very good at solving.

By converting sound into a series of spectrogram images, we give the AI a format it can understand. This journey—from continuous wave to discrete samples, then to sliced frames, and finally to spectrograms—is the foundation of how AI listens.