In machine learning, we work with numbers constantly—weights, biases, learning rates. But deep down, computers only understand switches that are on or off. This two-state system is the foundation of binary, or Base-2, where all numbers are represented using just two digits: 0 and 1. Each position in a binary number represents a power of 2, starting from $2^0$ on the right.

While we write code using decimal numbers, the hardware underneath converts everything into this binary format. Understanding this conversion is key to grasping how models handle memory and perform calculations efficiently.

Representing positive integers is straightforward, but what about negative numbers? Computers use a clever system called two's complement. Instead of just adding a minus sign, this method reinterprets the most significant bit (the leftmost one) as a negative value. This trick makes subtraction as simple as addition for the hardware, which is a huge performance win.

For an 8-bit number, the leftmost bit represents $-128$ instead of $+128$. All other bits remain positive powers of two. To find the two's complement of a positive number, you flip all the bits (0s become 1s and vice versa) and then add one.

Binary strings can get very long and difficult for humans to read. To simplify this, programmers often use hexadecimal, or Base-16. This system uses digits 0-9 and letters A-F. The magic of hex is its direct relationship with binary: one hexadecimal digit represents exactly four binary digits (a nibble). This makes it a convenient shorthand for representing binary data like memory addresses or model weights.

Integers are useful, but most values in machine learning—like the weights in a neural network—are fractional. To handle these, computers use a format called floating-point, which is essentially scientific notation for binary numbers. The IEEE 754 standard defines how these numbers are stored.

A standard 32-bit floating-point number (a float or FP32) is broken into three parts:

1. Sign bit (1 bit): Simple. 0 for positive, 1 for negative.
2. Exponent (8 bits): This determines the range of the number. It represents the power of 2 that the number is multiplied by, allowing for both very large and very small values.
3. Mantissa (23 bits): Also called the significand, this represents the actual digits of the number—its precision. It's the fractional part of the number in binary scientific notation.

This structure creates a fundamental trade-off. More bits for the exponent mean a wider range of numbers can be represented, while more bits for the mantissa mean higher precision for the numbers within that range.

This trade-off is critical in ML. Most models are initially trained using 32-bit floating-point numbers (FP32), which offer a good balance of range and precision. However, FP32 requires significant memory and computational power.

To optimize models for inference, especially on specialized hardware like GPUs or TPUs, we often use quantization. This is the process of converting a model’s weights from a higher precision to a lower one.

Different precisions offer different benefits:

• FP32 (Single Precision): The standard. High precision, large range. Safe but slow and memory-intensive.
• FP16 (Half Precision): Halves the memory usage of FP32. Faster computations, but has a much smaller numerical range, making it vulnerable to underflow (numbers becoming zero) or overflow (numbers becoming infinity) during training.
• BF16 (BFloat16): A compromise. It uses the same number of exponent bits as FP32 (8), giving it a similar range, but fewer mantissa bits (7), reducing its precision. This format is often better for training deep learning models than FP16 because it avoids the overflow/underflow problem.
• INT8 (8-bit Integer): Very fast and memory-efficient. Ideal for inference on deployed models, but the sharp drop in precision can sometimes degrade model accuracy if not handled carefully.

Understanding how numbers are represented at the binary level allows you to make informed decisions about model optimization. Choosing the right precision is a balancing act between performance, memory footprint, and the final accuracy of your model.