The core of TabNet is its encoder, which processes tabular data in a series of sequential steps. Think of it like a human expert analyzing a spreadsheet. Instead of looking at all the columns at once, they might first focus on a few key features, make a preliminary judgment, and then use that insight to decide which features to examine next. TabNet mimics this process. Each step takes the output of the previous one and decides which features to focus on, creating a more nuanced understanding of the data with each pass.

Within each step, two key modules work together. First, the Attentive Transformer analyzes the processed information from the previous step to decide which features are most important right now. It creates a "mask" that highlights these features and effectively ignores others. Second, the Feature Transformer takes this masked input and processes it, transforming the selected features into a more useful representation. This new representation is then passed on, ready for the next stage of the process.

After the Feature Transformer does its work, the resulting output is handled by a 'split' block. This block doesn't perform a complex calculation; its job is to route information. It divides the output vector into two parts with distinct purposes.

This split is controlled by two key parameters: $n_d$ and $n_a$. These define the dimensionality, or size, of the two resulting vectors.

• $n_d$ (decision dimension): This portion of the vector is sent to the final output layer of the model. Each step produces an $n_d$-sized vector, and these are all aggregated at the end to make the final prediction. It represents the step's contribution to the overall "decision."

• $n_a$ (attention dimension): This part is passed to the next step's Attentive Transformer. It carries the contextual information needed for the subsequent step to intelligently select its own features. It guides the model's "attention."

By stacking these decision steps, TabNet builds a deep, multi-layered understanding of the data. The number of steps is a hyperparameter you can tune. More steps allow for more complex reasoning but also increase computational cost. The separation of features into decision ($n_d$) and attention ($n_a$) streams is what allows the model to be both deep and interpretable, as we can inspect the attention masks at each step to see which features the model focused on.