A standard neural network learns by minimizing a loss function, which is typically a measure of how far its predictions are from the true data. For example, the mean squared error (MSE) simply averages the squared difference between the predicted and actual values. This works well when you have a lot of data. But what if your data is sparse, or you need the model to respect physical laws in regions where you have no data at all?

This is where Physics-Informed Neural Networks (PINNs) change the game. Instead of just learning from data, a PINN also learns from the governing equations of a system. The key idea is to encode the physical law directly into the loss function.

To do this, we first need to define the PDE's residual. Let's say we have a neural network, $N(x, t; \theta)$, that takes spatial coordinate $x$ and time $t$ as inputs and is parameterized by weights $\theta$. The network's job is to approximate the solution of a PDE, let's call it $u(x, t)$.

The residual, $f(x, t)$, is what you get when you plug the network's approximation into the governing PDE. If the network's output is a perfect solution, the residual will be zero everywhere.

You might wonder how we calculate those partial derivatives of the network's output with respect to its inputs, like $\frac{\partial N}{\partial t}$. This is accomplished using Automatic Differentiation (AD), a technique that is both precise and computationally efficient. AD is the same mechanism that enables backpropagation to train the network's weights, but here we apply it 'forward' to compute the physical derivatives needed for the residual.

The total loss function for a PINN is a composite of several terms. The first part is familiar: the data-driven loss. This is typically the Mean Squared Error (MSE) calculated on the known data points we have. It anchors the model to reality.

The second, and most important, part is the physics-based loss. This loss term is the MSE of the PDE residual, but it's calculated over a set of collocation points scattered throughout the problem's domain (in both space and time). These points don't need to have known solution values; their purpose is simply to enforce the PDE everywhere.

A physical system isn't just defined by a governing equation; it's also defined by its initial and boundary conditions. A PINN must respect these as well. For example, what was the state of the system at time $t=0$? What is happening at the physical edges of the domain?

We enforce these by adding more terms to our loss function. We define an initial condition loss ($L_{IC}$) and a boundary condition loss ($L_{BC}$). These are also MSE terms, calculated on points sampled from the initial time and the spatial boundaries, respectively.

The final, complete loss function is a weighted sum of all these components.

By minimizing this composite loss function, the network learns a function that simultaneously fits the observed data and obeys the underlying physics of the system across the entire domain. This allows it to make accurate predictions even in regions far from any training data points, overcoming a major limitation of purely data-driven models.