Probability is simply a way to measure how likely something is to happen. It's a number between 0 and 1, where 0 means it's impossible and 1 means it's certain. For example, the probability of a standard coin landing on heads is 0.5.

To talk about probability formally, we need a few key terms. An experiment is any process with an uncertain result, like rolling a die. An outcome is a single result of an experiment, like rolling a 4. The sample space is the set of all possible outcomes. For a six-sided die, the sample space is {1, 2, 3, 4, 5, 6}. An event is any collection of outcomes, such as rolling an even number {2, 4, 6}.

All of probability is built on three simple rules, called axioms. They are the bedrock principles that everything else follows from.

1. Probability is never negative. The likelihood of any event must be zero or positive. You can't have a -50% chance of rain. For any event A, its probability is:

2. The probability of all possible outcomes is 1. If you perform an experiment, one of the possible outcomes must occur. The probability of the entire sample space (S) is 1, or 100%.

3. You can add probabilities of mutually exclusive events. If two events can't happen at the same time (they are 'mutually exclusive'), the probability that one or the other occurs is the sum of their individual probabilities. For example, a single die roll can't be both a 1 and a 6. So, the probability of rolling a 1 or a 6 is:

Things get more interesting when we have extra information. This brings us to conditional probability, which is the probability of an event happening given that another event has already occurred. It's about updating our beliefs based on new evidence.

Imagine a standard deck of 52 cards. Let's find the probability of drawing a King. Since there are 4 Kings, the probability is $P(\text{King}) = 4/52$. Now, what if we know the card we drew is a face card (Jack, Queen, or King)? This new information changes the sample space. There are 12 face cards in total. Out of these 12, 4 are Kings. The probability of drawing a King given that it's a face card is $4/12$.

The notation for this is $P(A|B)$, which reads "the probability of A given B." The formula is:

Here, $P(A \cap B)$ is the probability of both A and B happening. In our example, $A$ is drawing a King and $B$ is drawing a face card. $P(A \cap B)$ is the probability of drawing a card that is both a King and a face card, which is just the probability of drawing a King: $4/52$. The probability of drawing a face card, $P(B)$, is $12/52$. So:

Sometimes, one event tells you nothing about another. If you flip a coin twice, the outcome of the first flip has no impact on the second. These events are called independent.

Two events A and B are independent if knowing that B happened doesn't change the probability of A. Mathematically, this means:

When events are independent, the probability of both happening is simply the product of their individual probabilities: $P(A \cap B) = P(A) \times P(B)$.

Finally, let's look at the Law of Total Probability. It's a way to find the probability of an event by breaking it down into different, mutually exclusive scenarios. It’s like finding the total area of a room by adding up the areas of all the non-overlapping rugs that cover its floor.

Suppose we have a set of events $B_1, B_2, ..., B_n$ that are mutually exclusive and their probabilities sum to 1 (they form a 'partition' of the sample space). The law states that the probability of another event A can be found by summing up the conditional probabilities of A across all those scenarios.

Let's say a factory has two machines. Machine 1 makes 60% of the products, and Machine 2 makes 40%. Machine 1 has a defect rate of 5%, while Machine 2 has a defect rate of 10%. What's the overall probability that a randomly selected product is defective?

Using the law of total probability: $P(\text{Defect}) = P(\text{Defect | Machine 1})P(\text{Machine 1}) + P(\text{Defect | Machine 2})P(\text{Machine 2})$ $P(\text{Defect}) = (0.05)(0.60) + (0.10)(0.40) = 0.03 + 0.04 = 0.07$

The overall probability of a product being defective is 7%.

These foundational concepts, from the basic axioms to the law of total probability, are the building blocks for understanding more complex ideas in statistics and data analysis.