Probability Rules and Relationships

Handling Independent and Dependent Events, Joint and Conditional Probability

Day 20 of

Raven-R

’s AI/ML Learning Journey! We’ll continue building our foundational understanding by exploring operational rules and relationships between events.

Let’s get into it. :)

Addition Rule

The addition rule is used to find the probability of either of two events occurring. There are two scenarios:

• For mutually exclusive events (remember? events that cannot happen at the same time), the probability of either event A or event B occurring is simply the sum of their individual probabilities -

  \(P(A \text{ or } B) = P(A) + P(B)\)

If you roll a six-sided die, the probability of rolling a 2 or a 5 is -

\(P(2 \text{ or } 5) = P(2) + P(5) = \frac{1}{6} + \frac{1}{6} = \frac{2}{6} = 0.3333\)

# Probability of rolling a 2 or a 5 on a six-sided die

P_2 = 1/6
P_5 = 1/6

P_2_or_5 = P_2 + P_5

P_2_or_5  # Output: 0.3333

On the other hand, for non-mutually exclusive events (events that can happen at the same time), the probability of either event A or event B occurring is -

\(P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)\)

This is because we must subtract the overlap where both A and B occur (otherwise, we'd be counting it twice).

If 30% of students like soccer, 40% like basketball, and 10% like both, then the probability of a randomly selected student liking soccer or basketball is -

\(P(\text{Soccer or Basketball}) = 0.3 + 0.4 - 0.1 = 0.6\)

# Probability of liking soccer or basketball

P_Soccer = 0.3
P_Basketball = 0.4
P_Soccer_and_Basketball = 0.1

P_Soccer_or_Basketball = P_Soccer + P_Basketball - P_Soccer_and_Basketball

P_Soccer_or_Basketball  # Output: 0.6

The Multiplication Rule

The multiplication rule is used to find the probability of two events occurring together. And as we probably already know, there are two cases -

• For independent events (events that do not affect each other), the probability of both A and B occurring is -

  \(P(A \text{ and } B) = P(A) \times P(B) \)

  If you flip a coin and roll a die, the probability of getting heads and rolling a 4 is -

  \(P(\text{Heads and 4}) = P(\text{Heads}) \times P(4) = 0.5 \times \frac{1}{6} = 0.0833\)

  # Probability of flipping heads and rolling a 4
  
  P_Heads = 0.5
  P_4 = 1/6
  
  P_Heads_and_4 = P_Heads * P_4
  
  P_Heads_and_4  # Output: 0.0833

• For dependent events (events where one affects the other), the probability of both A and B occurring is -

  \(P(A \text{ and } B) = P(A) \times P(B \mid A)\)

  Here, P (B | A) is the probability of B occurring given that A has occurred.

  Now, If there are 3 red and 2 blue marbles in a bag, and you draw a red marble without putting it back, the probability of drawing a red marble and then a blue marble is -

  \(P(\text{Red and Blue}) = P(\text{Red}) \times P(\text{Blue} \mid \text{Red}) = \frac{3}{5} \times \frac{2}{4} = 0.3\)

  # Probability of drawing a red marble and then a blue marble
  
  P_Red = 3/5
  P_Blue_given_Red = 2/4
  
  P_Red_and_Blue = P_Red * P_Blue_given_Red
  
  P_Red_and_Blue  # Output: 0.3

  Question!? You might wanna ask; How do we calculate the probability of B occurring given that A has already occurred (denoted as P (B | A))

  
  Let’s try it out with an example.

• There are 3 red marbles and 2 blue marbles in a bag, making a total of 5 marbles.

• You draw one red marble without putting it back, so now there are 4 marbles left in the bag (2 red and 2 blue).
  
  Solution

• Event A occurs: Drawing a red marble first. Now, there are 4 marbles left in the bag (2 red, 2 blue).

• Calculate P (B | A) -
  After drawing a red marble, the probability of drawing a blue marble is -

  \(P(\text{Blue} \mid \text{Red}) = \frac{\text{Number of remaining blue marbles}}{\text{Total remaining marbles}} = \frac{2}{4} = 0.5 \)

  Thus, the conditional probability P (B | A) = 0.5
  Which is the probability of B occurring, given that A has already occurred.
  
  Then, Finally! Calculation of P(Red and Blue)
  
  The probability of both events happening
  

  \(P(\text{Red and Blue}) = P(\text{Red}) \times P(\text{Blue} \mid \text{Red}) = \frac{3}{5} \times 0.5 = 0.3\)

  # Calculating conditional probability P(Blue | Red)
  
  remaining_blue_marbles = 2
  remaining_total_marbles = 4
  P_Blue_given_Red = remaining_blue_marbles / remaining_total_marbles
  
  # Probability of drawing a red marble first
  P_Red = 3/5
  
  # Probability of drawing a red marble and then a blue marble
  P_Red_and_Blue = P_Red * P_Blue_given_Red
  P_Red_and_Blue  # Output: 0.3

  - Python Implementation

Wrapping up, Let’s discuss Joint and Conditional Probability, really popular terms.

Joint Probability

Joint probability refers to the probability of two events happening at the same time. It is usually represented as P (A and B).

Conditional Probability

Conditional probability is the probability of an event occurring given that another event has already occurred. It is written as P(A | B) which means "the probability of A given B."
(We’ve already been through this, but it’s worth re-highlighting)

The formula for conditional probability is -

\(P(A \mid B) = \frac{P(A \text{ and } B)}{P(B)}\)

Recap Time!

• Addition Rule: Helps find the probability of either event occurring, with variations for mutually exclusive and non-mutually exclusive events.

• Multiplication Rule: Used to calculate the probability of two events happening together, with cases for independent and dependent events.

• Joint Probability: Represents the probability of two events occurring simultaneously.

• Conditional Probability: Determines the likelihood of an event occurring, given that another event has already happened.

Cheers! See you again.