1.4.3 Bayes' Rule
Now we are ready to state one of the most useful results in conditional probability: Bayes' rule. Suppose that we know $P(A|B)$, but we are interested in the probability $P(B|A)$. Using the definition of conditional probability, we have $$P(A|B)P(B)=P(A \cap B)=P(B|A)P(A).$$ Dividing by $P(A)$, we obtain $$P(B|A)=\frac{P(A|B)P(B)}{P(A)},$$ which is the famous Bayes' rule. Often, in order to find $P(A)$ in Bayes' formula we need to use the law of total probability, so sometimes Bayes' rule is stated as $$P(B_j|A)=\frac{P(A|B_j)P(B_j)}{\sum_{i} P(A | B_i) P(B_i)},$$ where $B_1, B_2,\cdots, B_n$ form a partition of the sample space.
- For any two events $A$ and $B$, where $P(A)\neq 0$, we have $$P(B|A)=\frac{P(A|B)P(B)}{P(A)}.$$
- If $B_1, B_2, B_3,\cdots$ form a partition of the sample space $S$, and $A$ is any event with $P(A)\neq 0$, we have $$P(B_j|A)=\frac{P(A|B_j)P(B_j)}{\sum_{i} P(A | B_i) P(B_i)}.$$
Example
In Example 1.24, suppose we observe that the chosen marble is red. What is the probability that Bag 1 was chosen?
- Solution
-
Here we know $P(R|B_i)$ but we are interested in $P(B_1|R)$, so this is a scenario in which we can use Bayes' rule. We have
$P(B_1|R)$ $=\frac{P(R|B_1)P(B_1)}{P(R)}$ $=\frac{0.75 \times \frac{1}{3}}{0.6}$ $=\frac{5}{12}$.
$P(R)$ was obtained using the law of total probability in Example 1.24, thus we did not have to recompute it here. Also, note that $P(B_1|R)= \frac{5}{12}>\frac{1}{3}$. This makes sense intuitively because bag 1 is the bag with the highest number of red marbles. Thus if the chosen marble is red, it is more likely that bag 1 was chosen.
-
Example (False positive paradox [5])
A certain disease affects about $1$ out of $10,000$ people. There is a test to check whether the person has the disease. The test is quite accurate. In particular, we know that
- the probability that the test result is positive (suggesting the person has the disease), given that the person does not have the disease, is only 2 percent;
- the probability that the test result is negative (suggesting the person does not have the disease), given that the person has the disease, is only 1 percent.
- Solution
-
Let $D$ be the event that the person has the disease, and let $T$ be the event that the test result is positive. We know $$P(D)=\frac{1}{10,000},$$ $$P(T|D^c)=0.02,$$ $$P(T^c|D)=0.01$$ What we want to compute is $P(D|T)$. Again, we use Bayes' rule:
$P(D|T)$ $=\frac{P(T|D)P(D)}{P(T|D)P(D)+P(T|D^c)P(D^c)}$ $=\frac{(1-0.01)\times 0.0001}{(1-0.01)\times 0.0001+0.02 \times (1-0.0001)}$ $=0.0049$
This means that there is less than half a percent chance that the person has the disease.
-
Discussion: This might seem somewhat counterintuitive as we know the test is quite accurate. The point is that the disease is also very rare. Thus, there are two competing forces here, and since the rareness of the disease (1 out of 10,000) is stronger than the accuracy of the test (98 or 99 percent), there is still good chance that the person does not have the disease.
Another way to think about this problem is illustrated in the tree diagram in Figure 1.25. Suppose 1 million people get tested for the disease. Out of the one million people, about $100$ of them have the disease, while the other $999,900$ do not have the disease. Out of the $100$ people who have the disease $100 \times .99=99$ people will have positive test results. However, out of the people who do not have the disease $999,900 \times .02=19998$ people will have positive test results. Thus in total there are $19998+99$ people with positive test results, and only $99$ of them actually have the disease. Therefore, the probability that a person from the "positive test result" group actually have the disease is $$P(D|T)=\frac{99}{19998+99}=0.0049$$
