## 0-Introduction Bayes' theorem is a fundamental concept in probability theory that describes how to update our beliefs about an event based on new evidence. It's a powerful tool used in various fields like medicine, statistics, machine learning, and even in today's AI. In this blog, I will give a brief introduction about Bayes' theorem (just note some formulas) ## 1-The Basics of Probability Before diving into the theorem, let's quickly review some probability basics: - Event: An outcome or a set of outcomes of an experiment(e.g., flipping a coin and getting heads) - Probability: A measure of the likelihood of an event occuring, ranging from 0 to 1. - Conditional Probability: The probability of an event happing given that another event has already occurred. We denoted this as $P(A|B)$ which reads "the probability of event A given event B" ## 2-Introducing Bayesian Theorem Bayesian Theorem states as follows: $ P(A|B) = \frac{P(B|A) \times P(A)}{P(B)} $ There are some components in this formula needed to be clarified: - $P(A|B)$ is posterior probability - The probability of event A occurring given that event B has occurred (what we want to find) - $P(B|A)$ is likelihood - The probability of event B occurring given that event A has occurred. - $P(A)$ is prior probability - The initial probability of event A occurring(our belief before considering new evidence) - $P(B)$ is marginal likelihood - The probability of event B occurring. ## 3-An example to illustrate Let's say we want to determine the probability of a person having a disease (event A) given that they tested positive for it(event B). - Prior $P(A)$: Let's assume the disease is rare, with a prevalance of 1% in the population. So, $P(A) = 0.01$. - Likelihood $P(B|A)$: Let's say the test is accurate, with 95% a chance of a positive result if the person has the disease. So, $P(B|A)=0.95$ - Marginal Likelihood: This is slightly tricker. We need to consider the probability of testing positive whether the person has the disease or not. We can calculate it using the law of total probability. $ P(B) = P(B|A)P(A) + P(B|-A)P(-A) $ Here, $-A$ means not A. So according to $P(A) =0.01$ and $P(B|A)=0.95$, we can get that $P(B|-A) = 1-0.95=0.05$ and $P(-A) = 1-0.01=0.99$; Now, we get : $ P(B) = 0.95 \times 0.01 + 0.05 \times 0.99 =0.059 $ - Posterior $P(A|B)$: Now, we can apply Bayesian Theorem: $ \begin{aligned} P(A|B) &= \frac{0.95\times 0.01}{0.059} =0.16 \end{aligned} $ ## 4-Conditional Probability on Multiple Events When dealing with more than two events, we enter the realm of multivariate Bayesian probability. The notation $P(A|B.C)$ represents the probability of event $A$ occurring given that both event $B$ and $C$ have occurred. The formula for the conditional probability of $A$ given $B$ and $C$ is: $ P(A|B,C) = \frac{P(A,B, C)}{P(B,C)} $ where $P(A,B,C$ is the joint probability of $A$, $B$, $C$ occurring together, and $P(B,C)$ is the joint probability of $B$ and $C$ occurring. We can also express this using the chain rule of probability: $ \begin{aligned} P(A|B,C) &= \frac{P(A,B,C)}{P(B,C)} \\ &=\frac{P(C|A,B)\times P(A|B)\times P(B)}{P(C|B) \times P(B)} \end{aligned} $ ## 5-Conclusion In this brief blog post, I introduce the renowned Bayes' Theorem. However, it's important to note that in advanced statistical research, the theorem's applications are far more complex. This post serves as a starting point, offering a glimpse into the fundamental concept while acknowledging that there is much more to explore and understand in the field of Bayesian statistics.