Probability — Frequentist and Bayesian approach

To understand two different approaches to probability, we should be initially very clear about Probability Theory.

Probability Theory is a mathematical framework for representing uncertain Statements.

In Computer science we rarely talk about uncertainty and take most of the entities to be certain and deterministic.

• Errors in hardware will not occur
• CPU will execute every command that you give to it

But unlike the case, in machine learning we deal with uncertain quantities and sometime with even non-deterministic (stochastic) quantities as well.

The three possible reasons for uncertainty are:-

1. Inherent stochasticity in the system
2. Incomplete observability
3. Incomplete modelling

To understand the reasons mentioned above, lets take two statements to account:-

1. Most birds fly
2. Birds fly, except for very young or very old, sick or injured, flightless species including ostrich, kiwi etc…..

Its more reasonable to use simple but uncertain rule instead of complex but certain one. Rule 1 lacks observability and…