The classical definition of probability

     Let us consider an urn U that contains n balls, from which m balls are white and n – m  balls are black. A ball is drawn at random. We have n elementary events. Let A be the event the drawn ball is white. This event can occur after m tests, m is less than or equal to n.

Definition:  Call the probability of event A the ratio between the number of situations (tests) favorable for A to occur and the number of equally possible situations. Therefore,  P = m/n.

This is the classical definition of probability. It can only be used in experiments having equally possible elementary events.

     Probability on a finite field of events

In the case of an arbitrary finite field of events{Ω, Σ}, a probability on this field is defined as follows:

Definition:  A finite field of events {Ω, Σ}, structured with a probability P, is called a finite probability field (finite probability space) and is denoted by {Ω, Σ, P}.

Probability as a measure

The definition corresponds to the definition of tribe in measure theory.

The two conditions from the definition of probability imply the axioms in the definition of measure.
     Therefore, the probability is a measure P with P(Ω) = 1, so it acquires all the properties of a measure.

Probability as a limit

There is an important theorem that illustrates the way in which probability models the hazard. We enunciate the Law of Large Numbers, not in its general mathematical form, in order to avoid having to define more complex concepts, but in an exemplified particular form, in a way that everyone can understand. The particular enunciation is the following classic result, known as Bernoulli’s Theorem: 

The relative frequency of the occurrence of a certain event in a sequence of independent experiments performed under identical conditions converges toward the probability of that event.

We exemplify this expression by considering the classical experiment of tossing a coin: Let A be the event the coin falls heads up. Obviously, P(A) = 1/2. Let us say that event A has the following occurrences: 

 – after the first throw, 0 occurrences, relative frequency 0/1;
       – after the first two throws, 0 occurrences, relative frequency 0/2;
       – after the first three throws, 1 occurrence, relative frequency 1/3;
       – after the first four throws, 1 occurrence, relative frequency 1/4;
       – after the first five throws, 2 occurrences, relative frequency 2/5;
  ……………………………………………………
     

  The Law of Large Numbers says that the sequence obtained at the right, namely 0, 0, 1/3, 1/4, 2/5, … is convergent toward 1/2. In other words, while n is growing, the relative frequency is approximating 1/2 with higher accuracy. Or we can say we can find a big enough number of experiments for the relative frequency of occurrences of A to approach 1/2 with any decimal place we want. The Law of Large Numbers confers on probability a property of limit.
     Of course, the theorem does not provide information about the terms of the sequence of relative frequencies, but only about its limit. In other words, we cannot make a precise prediction on the event at a certain chronological moment, but we can only know the behavior of the relative frequency of occurrences at infinity.

As a whole, the following ideas must be kept in mind:

●  Probability is nothing more than a measure; as length measures distance and area measures surface, probability measures aleatory events. As a measure, probability is in fact a function with certain properties, defined on the field of events generated by an experiment.

●  A probability is characterized not only by the specific function P, but by the entire aggregate the set of possible outcomes of the experiment – the field of generated events – function P, called probability field; probability makes no sense and cannot be calculated unless we initially rigorously define the probability field in which we operate.

●  Probability is not a punctual numerical value; textually given an event, we cannot calculate its probability without including it in a more complex field of events. Probability as a number is in fact a limit, respectively the limit of the sequence of relative frequencies of occurrences of the event to measure, within a sequence of independent experiments.

Sources

The book UNDERSTANDING AND CALCULATING THE ODDS: Probability Theory Basics and Calculus Guide for Beginners, with Applications in Games of Chance and Everyday Life is addressed to non-mathematicians and builds a clear image of the probability concept by reconstructing its mathematical definition step by step through its constituent notions, starting with fundamental notions like sets, functions, convergence and measure theory basics. You may find it in the Books section with a free sample.