Random Variables and Probability Distributions

Random variables and probability distributions are key concepts in probability theory and statistics.

A random variable is a variable whose possible values depend on the outcome of a random event. For example, if we flip a coin, the random variable could be the number of heads that come up – this value could be 0, 1 or 2 depending on the outcome of the coin flip.

Probability distributions describe the likelihood of each possible value of a random variable. There are two main types of probability distributions: discrete and continuous.

Discrete probability distributions are used when the random variable takes on a finite or countably infinite number of possible values. Examples of discrete probability distributions include the binomial distribution, which describes the number of successes in a fixed number of independent trials, and the Poisson distribution, which describes the number of events that occur in a fixed interval of time.

Continuous probability distributions are used when the random variable takes on a continuous range of possible values. Examples of continuous probability distributions include the normal distribution, which describes many natural phenomena such as heights and weights, and the exponential distribution, which is often used to model waiting times.

Random Variable Definition

A random variable is a function that associates a unique numerical value with every outcome of an experiment. The value of the random variable will vary from trial to trial as the experiment is repeated.

The outcome of an experiment need not be a number, for example, the outcome when a coin is tossed can be ‘heads’ or ‘tails’. However, we often want to represent outcomes as numbers.

Let X represent a function that associates a Real number with each and every elementary event in some sample space S. Then X is called a random variable on the sample space S.

  • If random variable can only equal a finite number of values, it is a discrete random variable. Probability distribution is known as a “probability mass function” or just p.m.f.
  • If a random variable can equal an infinite (or really really large) number of values, then it is a continuous random variable. Probability distribution is know as a “probability density function” or just p.d.f.
  • A probability distribution can be shown using tables or graph or mathematical equation.

Random Variable Examples

  • A coin is tossed ten times. The random variable X is the number of tails that are noted. X can only take the values 0, 1, …, 10, so X is a discrete random variable.
  • Examples of discrete random variables include the number of children in a family, the Friday night attendance at a cinema, the number of patients in a doctor’s surgery, the number of defective light bulbs in a box of ten.
  • A light bulb is burned until it burns out. The random variable Y is its lifetime in hours. Y can take any positive real value, so Y is a continuous random variable.
  • Examples of continuous random variables include height, weight, the amount of sugar in an orange, the time required to run a mile.

Probability Mass Function (PMF)

The probability distribution of a discrete random variable is a list of probabilities associated with each of its possible values. It is also sometimes called the probability function or the probability mass function.

More formally, the probability distribution of a discrete random variable X is a function which gives the probability p(xi) that the random variable equals xi, for each value xi:

p(xi) = P(X=xi)

It satisfies the following conditions:

  • 0 <= p(xi) <= 1
  • sum of all p(xi) is 1

This same concept was discussed in the previous page at the time of explaining Sample Space example with  one and two coin tosses. A probability distribution lists all possible outcome in the sample space and the probabilities with which they occur has given below.

As, sum of probability of all outcomes add up to 1 so, it is called probability mass function or p.m.f.

I already mentioned that a probability distribution can be shown using tables or graph or mathematical equation. If we plot the same distribution using a histogram then X axis represents the cases (HH, TT, HT, TH)  and Y- axis represents it probability value. But for probability density functions (p.d.f.) Y-axis presents the probability density value. In the below picture a probability distribution of a lottery spinner has been shown.

If we draw a histogram for this then it will look like this. Observed the X and Y axis.

Cumulative Distribution Function (CDF)

All random variables (discrete and continuous) have a cumulative distribution function. It is a function giving the probability that the random variable X is less than or equal to x, for every value x.

Formally, the cumulative distribution function F(x) is defined to be:

F(x) = P(X<=x)

for

-infinity < x < infinity

For a discrete random variable, the cumulative distribution function is found by summing up the probabilities as in the example below.

For a continuous random variable, the cumulative distribution function is the integral of its probability density function.

Example

Discrete case : Suppose a random variable X has the following probability distribution p(xi):

xi                             0              1              2              3              4              5

p(xi)                       1/32       5/32       10/32     10/32     5/32       1/32

This is actually a binomial distribution: Bi(5, 0.5) or B(5, 0.5). The cumulative distribution function F(x) is then:

xi                             0              1              2              3              4              5

F(xi)                       1/32       6/32       16/32     26/32     31/32     32/32

F(x) does not change at intermediate values. For example:

F(1.3) = F(1) = 6/32

F(2.86) = F(2) = 16/32

Consider another distribution example below. Now based on below picture what is the probability that X takes a value of either 2 or 3.

P( X=2 or X=3 ) = P(X=2) + P(x=3) = 0.3+0.4 =0.7

Here union of probabilities = sum of probabilities.

What is the probability that X is greater than 1 ?

P(X=2 or X=3 or X=4 ) = 0.3+0.4+ 0.2 =0.9

On the other way you can tell P(X=2 or X=3 or X=4 )= 1- P(x=1) = 1- 0.1 = 0.9 both the answer are same. Here we are finding from complementary rule.

Based on a probability distribution we can easily calculate probabilities for values that are less than or equal to a given value. The probability of X is less than or 1 is 0.1. Similarly, probability of X is less than or equal to 2 is (0.1+0.3) =0.4 and so on.

Probability histogram  of cumulative probability distribution has shown below for the above example.

Probability Density Function (PDF)

The probability density function of a continuous random variable is a function which can be integrated to obtain the probability that the random variable takes a value in a given interval. A probability density function will look like the below diagram. For p.d.f. Y-axis does represent the probability rather it represents the probability density value. To get probability from here you need to consider certain interval under the curve rather than height of the curve at certain location what we do for probability mass function. Here probability if given by the surface area under the curve with a interval.

More formally, the probability density function, f(x), of a continuous random variable X is the derivative of the cumulative distribution function F(x):

Since it follows that:

Since F(x) = P(X<=x) If f(x) is a probability density function then it must obey two conditions:

  • that the total probability for all possible values of the continuous random variable X is 1:
  • that the probability density function can never be negative: f(x) > 0 for all x.

Mean of a Random Variable

The mean of a random variable indicates its average or central value. It is a useful summary value of the variable’s distribution.

As it is the expected average outcome from many observation so we can tell it as Expected value also.

Stating the expected value gives a general impression of the behavior of some random variable without giving full details of its probability distribution (if it is discrete) or its probability density function (if it is continuous).

  • The mean of a discrete random variable is the probability-weighted average of all possible values that the random variable can take. Check the below formula for mean of discrete random variable.
  • The mean of a continuous random variable is the average of all possible values that the random variable can take. We can represent that as integral and function of X. Check the formula for mean of a continuous random variable.

The expected value of a random variable X is symbolized by E(X) or µ.

If X is a discrete random variable with possible values x1, x2, x3, …, xn, and p(xi) denotes P(X = xi), then the expected value of X is defined by:

where the elements are summed over all values of the random variable X.

If X is a continuous random variable with probability density function f(x), then the expected value of X is defined by:

Mean of a Random Variable – Example

For Discrete Case: When a die is thrown, each of the possible faces 1, 2, 3, 4, 5, 6 (the xi’s) has a probability of 1/6 (the p(xi)’s) of showing. The expected value of the face showing is therefore:

µ = E(X) = (1 x 1/6) + (2 x 1/6) + (3 x 1/6) + (4 x 1/6) + (5 x 1/6) + (6 x 1/6) = 3.5

Notice that, in this case, E(X) is 3.5, which is not a possible value of X.

Variance of a Random Variable

The variance of a random variable is a non-negative number which gives an idea of how widely spread the values of the random variable are likely to be; the larger the variance, the more scattered the observations on average.

Stating the variance gives an impression of how closely concentrated round the expected value the distribution is; it is a measure of the ‘spread’ of a distribution about its average value.

Variance is symbolized by V(X) or Var(X) or 

The variance of the random variable X is defined to be:

where E(X) is the expected value or mean  of the random variable X.

  • The larger the variance, the further that individual values of the random variable (observations) tend to be from the mean, on average;
  • the smaller the variance, the closer that individual values of the random variable (observations) tend to be to the mean, on average;
  • taking the square root of the variance gives the standard deviation, i.e.:
  • The variance and standard deviation of a random variable are always non-negative.

Take the following example where a distribution has been shown for a person to get involved in a traffic accident. The mean risk is 1/25 accident/ year

To calculate the variance follow the steps given in diagram.

So far, variance of random variable has been discussed in terms of discrete variable only.

Calculating variance of a continuous random variable is complex and difficult. We need to integrate the below function to get that.

Some examples of random variables and probability distributions

Coin flip

The random variable is the number of heads that come up in a single flip of a fair coin. The probability distribution is a discrete uniform distribution with two possible outcomes – heads or tails, each with a probability of 0.5.

Roll of a die

The random variable is the number that comes up when rolling a fair six-sided die. The probability distribution is a discrete uniform distribution with six possible outcomes, each with a probability of 1/6.

Number of customers in a store

The random variable is the number of customers who visit a store in a given day. The probability distribution could be a Poisson distribution, which would describe the probability of a certain number of customers arriving based on the average number of customers per day.

Height of people in a population

The random variable is the height of an individual in a population. The probability distribution could be a normal distribution, which describes the distribution of many natural phenomena including heights and weights.

Time between customer arrivals

The random variable is the time between when one customer leaves and the next customer arrives at a store. The probability distribution could be an exponential distribution, which is often used to model waiting times.

Understanding random variables and probability distributions is essential for a wide range of applications, from finance and economics to engineering and data science.

Elementary Probability

Normal Distribution, Binomial Distribution & Poisson Distribution