Tag: probability

Posted on

The Normal Distribution: The Basics

The NORMAL DISTRIBUTION (sometimes referred to as the Gaussian distribution) is a continuous probability distribution that can be found in many places: height, weight, IQ scores, test scores, errors, reaction times, etc.  Understanding that a variable is normally distributed allows you to:

  • predict the likelihood (i.e., probability) of observing certain values
  • apply various statistical techniques that assume normality
  • establish confidence intervals and conduct hypothesis tests

Characteristics of the Normal Distribution

There are several key characteristics of the normal distribution:

  • mean, median, and mode are equal and located at the center of the distribution
  • the distribution is symmetric about the mean (i.e., the left half of the distribution is a mirror image of the right half); “Scores above and below the mean are equally likely to occur so that half of the probability under the curve (0.5) lies above the mean and half (0.5) below” (Meier, Brudney, and Bohte, 2011, p. 132)
  • the distribution resembles a bell-shaped curve (i.e., highest at the mean and tapers off towards the tails)
  • the standard deviation determines the SPREAD of the distribution (i.e., its height and width): a smaller standard deviation results in a steeper curve, while a larger standard deviation results in a flatter curve
  •  the 68-95-99 RULE can be used to summarize the distribution and calculate probabilities of event occurrence:
    – approximately 68% of the data falls within ±1 standard deviation of the mean
    – approximately 95% of the data falls within ±2 standard deviations of the mean
    – approximately 99% of the data falls within ±3 standard deviations of the mean
  • there is always a chance that values will fall outside ±3 standard deviations of the mean, but the probability of occurrence is less than 1%
  • the tails of the distribution never touch the horizontal axis: the probability of an outlier occurring may be unlikely, but it is always possible; thus, the upper and lower tails approach, but never reach, 0%

Why the Normal Distribution is Common in Nature: The Central Limit Theorem

The CENTRAL LIMIT THEOREM states that the distribution of sample means for INDEPENDENT, IDENTICALLY DISTRIBUTED (IID) random variables will approximate a normal distribution, even when the variables themselves are not normally distributed, assuming the sample is large enough.  Thus, as long as you have a sufficiently large random sample, we can make inferences about the population parameters (what we are interested in) from sample statistics (what we often are working with).

What Does “IID” Mean?

Variables are considered independent if they are mutually exclusive.  Variables are considered identically distributed if they have the same probability distribution (i.e., normal, Poisson, etc.)

Do Outliers Matter?

In a normal distribution based on a large number of observations, it is unlikely that outliers will skew results.  If you are working with data involving fewer observations, outliers are more likely to skew results; in these situations, you should identify, invest, and decide how to handle outliers.

Example of a Normal Distribution: IQ Tests

Because the IQ test has been given millions of times, IQ scores represent a normal probability distribution.  On the IQ test, the mean, median, and mode are equal and fall in the middle of the distribution (100).  The standard deviation on the IQ test is 15; applying the 68-95-99 rule, we can say with reasonable certainty:

  • 68% of the population will score between 85 and 115, or ±1 standard deviation from the mean
  • 95% of the population will score between 70 and 130, or ±2 standard deviations from the mean
  • 99% of the population will score between 55 and 145, or ±3 standard deviations from the mean

Rarely will you encounter such a perfect normal probability distribution as the IQ test, but we can calculate z-scores to standardize (i.e., “normalize”) values for distributions that aren’t as normal as the IQ distribution.

Posted on

Probability Theory: The Basics

PROBABILITY is a branch of mathematics that deals with the likelihood or chance of different outcomes occurring in uncertain situations.  It quantifies how likely an event is to happen and is expressed as a number between 0 and 1, where 0 indicates impossibility and 1 indicates certainty. 

Probabilities are important to understand because they give us predictive capabilities: probabilities are the closest thing we have to being able to predict the future.  Of course, this is not foolproof; there is always a chance that we are wrong, which is why we couch results/findings in terms of a 95% confidence interval and margin of error. 

Basic Law of Probability

“The BASIC LAW OF PROBABILITY . . . states the following: Given that all possible outcomes of a given event are equally likely, the probability of any specific outcome is equal to the ratio of the number of ways that that outcome could be achieved to the total number of ways that all possible outcomes can be achieved” (Meier, Brudney, and Bohte, 2011, p. 113).  This means that we can predict the outcome of a specific event as long as the likelihood of a given event is known.  For example:

  • the probability of getting heads with one coin flip is 1/2
  • the probability of getting three with one roll of a six-sided dice is 1/6
  • the probability of drawing the ace of spades from a deck of cards is 1/52
  • the probability of drawing an ace from a deck of cards is 1/13 (although there are 52 cards in a deck, there are four aces; to calculate the probability of drawing an ace from a deck of cards, you would divide 52 by four)
  • the probability of drawing a heart from a deck of 52 cards is 1/4 (this is because there are four suits in each deck of cards)

All of these examples involve probabilities of the occurrence of single events.  As you can see, the process of calculating these probabilities is pretty straightforward: you divide the number of times a specific outcome can occur by the total number of possible outcomes.

Probability P(A) of a Single Event A

P (A) = Number of favorable outcomes / Total number of possible outcomes

This gets a little more complicated as we factor in other events; the manner in which we would calculate the probability of an event occurring differs depending on whether we are looking at mutually exclusive events (i.e., events that cannot occur at the same time), non-mutually exclusive events (i.e., events that can occur at the same time), independent events (i.e., events in which the occurrence of one does not affect the occurrence of the other), an event occurring given the occurrence of a different event (i.e., the conditional probability), etc.  Nevertheless, while different equations are used to calculate probabilities in these situations, the basic law of probability still exists: the probability of an event occurring falls between 0 (“never occurs”) and 1 (“always occurs”), and calculating this probability is based on possible outcomes.

A Priori and Posterior Probabilities 

A PRIORI PROBABILITIES are initial probabilities of an event based on existing knowledge, theory, or general reasoning about the event.  Everything we have discussed thus far are a priori probabilities, because we know the possible outcomes of a coin flip, die roll, or card draw.  By contrast, POSTERIOR PROBABILITIES are probabilities of an event after new evidence or information is taken into account.  A classic example of posterior probability is the Monty Hall problem.  Posterior probabilities are often calculated using BAYES’ THEOREM, which combines the prior probability with the likelihood of new evidence or information.  Frequency distributions provide the empirical data needed to estimate the probabilities used in Bayes’ theorem. 

Privacy Policy Proudly powered by WordPress