Confidence intervals and hypothesis tests are similar in that they are both inferential methods that rely on an approximated sampling distribution. The purpose of a confidence interval is to estimate a population parameter based on sample data, while a hypothesis test is used to evaluate a specific claim about a population parameter using sample data. It is important to note that hypothesis testing requires a pre-specified parameter value to be tested.

The simulation methods used to construct bootstrap distributions and randomization distributions are similar. One primary difference is a bootstrap distribution is centered on the observed sample statistic while a randomization distribution is centered on the value in the null hypothesis.

A confidence interval is a range of values within which a population parameter, such as a mean or proportion, is estimated to lie with a certain level of confidence. For example, a 95% confidence interval for the population mean would imply that if the sampling procedure was repeated many times, then 95% of the intervals generated would contain the true population mean. Confidence intervals are typically computed using the sample mean and standard deviation, and they provide a measure of the precision of an estimate.

Hypothesis testing, on the other hand, is a procedure for evaluating whether a claim about a population parameter is supported by the sample data or not. Hypothesis testing involves setting up null and alternative hypotheses, calculating a test statistic based on the sample data, and comparing the test statistic to a critical value or p-value. If the test statistic falls in the rejection region, then the null hypothesis is rejected in favor of the alternative hypothesis, which implies that the sample data provides evidence in support of the claim being tested.

Both confidence intervals and hypothesis testing are important tools for making statistical inferences from sample data. Confidence intervals provide information about the precision of an estimate, while hypothesis testing provides a formal procedure for evaluating claims about population parameters. Together, they provide a powerful framework for making data-driven decisions and drawing conclusions about populations based on sample data.

Selecting the Appropriate Procedure

Confidence intervals and hypothesis testing are two different but complementary approaches to making statistical inferences from sample data.

The decision of whether to use a confidence interval or a hypothesis test depends on the research question. If we want to estimate a population parameter, we use a confidence interval. If we are given a specific population parameter (i.e., hypothesized value), and want to determine the likelihood that a population with that parameter would produce a sample as different as our sample, we use a hypothesis test. Below are a few examples of selecting the appropriate procedure.

Example: Cheese Consumption

Research question: How much cheese (in pounds) does an average American adult consume annually?

What is the appropriate inferential procedure? 

It is important to note that both confidence intervals and hypothesis testing are based on assumptions about the underlying population, sample size, and sampling distribution, and the choice of procedure should be made based on the research question and the underlying assumptions. In practice, both approaches are often used together to provide a more comprehensive understanding of the population parameter being studied.