Math III · S-IC.2

Using Simulation to Decide Whether Data Are Consistent with a Proposed Model

Simulation lets students test whether an observed result is surprising or ordinary under a proposed chance model.

Concept Statistics and Probability

Domain Making Inferences and Justifying Conclusions

Read time 5 minutes

What this learning objective is really asking you to learn

This objective asks students to use simulation to decide whether data are consistent with a proposed model. A model is a claim about how a random process works. Data are what actually happened. Simulation lets students repeatedly imitate the model and see what kinds of results it typically produces.

For example, suppose someone claims a coin is fair. A fair coin model says heads should occur with probability 0.5 on each flip. If the coin is flipped 100 times and lands heads 62 times, is that result consistent with a fair coin? It is more than 50, but random variation is expected. Simulation can help.

To simulate, use a fair coin or random number generator to create many trials of 100 flips. Count the number of heads in each simulated trial. Then compare the observed result, 62 heads, to the simulation distribution. If results of 62 or more heads happen fairly often, the observed data are consistent with the fair coin model. If they almost never happen, the data provide evidence against the model.

This objective introduces the logic of simulation-based inference. Instead of relying only on formulas, students generate random outcomes under a proposed model. Then they ask: is the observed result typical or surprising?

The proposed model is sometimes called a null model in later statistics. It represents the claim being tested: fair coin, no treatment effect, random guessing, equal preference, no association, or some stated probability. Simulation shows the natural variability expected if that model were true.

Students should understand that simulation does not prove a model is true. It helps judge whether observed data are plausible under the model.

Why students should learn this math

Students should learn simulation because randomness is hard to reason about intuitively. People often overreact to unusual-looking results or underreact to genuinely surprising ones. Simulation gives a concrete way to see what chance alone can produce.

For example, if a basketball player makes 8 of 10 free throws, is that evidence the player is better than a 60% shooter? Maybe, but 8 out of 10 can happen by chance even for a 60% shooter. Simulation can estimate how often. If it happens often enough, the data are not strong evidence. If it is rare, the data are more surprising.

Simulation is also accessible. Students do not need advanced probability formulas to run repeated trials. They can use coins, dice, cards, spinners, random number generators, or computer simulations. This makes inference visible.

Modern science and data work use simulation constantly. Weather models, election forecasts, medical trials, financial risk models, traffic models, manufacturing quality checks, and machine-learning evaluations all use simulated or resampled data to understand uncertainty. Simulation is not a classroom shortcut. It is a serious computational tool.

This objective also teaches students to think in distributions. One simulated result is not enough. Many simulated results show a pattern of expected variation. The observed data are then compared to that pattern.

The “why” is that simulation helps students distinguish ordinary randomness from evidence against a model.

The historical machinery: computation changes statistics

Before computers, simulation was possible but slow. People could use dice, cards, random-number tables, or mechanical devices. With computers, simulation became a major statistical tool. Repeating thousands or millions of random trials became easy.

Simulation-based inference has become increasingly important because many real problems are too complicated for simple formulas. Computers can approximate distributions by generating many random outcomes. This approach supports modern statistics, risk analysis, scientific modeling, and data science.

The basic idea is simple: if you know or assume a model, generate many possible outcomes from it. Then see whether the real outcome looks typical. This logic is powerful even before students learn formal hypothesis tests.

The historical lesson is that probability plus computation gives a practical way to reason about uncertainty.

Where this fits in the big map of mathematics

This objective follows the introduction to statistical inference. Objective 179 explains samples, populations, parameters, and statistics. Objective 180 shows how simulation can evaluate whether observed sample data are plausible under a model.

It connects to probability models. Simulation requires assumptions about chance.

It connects to sampling variability. Simulations show how statistics vary from sample to sample.

It connects to conditional probability and independence. Models often assume independent trials or specified probabilities.

It connects to later hypothesis testing. Simulation is an intuitive path into p-values and statistical significance.

It connects to technology. Random number generators make simulation efficient.

The big-map role is evidence through repeated random trials. Students learn to ask whether data are surprising under a stated model.

How to execute the skill technically

Use this simulation routine:

State the proposed model.
Define the statistic to measure.
Simulate many trials under the model.
Record the statistic for each trial.
Build a distribution of simulated statistics.
Compare the observed statistic to the simulated distribution.
Decide whether the observed result is typical or surprising.
Interpret carefully.

Example: A multiple-choice quiz has 20 questions, each with 4 choices. A student claims they guessed randomly but got 10 correct. Is that surprising?

Model: random guessing, probability correct = 1/4 per question.

Statistic: number correct out of 20.

Simulate many trials of 20 guesses, where each question has 25% chance correct. Count correct answers each time.

If 10 or more correct happens rarely in the simulation, the result is surprising under random guessing. If it happens fairly often, it is consistent with guessing.

Technology can estimate the probability by running thousands of trials.

Coin example

Observed: 62 heads in 100 flips.

Model: fair coin.

Simulate 100 fair coin flips many times. Count heads.

Suppose in 1,000 simulations, 62 or more heads occurs 18 times. That is about 1.8% of simulations. This would be fairly surprising under the fair coin model and may provide evidence the coin is biased toward heads.

But if 62 or more occurred 120 times, or 12%, the result would not be very surprising. The conclusion depends on the simulation distribution.

What “consistent with the model” means

Consistent does not mean proven true. If data are consistent with a model, it means the observed result is not unusual under that model. Many models may be consistent with the same data. More data or different measurements may be needed to distinguish them.

Not consistent means the observed result is unusual under the proposed model. That gives evidence against the model, but students should still consider data quality, assumptions, and context.

For example, if a coin gives 90 heads in 100 flips, that is very unusual under a fair coin model. But before declaring the coin biased, check whether flips were recorded correctly, whether the coin was tossed properly, and whether the process was independent.

Problem Library

Problems in the App From This Objective

177 problems across 12 archetypes in the app.

define assumptions and outcomes.

15 problems Warmup Practice Mixed Review Assessment

Problem 1

State proposed probability model for simulation fair coin landing heads.

Problem 2

State proposed probability model for simulation roll fair die and count six.

Problem 3

State proposed probability model for simulation select colored marble from bag with 3 red and 2 blue.

Problem 4

State proposed probability model for simulation basketball free throw with 70 percent success.

Problem 5

State proposed probability model for simulation drawing the Ace of Spades from a standard 52-card deck.

Open in simulator

Problem 6

State proposed probability model for simulation rolling two fair dice and observing a sum of 7.

Problem 7

State proposed probability model for simulation flipping a biased coin that lands on heads 60% of the time.

Problem 8

State proposed probability model for simulation randomly selecting a girl from a group of 15 boys and 10 girls.

Problem 9

State proposed probability model for simulation a spinner with 4 equally sized sections labeled A, B, C, D landing on C.

Problem 10

State proposed probability model for simulation arriving at a traffic light that is green for 45s, yellow for 5s, red for 30s in an 80s cycle.

Problem 11

State proposed probability model for simulation guessing the correct answer on a multiple-choice question with 5 options, one correct.

Problem 12

State proposed probability model for simulation drawing a red card from a standard 52-card deck.

Problem 13

State proposed probability model for simulation a manufacturing process producing a defective item with a 2% rate.

Problem 14

State proposed probability model for simulation rolling a fair 8-sided die and getting an even number.

Problem 15

State proposed probability model for simulation selecting a vowel from the letters A, B, C, D, E, F, G.

map random digits/tools to outcomes.

15 problems Warmup Practice Mixed Review Assessment

Problem 16

Design simulation for probability question simulate 30 percent chance of rain each day.

Problem 17

Design simulation for probability question simulate rolling two dice and getting sum 7.

Problem 18

Design simulation for probability question simulate drawing from bag with 2 red,3 blue.

Problem 19

Design simulation for probability question simulate 65 percent success.

Problem 20

Design simulation for probability question simulate a 50 percent chance of an event.

Problem 21

Design simulation for probability question simulate a 15 percent chance of a product being defective.

Open in simulator

Problem 22

Design simulation for probability question simulate a 1/6 chance of winning a prize.

Problem 23

Design simulation for probability question simulate drawing a specific card from a deck of 10 unique cards.

Problem 24

Design simulation for probability question simulate a survey where 40% prefer A, 30% prefer B, 30% prefer C.

Problem 25

Design simulation for probability question simulate flipping two coins and getting two heads.

Problem 26

Design simulation for probability question simulate a 1/4 chance of success.

Problem 27

Design simulation for probability question simulate a spinner with sections for Red (1/2), Blue (1/4), Green (1/4).

Problem 28

Design simulation for probability question simulate a 20 percent chance of rain.

Problem 29

Design simulation for probability question simulate drawing a red marble from a bag with 3 red, 5 blue, 2 green marbles.

Problem 30

Design simulation for probability question simulate rolling two dice and getting a sum of 10 or more.

identify one trial and success condition.

15 problems Warmup Practice Mixed Review Assessment

Problem 31

Identify one simulation trial and success condition for roll die until first six.

Problem 32

Identify one simulation trial and success condition for simulate 10 births for at least 6 girls.

Problem 33

Identify one simulation trial and success condition for randomly assign 20 subjects to treatments.

Problem 34

Identify one simulation trial and success condition for flip a coin 5 times for exactly 3 heads.

Problem 35

Identify one simulation trial and success condition for flip a coin until the first tail.

Problem 36

Identify one simulation trial and success condition for draw 3 cards from a deck for at least one ace.

Problem 37

Identify one simulation trial and success condition for draw a card with replacement until a heart.

Problem 38

Identify one simulation trial and success condition for inspect 10 products for no more than 1 defective item.

Problem 39

Identify one simulation trial and success condition for simulate a best-of-7 series until one team wins 4 games.

Open in simulator

Problem 40

Identify one simulation trial and success condition for simulate customer arrivals at a store for 1 hour.

Problem 41

Identify one simulation trial and success condition for roll two dice 10 times for a sum of 7 at least twice.

Problem 42

Identify one simulation trial and success condition for simulate 5 offspring from heterozygous parents for 3 recessive traits.

Problem 43

Identify one simulation trial and success condition for survey 50 people about product preference for at least 30 preferring product A.

Problem 44

Identify one simulation trial and success condition for simulate a random walk of 10 steps starting at 0.

Problem 45

Identify one simulation trial and success condition for simulate bus arrivals until 3 buses have arrived.

use relative frequency.

15 problems Warmup Practice Mixed Review Assessment

Problem 46

Estimate probability from simulation results 23 successes in 100 trials.

Problem 47

Estimate probability from simulation results 146 successes in 500 trials.

Problem 48

Estimate probability from simulation results 0 successes in 50 trials.

Problem 49

Estimate probability from simulation results s successes in n trials.

Problem 50

Estimate probability from simulation results 1 success in 4 trials.

Problem 51

Estimate probability from simulation results 5 successes in 10 trials.

Problem 52

Estimate probability from simulation results 34 successes in 200 trials.

Problem 53

Estimate probability from simulation results 75 successes in 250 trials.

Problem 54

Estimate probability from simulation results 10 successes in 10 trials.

Problem 55

Estimate probability from simulation results 0 successes in 100 trials.

Problem 56

Estimate probability from simulation results 123 successes in 400 trials.

Open in simulator

Problem 57

Estimate probability from simulation results 3 successes in 4 trials.

Problem 58

Estimate probability from simulation results 250 successes in 1000 trials.

Problem 59

Estimate probability from simulation results 17 successes in 50 trials.

Problem 60

Estimate probability from simulation results 345 successes in 1500 trials.

decide whether observation is typical or unusual.

12 problems Warmup Practice Mixed Review Assessment

Problem 61

Compare observed data to simulated distribution observed statistic lies in middle of simulated values.

Problem 62

Compare observed data to simulated distribution observed statistic exceeds 98 percent of simulated values.

Open in simulator

Problem 63

Compare observed data to simulated distribution two-sided test, observed absolute difference larger than 4 of 200 simulations.

Problem 64

Compare observed data to simulated distribution observed count near simulation center.

Problem 65

Compare observed data to simulated distribution observed statistic is smaller than 99% of simulated values.

Problem 66

Compare observed data to simulated distribution observed value is greater than 97 out of 100 simulated values.

Problem 67

Compare observed data to simulated distribution observed mean falls within the middle 90% of simulated means.

Problem 68

Compare observed data to simulated distribution observed proportion is at the 45th percentile of the simulated distribution.

Problem 69

Compare observed data to simulated distribution two-sided test, observed absolute difference is less extreme than 180 of 200 simulated differences.

Problem 70

Compare observed data to simulated distribution observed increase is greater than 80% of simulated increases.

Problem 71

Compare observed data to simulated distribution observed minimum value is among the lowest 1% of simulated minimums.

Problem 72

Compare observed data to simulated distribution observed median is identical to the median of the simulated distribution.

compare observed outcomes to expected random variation.

15 problems Warmup Practice Mixed Review Assessment

Problem 73

Use simulation to test fairness for coin has 62 heads in 100 tosses.

Problem 74

Use simulation to test fairness for die rolls six 35 times in 120 rolls.

Problem 75

Use simulation to test fairness for game spinner lands prize 5 out of 200 when model says 10 percent.

Problem 76

Use simulation to test fairness for coin shows 30 heads in 40 flips.

Problem 77

Use simulation to test fairness for six-sided die rolls a 4 ten times in 30 rolls.

Problem 78

Use simulation to test fairness for spinner designed for 25% chance of green lands green 3 times in 20 spins.

Problem 79

Use simulation to test fairness for coin yields 15 heads in 50 tosses.

Open in simulator

Problem 80

Use simulation to test fairness for die rolls an even number 28 times in 40 rolls.

Problem 81

Use simulation to test fairness for bag with 40% red marbles (rest blue) yields 18 red marbles in 30 draws with replacement.

Problem 82

Use simulation to test fairness for standard deck of cards (with replacement) draws a face card (J, Q, K) 10 times in 25 draws.

Problem 83

Use simulation to test fairness for a game has a 1 in 5 chance of winning, but a player wins 8 out of 20 games.

Problem 84

Use simulation to test fairness for a die shows a number less than 3 (1 or 2) 25 times in 60 rolls.

Problem 85

Use simulation to test fairness for coin produces 45 heads in 100 tosses.

Problem 86

Use simulation to test fairness for a spinner with 50% chance of blue lands blue 15 times in 50 spins.

Problem 87

Use simulation to test fairness for a random number generator (1-10) produces the number 7 only once in 50 trials.

simulate under proposed population proportion.

15 problems Warmup Practice Mixed Review Assessment

Problem 88

Use simulation to assess sample proportion sample 58 out of 100 support; proposed p=0.50.

Problem 89

Use simulation to assess sample proportion sample 42 percent; proposed population proportion 0.40.

Problem 90

Use simulation to assess sample proportion poll result below claimed p.

Problem 91

Use simulation to assess sample proportion A new drug is effective in 75 out of 150 patients; the manufacturer claims it's effective in 60% of patients.

Problem 92

Use simulation to assess sample proportion In a random sample of 200 voters, 112 favor Candidate A; the candidate's campaign manager claims 55% support.

Problem 93

Use simulation to assess sample proportion A quality control check found 15 defective items in a batch of 300; the acceptable defect rate is 4%.

Problem 94

Use simulation to assess sample proportion A school claims 80% of its students graduate within four years. A sample of 120 students showed 90 graduates.

Problem 95

Use simulation to assess sample proportion A coin is flipped 50 times, resulting in 30 heads. Is the coin fair (p=0.5)?.

Problem 96

Use simulation to assess sample proportion A survey of 400 people found 220 prefer brand X. A competitor claims brand X has less than 50% market share.

Problem 97

Use simulation to assess sample proportion An experiment showed 65 successes out of 100 trials. The theoretical probability of success is 0.60.

Problem 98

Use simulation to assess sample proportion A genetics study observed a trait in 25 out of 100 offspring. The expected proportion is 0.30.

Problem 99

Use simulation to assess sample proportion A company claims 90% customer satisfaction. A random sample of 250 customers revealed 215 satisfied customers.

Problem 100

Use simulation to assess sample proportion A new teaching method resulted in 35 out of 50 students passing a test. The old method had a 60% pass rate.

Problem 101

Use simulation to assess sample proportion A political candidate believes they have 45% support. A poll of 600 voters shows 240 support.

Problem 102

Use simulation to assess sample proportion A manufacturing process is supposed to produce 5% defective items. A sample of 1000 items contained 60 defectives.

Open in simulator

simulate random assignment or random sampling.

15 problems Warmup Practice Mixed Review Assessment

Problem 103

Use simulation to assess difference between groups random assignment treatment/control with observed mean difference 4.

Problem 104

Use simulation to assess difference between groups two sample proportions differ by 0.12.

Problem 105

Use simulation to assess difference between groups experiment has fixed outcomes and random labels.

Problem 106

Use simulation to assess difference between groups two groups of test scores with an observed median difference of 5 points.

Problem 107

Use simulation to assess difference between groups treatment and control groups show a difference in standard deviation of 2.5.

Problem 108

Use simulation to assess difference between groups an observed difference in average reaction times between two randomly assigned drug dosages.

Problem 109

Use simulation to assess difference between groups two independent samples of voters show a 7% difference in support for a candidate.

Open in simulator

Problem 110

Use simulation to assess difference between groups a clinical trial with a 15% difference in recovery rates between a new drug and a placebo group.

Problem 111

Use simulation to assess difference between groups two teaching methods result in a 10-point difference in the interquartile range of student scores.

Problem 112

Use simulation to assess difference between groups an experiment comparing two fertilizers shows one yields 3 more bushels per acre on average.

Problem 113

Use simulation to assess difference between groups two populations have a 0.05 difference in the proportion of individuals with a certain characteristic.

Problem 114

Use simulation to assess difference between groups a small study with 10 participants, 5 in each group, shows a mean difference of 8 units.

Problem 115

Use simulation to assess difference between groups an observed 6-point average improvement in a training group compared to a control group.

Problem 116

Use simulation to assess difference between groups two different advertising campaigns result in a 3% difference in click-through rates.

Problem 117

Use simulation to assess difference between groups comparing the average commute times of employees from two different office locations, finding a 10-minute difference.

describe proportion of simulated results at least as extreme.

15 problems Warmup Practice Mixed Review Assessment

Problem 118

Interpret informal p-value from simulation 7 of 1000 simulations at least as extreme.

Problem 119

Interpret informal p-value from simulation 42 of 500 simulations at least as extreme.

Problem 120

Interpret informal p-value from simulation two-sided count includes both tails 18 of 600.

Problem 121

Interpret informal p-value from simulation 3 of 100 simulations at least as extreme.

Open in simulator

Problem 122

Interpret informal p-value from simulation 120 of 2000 simulations at least as extreme.

Problem 123

Interpret informal p-value from simulation two-sided count includes both tails 25 of 500.

Problem 124

Interpret informal p-value from simulation 1 of 2000 simulations at least as extreme.

Problem 125

Interpret informal p-value from simulation 150 of 1000 simulations at least as extreme.

Problem 126

Interpret informal p-value from simulation 28 of 700 simulations at least as extreme.

Problem 127

Interpret informal p-value from simulation two-sided count includes both tails 60 of 1000.

Problem 128

Interpret informal p-value from simulation 55 of 500 simulations at least as extreme.

Problem 129

Interpret informal p-value from simulation 350 of 5000 simulations at least as extreme.

Problem 130

Interpret informal p-value from simulation 23 of 200 simulations at least as extreme.

Problem 131

Interpret informal p-value from simulation two-sided count includes both tails 10 of 1000.

Problem 132

Interpret informal p-value from simulation 250 of 500 simulations at least as extreme.

use simulation evidence and threshold language.

15 problems Warmup Practice Mixed Review Assessment

Problem 133

Decide whether data are consistent with model using simulation evidence p-value about 0.35.

Problem 134

Decide whether data are consistent with model using simulation evidence p-value about 0.01.

Problem 135

Decide whether data are consistent with model using simulation evidence observed result near center of simulations.

Problem 136

Decide whether data are consistent with model using simulation evidence observed result in extreme tail.

Problem 137

Decide whether data are consistent with model using simulation evidence p-value of 0.28.

Problem 138

Decide whether data are consistent with model using simulation evidence p-value less than 0.05.

Problem 139

Decide whether data are consistent with model using simulation evidence observed value falls within the middle 90% of simulated values.

Problem 140

Decide whether data are consistent with model using simulation evidence observed statistic is more extreme than 98% of simulated statistics.

Problem 141

Decide whether data are consistent with model using simulation evidence p-value of 0.06.

Problem 142

Decide whether data are consistent with model using simulation evidence observed outcome is very close to the mean of the simulation distribution.

Problem 143

Decide whether data are consistent with model using simulation evidence observed value is in the far right tail of the simulation distribution.

Problem 144

Decide whether data are consistent with model using simulation evidence only 15 out of 100 simulations were more extreme than the observed result.

Open in simulator

Problem 145

Decide whether data are consistent with model using simulation evidence only 2 out of 1000 simulations were as extreme or more extreme than the observed result.

Problem 146

Decide whether data are consistent with model using simulation evidence the observed value falls within the central 95% of the simulation results.

Problem 147

Decide whether data are consistent with model using simulation evidence the observed value falls outside the central 95% of the simulation results.

critique assumptions, trial count, and model mapping.

15 problems Warmup Practice Mixed Review Assessment

Problem 148

Identify limitation of simulation only 20 simulated trials.

Open in simulator

Problem 149

Identify limitation of simulation uses fair coin for event with 70 percent chance.

Problem 150

Identify limitation of simulation ignores dependence between selections without replacement.

Problem 151

Identify limitation of simulation uses one-tail count for two-sided question.

Problem 152

Identify limitation of simulation defines one trial as 5 coin flips instead of 10.

Problem 153

Identify limitation of simulation assigns equal probability to each outcome when probabilities are different.

Problem 154

Identify limitation of simulation uses a 6-sided die to simulate an event with 8 equally likely outcomes.

Problem 155

Identify limitation of simulation uses a 10-sided die to simulate a 30% chance of success.

Problem 156

Identify limitation of simulation does not reset conditions after each trial in a 'without replacement' scenario.

Problem 157

Identify limitation of simulation simulates only 10 trials for a rare event.

Problem 158

Identify limitation of simulation counts exactly 3 successes when the question asks for at least 3 successes.

Problem 159

Identify limitation of simulation counts exactly 2 failures when the question asks for at most 2 failures.

Problem 160

Identify limitation of simulation assumes the outcome of one game does not affect the next game's outcome when player skill changes.

Problem 161

Identify limitation of simulation uses a random number generator for continuous values to simulate discrete counts.

Problem 162

Identify limitation of simulation ignores the order of selection when the problem implies order is important.

catch wrong model, trial definition, tail direction, and overclaiming.

15 problems Warmup Practice Mixed Review Assessment

Problem 163

Correct simulation-based inference error used observed proportion as simulation probability for null test.

Problem 164

Correct simulation-based inference error counted only high tail when question asks any difference.

Problem 165

Correct simulation-based inference error claimed p-value is probability model is false.

Problem 166

Correct simulation-based inference error called one random digit one trial for a 10-person sample.

Problem 167

Correct simulation-based inference error defined one random number as a full simulation trial for a multi-event scenario.

Problem 168

Correct simulation-based inference error used 0.5 as the probability of success when the null hypothesis stated 0.6.

Problem 169

Correct simulation-based inference error calculated p-value by counting results less than observed for a 'greater than' alternative.

Problem 170

Correct simulation-based inference error concluded that a very small p-value means the null hypothesis is definitely false.

Problem 171

Correct simulation-based inference error recorded only the number of successes per trial instead of the proportion for comparison.

Problem 172

Correct simulation-based inference error excluded the observed sample result when counting extreme outcomes for the p-value.

Problem 173

Correct simulation-based inference error simulated a fixed number of trials (binomial) when the problem described trials until first success (geometric).

Problem 174

Correct simulation-based inference error only counted outcomes in one tail for a two-sided significance test.

Problem 175

Correct simulation-based inference error stated there is a 95% chance the next sample mean will fall within the confidence interval.

Open in simulator

Problem 176

Correct simulation-based inference error used a uniform distribution to model a situation with a known skewed probability.

Problem 177

Correct simulation-based inference error simulated a sample of 5 when the problem specified a sample size of 20.