The following questions can be tailored to specific data sets, findings, or graphical representations of data.

The answers to these questions rely on both mathematical and humanistic thinking.  As such, they may not have "correct" answers, although some answers will be more convincing and involve more rigor, rationality and reflection.

Given a set of “raw” data—such as data from a primary source, or collected during the course of the class or by the students outside of class—how should we evaluate quality of the data set?  What are the potential problems of drawing statistical inferences from the data? 

Can we turn the data into an index?  What will be lost or gained if we collapse this data into an index?

What new questions can we generate based on a graphical representation of descriptive statistics?

What kinds of historical sources would provide data to help with a particular question?  How might we locate such data?  What should we do if we can’t find the ideal source? 

What is the difference between a statistically significant difference and a stereotype?  How should we react when a statistically significant finding appears to confirm (or refute) stereotypes?  When socially sensitive differences are found (such as, among genders, races, or religions), how should we evaluate the data quality, significance, causation and policy implications of these differences?  To what extent should our values and experiences inform our analysis of differences that are socially sensitive?

When we encounter variation within a system for which we are responsible—for instance, quality control as part of a job—what measures can and should be taken to control the variation?  Why would it be desirable to limit variation?

One principle of data analysis is that “an approximate answer to the right question...is better than an exact answer to the wrong question.” What does this mean?  Can you come up with examples of the “right question” that cannot be answered precisely?

Often it is impossible or unethical to divide a population into a treatment group and a control group in order to test
the null hypothesis.  How can researchers design studies to examine such difficult issues?  Brainstorm for creative designs using various techniques:

• Natural experiment: Comparing two groups that, due to random circumstances, can be considered to be "treatment" and "control" groups.
  
• Nonequivalent control: Comparing two groups that we hope are broadly similar, even though we haven’t been able to randomize.
• Difference in differences: Comparing before and after data for two groups that are broadly similar, but that had different treatments (such as two similar counties, but only one of which had a new unemployment program in place).
• Discontinuity analysis: Comparing “the outcomes for some group that barely qualified for an intervention or treatment with the outcomes for a group that just missed the cutoff for eligibility and did not receive the treatment."