A census involves collecting data for the entire population of interest. This is problematic for several reasons, including:

• It can be difficult to complete a census: there always seem to be some individuals who are hard to locate or hard to measure. And these difficult-to-find people may have certain characteristics that distinguish them from the rest of the population.
• Populations rarely stand still. Even if you could take a census, the population changes constantly, so it’s never possible to get a perfect measure.
• Taking a census may be more complex than sampling.

Sampling involves measuring a subset of the population of interest, usually randomly.

• Non-response: If only a small fraction of the randomly sampled people choose to respond to a survey, the sample may no longer be representative of the population.
• Voluntary response: Occurs when the sample consists of people who volunteer to respond because they have strong opinions on the issue. Such a sample will also not be representative of the population.
• Convenience sample: Individuals who are easily accessible are more likely to be included in the sample.

• Observational study: Researchers collect data in a way that does not directly interfere with how the data arise, i.e. they merely “observe”, and can only establish an association between the explanatory and response variables.
• Experiment: Researchers randomly assign subjects to various treatments in order to establish causal connections between the explanatory and response variables.

Source: XKCD 552 http://xkcd.com/552/

1. Control: Compare treatment of interest to a control group.
2. Randomize: Randomly assign subjects to treatments, and randomly sample from the population whenever possible.
3. Replicate: Within a study, replicate by collecting a sufficiently large sample. Or replicate the entire study.
4. Block: If there are variables that are known or suspected to affect the response variable, first group subjects into blocks based on these variables, and then randomize cases within each block to treatment groups.

Difference between blocking and explanatory variables

• Factors are conditions we can impose on the experimental units.
• Blocking variables are characteristics that the experimental units come with, that we would like to control for.
• Blocking is like stratifying, except used in experimental settings when randomly assigning, as opposed to when sampling.

• Placebo: fake treatment, often used as the control group for medical studies
• Placebo effect: experimental units showing improvement simply because they believe they are receiving a special treatment
• Blinding: when experimental units do not know whether they are in the control or treatment group
• Double-blind: when both the experimental units and the researchers who interact with the patients do not know who is in the control and who is in the treatment group