Types of Validity

The concept of validity applies to both whole studies (often called inference validity) and the measurement of individual variables (often called construct validity).

Detailed examination of each kind of validity

• inference validity  // study-level
• construct validity // variable-level

1. Construct validity

• validity of measurement. of variables

1.1 Translation validity (Trochim's term)

• subjective evaluation of whether a measure matches the construct it is meant to measure

1.1.1 Face validity

• does it mean the same thing as the concept.
• e.g., if you want to know if someone is a liberal, asking "are you a liberal?'

1.1.2 Content validity

• Do all of the elements of the measure seem connected to the concept.
• e.g., in determining if there is there is fire, asking Is there smoke? Destruction? Heat? Ash? Burnt stuff?

1.2 Criterion validity

• How well the measure relates to other measures and characteristics

1.2.1 Predictive validity

• Ability to predict future events.
• Divorce scale actually predicts divorces.

1.2.2 Concurrent validity

• Discriminate between groups
• e.g., engineers should do better in math test than poets

1.2.3 Convergent validity

• Correlates positively with other measures of the same construct, or measures of very similar constructs

1.2.4 Discriminant validity

• Correlated poorly with measures of different constructs.
• E.g., we don’t want emotional intelligence measure to correlate too well with self-monitoring

2. Inference validity

• the validity of conclusions drawn from a study

2.1 Internal validity

• whether claimed causality of findings is consistent with research design -- e.g., experiment vs field study

2.2. External validity

• Refers to generalizability of the results. Does it say anything outside of the particular case?
  
  A carpenter, a school teacher, and scientist were traveling by train through Scotland when they saw a black sheep through the window of the train.
       "Aha," said the carpenter with a smile, "I see that Scottish sheep are black."
       "Hmm," said the school teacher, "You mean that some Scottish sheep are black."
       "No," said the scientist glumly, "All we know is that there is at least one sheep in Scotland, and that at least one side of that one sheep is black."
   
• Three strategies for claiming external validity:
  • sampling. Select cases from a known population via a probability sample, then claim results for the population
  • proximal similarity model. Show the similarities between the cases you studied with a population you wish your results to be applied to
  • replication. repeat the study in different settings