HP 602, Spring, 2002
I. What is “operationalizing”?
Operationalizing a variable means finding a measurable, quantifiable, and valid index for your variable (independent and dependent variables), and (sometimes) finding a way to manipulate that variable in such a way as to have two or more levels.
Not all variables are easily measured. Factors that are objective, effort independent or involuntary, and concrete are more easily measured (with appropriate equipment). Factors that are subjective, effort dependent, or abstract are hard to measure.
Easily measured (with appropriate equipment): Weight, arterial oxygen saturation, venous lactate concentration, leg length, etc.
Fairly easily measured: Maximum joint angle, VO2max, vital capacity, 50 yard dash time, hearing frequency range, etc.
Not easily measured: Sportsmanship, aggressiveness, pain, basketball ability, love, hunger, frustration, readability, situation awareness, workload, etc.
II. Generalized Procedure for operationalization by Jean McMillian (Aptima Corp., Mass)
1. Identify the concept we hope to measure. For example, height, intelligence, or workload.
2. Determine one or more quantitative measures of the concept. For example, height can be measured in inches or centimeters; intelligence can be measured by the score on an IQ test; and workload can be measured by a score on the NASA Task Load Index (TLX) self-rating scale. There can be multiple measures of the same concept.
3. Determine the method for obtaining this measure. For example, the use of a ruler or tape measure to obtain a measure of height in inches; the specific paper-and-pencil instrument used to produce an IQ score (e.g., the Stanford-Binet) and the methods for administering it properly and producing the numerical IQ score from the results; the NASA TLX self-report instrument and the instructions for when/how people should fill it out as well as the methods for summing up the results.
III. What are some techniques for operationalizing a variable of interest?
Literature search (not really a technique, but a way to find techniques)
Example 1. Let’s say you hypothesize that male hockey players are more aggressive than female hockey players.
Example 2. Pain. Use of an anchored scale to measure the pain subjects (with DOMS after eccentric exercise) felt 24, 48, and 72 hours after the exercise. One problem that was mentioned is that pain is subjective and the same “level” of pain might be given a different ranking by two different subjects.
Example 3. How would you measure sportsmanship?
Example 4. Let’s say you hypothesize that greater situation awareness leads to better quarterback performance.
What is situation awareness?
How could you operationalize situation awareness?
Example 5. How would you measure attractiveness (of individual people)?
**No single operational definition of an abstract concept can encompass that concept completely
**First step is to see what has been done in the past (literature search), and if already-devised techniques for defining a variable are applicable to your study
**Reliability of the operational definition must be established – the operational scoring system should give the same result for the same sample
-Intra and inter-observer reliability
**Validity of the operational definition must be established
-multiple operational definitions – look to see if the results from different measures of the same concept give the same results
-correlation with another measure that has previously been shown to be valid
-advantageous if at least one operational definition is behaviorally based – more phone calls rather than just experts’ ratings
-When setting up different LEVELS of an operational variable, preliminary tests can be done to establish the validity of the levels
E.g., Low, Medium, High level of hunger – deprived of food for 4, 8, or 12 hours – check and see if subjects eat more after 12 than 8 than 4 hours of deprivation
E.g., Low, Medium, High levels of motivation – offered 20, 10, or 2 extra credit points for performing an extra task (or running 1.5 miles!) – Do more students complete the EC assignment when offered 20 points vs. 10 vs. 2?
**Operationalization by non-random selection of subjects: Sometimes levels of an independent variable can be set by pre-selection of subjects.
For example, let’s say you hypothesize that greater basketball knowledge leads to better basketball performance. To set up levels of basketball knowledge you take basketball referees (high level of knowledge), fans of basketball (moderate level of knowledge), and non-fans of basketball (low level of knowledge).
The problem with this style of operationalization is that there may be other variables correlated to the selection criteria that can threaten your internal validity. Maybe basketball referees are mostly former high-level basketball players. Maybe referees are more physically active than fans. Or maybe fans are more physically fit than non-fans, etc.