MEASUREMENT refers to the process of assigning numbers or labels to objects, events, or characteristics according to specific rules. Measurement is fundamental in transforming abstract concepts and complex social phenomena into observable and quantifiable data, allowing for empirical investigation and statistical analysis. Effective measurement enables public administrators to quantify and analyze variables, thereby:
- facilitating the examination of relationships, patterns, and trends
- enabling the comparison of variables across different groups, times, or conditions
- facilitating the testing of hypotheses by providing empirical data that can be analyzed to support or refute theoretical predictions
- informing evidence-based decision-making and policy formulation by providing measurable and reliable data
Measurement is not as simple as measuring a tangible object or substance. It involves defining, operationalizing, and quantifying variables in a manner that ensures consistency, reliability, and validity. We must be as precise and transparent as possible when discussing our measurements.
Conceptualization and Operationalization
The first step in empirical research is to identify CONCEPTS (sets of characteristics or specific information representing constructs or social phenomena) to study. Concepts help us organize and understand our data and can be thought of as the “building blocks” for constructing statistical models. Concepts can be abstract/unobservable (ex: public trust in government) or concrete/observable (ex: government spending, public service delivery). Once you have identified the concept you want to research, you need to narrow your concept to a more precise CONCEPTUAL DEFINITION, or an explanation of a concept based on theoretical constructs (i.e., what theory says about the concept you are researching) outlining the concept’s meaning and the essential characteristics and/or attributes related to different aspects or components of the concept. An effective conceptual definition:
- provides a theoretical framework for understanding the concept, establishing a common language, understanding, and foundation for measurement
- is clear enough that others can understand what you mean, facilitating the replication of studies
- distinguishes the concept being studied from other different (but related) concepts, reducing MEASUREMENT ERROR (i.e., the difference between the true value and the value obtained through measurement) and helping ensure we are using valid, reliable measures
- is not circular
Once you have developed a conceptual definition, you need to operationalize the concept. An OPERATIONAL DEFINITION is a clear, precise description of the procedures or steps used to measure a concept; in essence, it translates abstract concepts into measurable variables. Operational definitions are essential for enhancing the reliability and validity of data collection and measurement and for facilitating replication. In operationally defining the concept, you will identify one or more INDICATORS (i.e., specified observations used to measure the concept).
Variables
A VARIABLE refers to an empirical (observable and measurable) characteristic, attribute, or quantity that can vary (i.e., take on different values among individuals, objects, or events being studied). Variables are used to describe and quantify concepts in an analysis.
Variables are related to indicators. Multiple indicators may be used to represent a single variable (ex: a four-item scale, with each item associated with a distinct indicator, is often used to measure public trust in federal government). Alternatively, a single indicator may be used to represent a single variable OR to represent more than one variable, if it measures different aspects or dimensions of those variables.
Variables can be classified into three main types:
- DEPENDENT VARIABLES, or the outcome variables we are interested in explaining or predicting. Sometimes called outputs or responses
- INDEPENDENT VARIABLES, or the predictor variables we are using to explain or predict variation in the dependent variable(s). Sometimes called predictors, inputs, or features
- CONTROL VARIABLES, or variables that are theoretically related to the dependent variables of interest, which researchers hold constant or account for to isolate the relationship between the independent variable(s) and the dependent variable; control variables are important because they help ensure that the results are attributable to the variables of interest, rather than being influenced by extraneous factors
CODING refers to the process of preparing data for analysis. To allow for statistical analysis, variable values are coded numerically. Some of these numerical values represent categories or labels (ex: gender); other numerical values correspond to real numbers (ex: per capita state-level spending on secondary education).
Reliability and Validity
RELIABILITY is the extent to which a measure consistently captures whatever it measures. Simply put – reliability is repeatability. VALIDITY is the extent to which a measure captures what it is intended to measure. Simply put – validity is accuracy.
A measure CAN be reliable and not valid, but a measure CANNOT be valid if it is not reliable. To illustrate this point, let’s consider an example. Let’s say that your bathroom scale always displays your weight as ten pounds below your actual weight. Is the weight displayed by the scale a reliable measure? Yes: the scale consistently displays your weight as being ten pounds below your actual weight. Is the weight displayed by the scale a valid measure? No: the scale does not display your actual weight. The weight displayed by the scale is reliable but not valid.
Levels of Measurement
The level of measurement of a variable impacts the statistics that can be examined and reported and the types of statistical analyses that can be used. Levels of measurement are hierarchical: what applies to the lowest level of measurement will also apply to variables measured at higher levels.
Nominal
The lowest level of variable measurement is NOMINAL, in which observations are classified into a set of two (BINARY or DICHOTOMOUS variable) or more (MULTINOMIAL variable) categories that:
- have no direction (i.e., they are “yes/no”, “on/off”, or “either/or” in nature — there cannot be “more” or “less” of the variable)
- are MUTUALLY EXCLUSIVE (i.e., there is no overlap between categories)
- are EXHAUSTIVE (i.e., there is a category for every possible outcome)
When coding nominal variables, each category is assigned a numerical value to allow for statistical analysis; these numerical values have no meaning aside from what we assign them.
Ordinal
The next level of variable measurement is ORDINAL, in which observations are classified into a set of categories that:
- have direction
- are mutually exclusive and exhaustive
- have intervals between categories that cannot be assumed to be equal (i.e., there is no objective basis for differentiating numerical values between categories/responses; ex: the difference between “strongly disagree” and “disagree” may vary from one person to another)
Ordinal variables are often ranking variables measured using a LIKERT SCALE.
When coding ordinal variables measured using Likert scales, categories should be assigned numerical codes arranged in either:
- ASCENDING ORDER, i.e., from lower numbers (less of [variable]) to higher numbers (more of [variable])
- DESCENDING ORDER, i.e., from higher numbers (more of [variable]) to lower numbers (less of [variable])
Interval
The next level of variable measurement is INTERVAL, in which observations:
- have direction
- are mutually exclusive and exhaustive
- correspond to real numbers
- have equal intervals of measurement
- do not have an absolute zero
We rarely see true interval variables in public and non-profit administration. An example of an interval variable is temperature measured by Fahrenheit or Celsius. Fahrenheit and Celsius both lack an absolute zero (i.e., 0 degrees Fahrenheit or 0 degrees Celsius do not represent the absence of temperature). By extension, you can have negative values (-20 Fahrenheit or -10 Celsius), and you cannot assume that 80 degrees is twice as hot as 40 degrees: 80 degrees is certainly warmer than 40 degrees, and the distance between each degree is the same, but without an absolute zero, we cannot say it is twice as hot.
Ratio
The final level of variable measurement is RATIO, in which observations:
- have direction
- are mutually exclusive and exhaustive
- correspond to real numbers
- have equal intervals of measurement
- have an absolute zero
Because ratio variables have an absolute zero, we can make assumptions that cannot be made with interval variables. For example, an income of zero dollars indicates the absence of income, and someone who is 80 years old has twice as much age (in years) as someone who is 40 years old.
Replication and Generalizability
One of the hallmarks of social science research is REPLICATION (i.e., another researcher or administrator should be able to repeat the study or experiment). Replication is important for two reasons:
- it confirms the reliability of the research findings
- it helps us to determine the GENERALIZABILITY of the research findings (i.e., how well the results can be applied to other contexts, populations, or settings / the conditions under which the results hold; generalizability is also referred to as EXTERNAL VALIDITY)
Replication cannot occur without precise explanations of our research.