Continued article from the The Behavioral Measurement Letter, Vol. 2, No. 1, Fall 1994
David F. Gillespie
Measurement is an integral part of modem living. We have come to depend on measurement to a considerable extent, partly because measuring devices are so commonplace and pervasive in our society. We hang clocks on the wall and wear watches on our wrist to keep track of time. We keep scales in the bathroom to check our weight. Thermometers both indoors and outside give us temperature readings. Standard sized cans and packages help guide decisions in purchasing products. In addition, since package size alone can be misleading, usually the weight of the contents is included on the package to more precisely specify the contents. Various gauges and meters in our cars help monitor speed, the amount of gas in the tank, oil pressure, engine temperature, miles driven, and other things. Traffic lights tell us when to cross the street and when to wait. The number of measuring devices used today is immense and still growing. These and many more measuring instruments are constantly providing information that we use to make decisions. Without such information, our ability would be greatly hampered to make good decisions.
A byproduct of this pervasive use of measuring devices is a presumed confidence in their accuracy. To a large extent this confidence is justified, especially when applied to pressure gauges, clocks, thermometers, and other physical measuring instruments. They facilitate safety, efficiency, and comfort. Extensive checks by the federal government and other organizations help to maintain accurate measuring instruments. The National Institute of Standards and Technology, among its other activities, maintains the standards against which instruments for measuring distance and weight can be checked. Other governmental officials conduct regular checks of commonly used measuring devices such as the scales used in grocery stores and meters used in gasoline stations. Similarly, researchers are recurrently recalibrating the thermometers, burettes (graduated glass tubes with a valve at the bottom used for measuring liquid or gas), and weights used in physical science and engineering work to be certain that accuracy is maintained.
Measurement in the behavioral sciences has not yet achieved the degree of standardization that is characteristic of the physical sciences. In the physical sciences many of the factors that affect measurement readings have been identified. These factors are eliminated or minimized through adjustments. This means that a measurement reading by one scientist can be independently verified by others. In behavioral science research many of the variables that affect measurement have not been identified, and those that have often resist control or elimination. For example, responses coded by interviewers from the Census Bureau are partly a function of the specific question asked – What is your income? Marital status? Number of children? – and partly a function of social circumstances. These circumstances include the interviewer’s appearance, the respondent’s relations to the law, relations with neighbors, with employers, and so forth.
Despite the differences in standardization, there is increasing use of psychological, educational, and social measurement to help us make decisions. Aptitude and achievement tests as well as various psychological inventories are now in widespread use at all levels of education. Employers use skill tests and attitude scales to screen personnel. The use of honesty tests is expanding in banking, retailing, and other industries where opportunities for deceit and fraud are inherent in the work. Salaries in many organizations are tied to measures of work productivity. Insurance claims and other forms of accountability depend on measures of risk probability and disaster preparedness. The use of educational, psychological, and sociological instruments is almost as commonplace as the others. As we stop to think about these measurements, clearly they too have become woven into the fabric of modern living.
Although the measurement of psychological, educational, and sociological phenomena is less firm than the measurement of physical phenomena, the presumed confidence in measurement is carried over and applied by many people in good faith to measures of mental abilities, self-esteem, honesty, marital satisfaction, and other concepts studied by behavioral scientists. This confidence is justified sometimes, other times it is not.
For example, in my own research on disaster preparedness, my colleagues and I have encountered a rich breadth of meanings associated with preparedness, along with the resulting scattered and atheoretical understanding we have of the concept of preparedness. Yet, only when scientists can agree on what preparedness is will it be possible to study systematically its causes and consequences, and to design research that assesses costs and benefits associated with preparedness.
We have developed a 7-item summative measure of preparedness that asks respondents: (a) whether their organization has a specific emergency response plan to guide its operation in a disaster; (b) if yes, how long it has been since the emergency response plan was reviewed and updated; (c) during the past three years, how many different times a representative of their organization has participated in a training session related to disaster preparedness; (d) during the past three years, how many different times a representative of the organization has participated in simulated disaster exercises; (e) during the coming year, how often a representative will participate in a disaster response training session; (t) during the coming year, how often a representative will participate in a field disaster exercise; and (g) whether respondents are familiar with the meaning of the term ”Integrated Emergency Management System” (Journal of Mass Emergencies and Disasters, 1987). We have had to sharpen our conceptualization of preparedness as an initial step toward a more refined construct. This process has reinforced our respect for the complexities inherent in deriving clear and precise operationalizations of social science constructs, and it is this process that will ultimately contribute to the advancement of a theory of disaster preparedness.
In conclusion, an understanding of how measures in behavioral research are constructed is essential to evaluating the worth of that research. If a study bases its conclusions either all or in part on measures that are inappropriate, out of date, or of dubious validity, the conclusions drawn from the research, even if otherwise adequately executed, will be suspect. Being able to recognize good measurement and make use of it is important in modern society, both in our personal and professional lives.
David F. Gillespie is Professor of Social Work at the George Warren Brown School of Social Work, Washington University, St. Louis. He received his PhD in Sociology from the University of Washington, Seattle. His career in the social work field has concentrated on organizational theory and disaster preparedness. Recent publications include Partnerships for Community Preparedness (Natural Hazards Research and Applications Information Center) and Quantitative Methods in Social Work: State of the Art, edited with Charles Glisson (Haworth Press). Dr. Gillespie is current Editor of the Journal of Social Service Research.
“For those who don’t know, even the prairie is a jungle.”
(Ethiopian proverb)
Read additional articles from this newsletter:
Looking Inside HaPI: Searching for Instruments on a Particular Topic
Clinical Measurement for Primary Care
2-1-fall-1994
