• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 



• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 

Measurement Issues in Biobehavioral Studies

Vol 10, No 1 – Winter 2008


Introduction to the Winter 2008 Issue of The Behavioral Measurement Letter

Nursing research covers a broad area, with nurse scientists involved in studies that often examine not only theoretical but also behavioral, clinical, and physiological aspects of patient care, sometimes all within the same study. As a result, instrumentation may not always get the focus that is needed to ensure quality measures to make the study meaningful or to get the study published.

In this issue, Molly Dougherty, in Observations of a Nursing Research Editor, offers some insight into common problems she has observed with submitted manuscripts dealing with behavioral instruments, such as scope, copyright, literature review, and sampling. Dougherty dispenses some useful tips to potential authors for overcoming these problems, including reminding authors to read for both personal and professional reasons in order to overcome some common writing problems, considering publishing in on-line journals, and querying editors on the appropriateness of an article before submitting it to that journal. Finally, she shares with us some of the trends in the publication of behavioral instruments, including increased submissions, internationalization of nursing publications, and ‘open access publication.’

Next, in Measurement Error in Clinical Investigations, Louise Jenkins reminds us of the complexity of clinical trials and how measurement of behavioral variables may not get the attention that it needs because of the focus on other key aspects, such as sample size, randomization, etc. Jenkins cites some specific sources of behavioral measurement error found in clinical trial failures as found by the Food and Drug Administration and also asks us to consider other factors which can increase measurement error in a study. Finally, she offers a practical approach, the development of a measurement protocol, to help minimize measurement error.

Lastly, Maureen Groer, in her article Measurement Issues in Biobehavioral Studies, compares and contrasts measurement issues facing both behavioral and biological scientists. Groer reviews some important areas that need to be addressed when planning and implementing biobehavioral research. Lastly, while recognizing the complexities involved in biobehavioral studies, she reminds us of the value of an integrated, holistic approach.

Just a reminder, most college and university libraries in the US, as well as many colleges and universities in Canada, subscribe to both the HaPI database service and the newsletter. If you have questions with regard to how you might be able to take advantage of these services, don’t hesitate to contact the staff at Behavioral Measurement Database Services (BMDS). Also, since the last issue, I am happy to report that the titles of the articles appearing in The Behavioral Measurement Letter (BML) are now listed in the Cumulative Index to Nursing and Allied Health Literature (CINAHL). So if you missed Introduction to the Winter 2008 Issue of The Behavioral Measurement Letter The Behavioral Measurement Letter 2 Vol. 10, No. 1, Winter 2008 Introduction (continued from Page 1) a particular issue or would like to see what articles have been printed since 1993, just check out the listings in CINAHL.

Please address comments and suggestions to The Editor, The Behavioral Measurement Letter, Behavioral Measurement Database Services, PO Box 110287, Pittsburgh, PA 15232-0787. We also accept short manuscripts for The BML. Submit, at any time, a brief article, opinion piece, or book review on a BML-relevant topic to The Editor at the above address. Each submission will be given careful consideration for possible publication.

HaPI reading…

Deidre M. Blank, RN, DSN, FAAN Guest Editor


Measurement Issues in Biobehavioral Studies

Maureen Wimberly Groer

Research that simultaneously explores both psychosocial and biological processes provides a lens into integrated, holistic mechanisms by which human beings organize responses to external and internal events. In fact, studies that explore only a single aspect of those human responses provide only partial and often very in-complete data. By focusing on a single dimension, the complexity of a response is certainly obscured. But the exploration of biobehavioral responses is not easy, because there are multiple methodological issues that the investigator must consider. This article will describe several of the major issues and suggest solutions that will assist those scientists who wish to use a more holistic approach in their human studies.


General Biobehavioral Measurement Issues

Behavioral and biological scientists share common as well as disparate paradigms, scientific concerns, and training. While the behavioral scientist tends to seek answers to questions about relationships between large variables such as personality, environment, and behavior, the biological scientist is often focused on a particular molecule or biochemical process. The scale of inquiry is therefore quite different and the degree of specialization can appear to be extreme in some cases.

Issues related to reliability and validity, while always important in science, are of differing dimensions in the two fields of inquiry. The behavioral scientist worries about the performance of a questionnaire within a population. The biologist worries about the performance of an assay or a piece of electronic equipment. A variable such as depression may be measured by a cut-off score on a well validated instrument that has a range of 1-200. The biologist may be measuring a particular molecule in a body fluid that has a range of 1-5 picograms/ml. Measurement error on the order of a standard deviation becomes extremely worrisome when one is measuring such small and narrowly distributed values.

Biologists are often envied for the fact that they are doing “hard” science, with the faulty assumption being made that error and bias is less likely to creep into studies than when psychosocial measurements are being made. Many times the variables of interest to the behavioral scientist are collected by self-report paper and pencil testing and recall, which are notorious sources of error in studies. But biology is also fraught with similar problems. Every biologist knows that one’s laboratory and personnel are major sources of error and bias and therefore every lab establishes its own quality control standards and controls. The general working environment, the training and level of competency of laboratory personnel, the handling of specimens, the guidelines for procedures, the proper operation and maintenance of equipment, and the accuracy of data collection and recording are all possible sources of error in biological measurement. Modern automation and robotic procedures can help reduce errors, but the equipment must be properly used, and many research labs do not handle the large number of samples that make this equipment cost effective.

Another difference between the two disciplines is related to training. Behavioral scientists tend to have much stronger training in measurement theory, research design, and statistics than classically trained biologists. The apprenticeship model for advanced training is still widely in use in the biological sciences and students are expected to learn by example and exposure in their mentor’s laboratory as well as in classroom settings. More sophisticated statistical approaches are seen in the psychosocial compared to the biological literature. Data are not only analyzed but also often presented quite differently as well. Biologists often employ graphs and tables while behavioral scientists are leaders in the use of modern statistical approaches such as structural equation modeling and hierarchical linear modeling.

While the differences described above are significant, they do not present insurmountable obstacles to interdisciplinary collaboration. These different ways of thinking and doing research can inform biobehavioral studies. In order for truly interdisciplinary work to be carried out in the future, some aspects of the training of young scientists in both disciplines should be considered. An approach then could be coursework in biobehavioral paradigms with both types of students engaging in dialogue, thinking, and planning for research. At a later time, when thinking about a biobehavioral type of study, the scientist so trained would have a firmer ground for planning and working with others to develop ideas.

The next section of this paper will review some areas for attention and concern in the planning and implementation of biobehavioral research, particularly focusing on biological issues known to influence biobehavioral data collection and analysis.


Metabolic Pathways

Oftentimes the goals of research are to examine relationships between a particular psychological state and biological variables. The psychological state is usually measured by a paper and pencil self-report instrument, or occasionally by qualitative approaches. The researcher may be interested in a short term concurrent behavioral state or a more chronic condition. The biological variable (biomarker) is often the measure of a particular molecule in some body fluid at a single point in time, unless the study is prospective in design. Behaviors, emotions, moods, perceptions of stress are all potentially associated with neurohumoral, endocrine, or immune states, which can be assessed through measurement of well chosen biomarkers. The difficulty lies in being aware of the timing of metabolic pathways involved in the secretion, release, uptake, binding, and removal of the biomarker. An example would be the measurement of Interleukin-1 β, which is a proinflammatory cytokine, which has been shown to be increased in depression and stress (Anisman & Merali, 2002). Direct serum measures of this cytokine provide a value which ultimately reflects influences of multiple biological and biochemical pathways and influences. First is the actual production and secretion of the cytokine, mainly by cells of monocytic lineage. IL-1 is released from these cells after cleavage of its aminoterminal region by caspase-1 (Jacques, Gosset, Berebaum & Gabay, 2006). The effects of serum IL-1 are modified in vitro and in vivo by natural inhibitors such as IL-1 receptor antagonist (IL-1ra) and soluble IL-1 receptors. IL-1 ra inhibits the effect of IL-1 by preventing its attachment to IL-1 cell receptors. Thus, the measurement of IL-1 as a single biomarker of inflammation yields only partial information. A full picture requires measurement of soluble receptors and antagonists. In addition, IL-1 is usually in very low concentration in serum, so a high sensitivity assay may be necessary to produce values in the 1-3 picogram/ml range. Many cytokines have similar pathways. There are certainly multiple physiological influences on levels of most biomarkers of interest to the biobehavioral researcher, many of which are not yet well described.

Another issue is the compartmentalization of some biomarkers. Large molecules (many hormones, drugs) may not be able to cross epithelial barriers and be accurately measured in secretions such as saliva and tears. Other influences must also be considered. For example, more than 90% of serum cortisol is bound to cortisol binding globulin (CBG), and changes in serum proteins may alter bioavailability. The hormone is present in an unbound state in saliva, and thus is more reflective of bioavailable cortisol. On the other hand, the signature molecule of hypothalamic-hypophyseal-adrenocorticotrophic (HPA) activation is corticotrophin releasing hormone (CRH), which is not measurable in the serum (except during pregnancy when the placenta secretes it) (Mastorakos & Ilias, 2003). So indirect measures of HPA activation must be used, such as levels of serum ACTH or salivary cortisol. ACTH and many other hormones may be difficult to measure in body fluids because natural proteases may degrade them after collection of the sample. While it is often best to add a protease inhibitor such as apoprotein to samples that are collected, it is a procedure that is rarely done and the choice of the protease inhibitor has to be carefully planned so as to not interfere with the concentration of the protein actually being measured.

Circadian influences also markedly influence the concentration of many of the biomarkers of interest to the biobehavioral researcher. Most notable are the hormones of the HPA axis, but also many of the cytokines (IL-6, TNF-α) are now being identified as being influenced by the time of day and involved in sleep/wake cycles (Kapsimalis, Richardson, Opp, & Kryger, 2005). When comparing individuals within a stress paradigm, the levels of these hormones and cytokines cannot be compared among individuals measured at different time points across the course of a day. Time must be constant or controlled in some way.


Demographic Influences

Influences of demographics and environment cannot be ignored in biobehavioral studies. Socioeconomic status, gender, age, and body mass index are all important mediators or moderators of psychosocial-biological relationships. For example, reactivation of latent herpes viruses is often used as a marker of cell mediated immunity. Older females, compared to males, are more at risk for having reactivation of herpes zoster virus, which produces shingles, and this may be related to a lower percentage of herpes zoster memory T cells in females (Klein et al., 2006). Thus, in experiments evaluating the influence of stress of herpes zoster reactivation, gender must be considered. Stressors from multiple potential sources must be accounted for in biobehavioral research. Vulnerabilities to stressors may be exacerbated by environmental influences, such as may occur for pediatric asthma. Combinations of both life stress and exposure to allergens and indoor pollutants together may produce the biological effect (Friedman & Lawrence, 2002).

Another potential source of error is the presence of a concomitant disease process or multiple risk behaviors in individuals being studied with a biobehavioral approach. Many of the markers associated with stress states and dysphoric moods are also associated with certain illnesses. The relationship between cardiovascular risk and C-reactive protein (CRP) is well established, but CRP is also associated with other inflammatory diseases, generally rises with age, and may be related to other multiple factors associated with cardiovascular disease (Lowe & Pepys, 2006). Another example would be the finding of a relationship between stress and a particular cytokine. Many other intervening variables could potentially account for the cytokine elevation, such as genetic polymorphisms in genes that regulate cytokine expression, or even something as simple as a lack of sleep the night before.


Measurement Problems

Ease of measurement is often a concern, so biological fluid which can be collected without invasive procedures may be an option. However, many times the most accessible fluids provide limited information. Careful controls over the actual collection of fluids such as saliva or breast milk are necessary to avoid spurious results. For example, smoking, eating, or oral hygiene may lead to microscopic bleeding in the mouth and if these are done prior to saliva collection, then measurements of certain biomarkers will not be accurate. Subject burden may result in spurious or inaccurate biological data. In this researcher’s experience, when collecting saliva samples in people with very dry mouths, the wish to produce something in the tube may lead to undesirable sputum collection rather than saliva. The collection method may actually interfere with the accuracy of certain biomarkers. The use of cotton-based saliva collection has been found to lead to errors. Salivary assay results for testosterone, DHEA, progesterone, and estradiol were artificially high, and for secretory Immunoglobulin A (sIgA) artificially low in studies, when samples were collected using cotton absorbent materials (Shirtcliff, Granger, Schwartz & Curran, 2001). Timing of the biological collection is also important. Many times salivary markers require accurately timed collection, as not only absolute concentration is of interest, but also secretion rate. Sometimes a protein marker of secretion is used so that the molecule of interest is expressed in units per mg of a protein that is secreted at a constant rate. With regard to measurement of human milk molecules, the time of milk expression in relationship to the production of milk may be important, as fore milk, produced at the beginning of a feed, is quite different from hind milk.

When data are collected in the fi eld, the samples must be transported to the laboratory at appropriate temperatures to assure viability of the samples, but also with regard to the possibility of cold damage to samples. An example would be packing and transporting blood collected in plasma collection tubes in ice buckets which may cause cryodamage through spotty icing along the inside surface of the collection tube which may result in platelet clumping and aggregation onto white blood cell surfaces.

Laboratory assays of biomarkers also may not be accurate for one reason or another. Appropriate controls, and intra and inter assay coefficients of variation are important to assess in order to be confident of the results. Many biological variables are positively skewed and require mathematical transformation in order to be amenable to parametric statistical analysis. Another problem is the limit of the assays available to measure biomarkers. Some cytokines may be present in concentrations lower than the lowest limit of the assay’s standard curve, so that the results are all essentially zero if one uses these assays and biologically significant differences are easily missed. The use of high sensitivity assays may be necessary to discover these differences. The reverse problem occurs when the amount of the biomarker is very high and the assay measures at a much lower range. This requires serial dilutions of samples to achieve the range of the assay, but with every serial step, there is potential for error. Another consideration is the statistical manipulation of data, when outliers in particular biomarkers are discarded from the analysis. Individual differences, which may be of great interest, may be obscured by this approach (Cohen, 2004). An experience this researcher had was the discovery of extremely high levels of proinflammatory cytokines in a participant in a study which would have skewed the data considerably. However, this participant then was diagnosed with an autoimmune disease. So mindlessly eliminating data that is way beyond the means and ranges in the study may result in statistically more homogeneous data, but care should be taken to assure that the “outlier” is not a methodological aberration, but a real person with an illness needing treatment.



While biomarkers clearly add important information to studies, a theoretical approach which provides carefully considered and justified rationale for the choice of the biological molecule is essential. Simply adding a “stress” marker, without the framework to guide its selection, is casting a net out for biological data without a plan for integrating that data into a theoretically sound analysis plan.

The future of biobehavioral research is exciting as technology will allow great ease of measurement and extensive amounts of data to be collected. Currently the use of gene arrays and multiplex protein assays provide this opportunity. This reinforces the need for training young behavioral scientists in physiology and molecular biology, as well as educating biology students in behavioral measurement, so that future studies will be framed in the most holistic, theoretically sound, and robust manner. This type of research is strongest when interdisciplinary collaboration is employed to develop and implement the study.