of the Real Colors® Instrument
INTRODUCTION
The value of a temperament assessment lies in its reliability and validity. Does it measure what it intends to measure and does it do so consistently?
For centuries, people have searched for a convenient way to peek into the human mind. The Greek philosopher, scientist, and physician Hippocrates wrote over twenty-three hundred years ago that all human beings could be
divided into four distinct personality types. Since the time of Hippocrates, researchers have continued to build on the body of research furthering the desire to better understand themselves and each other.
It is from the innate desire to understand human behavior and the motivations behind it that the Real Colors® instrument was developed. Since 1993, Real Colors® has sought to be a provider of simple, easy to apply, research-based tools and training that will allow people to better understand themselves and others. Because of our commitment to ongoing research and development, Real Colors® finds it necessary to periodically assess the reliability and validity of its own assessment. When assessing reliability, one must ask, “Does the instrument give consistent results?”. And when assessing validity, one must ask, “Does the instrument measure what it intends to measure?”. On both these criteria, the Real Colors® instrument performs well.
TYPES OF RELIABILITY AND VALIDITY
Reliability and validity are two ways researchers can evaluate the quality of assessment tools. They indicate how well a method, technique, or test measures what it intends to measure. Reliability provides information on the consistency of a measure, and validity provides information about the accuracy of a measure.
Reliability definition. Reliability is the extent to which results can be reproduced when research is repeated under the same conditions. It is also known as consistency. The more consistent results are when achieved
under the same conditions, the higher the reliability of a test. Two important types of reliability are individual test reliability and test-retest reliability.
Individual test reliability. Individual test reliability (also known as internal consistency) provides information on the consistency of items within a single test. That is, it is a measure of how closely related a set of items are as a group within a test. Individual test reliability is measured using a statistic called Cronbach’s Alpha. If this statistic Alpha is larger than .7, this indicates high individual test reliability.
Test-retest reliability. Test-retest reliability provides information on how consistent the same test is over time. This is important for getting results to replicate with a single test under similar conditions. This can be measured by administering the same test twice over a period of time to the same group of individuals. The scores from Time 1 and Time 2 can then be correlated in order to evaluate the test for test-retest reliability using a correlation statistic. A correlation larger than .7 with a statistical significance of p < .05 is evidence of test-retest reliability.
Validity definition. Validity is the extent to which a test that is designed to measure a specific concept accurately measures that concept. It is the extent to which results of a test truly measure what they are supposed to measure. This study focuses on one type of validity called convergent validity.
Convergent validity. Convergent validity measures the degree to which two distinct but related theories agree with each other, or how they converge. Our study was designed to measure the convergent validity of Real Colors® with respect to aspects of the Myers-Briggs Type Indicator (MBTI), specifically how each color relates to specific aspects of the MBTI. Convergent validity is measured with correlations between items that are thought to be related. Significant correlations (p < .05) between items that are hypothesized to be related are evidence of convergent validity.
The next section describes this study’s findings for these types of reliability and validity.