Copyright © 2007-2018 Russ Dewey
The Theory of Signal Detection
A funny thing happened to the concept of threshold on the way to the second half of the 20th Century: it disappeared. Or maybe we should say it became mobile.
Experiments showed there was no magic line which, when crossed, made a stimulus perceivable. Instead, people acted like the threshold was a decision point, variable in nature, that could be adjusted depending upon different circumstances.
This conclusion came from a new field called information theory or communications theory. Information theory started after World War II as scientists tried to improve communication systems such as the telephone.
Information theorists found that detectability of a signal depended upon several factors that could be manipulated independently: (1) the level of background noise, (2) the strength of the stimulus, and (3) the redundancy (amount of repeated information) in the stimulus.
What factors determine the detectability of a signal?
If a person is trying to detect very weak signals in background noise (for example, locating aircraft or missiles on a radar screen) the problem is to pick out the signal from the noise. But if the signal is very faint, or the noise level is very high, the observer might make errors. There are two types of errors a person can make.
1. False positives occur if a person says yes (a positive response) but this is wrong (false) because no signal was presented. A false positive response can also be called a false alarm. If you thought you heard somebody call your name, but nobody actually did, that is a false positive.
2. False negatives occur when a person says no (a negative response) but this is false. Actually a signal was present. An example would be failing to detect a blip on a radar screen.
What are false positives and false negatives?
Psychologists found people could vary in how sure or confident they were of a signal detection. People could be asked to avoid saying they had detected a signal unless they were very sure, and that would reduce false positives.
Or people could be asked to avoid missing any signals, even if it meant making false positive errors. People could do that, too; they reduced false negatives to near zero, but they made lots of false positives in the process.
There was a clear trade-off. You could minimize false positives or false negatives, but not both at the same time.
In some situations, false positives do not cause as much harm as false negatives. Consider a blood bank screening samples for the AIDS virus (HIV).
The initial screening of blood samples uses a very sensitive test designed to eliminate false negatives, even though that means there will be some false positives. False positives, in this case, are blood samples that test positive for the HIV virus, even though later testing shows they are not really infected.
These false positives have a cost: some blood is wasted. But that is a small price to pay for insuring no infected blood is given to people receiving transfusions. In other words, there must be no false negatives in this situation.
Why use different thresholds in different situations?
In other situations, a more urgent goal is to avoid false positives. A hunter must learn not to shoot at everything that moves, because a moving object might be a human or a dog.
The hunter must wait until the form of the object becomes clear. The threshold for pulling the trigger and shooting must be raised. False negatives (failing to shoot) are less important than false positives (shooting the wrong thing).
D-prime
In the 1960s, researchers found that they could influence people to move their thresholds for detecting a signal up or down, in effect. This was done by manipulating the payoffs, or costs, of different decisions.
For example, a subject could be promised $10 for every successful act of signal detection but penalized $1 for every false positive. That would create a response bias for saying Yes.
If the incentives were reversed, so experimental subjects were promised $1 for each hit but penalized $10 for each miss or false positive, the subject became more conservative. In effect, the subject raised the threshold for saying "Yes, I see a signal."
How did signal detection researchers lead subjects to change their signal detection thresholds?
Then researchers in signal detection theory had an important insight. They realized if they collected enough signal detection data, under a range of biasing conditions, they could statistically filter out or eliminate the effects of bias.
The result is d' (d prime). This statistic that gives a pure measure of the observer's sensitivity or ability to detect weak signals.
To obtain the d' statistic, an observer's bias or tendency to say Yes or No is manipulated. This is done by arranging a series of trials (sequences of signal detection tests) with incentives to minimize false positives, then false negatives.
After the participant goes through a range of biasing conditions, the true sensitivity of the observer can be calculated. The result is a statistic called d-prime (d').
What was "the important insight" from early research on signal detection theory?
D prime gives a bias-free estimate of a person's sensitivity to stimuli. It filters out the effect of being bold or cautious, by putting a participant through the full range of biasing conditions.
A bias-free measure of observer sensitivity is a real asset in situations where people might be inclined to give a Yes or No answer. For example, Clark and Yang (1974) used signal detection theory to study acupuncture. That is the Chinese medical treatment in which tiny needles are inserted into the body to relieve pain.
Most people who use acupuncture are inclined to believe in it. They show a bias toward reporting less pain after acupuncture treatment.
To find out if sensitivity to pain really decreased, Clark and Yang arranged a variety of biasing conditions and calculated the d' statistic for painful stimulation before and during acupuncture. The results showed no change in sensitivity to pain.
Clark and Yang concluded people were "less inclined to report pain" after the acupuncture needles were inserted. But their actual sensitivity to pain was unchanged. This would be difficult to determine without a bias-free measuring technique.
When is d' (d prime) especially useful? What did a signal detection analysis of acupuncture reveal?
Similarly, using the Theory of Signal Detection, researchers were able to study the accuracy of memory under hypnosis. Using signal detection theory, researchers showed that memory is not more accurate under hypnosis.
The d prime statistic did not change under hypnosis. What changed was the willingness to say Yes. Under hypnosis more memories (both true and false ones) are accepted as true, resulting in more true memories and also more false positives.
What did signal detection analysis reveal about memory under hypnosis?
Swets, Dawes, and Monahan (2000) point out that the d-prime statistic is useful whenever diagnostic decisions must be made. That includes many important decision-making situations:
Is a cancer present? Will this individual commit violence? Are there explosives in this luggage?
Is this aircraft fit to fly? Will the stock market advance today? Is this assembly line item flawed?
Will an impending storm strike? Is there oil in the ground here? Is there an unsafe radiation level in my house?
Is this person lying? Is this person using drugs? Will this applicant succeed? Will this book have the information I need? (Swets, Dawes, and Monahan, 2000)
Sound important? Psychologists thought so. When the Association for Psychological Science started a new journal, Psychological Science in the Public Interest, the Swets, Dawes, and Monahan was the first article published, and it took up the entire inaugural issue.
What did the Theory of Signal Detection turn into?
Psychophysics was transformed (in the modern era) into the Theory of Signal Detection. It became a general theory of decision-making in Yes/No situations. What started as an attempt to refine psychophysical functions turned out to have many applications beyond the exploration of human sensory ability.
---------------------
References:
Clark, W. C. & Yang, J. C. (1974). Acupunctural analgesia? Evaluation by signal detection theory. Science, 184, 1096-1097.
Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest, 1, 1-26.
Write to Dr. Dewey at psywww@gmail.com.
Don't see what you need? Psych Web has over 1,000 pages, so it may be elsewhere on the site. Do a site-specific Google search using the box below.