A teacher said that she had observed that how well a student does on a particular quiz depends on how well or poorly he or she did on the last quiz. Then she gave the following statistics:

If you did well on a quiz, there is an 80% chance you will do well on the next quiz, a 15% chance you will do so-so, and a 5% chance you will do poorly.

If you did so-so on a quiz, there is a 20% chance you will do well on the next quiz, a 60% chance you will do so-so, and a 20% chance you will do poorly.

If you did poorly on a quiz, there is a 3% chance you will do well on the next quiz, a 15% chance you will do so-so, and an 82% chance you will do poorly. The teacher then asked the following question (which she said we'd be able to answer once we had successfully completed the class):

If you did well on the first quiz, what is the probability that you will do well on the fifth quiz in the class?

The first question to be answered is whether, in fact, the mere statistics that the teacher presented actually back up her claim that "how well a student does on a particular quiz is affected by how well or poorly he or she did on the last quiz."

What immediately comes to mind is the likelihood that different students, whether due to motivation levels, different aptitudes for the subject, and other factors, tend consistently to do better or worse than others on these tests. Then, rather than being evidence for a test-to-test correlation for a given student, the statistics merely reflect that good students usually do better and poorer students poorer on the tests, and the previous test is a barometer of the aptitudes of the students.

In that case, and if the scores on the tests are actually independent, the probability of doing well on the fifth quiz given a good score on the first, would be the same as the probability of doing well on the second test given a good score on the first: 80%. (That's 80%, in case the html bold indication makes the number disappear entirely.)

A scenario which approximates the statistics given, and consistent with the independence hypothesis is the following:

If the students are grouped into five groups, labeled Bright to Low, they could have the following probabilities, each, of scoring well, so-so or poorly:

                 Well      Soso      Poorly


Bright                1         0         0    0.16


                   0.74      0.18      0.08    0.19


Avg                0.14      0.79      0.07    0.25


                      0      0.16      0.84     0.2


Low                   0      0.05      0.95     0.2

--------
The last column is the fraction of the class that is in each of the five groups.

Those statistics indicate the following as the probabilities that a student is at a given performance level given that he or she has done well, so-so or poorly on a given test:

p(bright)      0.476758         0         0


               0.418951  0.124954  0.038905


p(avg)         0.104291  0.721593  0.044791


                      0  0.116916  0.429997


p(low)                0  0.036536  0.486307

-------
To take one example of how this was figured: p(bright given well) = 1*.16 / (1*.16 + .74*.19 + .14*.19) = .476758.

Then, the probability of doing well on some other test given doing well on the first test would be
.476758*1 + .418951*.74 + .104291*.14 = .801383, which might be quoted, to the nearest percent as 80%. The full distribution would be:

                p(well)   p(soso)  p(poorly)


had done well  0.801383  0.157801  0.040816





had done soso  0.193489  0.613084  0.193427





had done poorly 0.03506  0.135503  0.829437

-----
which is close to the observed statistics.

The Excel solver wasn't able to come up with a scenario exactly matching all the given statistics. Perhaps the independence hypothesis is not enough to completely account for the statistics, but it certainly could account for the major tenor of them. In fact there could be mixed motivations in the student body, some being buoyed by a good score and depressed by a bad score, while others take a good score as a reason for "resting on their laurels" and tend to slack off or see a poor score as a "wake up call" to "get cracking". The combined mixture of student types and abilities could account for the statistics shown.

The alternative, from the teacher's opinion:

But, to take the teacher's point of view that the statistics are completely the result of the psychologically buoying effects of a good score and the depressing effects of a poor score, and they affect only the next test:

If the probabilities are as given by the statistics, from one test to the next, then for each successive test, the probability of doing well equals the probability of having done well on the previous test times .8 plus the probability of having done soso times .15 plus the probability of having done poorly times .05. Similar conditional probabilities, as given by the statistics, applied to the previous test's probabilities give the new test's probabilities. In the first test, we know the probability of having done well is 1, as that has already happened. The following program goes through the 4 additional generations of tests:

DEFDBL A-Z
CLS

well2well = .8#
well2soso = .15#
well2poor = .05#

soso2well = .2#
soso2soso = .6#
soso2poor = .2#

poor2well = .03#
poor2soso = .15#
poor2poor = .82#

DIM results(5, 3)
results(1, 1) = 1
FOR i = 2 TO 5
  results(i, 1) = results(i - 1, 1) * well2well + results(i - 1, 2) * soso2well + results(i - 1, 3) * poor2well
  results(i, 2) = results(i - 1, 1) * well2soso + results(i - 1, 2) * soso2soso + results(i - 1, 3) * poor2soso
  results(i, 3) = results(i - 1, 1) * well2poor + results(i - 1, 2) * soso2poor + results(i - 1, 3) * poor2poor
  PRINT USING " #.########"; results(i, 1); results(i, 2); results(i, 3)
NEXT

PRINT results(5, 1)

The results are

  0.80000000  0.15000000  0.05000000


  0.67150000  0.21750000  0.11100000


  0.58403000  0.24787500  0.16809500


  0.52184185  0.26154375  0.21661440


 .5218418500000002

-------
with internal rounding causing that excess 00000002.

With the teacher's assumptions in mind, the answer would be 52.18%. (That's 52.18% in case the html bold indication makes the number disappear altogether.)

So the answer is somewhere between 52.18% and 80%, depending on how much general aptitude actually played in the statistics observed by the teacher. Similarly to the probability of broken sticks forming triangles, or random chords being larger than a circle's radius, or other coin's matching a reported coin's tails status, the probability depends on how the observed statistics got there.

Posted by Charlie on 2003-12-26 14:49:56