Three points have been chosen randomly from the vertices of a n-sided regular polygon.

Determine the probability (in terms of n) that the center of the given polygon lies in the interior of the triangle determined by the three chosen points.

Case 1--odd number of vertices:

After the initial vertex is chosen, there are (n-1)/2 possible distances to the second vertex, after which there are n-2 vertex choices for the third point.

The probability of a successful (center within the interior of the triangle) third point depends on which of the (n-1)/2 distances that the second point is from the first. If the first distance is only 1 unit, there is only one position for the third point that leads to success. If the first distance is 2 units, there are two points, etc., until, when the first distance is (n-1)/2, there are (n-1)/2 choices that lead to success.

The number of ways:

1, 2, 3, ... , (n-1)/2

In each case the conditional probability is the given number divided by (n-2). The average of the above numbers is ((n-1)/2 + 1) / 2 = (n+1)/4, and therefore the average of the conditional probabilities is (n+1)/(4*(n-2)), which is thus also the overall probability.

Note that even for a 3-gon (triangle), the formula yields a 1, which is correct. For the pentagon, it's 6/12 = 1/2.

Case 2--even number of vertices:

After the initial vertex is chosen, the n/2 possible distances to the second vertex are not equally likely. If the second vertex is directly opposite the first (probability 1/(n-1)), the resulting triangle will not contain the center of the n-gon strictly on its interior as it will lie on a side. Therefore we shall first consider the cases where the second vertex is not directly opposite the first, and then multiply the probability found by (n-2)/(n-1), which is the probability that the second vertex is not directly opposite the first.

Given that the second is not directly opposite the first:

The probability of a successful third point depends now on which of the n/2 - 1 distances that the second point can be from the first. If the first distance is only 1 unit, there is no position for the third point that leads to success. If the first distance is 2 units, there is one point, etc., until, when the first distance is n/2 - 1, there are n/2 - 2 choices that lead to success.

The number of ways:

0, 1, 2, ... , n/2 - 2

In each case the conditional probability is the given number divided by (n-2). The average of the above numbers is (n/2 - 2)/2 = n/4 - 1, and therefore the average of the conditional probabilities is (n/4 - 1)/(n-2), which is thus also the overall probability assuming that the second vertex is not directly opposite the first.

Applying the correction factor of (n-2)/(n-1), this becomes (n/4 - 1)/(n-1).

In the 4-gon (square) case, this becomes trivially zero. A more pertinent test case is for the hexagon, where the formula gives (6/4 - 1)/5 = 1/10. This agrees with the fact that only two triangles--the inscribed equilateral triangles--out of the C(6,3)=20 possible triangles, meet the criterion.

In summary:

Odd n: (n+1)/(4*(n-2))
Even n: (n/4 - 1)/(n-1)

In the limiting case of the circle, both approach 1/4.

Posted by Charlie on 2011-10-11 12:28:02