An inverted cone of radius a and height h is filled with water. A sphere made of a material denser than water is placed in the cone. Find the radius of the sphere, r, that will displace the largest volume of water.

There are four possible relationships between the sphere and the cone.

First, the sphere is entirely inside the cone.

Second, the sphere is mostly in the cone with the center still inside the cone.

Third, the sphere is mainly outside the cone along with its center but the ring of contact (where the cone is tangent to the sphere) is still inside the cone.

Fourth, the sphere is almost entirely outside the cone.

The boundary condition between the first and second cases is the sphere is the insphere of the cone. This happens when r=a*h/(a+sqrt(a^2+h^2).

The boundary condition between the second and third cases is when the center of the sphere coincides with the center of the cone's base. This happens when r=a*h/(sqrt(a^2+h^2).

The boundary condition between the third and fourth cases happens when the edge of the cone is the ring of contact. This happens when r=(a/h)*sqrt(a^2+h^2).

The first and fourth cases are easy to dispose of. In the first case the largest volume displaced occurs at the first boundary and in the fourth case the largest volume displaced occurs at the third boundary. The stuff in between is a lot tougher.

Just comparing the spheres of the first two boundaries finds that if h:a is about 3.7151:1 (assuming my math is right) then the insphere volume of the first boundary is the same as the half-sphere in the second boundary.