ABC is a right triangle with legs a and b and hypotenuse c. Two circles of radius r are placed inside the triangle, the first tangent to a and c, the second tangent to b and c, and both circles externally tangent to each other. What is the smallest possible area of the triangle if a, b, c and r are distinct integers?
Without loss of generality assume that AC is the shorter leg of length b.
The segment joining the centers of the two circles is parallel to the hypotenuse AB. Draw the common tangent to the two circles. This tangent is the bisector of the segment joining the two circles centers and is perpendicular to the hypotenuse.
Let D be the point where the tangent line intersects AB, E be the point where the line intersects BC and F be the point where the line intersects the extension of AC.
Triangles AFD and EBD are similar to ABC. The two circles are the incircles of AFD and EBD. Since the circles are the same size then the two triangles must be congruent.
Trivially, AD+DB=AB=c. From similar triangles ABC and EBD, DE/DB = AC/CB = b/a. From congruent triangles AFD and EBD, AD=DE. Combining these results implies DE+DB=AB and then ED=b*c/(a+b) and BD=a*c/(a+b).
Then area = semiperimeter*radius implies radius r = a*b*c^2 / ((a+b+c)*(a+b)^2). Now the problem has been reduced to finding a Pythagorean triple (a,b,c) such that r is an integer.
Some progress
