Let A^2 = X^2 + Y and B^2 = Y^2 + 17X
A must be at least X+1 and B must be at least Y+1. Substituting these inequalities yields Y >= 2X+1 and (17X-1)/2 >= Y.
Substituting the latter inequality into the first square then implies X^2 < A^2 <= X^2 + (17X-1)/2 < (X+5)^2. Then A is equal to one of X+1, X+2, X+3, or X+4.
Case 1: A=X+1
Then Y=2X+1, which implies (2X+1)^2 < B^2 = 4x^2 + 21x + 1 < (2X+6)^2. B is then one of 2X+2, 2X+3, 2X+4, or 2X+5. Two of these yield integer values: B=2X+4 yields X=3 and B=2X+5 yields X=24. Then (X,Y) = (3,7) or (24,49) are solutions.
Case 2: A=X+2
Then Y=4X+4, which implies (4X+4)^2 < B^2 = 16x^2 + 49x + 16 < (4X+7)^2. B is then one of 4X+5 or 4X+6. Both of these yield integer values: B=4X+5 yields X=1 and B=4X+6 yields X=20. Then (X,Y) = (1,8) or (20,84) are solutions.
Case 3: A=X+3
Then Y=6X+9, which implies (6X+9)^2 < B^2 = 36x^2 + 125x + 81 < (6X+11)^2. B is then 6X+10. This does not yield an integer value.
Case 4: A=X+4
Then Y=8X+16, which implies (8X+16)^2 < B^2 = 64x^2 + 273x + 256 < (8X+18)^2. B is then 6X+10. This yields an integer value X=33. Then (X,Y) = (33,280) is a solution.
Then the set of all positive integer solutions is (X,Y) = (1,8), (3,7), (20,84), (24,49), or (33,280).