(In reply to re(5): Solution
Nice job on the factoring, but now you got me curious about another set of possible solutions, those which satisfy the other factor you created: (2r^2 - 2a^2 - b^2) + 2ab = 0.
I rewrote this as 2r^2 = a^2 + (a-b)^2. From the identity 2*5^2 = 7^2 + 1^2, I found sets (a,b,r)=(7,6,5) or (7,8,5).
Since r is the smallest value, I wonder if these represent an improper octagon, joining every other corner or every third corner?!