x²z + y²z + 4xy = 40,

x² + y² + xyz = 20

(1, 1, 18) and (-1, -1, 18) work.  I don't have an extensive math background, so I don't always know how these are supposed to be solved, or how to tell if there might be any other solutions available, I just kind of looked at the equations and those combinations popped out at me.

It's pretty obvious that z could = 2, but I couldn't find any integer solutions for x and y if that was the case.

As far as smaller integers go, I didn't immediately see any other solutions.  However, I suppose it is possible that there are larger values of x and y that would work as long as one of them is negative and the other is positive (doesn't matter which is which, of course).