Steve Herman showed that by letting f(x) = ax, then applying the above condition one arrives at a quadratic equation. Note if the form was f(x) = ax + c, then one can easily prove c = 0, and that a = 3 or -4. Steve correctly stated only a=3 satisfies the problem, since a = -4 doesn't map from R+ to R+.
Next, assume f(x) = ax^2 + bx + c.
Then, f(f(x)) = a(ax^2 + bx + c)^2 + b*(ax^2 + bx + c) + c.
From examination of the above expression, there exists one x^4 term in the expression. This term is a^3x^4. Since f(x) doesn't possess an x^4 term however, the condition f(f(x)) = 12x - f(x), implies that x^4 term of ff(x) must be 0, and thus a = 0. If a = 0, then we find ourselves considering the equation of the form f(x) = ax + c, which Steve Herman already solved.
By continuing this reasoning, one can show through induction that f(x) cannot be a polynomial of degree 2 or more.
Thus any expression that can be represented by an infinite Taylor expansion, or any polynomial of degree greater than 2 cannot satisfy the expression.
Edited on September 10, 2012, 6:03 am
Explanation of my solution.
