Then I started playing around by splitting the quartic into two functions: f(x) = x^4+bx^2+1 and g(x) = (-a)*(x^3+x). f(x) and g(x) intersect exactly when the original polynomial has a real root.

I noticed that when fixing b, then the minimum choice of a occurs when f(x) is tangent to g(x).  Or equivalently the original x^4+ax^3+bx^2+ax+1=0 has a double root.

x^4+ax^3+bx^2+ax+1=0 is symmetric, so for any root its reciprocal is also a root.  Then for a double root to occur then either the whole equation is a perfect square or one of 1 or -1 is a double root.

Case 1: the whole equation is a perfect square

Lets start by completing the square:

x^4+ax^3+bx^2+ax+1 = (x^2+(a/2)x+1)^2 + (b-2-a^2/4)x^2 = 0.  But for this to be a perfect square the leftover x^2 must be zero; then b-2-a^2/4 = 0.

Isolating b yields b = 2+a^2/4, then we must have b>=2, and then a^2+b^2 >= 4.  But this is already larger than the first guess where a^2+b^2=1.

Case 2: 1 or -1 is a double root

Having 1 as a double root is identical to having -1 as a root, just by making the substitution x->-x (reflection over the y-axis).  So these are just two ways of expressing the same thing.

So wlog let x=-1 be the double root.  Then (x^2+2x+1)*(x^2+cx+1) = x^4+ax^3+bx^2+ax+1.  Expanding and equating coefficients gives c+2 = a and 2c+2 = b.

Then minimizing a^2+b^2 is equivalent to minimizing (c+2)^2 + (2c+2)^2 = 5c^2+12c+8 = 5*(c+6/5)+4/5.  In this last form it is easy to see the minimum value is 4/5, which occurs when c=-6/5, which makes a=4/5 and b=-2/5.