Hi,
There are several methods of factoring polynomials. Here is the way that I personally tend to do it:
(1) Factor out any obvious factors (what is obvious will be different for different people). If you like, this could mean factoring out any common constant factors. This step also covers known factorizations, such as perfect squares, a difference of squares, a difference of cubes, factoring by grouping, etc.
(2) If this is a quadratic, try the standard factoring tricks. If not, move on.
(3) Test for simple roots, like 1 or -1. Due to the Root-Factor Theorem, each root corresponds to a binomial factor.
(4) Use the Rational Roots Theorem to test for all possible rational roots.
(5) If there are no rational roots left, all remaining roots are either irrational or complex. As proven by Abel, if the remaining unfactored polynomial is of 5th degree or higher, and it is not a special case (like a difference of fifths), then there is no finite general formula involving elementary algebra operations to find its roots. If the remaining polynomial is of 4th degree or lower, then we can use one of the standard formulas to find its roots, and thereby factor it.
Looking at x⁶ - 14x⁴ + 49x² - 36, we can quickly pass over steps (1) and (2).
For step (3), we note that 1⁶ - 14∙1⁴ + 49∙1² - 36 = 0, so 1 is a root of this polynomial. So is -1, via the same test.
Thus, this polynomial has (x + 1)(x - 1) = x² - 1 as a factor. Let us divide to get the quotient:
(x⁶ - 14x⁴ + 49x² - 36) ÷ (x² - 1) = x⁴ - 13x² + 36
This means x⁶ - 14x⁴ + 49x² - 36 = (x + 1)(x - 1)(x⁴ - 13x² + 36).
Now we perform the same steps on x⁴ - 13x² + 36. We note that this is a quadratic in x², since it can be written as (x²)² - 13x² + 36. So we can factor it using the usual quadratic tricks.
In particular, 4∙9 = 36 and 4 + 9 = 13, so performing some sign changes, we can see that (x²)² - 13x² + 36 = (x² - 4)(x² - 9).
Thus, x⁶ - 14x⁴ + 49x² - 36 = (x + 1)(x - 1)(x² - 4)(x² - 9).
The final two factors are differences of squares, which have a known factorization:
x⁶ - 14x⁴ + 49x² - 36 = (x + 1)(x - 1)(x + 2)(x - 2)(x + 3)(x - 3).
These are all linear factors, so we are done.
You can use the same steps for factoring over the complex numbers.