The design of the inverse system is a common issue on the design of a digital signal processing and communication systems. if the system transfer function is not minimum phase, the inverse system is unstable based on the least square error criterion, employing the calculation of variation technique and spectral factorization, a stable inverse transfer function is designed both fir and IIR filter design examples are given to illustrate the design procedure.