With the progress in the analytical technique of non-linear material problem by the finite element method, the application of this method to thermal elasto-plastic problems have been carried out by many researchers. However, in case of thermal stress analysis during welding, the valid solution is not always guaranteed because the occurance of yielding, unloading and re-yielding is very complicated. This is due to the temperature dependence of material properties and unstational temperature change. Usually, constitutive equation is derived under the assumption that the stress increment is infinitesimal and the terms higher than second order can be omitted, and is expressed in differential form. In thermal elasto-plastic problems, however, the higher order of stress increment can not be omitted because the yield stress becomes very small at high temperature. In this paper, constitutive equation is derived taking into account the second order of the stress increment. Then d λ, the scalar in Prandtl-Reuss flow rule, can be expressed in quadratic equation with respect to strain increment. The valid solution can be obtained when the time increment is controlled by the discriminant of this quadratic equation. As numerical examples, some typical thermal elasto-plastic problems are analyzed by the initial strain method based on the finite element method. The convergence of the iterative approach is fairly good and the results obtained are quite satisfactory. The residual stress of actual stiffened panel is also analyzed and a formula that estimates the residual stress distribution for a given panel is proposed.