In this paper, a ship's tracking controller using an optimal tracking algorithm by the Bryson-Ho algorithm is proposed. In this system, the optimal controller by using the algorithm of time-varying linear-quadratic controller (TVLQC) steers a rudder so that the ship navigates on the desired track. When stationary disturbances like wind or current occur, a virtual track route in which the true desired track is appropriately shifted, is introduced. In the latter part of this paper, as a tracking by TVLQC in low speed a tracking using bow thruster is discussed. The possibilities of the proposed algorithm are tested in simulations and actual ship's tracking experiments using an actual ship.