Recent cross-sectional and randomized controlled studies of small sample sizes revealed that regular exercise is effective for improving nonalcoholic fatty liver disease. However, there has been no large-scale longitudinal study addressing the effect of regular exercise on remission of nonalcoholic fatty liver disease. Thus, we investigated the impact of exercise on the natural history of nonalcoholic fatty liver disease. We analyzed 1,010 (860 men and 150 women) Japanese participants who received health checkups repeatedly over 10 years by a historical cohort study and were diagnosed with nonalcoholic fatty liver disease at baseline. Regular exercise was defined as participating in any kind of sports at least once a week. Nonalcoholic fatty liver disease was diagnosed by ultrasonographic images. During 10 years of follow-up, remission of nonalcoholic fatty liver disease was observed in 46.0% (396/860) of men and 48.7% (73/150) of women. In men, the adjusted hazard ratio of regular exercise for remission of nonalcoholic fatty liver disease was 1.46 (95% confidence interval 1.10–1.95, p = 0.010). However, this was not significant in women. Exercise at least once a week is implicated in the remission of nonalcoholic fatty liver disease in men.