In generalized additive models (GAMs), the extent of smoothing is controlled by the user-defined degrees of freedom (df). Often, to choose the df, alternative models are estimated with different df and the best-fitting model based on some data-dependent criterion (e.g. AIC) is identified. Through simulations, we estimated the type I error of the GAM-based tests of (i) no association, and (ii) linearity, while using this approach. Overall empirical type I error rates of the tests, conditional upon the AIC-optimal df, were two to three times higher than nominal levels. We proposed new critical values, calculated from the empirical distribution of the “conditional” test statistic generated from simulations with the true null hypothesis that resulted in type I error near 5%. Finally, we compared the power to detect non-linear associations of several AIC-based selection strategies to (i) GAMs with df selected a priori and (ii) conventional parametric logistic models.