The objective of this article is to evaluate the accuracy of the New Ballard (NB) score in detecting prematurity. Date of last menstrual period (LMP) and ultrasonography (USG) are used as reference procedures. First, an assessment of inter-observer reliability of the NB score measurements and the USG reports was carried out. Regarding the NB scores, the kappa coefficient and the Intraclass Correlation Coefficient were, respectively, 0.74 (CI 95% 0.49-0.99) and 0.88 (CI 95% 0.78-0.93). For the USG analysis an Intraclass Correlation Coefficient of 0.96 (CI 95% 0.91-0.98) was found. Turning to validity, sensitivity of the NB score was below 0.70 in all but one comparison, irrespective of which procedure was taken as reference. Although specificity was consistently above 0.90, results suggest that the diagnostic ability of the NB score to detect prematurity is far from ideal. The proportion of false negatives may be a warning against its use alone, both in clinical practice and in epidemiological research contexts. Rather, the procedure should be recommended mainly as an accessory diagnostic tool.