During a five-year period the authors taught over 100 students in a graduate course (The Nature of Science) counting toward teacher certification at the Harvard Graduate School of Education. Despite the fact that students had undergraduate degrees in the sciences, most of them found the application of models in science challenging and the epistemological consequences unsettling. Moreover, students found it especially difficult to use a model to correctly generate predictions, which was starkly illustrated with the application of Archimedes’ principle during our unit on floating and sinking. We examine the deceptive belief that student success with algorithms and word problems leads to conceptual understanding as well as the conceptual change necessary to understand the relationship between evidence and inference as explored in the nature of science. Considering the apparently strong science backgrounds of our students, we doubt that typical pre-college students can achieve the goals described in the National Science Education Standards in the short time typically allotted for their science studies. We explore the issues students face in “understanding” science as well as the impact of science education on students and teachers, and implications for policy makers and pre-service programs.