The Brazilian National Guidelines for Undergraduate Medical Education foster changes in curricula and the teaching-learning-evaluation processes to develop professional competence and skills. This article is an initiative by the Brazilian Association for Medical Education (ABEM), focusing on the meaning and use of such concepts according to experts in health education evaluation. Ten interviews with open-ended questions were conducted with researchers, selected according to their academic background. Their narratives were submitted to thematic content analysis. The results highlighted the meaning ascribed to competence, the role of subjects involved in the process, the context in which skills and competence are developed, and the relationship between competence assessment in medical education and the provision of comprehensive health care. Although adopting diverse approaches, researchers in the area have embraced the concept of competence creatively. Nevertheless, this study suggests that strengthening a "culture of evaluation" requires investment in study groups and research to deepen the meaning of concepts, objectives, content selection, definition of criteria, development of appropriate tools, and evaluators' training in Brazilian medical schools.