Color categories sit at the intersection of two central topics in the study of human cognition: 1) the analysis of vision, and 2) the study of semantic categories, or more generally processes of classification. Using as data videotape of archaeologists filling out a coding sheet that requires them to systematically describe the color of the dirt they have excavated, this paper describes the practices required to competently classify color within the work life of their profession. The task of color classification is embedded within a situated activity system, which includes not only several different ways of identifying the same color (each designed for alternative uses), but also cognitive artifacts, such as a Munsell color chart, and specific embodied practices. The chart creates an historically constituted architecture for perception, a heterotopia that juxtaposes in a single visual field two very different kinds of space. As multiple parties fill out the coding sheet together the full resources of the organization of talk-in-interaction are brought to bear on the contingent tasks they are charged with accomplishing. The present investigation of a situated activity system encompassing not only semantic categories, but also physical tools and embodied practices, contrasts with most previous research on color categories, which has focused almost exclusively on mental phenomena, and not on how people perform color classification to pursue a relevant course of action in the consequential settings that make up their lifeworld.