In this paper we address a number of issues concerning how the modular structure of the human cortex arises. In contrast to the prevailing view that these modules are prespecified, we discuss evidence from developmental neurobiology that this specificity emerges through activity-dependent processes during pre and postnatal development. We present some examples of this process with data from human infants, and illustrate some of the mechanisms involved with neural network simulations.