In order to assess the development of children’s cognitive and social skills and to clar- ify the structure of family conversations as an area in which the developmental process is promoted, this study examines children’s behaviors in interactions with their family members. From videotaped, naturally occurring conversations in a Japanese family (including a parent, four and two-year-old boys), scenes in which the child avoids join- ing the conversations were extracted. An interactional analysis of the scenes reveals that family members collaboratively assigned the “side-participant”status to the child. Meanwhile, the child voluntarily gave up the role of addressee or next speaker. Our analysis of the scenes revealed that the gaze direction of the speaker was used as a cue to determine who the addressee was. Furthermore, the child could extract the meaning from the father’s utterances without gazing and nomination in order to speculate the child’s own participation status.