Indigenous children's multimodal communication of emotions through visual imagery.

Mills, Kathy A. ; Bellocchi, Alberto ; Patulny, Roger 等

Introduction

In recent years, emotions have become a major focus of language and literacy research nationally and internationally (Mills, Unsworth, Bellocchi, Park & Ritchie 2014; Macken-Horarick & Martin, 2003; White, 2005). Children's ability to express their personal emotions by writing, talking, or via other modes is associated with long-term social and academic success (Durlak, Dymnicki, Taylor, Wesissberg & Schellinger, 2011). Conversely, the inability to articulate and interpret emotions is linked to poor social outcomes (Banninger--Huber, 1992). In terms of student outcomes, strategies to identify emotions in self and others in ways that support language, learning and participation in school, are of central importance (cf. Durlak, Domitrovich, Weissberg & Gullotta, 2015).

Teaching students how to express emotions is of national and international significance because there is a critical problem for the teaching profession--worldwide, under-preparedness to teach multimodal literacy, including the grammar of visual representation of emotional expression (Hudson & Whalmsley, 2005). Those entering the profession are often anxious about the language of emotions (Macken-Horarik, 2009). Multimodal research to date tells us how emotions are depicted in a range of print and digital texts, but not how to teach students the multimodal expression of emotions and the associated grammar.

Visual imagery is a powerful form of communication, even in the absence of written words. A considerable number of scholars have provided accounts of how images convey meanings across a range of social contexts, such as in visual ethnographic research (Pink, 2007), picture books (Painter, Martin & Unsworth, 2013), classroom practice and e-literature (Unsworth, 2001, 2006a), and in children's visual compositions (Mills, Comber & Kelly, 2013). The circulation of images via the social web--the set of social relations that connect web users together--has reached unprecedented velocity via online photo-sharing and social networking sites, such as Snapchat, Instagram, Pinterest, Tumblr and Flickr (Mills, 2015). Yet little research has examined the way in which expert knowledge of the visual language of emotions can be understood, developed, taught and applied by children and young people in the context of multimodal literacy pedagogy and curriculum.

Theorists of multimodal semiotics, like Kress (2014), see that multimodal language must be interpreted in relation to the culturally negotiated semiotic resources that are not universally understood. Kress and van Leeuwen (2006, p. 4) acknowledge that their' ... investigations have been restricted, by and large, to Western visual communication'. Indigenous students' multimodal representations remain largely under-examined (Mills et al, 2016), and studies of the emotional, multimodal language of Indigenous students are rare. The term 'Indigenous' in this paper refers to the Aboriginal and Torres Strait Islander peoples of Australia. It is acknowledged that there are many First Nations people, each with their own cultures. Indigenous researchers have identified that epistemologies of Indigenous groups have both similarities and distinctions to other groups (e.g. Martin, 2003), but researchers have not examined how the integration of such knowledge principles can be embedded in the multimodal literacy learning of Indigenous children in school settings.

This paper applies an appraisal framework--a toolkit for analysing the expression of emotions in discourse--and extends it here to examine students' visual texts. Martin and White (2005) developed the present appraisal framework as an outcome of the Disadvantaged Schools Program Write it Right literacy project. Systemic Functional Linguistics theorists, Rothery and Stenglin (2000), Poynton (1990), Macken-Horarik & Martin (2003) and Hood (2004), among others, were also part of the ongoing development of the work through applied research. Unsworth (2006b), White (2005) and Mills and colleagues (2014) have applied the appraisal framework to accounts of the visual image.

Guiding students' communication of emotions is particularly timely in Australia because the Australian Curriculum: English specifically addresses a strand of outcomes called 'Language for Interaction' taught from Prep to Year 10. Students must 'recognise ways that emotions and feelings are conveyed and influenced' in expression (ACARA, 2014). Students need to speak, write and communicate in digital and non-digital ways to generate empathy.

Research questions

In the context of working with Indigenous Australian students across four grade levels and through a series of professional visual imagery workshops, this research addresses the following questions:

1. How can Indigenous students construct affective meanings through still digital imagery?

2. What affective meanings are evident when Indigenous students express emotions using still digital imagery?

These questions are significant because communicating sophisticated affective meanings in visual texts is an under-researched, yet important life skill for Indigenous students who participate in image-based, globalised communication environments. Affect or affective meanings are defined and analysed in accordance with Martin and White's (2005) language of evaluation--the appraisal framework in English. The research questions are focused on digital imagery, which includes representations of gaze, gestures, posture and other semiotic elements.

Research design

This article reports participatory research conducted at an independent school in Queensland, Australia. Cross-cultural, participatory research is a methodology that involves collaboration between researchers often belonging to a different culture to the research participants, to address an identified area of need and development in the community (Stoecker, 2005). A key feature of participatory research is that the focus of community development is negotiated with the community leaders, rather than driven by the researchers' agenda. In the present study, the Indigenous principal of the school had identified multimodal literacy as an area of importance in the curriculum, and recommended the teachers to be involved in the research. The school teachers in the study were also interested in achieving the goals of the national curriculum through engaging with the research. The principal invited the lead researcher to engage in a number of 'yarning' sessions to discuss the program--a form of inclusive dialogue to reach common understandings which is central to Aboriginal cultures in Australia (see, for example: Mills, Sunderland and Davis-Warra, 2013). One of the researchers participated in orientation workshops to understand the community's approach to Indigenous knowledge, and was invited to engage in their cultural and community ceremonies and celebrations at the school.

Site description

Hymba Yumba Community Hub in Springfield, Queensland, is an independent school that had been operating for five years at the time of the field work. The students identify as having Aboriginal and/or Torres Strait Islander backgrounds. The teachers and students spanned four grade levels from two composite classrooms: Years 4-5 (students aged 9-10) and Years 5-6 (students aged 10-11). The Guidelines for Ethical Research in Australian Indigenous Studies was followed in all relevant aspects of this research (AIATSIS, 2012).

Digital imagery workshops

The workshops supported students to communicate a range of emotions through digital image composition. The teachers and the director of a digital imagery studio, Big Picture Industries Inc (http://www.shed11. com.au/#!workshops/ckiy), were involved in the planning of the program with a university academic. The two teachers identify as Aboriginal. One teacher had four years of teaching experience and the other is a co-author of this paper, in her second year of teaching. A series of four full-day visual imagery workshops were conducted over several weeks with 56 urban Indigenous, upper primary school students.

In the words of the Indigenous teacher, the workshops formed part of a larger unit of work designed to integrate '... the idea of creating their place in a visual way, or themselves and their identities, particularly to counter deficit images of Indigenous young people'. The teacher used selected images from the workshops to later create a book of the children's faces, and organised for each portrait to be supplemented by written text about themselves.

The workshops were also connected to a science unit about light and colour, and as part of this, the students were guided by a professional visual imagery director to capture over one thousand indoor and outdoor images. Students were taught how to create studio-quality portraits of their peers, fast-moving action, still life and nature photography. In some workshops, students were given a list of emotions to communicate through photography, such as 'powerful', 'miserable', 'lonely' and 'scared'. This list was elicited from the students in the first instance, with additional emotions suggested by the teachers and researchers to extend their repertoire of affective vocabulary. The list included examples of meanings across three clusters of emotions in Martin & White's (2005) attitude network--un/happiness, dis/ satisfaction and in/security--while dis/inclination was not included (see further discussion in Data Analysis below). Students were then shown models of exemplary images taken by other groups of students.

When photographing their peers, the students were given significant autonomy to choose to portray each emotion on the list using their own creative interpretation, including who would be in each photo, location, positions, gaze, facial expressions, body language, movements and other compositional elements (e.g. camera angles, framing). Each day of the workshops was divided into a morning session in the studio and an afternoon session outdoors. There was sufficient time for the students to represent all the emotions on the list.

In the indoor format, the studio directors set up four studio areas as workstations for groups within the class: (a) Still life station for children to bring items of nature to photograph close up with professional lighting; (b) Action shot area with a floor to wall white background and bean bags for landings; (c) Portrait area with timber background and direct natural light from a window; and (d) Portrait area with black background and professional lighting. The outdoor photography was conducted in the adjoining parklands, in small groups led by an adult, with a camera shared among the group.

Some workshops focused on particular skills, such as capturing peers with moving water from strategically bursting water bombs, trick photography, or still portraits with moving participants in the background. Assisting adults included two digital imagery experts, the lead researcher, teachers and Indigenous education workers who provided supervision, but who did not take the photos. Throughout the workshops all images were taken by the students--in the words of the students: 'The person that was taking the picture directed the action' in the narrative. As the workshop director noted, 'If I come in, Mr Photographer ... and I'm telling their story, I'm never going to get there, because the only way a story can be told is from their perspective'.

Data collection

Three data sets were collected to address the research question concerning the teaching of emotional expression through still digital imagery. The data sets were: (i) over one thousand still digital images produced by the students; (ii) semi-structured interviews with the students about their digital imagery practices during the workshops; and (iii) semi-structured teacher interviews and meetings about the digital imagery program. As participant researchers, two of the authors observed and interacted with the students to learn about their intentions for the designs. The project received ethical clearance from a human research ethics committee and obtained voluntary, informed, understood and signed permission from the parents and students to share their facial images as part of the ethics approvals.

The digital images were created in student groups, with each group carrying one camera and taking turns. Multiple cameras were in use throughout the workshops, with opportunities for all students to use the cameras. A small sample of the best images were lightly edited by the workshop director through techniques such as cropping, desaturation and some use of a sharpening tool; an illustrative collection of these have been included in this article.

The semi-structured student interviews were conducted with a roving audio-recording device, to sample the students' intents and insights about the construction of the visual images as they worked with the cameras. This data set was used to identify patterns in the children's thinking about how they communicated their emotions through digital photography. Example questions included: 'What are you thinking about when you take these photographs?'; 'What emotions are you trying to show?'; and 'How are you going to show this emotion?'

The teacher interviews and meetings were held during the planning of the visual imagery workshops with the studio director and audio-recorded over several days during the workshops. These data were similarly aligned to the research question, and were aimed at understanding the pedagogical and curriculum dimensions pre-identified by the teachers as enablers of student emotional communication through still digital imagery. Example questions included: 'How will the digital imagery work be integrated with your curriculum, and for what kinds of outcomes?' and 'What affective meanings were the students able to communicate through the digital images?'

Data analysis

The student and teacher interviews were replayed, transcribed and the key themes coded and analysed for repeated themes, to understand the student and teacher intentions and interpretations of the digital imagery practice (Silverman, 2014). Analysis of the student photographs and the interview data sets involved applying the appraisal framework to the images, as outlined in The Language of Evaluation: Appraisal in English (Martin & White, 2005). There are three modes of meanings that operate concurrently in all language--the interpersonal (i.e. construction of relationships), the ideational (i.e. construction of the nature of events) and the textual (i.e. relative information value among textual elements) (Matthiessen, 1995). The appraisal framework develops the Hallidayan account of the interpersonal metafunction of language, with a secondary focus on the ideational and textual meanings (Halliday, 1994). The interpersonal is focal because appraisal in English analyses the linguistic resources used by interlocutors to represent feelings and attitudes within the nature of relationships between the author, the audience and the represented participants (Mavers, 2014; Mills et al, 2014).

It was necessary to extend Martin and White's linguistic appraisal framework to address the representation of affect through images. This system was chosen because it is a leading framework for mapping feelings or attitudes in discourse semantics. It attends to three axes--affect, judgment and appreciation through which an author's inter-subjective position is communicated (Martin & White, 2005; Mills et al., 2014).

The axis of the framework that concerns emotions in language is affect--language resources to convey positive and negative feelings. The coding framework applied three of Martin and White's (2005) broad categories of affect (un/happiness, dis/satisfaction, in/ security) and relevant elements of multimodal design, such as facial expression, posture, movement, camera angles, lighting, framing and composition (Exley & Mills, 2012). Note that Martin and White posit a fourth cluster based on 'irrealist affect' called dis/inclination, a cluster of emotions associated with fear and desire that always implicate a trigger. These were not included because the many specific 'desires' that are lexicalised in English are not considered by most researchers to be emotions, but cognitive states, such as hunger (desire for food), thirst (desire for drink), or curiosity (desire for information) (Ortony & Turner, 1990). Furthermore, anxiety or fear is already addressed in the in/ security cluster of emotions (Martin & White, 2005).

Affect meanings may be descriptions of emotional states (e.g. I feel excited) or behaviours that indicate emotional states (e.g. her eyes misted with tears). This approach extends beyond some existing accounts of affect because it addresses not only the funds of language through which authors overtly encode what they present as attitudes, but also how they indirectly or directly communicate feelings. Different intensities of the same emotion can also be expressed grammatically using different lexical terms. For example, gradations of 'scared' from low to high could be expressed as scared (low), terrified (median), or petrified (high) (Martin & White, 2005).

Gradation values available in the visual semiotic resources of photographic images have been demonstrated to be a richer and more powerful source than in written language, with multiple values able to be simultaneously infused with different kinds of meaning. Economou (2008) provides a relevant example of a journalistic photo intended to instantiate factual news, which shows a refugee couple emerging from a disaster scene, embracing one another in a side-by-side hug, and walking toward the camera like hero survivors in a Hollywood movie poster. A visual image can thus depict a factual event, while simultaneously cinematising the depicted subjects to provoke positive appreciation, satisfaction and empathy.

Findings

The research findings demonstrated that the students were able to provoke (directly) or invoke (indirectly) a range of positive and negative emotions across the six major sets of meaning in Martin and White's (2005) typology of affect groups in the attitude network unhappiness or happiness, insecurity or security, and dissatisfaction or satisfaction. Further, the students produced examples of associated meanings of affect. These are systematically covered in the order in which they are presented in the attitude network: misery and antipathy (examples of unhappiness), cheer and affection (examples of happiness), boredom and displeasure (examples of dissatisfaction), interest and pleasure (examples of satisfaction), scared and surprised (examples of insecurity), and confidence and trust (examples of security). These meanings of affect are listed in the attitude network as distinct from their associated behaviours (e.g. fidget or yawn to show boredom or ennui). At times a single image simultaneously provoked several affective meanings. As the photographic subjects, the students were able to perform as social actors using facial expressions, body language, posture, haptics and movement to suggest or communicate certain emotions from their perspective. As photographers, they authored narrative representations that drew on a wide range of visual grammatics, such as different narrative structures, camera angles, shot types, framing and settings to typify particular emotions. As the Indigenous teacher said somewhat metaphorically, 'I think the visual way is a way of finding your voice ...'.

Reading unhappiness in images

'Unhappiness' is the first of six major affect groups in the attitude network. In the visual imagery workshops, the students were challenged not to resort to the first emotions that come to mind, such as feeling happy, because the perpetually smiling face has become a cultural and social convention. The students were asked to think about how to provoke or invoke a sense of unhappiness through the visual image. Unhappiness meanings include feelings of sadness or antipathy that may be internal or directed at someone (Martin & White, 2005). Figure 1 is a student's photograph of a peer who adopts a forlorn posture of slumping shoulders, downward gaze and forehead rubbing, which provokes the negative affect of internal sadness, struggle, or discomfort. As the students confirmed, the photo portrays a subject that is 'sad and emotional'. The researchers noted that she continued to take many photos to portray sadness, because she wanted to make it clearer. After taking this image, she directed her peers toward depicting a different emotion.

The student's posture is telling. The weighed-down body, and the downward head-tilt and gaze form a vector that points to the ground. The student photographer has encoded an oblique angle, which is typically used to suggest detachment from the represented participants, rather than involvement. As Kress and van Leeuwen (2006, p. 136) explain, 'The oblique angle says, "What you see is not part of our world, it is their world, something we are not involved with".' This contrasts with the frontal angle, which provokes direct involvement with the represented participants. Yet at the same time, the photographer has used a close-up shot--typically used to create closer social distance than the medium or long shot (Kress & van Leeuwen, 2006). In this image, the mood is undirected, rather than a reaction to another participant, given the non-transactional nature of the image. There is an actor, but there is no clear phenomenon to which the actor is reacting (Kress & van Leeuwen, 2006). This invokes a sense of empathy with the girl, leaving the event that has triggered the sadness beyond the frame, inviting the viewer to interpret this meaning. The studio setting was used to create a cinematising, emotional effect, as one student described: 'It made some good lighting ... made half of the face look dark'.

Figure 2 is a further example of the visual communication of unhappiness and the evocation of the emotion of antipathy directed at someone else. This is a transactional image because there is an actor and a goal. The girl on the right (from the perspective of the viewer) is the actor, whose body forms a vector pointed at the goal--the other girl whose back is turned away from the actor. The actor is always the participant in a visual action process from whom the vector extends, or which is fused with the vector (Kress & van Leeuwen, 2006).

In this image, her whole body is tilted away from the goal with arms crossed, head tilted along the same plane, with a clearly narrowed, sideways gaze. This forms an eye line vector, which connects two participants: the reactor with the participant, who becomes the phenomenon. The actor's facial expression is negative, and could be described as perhaps looking down her nose in derision, contempt or disdain, accompanied by the turning of the shoulder and tightening of the lips. This meaning is confirmed by the students, who wanted to show the subject as 'moody'. In this narrative image structure, the student photographer has positioned the subjects so that the umbrella lighting does not permit the face of the phenomenon to be seen clearly.

The phenomenon or emotional trigger depicted in the photograph is the student with arms folded to close herself from emotional engagement, as if the two have perhaps been engaged in an emotional conflict. This interpretation is confirmed by the interactions between the girls who contextualised the social context, 'Let's pretend to be having a fight'. This was accompanied by instructions from the student photographer on how to stand, pose and so on. The action of turning one's back can signify confused emotions, walking away, while making oneself vulnerable to the other, signifying a measure of trust (Kress & van Leeuwen, 2006). The photographer explained, 'I used the black background which helped', and the light and dark was 'powerful'.

Reading happiness in images

In the attitudinal network, happiness can be expressed as cheer (internal mood) or affection towards another person (Martin & White, 2005). Cheer is associated with behaviour such as laughing, with dispositions that can be graded from cheerful (low), to buoyant (medium) to jubilant (high). To compose Figure 3, the student photographer directed the peer to lie on the lush green grass and to smile and gaze upwards at the blue, sunny sky. As the photographer reflected, the theme was 'happy'.

The actor's genuine, full smile and arms spread wide provoke happiness or cheer, accompanied by the use of strong natural daylight to bring out the bright colours in the scene. As this student photographer confirmed, 'I use lots of colour--red, green and orange.' The image is framed as a medium shot, which implies a greater degree of personal involvement between the subject and the viewer than long shots. The actor's gaze is directed toward the viewer. The interactive meaning between viewer and participant is influenced by the 'demand' gaze: where the gaze is directed at the viewer to gain their attention. As the workshop director explained, 'As humans, that's where we look--the face--it is a most powerful place of communication'.

The students also attempted to communicate happiness as affection between two or more participants in their narrative visual representations. The girls said that this and other similar images were to show the emotion of 'happy' and 'best friends'. For example, in Figure 4, an image was taken as the resolution to the emotional conflict depicted in Figure 2. This time, the girls engage in an upper-shoulders embrace of friendship. The girl on the left, who was previously the phenomenon, takes on the role of the actor, her body forming a vector that points toward the object of her affection--the goal. Her smiling mouth and closed eyes invoke inner satisfaction. The friend uses an open-mouthed, cheek-raised smile, which can be interpreted as an intensified level of happiness compared to a faint smile. This time, the lighting enables both of the girls' facial expressions to be more visible than in Figure 2. The photographer reflected on the lighting: 'The studio light cube was special'. The medium shot invites the viewer to have a greater degree of empathy for the girls' friendship than a long shot, and the demanding gaze of the girl on the right similarly provokes the viewer's direct attention.

Reading dissatisfaction in images

Less common than images of positive affect were images that communicated a sense of 'dissatisfaction'. Dissatisfaction and satisfaction concern emotions associated with achievement or frustration about the activities in which we are engaged, either as participants or spectators (Martin & White, 2005). The students constructed a small number of thoughtful compositions to suggest dissatisfaction. For example, in Figure 5, the student photographer arranged three peers to pose slouched on the ground, each portraying facial expressions and body language of boredom or disinterest. The sense of ennui is provoked through the use of their hands to support the face.

An interesting feature of this image is that the girls are directing their gaze in three contrasting directions, forming eye-line vectors that diversify, typifying the notion of distraction. One child on the right directs her gaze at the camera to gain attention (demand). The child in the centre looks downcast and the lids are lowered, suggesting an inward thought pattern, rather than outward openness. The girl on the left looks away and to the upper right, as if imagining an alternative place. The mouths of the participants are in either downwards or neutral positions. The size of the frame communicates a medium degree of social distance. The use of dramatic lighting and shadows blur the edges of the figures invoking the generalised essence or mood of the image. The photographer commented, I liked '... how the light reflected and shined on them and created a different effect'.

In another example, the student photographer has worked with the represented lone participant to invoke a sense of dissatisfaction, in particular, displeasure (See Figure 6). The subject is depicted from an oblique angle, with the viewfinder turned diagonally so that the baseline is not parallel with the represented figure. As one student indicated in her interviews, to create interest, 'You can put it [the camera] upwards, tilted, downwards, or sideways.'

The girl casts a sideways glance at the viewer, with slightly narrowed eyes and tightly closed, unsmiling lips to create negative affect. Again, the demand gaze invokes a visual address to the viewers, with the participants' eye-line connecting the subject and the viewer. As Kress and van Leeuwen (2006, p. 117) explain about demand images, 'Contact is established, even if it is only on an imaginary level'. The girl is depicted with both hands in her pockets, her body slightly turned away from the camera, and her shoulders a little slouched.

Reading satisfaction in images

The sheer satisfaction in throwing water is clearly provoked in Figure 7, which depicts a boy with arms fully extended, releasing a bucket of water toward imagined (or real) participants. In the attitude network (Martin & White, 2005), interest is demonstrated through behaviours, such as attentiveness and involvement, as provoked in this image. The boy's tongue escapes the corner of his smiling mouth, perhaps indicating deep concentration as the throw is aimed. The boy's gaze is focused on the movement of water via the bucket, similarly indicating interest and engagement. The two boys talked about the way their bodies were positioned to communicate the mood and to create interest: 'You have to move around'.

In this image, the posture is deictic, since the boy's action--his arm, the bucket and the flow of water droplets --point to non-existent or imaginary referents that are beyond the frame (Jaworski & Thurlow, 2014). The deictic gesture is both embodied and mediated prosthetically, by means of the bucket which functions as an extension of the body.

A different variant of satisfaction can be invoked from the Figure 8, in which three participants experience the rapid flow of water splashes from two directions. The child in the centre provokes pleasure (a form of satisfaction) through his postural stance. His arms are outstretched wide from both sides of the body, the head is tilted back, and his face points upwards with a discernible smile, as if to capture the full pleasure of the water experience. His arms form vectors that point to the two flows of water, making him the central actor. The boys commented about the water experience and photography, 'It was awesome', 'cool', and 'fun'.

The two figures behind the boy show varied responses to the water, perhaps with differing degrees of intensity or gradation of affect. The girl on the left shields her head with her hands to protect her face from the water, while invoking a sense of satisfaction by smiling broadly with high cheeks. The girl on the right cups her hands, reaching forward slightly and invoking interest, as if to experience the water within arm's reach. The overall arrangement provokes or activates a sense of satisfaction and interest, which contrasts the dissatisfaction and ennui provoked or even perhaps inscribed in Figure 5.

Reading insecurity in images

Insecurity and security together form one of the major areas of affect in the attitudinal network, which pertains to feelings of peace or anxiety in our social context and environment (Martin & White, 2005). Fear was one of the emotions the children were specifically asked to represent, and were given examples. While many of the children's representations of insecurity were not always clear, Figure 9 functions to invoke a sense of surprise as the girl experiences the sudden sensation of cold water. This is suggested primarily by her wide open mouth, which may suggest an accompanying vocalisation or scream.

She squeezes her eyes shut tightly, perhaps to avoid water entering her eyes. Her hands are concealed in her pockets, pulled tightly against her body to recoil from the cascade of water, also indicating a sense of anticipation. The boy is the actor in this transactional image. His arm, the metal bucket handle, and the flow of water form a vector that points toward the girl, who is the goal (Kress & van Leeuwen, 2006). His position outside the frame of the composition perhaps assists in evoking the element of surprise, or as the students described, even a bit 'funny'.

Similarly, Figure 10 provokes meanings of high level intensity emotions of insecurity, such as being petrified. The children were recorded planning the photo: 'Do a chilling one, like really, scary'. The large, menacing hand with fingers out-stretched is used as the trigger for insecurity for the boy.

The tree obscures the body, creating a sense of the unknown, and making the hand and human appendage to act of its own volition. The boy's facial expression provokes fear by displaying bared, bottom teeth, and the downward gaze of his eyes acting as an eyeline vector to the hand. The expression of his mouth suggests that there may have been a vocalisation associated with this posture of fear, such as a cry.

Reading security in images

In contrast, security is concerned with positive ecosocial wellbeing, such as confidence and trust. In Figure 11, the boy is photographed behind a screen of water thrown from the right, concealing his facial expression.

However, his postural stance provokes confidence through maintaining an upright body and head, arms held outwards and upwards, and palms facing the viewer.

The hands speak powerfully in this image with the 'ILY' symbol for love, formed by displaying palms vertically, and extending the thumb, index finger and little finger, while the middle and ring finger touch the palm of each hand. The overall mood of the subject is one of confidence and trust amidst the onslaught of water. Again, the dark background creates an interesting mood, as the students expressed: 'The studio environment lets you see people in special lighting'.

Figure 12 also invokes a sense of happiness, security, confidence and self-assurance of the child in the centre. The photographer provided instructions: 'Run, go further back, put your hand in the air, strike a pose, got it!' The girl in the centre holds an upright posture, with her left arm outstretched and a clenched fist, communicating victory and confidence. There is also a potential, invoked meaning of positive judgment conveyed through the recognisable social act of the peers chairing the girl off the field. Judgment is a region of meaning concerning attitudes toward people and to their behaviour --how they measure up (Martin & White, 2005). In the interviews, the students confirmed that positioning of subjects was important: 'You could have a close person, and one goes further back, so that they look smaller ... We would say things like, 'Move back a bit', so you change their size'. The girl in the centre, although the shortest, becomes the tallest object and therefore, more visually salient. The girls who prop her up have their bodies turned to face the girl in the centre, and the vertical alignment of their bodies form an apex that peaks at the top of the girl's head in the centre. The expressions on the girls' faces--broad smiles--are potentially aligned with both confidence, but also with positive judgment and affect.

Digital imagery in the students' everyday lives Many of the students indicated that they use digital devices to take images at home, noticing a continuity between the digital imagery workshops, and their out-of-school photography practices. These included 'taking family photos', 'photos of sisters, brothers, cousins', and 'Christmas and birthday photos'. As one student confessed: 'We know every pose. We do poses at home--selfies'. Some of the boys described what kinds of devices they used: 'We use a camera or iPad at home'; 'We use a camera like the ones we used today, and sometimes an iPhone.' Some of the girls continued: 'We like applying special effects, and putting frames around them using apps'. These data confirm existing research of the 'digital turn' in literacy studies, and the centrality of the visual image in contemporary digital lives (Mills, 2010a; Mills 2015).

At the same time, the students were amazed at the professional quality of the images that they were able produce in the workshops with access to new knowledge and professional photography and studio equipment. Mills (2010b) has argued that 'teachers of English need to do more than incorporate the out-of-school literacy practices, interests, and predilections of youth . Teachers have a key responsibility to scaffold multimodal literacies and model new technical proficiencies. They can lead students to engage in sophisticated, mature forms of communication that are unattainable for many students without intervention and expert guidance (pp. 41-42).'

Discussion

This research examined how Indigenous students express affect through still digital imagery, applying the appraisal framework to the analysis of affective meanings in the student's multimodal texts. This is significant because one of the keys to enhancing understandings about one's own emotions and those of others is the ability to identify emotions through various modalities (Durlak et al., 2015; Southam-Gerow, 2013). In part, this requires a combination of an emotional vocabulary and the ability to recognise bodily (and facial) expressions, including contextual information from the environment. The findings are also significant in terms our adaption of the appraisal framework originally intended for spoken and written language, to interpret the multimodal and visual representation of affective meanings.

The study has documented some of the complex decisions the students made to communicate affective meanings through digital image composition. Students were discerning about the elements of their multimodal representations of emotions, such as facial expression, gesture, posture, background, lighting, and framing. When initial compositions were not adequate, the students revised their settings and their participants' expressions and postures until they had achieved their purposes. At times, the students' approach to emotional representations showed a sophisticated understanding of affective meanings. Rather than lacking affective competency, as some have claimed of children (cf. Durlak et al., 2011), the young students in our study demonstrated an awareness of many different emotions and ways to communicate this awareness, both visually and through their accompanying words.

In light of the findings, we propose that having students compose still, digital images to represent affect is a useful approach to developing emotional knowledge and multimodal language. Although existing interventions to scaffold student social-emotional learning commonly use images of people (i.e., not the students) expressing emotions, or role-play style activities simulating emotional experiences (see Durlak et al., 2015; Southam-Gerow, 2013), there is scope for integrating the approach we report using digital still image composition. An advantage of this approach is that the students themselves are represented in the images, potentially increasing identification with the emotions represented. The final images become a permanent record for future reference and further teaching (e.g. identifying emotions from other's images; grading intensities of emotions by images).

It is important to contextualise the social milieu in which these student images were composed. While the meanings of gestures and body language are often backgrounded (Goffman, 1974), posing for photographs constitutes a form of gestural communication that is intended to place participants explicitly in the field of vision of the intended viewer (Jaworski & Thurlow, 2014). The social context of the educational activities set up in this study provides an interesting case because the children as authors or photographers chose to arrange the compositions to convey particular expression of affect. As one student explained 'You have to pretend', while another explained, 'You're using your imagination'.

Photographs taken in such contexts may have features of both inscription or invocation (White, 2005), and this point is also true of affect in written language. Affect can be inscribed (e.g. She was happy) or invoked (e.g. She smiled), but using different signs than visual images. For example, Figure 12 discussed above provokes a number of possible affect meanings, from happiness to security, while simultaneously invoking positive judgement of social esteem. It is important to appreciate here, as has been argued by Scollon and Scollon (2011), that it is wrong to assume that all modes of communication are structurally analogous to language and are patterned with analogous grammars. There is a certain ambiguity associated with the range of affect meanings intended by the participants, and the positioning of the viewer in images.

Asking students in this study to create scenarios and poses to represent their own or others' emotional states offers scope for teachers and researchers to provide instruction on the labels accepted more widely in the literature. It invites them to develop a language of emotions and to think about how they are expressed through gestural, facial, and vocal modalities of emotion. While the metalanguage of Martin and White's (2005) appraisal framework was used to analyse the images, it raises the vital question: Can students be taught to comprehend and apply the appraisal framework to the design and analysis of emotional expression in digital imagery? This is a potentially important area for future research. As the workshop director explained, 'We spend a lot of focus in schools on the grammars of teaching spoken and written language, but ... the visual communication should be valued at the same level'.

We note that Indigenous cultures are dynamic and changing, rather than regarded as static. Indigenous young people report being engaged in social media and other image sharing in their out-of-school lives as other groups in their generation (Kral, 2011). Care has been given in this analysis to respect Indigenous identities by avoiding interpretations of ethnicity in ways that would depict these young people as culturally 'different' or non-White, or in ways that might project any fascination with, or false characterisation of, Indigenous young people as 'other' (Picart, 2013). The teacher's vision for the project was that: 'This [digital imagery work] is a way of breaking new ground, of making this generation the generation that disconnects from the trauma [of past generations], that takes their identity and their culture with their hands and can be proud'.

Conclusion

This paper has documented Indigenous students' representation of affect through digital photography, with a detailed analysis of the visual techniques for emotional expression. In particular, due to the vast collection of images, we have been able to systematically elaborate visual meanings across six major categories of meanings in the attitude network--happiness, unhappiness, satisfaction, dissatisfaction, security and insecurity (Martin & White, 2005). The attitude network remains the most cited model for analysing affect in systemic functional linguistics.

In extending these meanings to the visual mode, we have also mapped the complex sets of visual narrative processes that were used by students to express affect. These include visual elements, such as participant gaze, facial expression, posture, framing and social distance, actor-goal relations and vectors, frontal and oblique angles, backgrounds, props, lighting, shadows, and colour (Kress and van Leeuwen, 2006). This is a unique contribution to theory, as we have demonstrated how multiple affective meanings can be provoked or invoked in multimodal texts. This analysis is potentially useful for reading and teaching the representation of positive and negative affect across six main affect groups of the attitude network.

Despite scant attention to emotions in educational research in the past, the last fifteen years has seen a distinct recognition of their importance in numerous dimensions of educational experiences. Studies have now confirmed that emotions impact upon engagement, performance, and personality development, and have the capacity to shape efficacy within educational settings and in broader societal contexts (Pekrun & Linnenbrink-Garcia, 2014). Notwithstanding these recent advances, Pekrun and Linnenbrink-Garcia (2014) have argued that there is more we need to know about emotions including better theorisation, how to study emotions in educational settings effectively, and which emotional phenomena should be central to such research.

This research has provided an important example or model for teachers using the appraisal framework, to begin to address the emotional work of lifelong learning due to the inevitable influence and frequent encounters with emotions in educational situations. Further research is needed to elaborate the visual expression of judgement and appreciation, which are the axes of meaning in the attitude network concerning judgement of social esteem and social sanction, and evaluations or appreciation of things that are composed or of natural phenomena, including reactions, composition and valuation (Martin & White, 2005).

A key to learning about emotional expression that was illustrated in our analysis is the centrality of the facial and gestural components that make emotions accessible for others during interactions (cf. Ekman 2007). Although we all experience these changes in our bodies and externalise or communicate our emotions through facial, gestural and prosodic features of our voices, many of these aspects of emotional experience can occur without students' conscious awareness of these changes taking place (Turner 2007). We cannot assume that emotions and appropriate emotional expression will develop without guidance from teachers, particularly across a variety of modes of expression.

Further research is needed to understand how students can represent judgment and appreciation across multiple modes (e.g. visual, gestural, movement etc.) and media, such as through written narratives, moving images, film, graphic novels, video games, and animations. Increasingly, digital imagery practices feature prominently in the lives of young people who have ubiquitous access to the internet via mobile devices, such as phones and tablets, so it is timely for educators to take an active role in guiding skilful and appropriate use of digital imagery.

Research on the appraisal framework applied to images is still emergent, particularly in relation to attitudinal meanings of appreciation and judgment (Economou, 2008; Unsworth, 2015; White, 2005). Teachers and students can build up broadened repertoires of multimodal resources to communicate gradations of attitudinal meanings in expert ways, particularly at a time when the production of multimodal texts is now part of everyday social practice in the digital times (Mills, 2015). The changes to the digital communications environment and the very nature of image construction, modification and sharing points to the need for schooling to seriously address students' repertoires of emotional language expression.

Acknowledgement

Professor Kathy A. Mills is a recipient of an Australian Research Council DECRA Fellowship (DE140100047) funded by the Australian Government. The views expressed herein are those of the authors and are not necessarily those of the Australian Government or the Australian Research Council.

References

Australian Curriculum Assessment and Reporting Authority. (2014). Australian Curriculum: English (Version 6.0). Canberra, ACT: Australian Government. Retrieved February, 27, 2017 from: http://www.australiancurriculum.edu.au/english

Australian Institute of Aboriginal and Torres Strait Islander Studies. (2012). Guidelines for Ethical Research in Australian Indigenous Studies. Canberra, ACT: Australian Institute of Aboriginal and Torres Strait Islander Studies. Retrieved February, 27, 2017 from: http://aiatsis.gov. au/sites/default/files/docs/research-and-guides/ethics/ GERAIS.pdf

Banninger-Huber, E. (1992). Prototypical affective microsequences in psychotherapeutic interaction. Psychotherapy Research, 2 (4), 291-306.

Durlak, J.A., Domitorvich, C.E., Weissberg, R.P. & Gullotta, T.P. (2015)(Eds.). Handbook of social and emotional learning: Research and practice. New York: The Guilford Press.

Durlak, J.A., Dymnicki, A.B., Taylor, R.D., Weissberg, R.P. & Schellinger, K.B. (2011). The impact of enhancing students' social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432.

Economou, D. (2008). Pulling readers in: News photos in Greek and Australian broadsheets. In Communicating Conflict: Multilingual Case Studies of the News Media, edited by Peter R.R. White and Elizabeth A. Thomson, 253-280. London: Continuum.

Ekman, P. (2007). Emotions revealed. New York, NY: St Martin's Griffin.

Exley, B. & Mills, K.A. (2012). Parsing the Australian English curriculum: Grammar, multimodality and cross-cultural texts. Australian Journal of Language and Literacy, 35 (1): 192-205.

Goffman, E. (1974). Frame analysis: An essay on the organisation of experience. New York, NY: Harper and Row.

Halliday, M. (1994). An introduction to functional grammar (2nd ed.). London, UK: Edward Arnold.

Hood, S. (2004). Appraisal research: Taking a stance in academic writing. Sydney, Australia: University of Technology Sydney.

Hudson, D. & Whalmsley, J. (2005). The English Patient: English grammar and teaching in the twentieth century. Journal of Linguistics, 41 (3), 593-622.

Jaworski, A. & Thurlow, C. (2014). Gesture and movement in tourist spaces. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (pp. 365-374). London, UK: Routledge.

Kral, I. (2011). Youth media as cultural practice: Remote Indigenous youth speaking out loud. Australian Aboriginal Studies, 4 (1): 4-16.

Kress, G. & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.). London, UK: Routledge.

Kress, G. (2014). What Is Mode? In C. Jewitt (Ed.), The Routledge Handbook of Multimodal Analysis (2nd ed., pp. 54-67). London, UK: Routledge.

Macken-Horarik, M. (2009). Navigational metalanguages for new territory in English: The potential of grammatics. English Teaching: Practice and Critique, 8 (3), 55-69.

Macken-Horarik, M. & Martin, J.R. (2003). Negotiating heteroglossia: Social perspective on evaluation. Text, 23 (2), 285-312.

Martin, J.R. & White, P.R.R. (2005). The language of evaluation: Appraisal in English. New York, NY: Palgrave, Macmillan.

Matthiessen, C.M. (1995). Lexicogrammatical cartography: English systems. Tokyo, Japan: International Language Sciences Publishers.

Martin, K.L. (2003). Ways of knowing, being, and doing: A theoretical framework and methods for Indigenous and Indigenist research. Voicing dissent new talents 21st century: Next generation Australian studies. Journal of Australian Studies, 76, 203-214.

Mavers, D. (2014). Image in the multimodal ensemble: children's drawing. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (2nd ed., pp. 431-439). New York, NY: Routledge.

Mills, K.A. (2010a). A review of the digital turn in the New Literacy Studies. Review of Educational Research, 80 (2), 246-271.

Mills, K.A. (2010b). Shrek meets Vygotsky: Rethinking adolescents' multimodal literacy practices in schools. Journal of Adolescent and Adult Literacy 54 (1), 35-45.

Mills, K.A. (2015). Doing digital composition on the social web: Knowledge processes in literacy learning. In B. Cope & M. Kalantzis (Eds.), A pedagogy of multiliteracies: Learning by design (pp. 172-185). New York, NY: Pagrave, Macmillan.

Mills, K.A., Comber, B. & Kelly, P. (2013). Sensing place: Embodiment, sensoriality, kinesis, and children behind the camera. English Teaching: Practice and Critique 12 (2), 11-27.

Mills, K.A., Sunderland, N. and Davis-Warra, J. (2013). Yarning circles in the literacy classroom. The Reading Teacher, 67(4), 285-289.

Mills, K.A., Unsworth, L., Bellocchi, A., Park, J. & Ritchie, S.M. (2014). Children's multimodal appraisal of places: Walking with the camera. Australian Journal of Language and Literacy, 37(3), 171-181.

Mills, KA., Davis-Warra, J., Sewell, M. & Anderson. A. (2016). Indigenous ways with literacies: Transgenerational, multimodal, placed, and collective. Language and Education, 30(1), 1-21.

Ortony, A. & Turner, T.J. (1990). What's basic about basic emotions. Psychological Review, 97(3), 315-331.

Painter, C., Martin, J.R. & Unsworth, L. (2013). Reading visual narratives: Image analysis of children's picture books. Sheffield, UK: Equinox Publishing

Pekrun, R. & Linnenbrink-Garcia, L. (2014). International handbook of emotions in education. New York, US: Routledge.

Picart, C.J. (2013). Critical race theory and copyright in American dance: Whiteness as status. New York, NY: Palgrave Macmillan.

Pink, S. (2007). Doing visual ethnography: Images, media and representation in research (2nd ed.). London, UK: SAGE.

Poynton, C. (1990). The privileging of representation and the marginalising of the interpersonal: a metaphor (and more) for contemporary gender relations. In T. Threadgold & A. Cranny-Francis (Eds.), Feminine/masculine and representation (pp. 231-255). Sydney, Australia: Allen & Unwin.

Rothery, J. & Stenglin, M. (2000). Interpreting literature: the role of appraisal. In L. Unsworth (Ed.), Researching language in schools and communities: Functional linguistic perspectives (pp. 222-244). London, UK: Cassell.

Scollon, R. & Scollon, S.W. (2011). Multimodality and language: A retrospective and prospective view. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (2nd ed., pp. 170-180). New York, NY: Routledge.

Silverman, D. 2014. Interpreting qualitative data. (5th ed.). London, UK: Sage.

Southam-Gerow, M.A. (2013). Emotion regulation in children and adolescents: A practitioner's guide. New York: The Guilford Press.

Stoecker, R. (2005). Research methods for community change: A project-based approach. Thousand Oaks, CA: SAGE.

Turner, J.H. (2007). Human emotions: A sociological theory. London, UK: Routledge.

Unsworth, L. (2001). Teaching multiliteracies across the curriculum: Changing contexts of text and image in classroom practice. Buckingham, UK: Open University Press.

Unsworth, L. (2006a). E-Literature for children: Enhancing digital literacy learning. London, UK: Routledge.

Unsworth, L. (2006b). Towards a metalanguage for multiliteracies education: Describing the meaningmaking resources of language-image interaction. English Teaching: Practice and Critique, 5 (5), 55-76.

Unsworth, L. (2015). Persuasive narratives: Evaluative images in picture books and animated movies. Visual Communication, 14 (1), 73-96.

White, P.R.R. (2005). The attitudinal work of news journalism images--a search for visual and verbal analogues. In Prefazione agli Occasional Papers del CeSLiC (pp. 5-44). Italy: Universita di Bologna.

Kathy A. Mills is a Professor of literacies and digital culture at the Learning Sciences Institute Australia, Australian Catholic University, Brisbane, Australia. Her research publications and nationally competitive fellowships and grants, have made an international contribution to knowledge of children multimodal literacies in digital contexts. Kathy's most recent book, Literacy Theories for the Digital Age, received the 2017 Edward B. Fry Book Award of the Literacy Research Association, Florida.

Alberto Bellocchi is a Senior Research Fellow with an ongoing position at the Queensland University of Technology, Brisbane, Australia. His current research programme focuses on the interplay between social bonds, emotions and learning. More broadly, his research addresses teaching and learning within university preservice teacher education classes, high school science classrooms, and teacher and student emotion management. Alberto's work contributes to micro-sociological perspectives of learning and the sociology of emotions.

Roger Patulny is a Senior Lecturer at the University of Wollongong, Australia. He is the co-founder and current co-convener of the Contemporary Emotions Research Network; co-founder and former convener of The Australian Sociological Association thematic group on the Sociology of Emotions and Affect; has been awarded several ARC grants (DP098810; LP140100033); and edited three special editions on emotions for the Australian

Journal of Social Issues and Emotion Review.

Jane Dooley is a proud Kamilaroi woman and PhD candidate at Deakin University, Geelong, Victoria, with degrees in contemporary arts (visual art) and education. Jane's thesis examines Indigenous knowledge in academic processes, providing frameworks for the Indigenous acquisition of knowledge, values and skills in culturally inclusive learning environments. Her teaching practice draws on students' funds of cultural knowledge to develop positive Indigenous identities.

Caption: Figure 1

Caption: Figure 2

Caption: Figure 3

Caption: Figure 4

Caption: Figure 5

Caption: Figure 6

Caption: Figure 7

Caption: Figure 8

Caption: Figure 9

Caption: Figure 10

Caption: Figure 11

Caption: Figure 12