Multimodal ensembles for storytelling sessions

During storytelling sessions narrators use (reflectively or spontaneously) communication to involve the children, motivate them to participate actively in ‘multimodal orchestration’ and help them co-create meaning by providing ‘multimodal scaffolding’ through cohesive multimodal ties. To analyze these intermodal relations we focus on ‘multimodal clusters’:

  1. the setting of the event: the physical (or digital) meeting space;
  2. the spatial/visual embodied story: gestures, facial expressions, movements in space, use of images and props;
  3. the oral/aural embodied story: language choices, voice quality and sound effects.

These three clusters co-occur and influence the interaction through co-deployed modes: the narrators need to be aware how spatial choices and layout influence the multimodal orchestration of embodied aspects of storytelling and viceversa. They also need to anticipate potential children’s responses in relation to the three clusters. 

Our research is based on the analysis of videos of storytelling events, and focuses on key aspects of the multimodal clusters and on the collaborative reflection between narrators and educators: verbal aspects (storybook language and/or narrator’s language choices), voice quality (volume, speed of delivery, pitch, type of intonation), soundtrack (music, sound effects), gestures, gaze and use of space.

Discussion of video analysis is complemented by field notes manually taken by the educator who video-recorded the events.

Additional data are the questionnaires administered to the narrators after the experience in the library and the field notes of the educator/observer during the university workshop, and online exchanges with the narrators for the library events.