The effect of different question types during shared book reading on children’s narrative comprehension
Dublin Core
Title
The effect of different question types during shared book reading on children’s narrative comprehension
Creator
Nicola Pooley
Date
2010
Description
This study investigated the effect of different question types on narrative comprehension in young children. Forty one five year olds participated in this study. One group (N=14) received three sessions of shared storybook reading in which they practised answering questions about literal information in the story, during the course of the storybook reading. A second group (N=13) practiced answering questions about information that had to be inferred. A third group of controls (N=14), did not receive any intervention. All groups completed two comprehension assessments before and after the intervention: one was a measure of general listening comprehension, the other included measures of both literal and inferential comprehension. Children’s engagement during the storybook reading was also assessed. Contrary to predictions, neither intervention benefitted post-test comprehension significantly. In addition engagement levels did not change over the course of the study. However, a consistent pattern was found for each comprehension measure: the group who received practice with answering inferential questions made the greatest gains. Implications for early literacy experiences are discussed.
Subject
reading comprehension
Source
Design
The study was an intervention design with three phases: a pre-test, training phase and post-test. There were three groups: two experimental groups who participated in all phases and a control group who only completed the pre and post tests. The design is shown in Table One. In the pre and post test sessions, participants completed a general measure of listening comprehension (adapted from the Neale Analysis of Reading Ability II, Neale, 1997) and a bespoke measure of listening comprehension with questions to tap literal and inferential comprehension. Participants were assigned to groups on the basis of their scores in the pre-test so that the three groups (two intervention and one control) did not differ in their performance on these measures (see Table Five). Children in the intervention conditions listened to three stories in separate sessions and either received practice at answering literal or inferential questions throughout the stories. In the post test all children were again assessed on alternate forms of the same measures used in the pre-test.
Table 1. Intervention design used.
Group
Pre Test
1
Training
2
3
Post test
Control
General + Bespoke Listening Comprehension.
x
x
x
General + Bespoke Listening Comprehension.
Literal
General + Bespoke Listening Comprehension.
Literal
Literal
Literal
General + Bespoke Listening Comprehension.
Inferential
General + Bespoke Listening Comprehension.
Inferential
Inferential
Inferential
General + Bespoke Listening Comprehension.
Materials: Pre and Post test.
General Measure of Listening Comprehension. The following stories taken from the Neale Analysis of Reading Ability (NARA, Neale, 1997) were read to each participant in either the pre or post test: Toys, Tree house, Lost and Found, Road Safety. Toys or Lost and Found were used as practice tasks at the beginning of the pre/post test to help develop rapport. The stories were chosen so that the level of difficulty was consistent pre and post test. Comprehension questions that went with each story were asked at the end of the story to obtain the general measure of listening comprehension score. Table Two shows an example of a story and some of the questions given.
Table 2. Example of general listening comprehension story used.
General Comprehension Example
Sample of Questions asked
My friend and I made a tree house. We like to hide in it. We climb up the rope and pull it up after us. Then no-one knows where we are. We play space-ships. At tea time we slide down fast and we are always first for tea.
What would you say was the best name for that story?
Who built the tree house?
How did the children always manage to be first for tea?
Bespoke Measure of Listening Comprehension. The stories used in this study were a series of books about a dog named ‘Harry’ written by Gene Zion. These stories were chosen as they were first published between 1956 and 1965 and so were suitable for this age group but the children were not likely to be familiar with them. The pictures from the stories were scanned then printed on A4 sheets and laminated to make a set of wordless picture books. The original text was retained for each story, however, small sections of some of the stories were omitted to try and keep each story the same length.
Two different types of questions were used in the bespoke measure of listening comprehension: literal and inferential questions. In the pre and post tests each child received eight literal questions and eight inferential questions after each story reading. The literal questions required the participants to recall facts from the text. The inferential questions tapped children’s ability to make inferences about information that was not stated explicitly in the text. These questions were designed to address: causality (why an event happened), emotions (how a character was feeling) and future events (what might happen next in the story). The Inferential questions in the pre and post test, however, consisted of four emotion and four causality questions as prediction questions could not be used at the end of the story. Table Three gives examples of literal and inferential questions used.
Table 3. Examples of literal and inferential questions used.
Extract
Question
1. Harry was a white dog with black spots who liked everything except having a bath. So one day when he heard the water running in the tub he took the scrubbing brush and buried it in the back garden.
Literal: What did Harry bury in the back garden?
Forced choice: the scrubbing brush/a sponge.
Causal Inferential: Why do you think Harry buried the scrubbing brush in the back garden?
Forced choice: Because the family told him to/Because he did not want a bath.
2. That night Harry slept in the dog house – again.
Literal Question: Where was Harry made to sleep again? In the Kitchen/in the dog house.
Emotion Inference Question: How do you think Harry felt about sleeping in the dog house?
Forced choice: happy/sad.
3. (After a sequence of events that lead to Harry being covered in seaweed and thinking the hot dog man was calling his name.) Harry still thought the man was calling his name. He barked and jumped with joy. He jumped so much that suddenly…
Literal Question: (before - he jumped so much…) What was the hot dog man really shouting? Hurry/Harry
Prediction Inferential question*: What do you think happened next?
Forced choice: Everyone ran away/ the seaweed fell off him.
*Please note. These were only used during the intervention sessions.
Materials: Intervention
Three of the stories were used for the intervention sessions. Scripts were produced that incorporated the questions for the intervention sessions during the stories. In the inferential intervention group there were four of each question type: causal, emotion and prediction. The inferential and literal questions were always placed at the same point in the story.
Procedure
Phase One: Pre-test. Children in all groups completed the general listening comprehension measure and the bespoke measure of literal and inferential comprehension. Each child was tested individually in a quiet space away from the classroom. The pre-test session was audio and video recorded. The video recorder was set in front of the participant to capture their direction of eye gaze. The experimenter explained the task to the child and obtained verbal consent. In the pre-test the experimenter asked the child if they had heard any stories about Harry the dog while showing them the front cover. One child reported recognising the story, but could not remember any details.
General Listening Comprehension Measure. Each participant was read two stories, the first acted as a practice task to help develop rapport. Immediately after each story the children were asked the comprehension questions for that story. If a child could not answer a question then the experimenter offered the correct response and moved onto the next question. If the child gave the incorrect answer then the experimenter did not highlight that this was incorrect but simply moved onto the next question. The decision to respond to answers in this way was based on the pilot of the procedure. This age group seemed to become easily disengaged if they supplied no answer on a number of occasions or incorrect answers and it was felt that this way of responding helped to maintain their confidence and interest in the task. Responses were scored as correct or incorrect. Acceptable answers were provided in the NARA manual.
Bespoke Listening Comprehension Measure. After the assessment of general listening comprehension each child completed the bespoke listening comprehension task. The experimenter read out the story whilst the child followed the pictures in a wordless picture book version. At the end of the story sixteen questions were asked: 8 literal and 8 inferential, of which four were causal and four were emotion related. If the child could not answer a question or gave the wrong answer then s/he was offered a forced choice of two possible answers (examples in Table Three). One option was the correct target answer and one was incorrect. The forced choices were included in the pre/post test as they were also used during the intervention; however, answers based on a forced choice were not included in the analysis. In the pre and post-test if the child chose the correct response then the experimenter agreed with the child and moved onto the next question. If the child selected the incorrect option, the experimenter also continued with the next question. The decision was taken not to correct the child at this stage as if the child was still getting the answer incorrect despite assistance then giving them the correct answer may change the representation they had created of the story and also have an effect on their confidence as mentioned earlier. The forced choices were alternated so that the correct answer occurred equally in first and second positions across items. When scoring the responses if the child gave the correct answer unaided (i.e. without the forced choice option) then they were given one point. All other responses were scored zero.
Phase two: Intervention (Intervention groups only). The intervention sessions took place the week after the selection phase, on three consecutive days. On each day, each child in the intervention groups was tested individually in a quiet space away from the classroom and the session was audio-recorded. Different stories were used in each session. As the stories were read to the participant they were asked questions (either literal or inferential depending on group assignment) about the story content. Children in the control group were not read to by the experimenter during this phase.
Literal Questions Intervention Group. Children in this condition were read one story in each of the three intervention sessions and asked twelve questions that assessed their understanding of explicit details in the story, e.g., ‘What did the lady next door sing louder than?’ The questions were positioned throughout the text and related directly to information that had just been given in the story. If the children gave no response or an incorrect response they were offered the forced choice. If a child still gave an incorrect answer after being given the forced choices then the experimenter corrected them and offered the correct answer. This was to try and ensure that the children were building accurate representations as they listened to the stories.
Inferential Questions Intervention Group. The same stories and question-response technique were used as outlined in the literal questions condition. Questions were also placed at the same position in the text, however, children in this condition were asked twelve inferential questions throughout each story that required them to think beyond the facts present in the text. In each story there were four causal inferential questions, e.g., ‘Why were Harry’s ears hurting?’ four prediction questions, e.g., ‘What do you think Harry did next?’ and four questions assessing understanding of the emotions of the characters. e.g., ‘How do you think Harry felt when the old lady told him to go away?’
Phase three: Post-test. This session took place between five and seven days after the final intervention session and followed the same format as the pre-test. Children in all three groups completed the general listening comprehension story and the bespoke listening comprehension story with literal and inferential questions asked at the end of the story.
Measure of Engagement. The video recordings from the pre and post-test were analysed for the children’s level of engagement. This was only based on the child’s behaviour during the reading of the bespoke listening comprehension story. The coding scheme used for this analysis is shown in Table Four. A second rater scored 20% of the pre-test videos. There was 100% agreement between raters.
Table 4. Coding scheme used to analyse level of engagement while listening to the bespoke story.
Code
Description of Behaviour
1
Limited Engagement. The child appears off-task and makes a large number of unrelated comments or is distracted and looking away for a large part of the story reading.
2
Engaged- Quiet. The child looks at the pictures and listens well throughout the story but does not make any independent comments.
3
Engaged – Interactive. The child looks at the pictures and listens well throughout the story. They also make independent comments relating the events in the story to their lives/elaborate on the text/ ask questions about the text.
Group Assignment.
Scores on the pre-test measures were used to assign the children to groups to ensure an equal range of scores in each. One-way Analysis of Variance was carried out on the general comprehension scores, literal and inferential scores. All F<1.0 and all p>0.1. In addition, where possible, an equal number of boys and girls were assigned to each group. Table Five shows the ages, number of boys and girls and pre-test scores for each group.
Table 5. Distribution of gender, age and pre-test scores across groups.
Variable
Control
Literal
Inferential
Gender Male
Female
8
6
7
7
7
6
Age (years; months)
5;5
5;5
5;4
General Comprehension (proportion)
0.43
0.46
0.46
Bespoke Literal (max=8)
3.79
3.79
4.15
Bespoke Inferential (max=8)
5.0
4.50
4.77
The study was an intervention design with three phases: a pre-test, training phase and post-test. There were three groups: two experimental groups who participated in all phases and a control group who only completed the pre and post tests. The design is shown in Table One. In the pre and post test sessions, participants completed a general measure of listening comprehension (adapted from the Neale Analysis of Reading Ability II, Neale, 1997) and a bespoke measure of listening comprehension with questions to tap literal and inferential comprehension. Participants were assigned to groups on the basis of their scores in the pre-test so that the three groups (two intervention and one control) did not differ in their performance on these measures (see Table Five). Children in the intervention conditions listened to three stories in separate sessions and either received practice at answering literal or inferential questions throughout the stories. In the post test all children were again assessed on alternate forms of the same measures used in the pre-test.
Table 1. Intervention design used.
Group
Pre Test
1
Training
2
3
Post test
Control
General + Bespoke Listening Comprehension.
x
x
x
General + Bespoke Listening Comprehension.
Literal
General + Bespoke Listening Comprehension.
Literal
Literal
Literal
General + Bespoke Listening Comprehension.
Inferential
General + Bespoke Listening Comprehension.
Inferential
Inferential
Inferential
General + Bespoke Listening Comprehension.
Materials: Pre and Post test.
General Measure of Listening Comprehension. The following stories taken from the Neale Analysis of Reading Ability (NARA, Neale, 1997) were read to each participant in either the pre or post test: Toys, Tree house, Lost and Found, Road Safety. Toys or Lost and Found were used as practice tasks at the beginning of the pre/post test to help develop rapport. The stories were chosen so that the level of difficulty was consistent pre and post test. Comprehension questions that went with each story were asked at the end of the story to obtain the general measure of listening comprehension score. Table Two shows an example of a story and some of the questions given.
Table 2. Example of general listening comprehension story used.
General Comprehension Example
Sample of Questions asked
My friend and I made a tree house. We like to hide in it. We climb up the rope and pull it up after us. Then no-one knows where we are. We play space-ships. At tea time we slide down fast and we are always first for tea.
What would you say was the best name for that story?
Who built the tree house?
How did the children always manage to be first for tea?
Bespoke Measure of Listening Comprehension. The stories used in this study were a series of books about a dog named ‘Harry’ written by Gene Zion. These stories were chosen as they were first published between 1956 and 1965 and so were suitable for this age group but the children were not likely to be familiar with them. The pictures from the stories were scanned then printed on A4 sheets and laminated to make a set of wordless picture books. The original text was retained for each story, however, small sections of some of the stories were omitted to try and keep each story the same length.
Two different types of questions were used in the bespoke measure of listening comprehension: literal and inferential questions. In the pre and post tests each child received eight literal questions and eight inferential questions after each story reading. The literal questions required the participants to recall facts from the text. The inferential questions tapped children’s ability to make inferences about information that was not stated explicitly in the text. These questions were designed to address: causality (why an event happened), emotions (how a character was feeling) and future events (what might happen next in the story). The Inferential questions in the pre and post test, however, consisted of four emotion and four causality questions as prediction questions could not be used at the end of the story. Table Three gives examples of literal and inferential questions used.
Table 3. Examples of literal and inferential questions used.
Extract
Question
1. Harry was a white dog with black spots who liked everything except having a bath. So one day when he heard the water running in the tub he took the scrubbing brush and buried it in the back garden.
Literal: What did Harry bury in the back garden?
Forced choice: the scrubbing brush/a sponge.
Causal Inferential: Why do you think Harry buried the scrubbing brush in the back garden?
Forced choice: Because the family told him to/Because he did not want a bath.
2. That night Harry slept in the dog house – again.
Literal Question: Where was Harry made to sleep again? In the Kitchen/in the dog house.
Emotion Inference Question: How do you think Harry felt about sleeping in the dog house?
Forced choice: happy/sad.
3. (After a sequence of events that lead to Harry being covered in seaweed and thinking the hot dog man was calling his name.) Harry still thought the man was calling his name. He barked and jumped with joy. He jumped so much that suddenly…
Literal Question: (before - he jumped so much…) What was the hot dog man really shouting? Hurry/Harry
Prediction Inferential question*: What do you think happened next?
Forced choice: Everyone ran away/ the seaweed fell off him.
*Please note. These were only used during the intervention sessions.
Materials: Intervention
Three of the stories were used for the intervention sessions. Scripts were produced that incorporated the questions for the intervention sessions during the stories. In the inferential intervention group there were four of each question type: causal, emotion and prediction. The inferential and literal questions were always placed at the same point in the story.
Procedure
Phase One: Pre-test. Children in all groups completed the general listening comprehension measure and the bespoke measure of literal and inferential comprehension. Each child was tested individually in a quiet space away from the classroom. The pre-test session was audio and video recorded. The video recorder was set in front of the participant to capture their direction of eye gaze. The experimenter explained the task to the child and obtained verbal consent. In the pre-test the experimenter asked the child if they had heard any stories about Harry the dog while showing them the front cover. One child reported recognising the story, but could not remember any details.
General Listening Comprehension Measure. Each participant was read two stories, the first acted as a practice task to help develop rapport. Immediately after each story the children were asked the comprehension questions for that story. If a child could not answer a question then the experimenter offered the correct response and moved onto the next question. If the child gave the incorrect answer then the experimenter did not highlight that this was incorrect but simply moved onto the next question. The decision to respond to answers in this way was based on the pilot of the procedure. This age group seemed to become easily disengaged if they supplied no answer on a number of occasions or incorrect answers and it was felt that this way of responding helped to maintain their confidence and interest in the task. Responses were scored as correct or incorrect. Acceptable answers were provided in the NARA manual.
Bespoke Listening Comprehension Measure. After the assessment of general listening comprehension each child completed the bespoke listening comprehension task. The experimenter read out the story whilst the child followed the pictures in a wordless picture book version. At the end of the story sixteen questions were asked: 8 literal and 8 inferential, of which four were causal and four were emotion related. If the child could not answer a question or gave the wrong answer then s/he was offered a forced choice of two possible answers (examples in Table Three). One option was the correct target answer and one was incorrect. The forced choices were included in the pre/post test as they were also used during the intervention; however, answers based on a forced choice were not included in the analysis. In the pre and post-test if the child chose the correct response then the experimenter agreed with the child and moved onto the next question. If the child selected the incorrect option, the experimenter also continued with the next question. The decision was taken not to correct the child at this stage as if the child was still getting the answer incorrect despite assistance then giving them the correct answer may change the representation they had created of the story and also have an effect on their confidence as mentioned earlier. The forced choices were alternated so that the correct answer occurred equally in first and second positions across items. When scoring the responses if the child gave the correct answer unaided (i.e. without the forced choice option) then they were given one point. All other responses were scored zero.
Phase two: Intervention (Intervention groups only). The intervention sessions took place the week after the selection phase, on three consecutive days. On each day, each child in the intervention groups was tested individually in a quiet space away from the classroom and the session was audio-recorded. Different stories were used in each session. As the stories were read to the participant they were asked questions (either literal or inferential depending on group assignment) about the story content. Children in the control group were not read to by the experimenter during this phase.
Literal Questions Intervention Group. Children in this condition were read one story in each of the three intervention sessions and asked twelve questions that assessed their understanding of explicit details in the story, e.g., ‘What did the lady next door sing louder than?’ The questions were positioned throughout the text and related directly to information that had just been given in the story. If the children gave no response or an incorrect response they were offered the forced choice. If a child still gave an incorrect answer after being given the forced choices then the experimenter corrected them and offered the correct answer. This was to try and ensure that the children were building accurate representations as they listened to the stories.
Inferential Questions Intervention Group. The same stories and question-response technique were used as outlined in the literal questions condition. Questions were also placed at the same position in the text, however, children in this condition were asked twelve inferential questions throughout each story that required them to think beyond the facts present in the text. In each story there were four causal inferential questions, e.g., ‘Why were Harry’s ears hurting?’ four prediction questions, e.g., ‘What do you think Harry did next?’ and four questions assessing understanding of the emotions of the characters. e.g., ‘How do you think Harry felt when the old lady told him to go away?’
Phase three: Post-test. This session took place between five and seven days after the final intervention session and followed the same format as the pre-test. Children in all three groups completed the general listening comprehension story and the bespoke listening comprehension story with literal and inferential questions asked at the end of the story.
Measure of Engagement. The video recordings from the pre and post-test were analysed for the children’s level of engagement. This was only based on the child’s behaviour during the reading of the bespoke listening comprehension story. The coding scheme used for this analysis is shown in Table Four. A second rater scored 20% of the pre-test videos. There was 100% agreement between raters.
Table 4. Coding scheme used to analyse level of engagement while listening to the bespoke story.
Code
Description of Behaviour
1
Limited Engagement. The child appears off-task and makes a large number of unrelated comments or is distracted and looking away for a large part of the story reading.
2
Engaged- Quiet. The child looks at the pictures and listens well throughout the story but does not make any independent comments.
3
Engaged – Interactive. The child looks at the pictures and listens well throughout the story. They also make independent comments relating the events in the story to their lives/elaborate on the text/ ask questions about the text.
Group Assignment.
Scores on the pre-test measures were used to assign the children to groups to ensure an equal range of scores in each. One-way Analysis of Variance was carried out on the general comprehension scores, literal and inferential scores. All F<1.0 and all p>0.1. In addition, where possible, an equal number of boys and girls were assigned to each group. Table Five shows the ages, number of boys and girls and pre-test scores for each group.
Table 5. Distribution of gender, age and pre-test scores across groups.
Variable
Control
Literal
Inferential
Gender Male
Female
8
6
7
7
7
6
Age (years; months)
5;5
5;5
5;4
General Comprehension (proportion)
0.43
0.46
0.46
Bespoke Literal (max=8)
3.79
3.79
4.15
Bespoke Inferential (max=8)
5.0
4.50
4.77
Publisher
Lancaster University
Identifier
Pooley2010
Contributor
John Towse
Language
English
Type
Project description
Coverage
LA1 4YF
LUSTRE
Supervisor
Kate Cain
Project Level
MSc
Topic
Cognitive Psychology
Sample Size
43 children (23 boys, 20 girls, mean age 5 years 4 months and range 4 years 9 months to 5 years 9 months) in their first year of primary school
Statistical Analysis Type
Chi-squared
Mcnemar test
Mcnemar test
Files
Collection
Citation
Nicola Pooley, “The effect of different question types during shared book reading on children’s narrative comprehension,” LUSTRE, accessed April 29, 2024, https://www.johnntowse.com/LUSTRE/items/show/22.