This study analyzed whether the relationship between academic performance and homework follow-up practices depended on the type of homework follow-up practice used in class. We found that the five types of homework feedback were associated with student academic performance, despite the unbalanced number of teachers in each condition, and the low number of sessions (six sessions). The magnitude of the effects found was small, which may be due to the two previously mentioned limitations. Data from the ancillary analysis collected in the two focus groups run to identify the types of homework follow-up used by EFL teachers in class, and data from the post-research evaluation meeting run with the participating teachers contributed to the discussion of our findings.
Types of EFL teachers' homework follow-up practices and academic performance
As Model C (see Table 4) shows, and once the effect of the pretest was controlled for, the differences among the types of EFL teachers' homework follow-up practices on students' performance were statistically significant, as hypothesized. Moreover, considering the positive value of the coefficients shown in Table 4, the data indicate that students' performance improved from homework follow-up types 1–5 (see also Figure 2), and also that the differences between the five homework follow-up types are not of the same magnitude. In fact, after checking the error rate for comparison family using the FDR procedure, two homogeneous subsets of treatment means were identified. The first subset encompassed homework follow-up types 1 and 2, whereas the second accounted for homework follow-up types 3–5. As shown in Table 5, significant differences were found between adjusted treatments' means for both subsets (homework follow-up types 1 and 2 vs. homework follow-up types 3–5).
What are the commonalities and differences between these two subsets of homework follow-up types that could help explain findings? Homework follow-up types 1 and 2 did not yield differences in school performance. One possible explanation might be that neither of these types of homework follow-up provides specific information about the mistakes made by students; information which could help them improve their learning in a similar way to when EFL teachers provide feedback (Hattie and Timperley, 2007). Besides, as the control for homework completion is low for these two types of homework follow-up practices, students may not have put the appropriate effort to complete the homework. The following statement was shared by most of the teachers that participated in the focus group and may help explain this latter finding: “[in class] I only ask students if they have done their homework. I know that this strategy does not help them correct their mistakes, but if I don't do it, I suspect they will give up doing their homework …” (F2P3).
In homework follow-up type 2, EFL teachers only addressed difficulties mentioned by the students, so some mistakes may have not been addressed and checked by the EFL teachers. This type of practice does not provide feedback to students. As the following quotation from a participant in the focus group revealed: “At the beginning of the class, I specifically ask students if they have any questions about their homework. The truth is, students who struggle to learn seldom ask questions…I guess that they don't do their homework, or they copy the answers from peers during the break, and just asking questions does not help a lot…but they are 28 in class.” (F2P4).
The second group of homework follow-up practices includes types 3–5. Our data indicate that there were no statistically significant differences among these three types of homework follow-up (intra-group comparisons) at posttest performance (see Table 4). Under each of these three conditions (homework follow-up types 3–5) homework contents were checked by the teacher. In these three types of homework follow-up, students experienced opportunities to analyze EFL teachers' explanations and to check their mistakes, which may help explain our findings and those of previous studies (see Cardelle and Corno, 1981; Elawar and Corno, 1985).
According to Cooper's model (1989, 2001), homework follow-up type 5 may be considered the homework feedback practice, because when EFL teachers grade students' assignments and provide individual feedback, students' learning improve. This idea was mentioned by one of our participants: “I collect students' exercise books, not every day, but often enough. That is because I've learned that my students improve whenever I comment upon and grade their homework assignments. I wish I had time to do this regularly…That would be real feedback, that's for sure.” (F1P6).
When analyzing students' conceptions of feedback, Peterson and Irving (2008) concluded that students believe that having their reports graded is a “clearer and more honest” (p. 246) type of feedback. These authors also argued that good grades generate a tangible evidence of students' work for parents, which may also give way to another opportunity for feedback(e.g., praise) delivered by parents and peers (Núñez et al., 2015). It is likely that students see graded homework more worthwhile when compared to other types of homework follow-up practices (e.g., answering questions about homework). This idea supports studies which found a positive association between homework effort and achievement (e.g., Trautwein et al., 2006b, 2009b). Walberg et al. (1985) claimed that graded homework has a powerful effect on learning. However, Trautwein et al. (2009a) alerted that graded homework may have a negative impact whenever experienced as overcontrolling, as “…students may feel tempted to copy from high-achieving classmates to escape negative consequences” (p. 185). These findings (Trautwein et al., 2006b, 2009a,b), aligned with ours, suggest the need to analyze homework feedback in more depth. For example, there are several variables that were not considered in the current research (e.g., number of students per class, number of different grade levels teachers are teaching or number of different classes teachers teach, different level of students' expertise in class, type of content domain; but also career related issues such as frozen salaries, reduced retirement costs), which may help explain our results.
We also noticed that the effect of EFL teachers' homework follow-up practices on performance was affected by students' prior performance, confirming our second hypothesis, but not by the number of homework follow-up sessions (i.e., the number of homework follow-up sessions was only marginally non-significant as a secondary factor, not as the principal factor). A quotation from a teacher under the third condition may help illustrate this finding: “reflecting on my experience under condition 3 [checking homework orally], I can tell that students' prior knowledge was very important for explaining the variations in the efficacy of this strategy. Some of my students, for example, attend language schools and master vocabulary and grammar, but others clearly need extra help. For example, checking homework on the board so that students may copy the answers and study them at home would be very beneficial for many of my students” (M15).
The results of this preliminary study were obtained in a real learning environment and focused on homework follow-up practices commonly used by EFL teachers. We acknowledge the difficulties to set up and run a randomized-group design in a real learning environment (i.e., motivating teachers to participate, training teachers to follow the protocol, control the process). Still, we believe in the importance of collecting data on-task. Plus, we consider that our preliminary findings may help teachers and school administrators to organize school-based teachers' training and educational policies on homework. For example, studies conducted in several countries (e.g., Germany, Hong Kong, Singapore, Israel) reported that checking homework completion is the homework follow-up practice most often used by teachers to keep track of students' homework (e.g., Trautwein et al., 2009a; Kaur, 2011; Zhu and Leung, 2012), and in some cases the only homework follow-up practice used in class (e.g., see Kukliansky et al., 2014). However, this type of homework follow-up does not provide students with appropriate information on how they may improve their learning. Our data show that, when EFL teachers offer individual and specific information to help student progress (e.g., homework correction, graded homework), the impact on school performance is higher, even when this help is provided for only 6 weeks. This main finding, that should be further investigated, may help teachers' in class practices and contribute to foster students' behaviors toward homework and school achievement.
In sum, our findings indicate that the time and effort teachers devote assessing, presenting, and discussing homework with students is worth the effort. In fact, students consider limited feedback an impediment to homework completion, and recognize teacher's feedback as a homework completion facilitator (Bang, 2011).
During the focus group interviews, and consistent with findings by Rosário et al. (2015), several EFL teachers stressed that, despite their positive belief about the efficacy of delivering feedback to students, they do not find the necessary time to provide feedback in class (e.g., comment on homework and grading homework). This is due to, among other reasons, the long list of contents to cover in class and the large number of students per class. Pelletier et al.'s (2002) show that the major constraint perceived by teachers in their job is related to the pressure to follow the school curriculum. Data from the focus group helped understand our findings, and highlights the need for school administrators to become aware of the educational constraints faced daily by EFL teachers at school and to find alternatives to support the use of in class homework follow-up practices. Thus, we believe that teachers, directly, and students, indirectly, would benefit from teacher training on effective homework follow-up practices with a focus on, for example, how to manage the extensive curriculum and time, and learning about different homework follow-up practices, mainly feedback. Some authors (e.g., Elawar and Corno, 1985; Epstein and Van Voorhis, 2012; Núñez et al., 2014; Rosário et al., 2015) have warned about the importance of organizing school-based teacher training with an emphasis on homework (i.e., purposes of homework, homework feedback type, amount of homework assigned, schools homework policies, and written homework feedback practices). With the focus group interviews we learned that several EFL teachers did not differentiate feedback from other homework follow-up practices, such as checking homework completion (e.g., see F2P7 statement, Table 1). EFL teachers termed all the homework follow-up practices used in class as feedback, despite the fact that some of these practices did not deliver useful information to improve the quality of students' homework and promote progress. These data suggest a need to foster opportunities for teachers to reflect upon their in-class instructional practices (e.g., type and purposes of the homework assigned, number and type of questions asked in class) and its impact on the quality of the learning process. For example, school-based teacher training focusing on discussing the various types of homework follow-up practices and their impact on homework quality and academic achievement would enhance teachers' practice and contribute to improve their approaches to teaching (Rosário et al., 2013).
Limitations of the study and future research
This study is a preliminary examination of the relationship between five types of EFL teachers' homework follow-up practices and performance in the EFL class. Therefore, some limitations must be addressed as they may play a role in our findings. First, participating EFL teachers were assigned to one and only one of five homework follow-up conditions, but 19 of them were excluded for not adhering to the protocol. As a result, the number of EFL teachers under each condition was unbalanced, especially in the case of homework follow-up condition number 5. This fact should be considered when analyzing conclusions.
Several reasons may explain why 19 EFL teachers were excluded from our research protocol (i.e., three were laid off, six did not report the work done correctly or submitted the data requested, and ten did not followed the protocol closely). Nevertheless, during the post-research evaluation meeting the EFL teachers addressed this topic which helped understand their motives for not adhering to the protocol. For example: “I'm sorry for abandoning your research, but I couldn't collect and grade homework every week. I have 30 students in class, as you know, and it was impossible for me to spend so many hours grading.” (M7). Our findings suggest that teachers' attitudes toward homework follow-up practices are important, as well as the need to set educational environments that may facilitate their use in class.
We acknowledged the difficulty of carrying out experimental studies in authentic teaching and learning environments. Nevertheless, we decided to address the call by Trautwein et al. (2006b), and investigate teachers' homework practices as ecologically valid as possible in the natural learning environment of teachers and students.
Future studies should find a way to combine an optimal variable control model and an authentic learning environment.
Second, a mixed type of homework follow-up practices (e.g., combining homework control and checking homework on the board) was not considered in the current study as an additional level of the independent variable. In fact, some of the excluded EFL teachers highlighted the benefits of combining various homework follow-up practices, as one EFL teacher remarked: “I was “assigned” condition 5 [collecting and grading homework], but grading and noting homework every week is too demanding, as I have five more sixth grade classes to teach. So, although I am certain that giving individualized feedback is better for my students, I couldn't do it for the six homework assignments as required. In some sessions I checked homework orally.” (M24). Thus, future studies should consider the possibility of analyzing the impact of different combinations of types of homework follow-up practices. Our research focused on sixth grade EFL teachers only. To our knowledge, there are no studies examining the impact of homework follow-up practices in different education levels, but it is plausible that the type and intensity of the homework-follow up practices used by teachers may vary from one educational level to another. Hence, it would be interesting to examine whether our findings may be replicated in other grade levels, or in different subjects. Furthermore, it would be beneficial to conduct this study in other countries in order to explore whether the follow-up practices identified by EFL Portuguese teachers match those found in other teaching and learning cultures.
Third, the fact that in our study the differences found were small suggests the importance of examining the type of homework follow-up used and students' interpretation of teachers' practice. Future studies may analyze the hypothesis that students' behavior toward teacher homework follow-up practices (e.g., how students perceive their teachers' homework follow-up practices; what students do with the homework feedback information given by teachers) mediates the effect of homework on student learning and performance. In fact, the way students benefit from their teachers' homework follow-up practice may help explain the impact of these practices on students' homework performance and academic achievement. Future studies may also consider conducting more large-scale studies (i.e., with optimal sample sizes) using multilevel designs aimed at analyzing how student variables (e.g., cognitive, motivational, and affective) mediate the relationship between teacher homework follow-up type and students' learning and academic performance.
Finally, future research could also consider conducting qualitative research to analyze teachers' conceptions of homework follow-up practices, mainly feedback (Cunha et al., 2015). This information may be very useful to improving homework feedback measures in future quantitative studies. Investigating teachers' conceptions of homework follow-up practices may help identify other homework feedback practices implemented in authentic learning environments. It may also help understand the reasons why teachers use specific types of homework feedback, and explore the constraints daily faced in class when giving homework feedback. As one teacher in the focus group claimed: “Unfortunately, I don't have time to collect and grade homework, because I have too many students and the content that I have to cover each term is vast. So I just check whether all students completed their homework” (F2P1).
byGrant Wiggins, Ed.D, Authentic Education
I received a nice query by email the other day and I thought readers might find my reply helpful.
Within my district there is quite a debate going on about the difference between formative and summative assessments. Specifically, the administration in my school district has developed a common syllabus for our teachers this year which states that 30% of a student’s grade should be made up of formative assessments (homework, journals, etc.) and the remaining 70% should be made up of summative assessments (quizzes, tests, exams). I am on the side that is arguing that a formative assessment should never be graded – it is supposed to be used as a tool to evaluate teaching so that adjustments can be made to instruction.
I write you in hopes of getting a clear and concise definition of a formative assessment. Can they be graded? If so, how are graded and ungraded formative assessments different from one another?
I’ll need to qualify my answers by distinguishing between ‘theory’ and ‘practice’ and in terms of the meanings of the key terms.
In theory, I would define ‘formative’ assessment as “useful feedback with an opportunity to use that feedback” to perform optimally on later summative assessments. A simple example: a pre-season of 4 games in soccer (as well as ongoing “scrimmages”) in which the games don’t ‘count’ is formative assessment to get teams ready for the games that do count in the regular season. So, your comment about “use the information to evaluate teaching” introduces a slightly different purpose for me. Formative assessments provide feedback – for students first, then teachers; that’s their purpose.
A 2nd simple example, more complicated: Suppose as an English teacher I do a pre-test writing on day 1 of my course, the same test is given twice during the year, and a post-test is given on the last day of the course – and it is the SAME prompt each time (say, a writing prompt on the course Essential Question “How well do we know ourselves?”). I will most likely NOT grade the pre-test, but I might grade it while not ‘counting’ the grade, i.e. tell students where they stand (e.g. what teachers do when they use a practice AP test or use the state rubric). But I also might grade and thus ‘count’ the ‘formative’ writing prompts DURING the year as well as the post-test at year’s end IF I felt that students should be ‘ready’ to be tested on their understanding and writing ability thus far.
Another not so simple example: I give you feedback on your college essay before you submit the final ‘real’ essay to your college. I might “grade” your college essay draft with a B- as well as giving you feedback and advice on how to improve it for the final “summative” version you hand in to the colleges of your choice since we have been working on essays all year. And I might even put that B- in my gradebook as one grade of many for you in English 12 this year. (But, then, I might also raise your grade once I see the ‘final’ version you sent to the colleges and count only the final’ grade on that personal essay in my gradebook.
In short, no matter the pure definition, I don’t think it is accurate to say that formative assessments can’t ever be graded. What matters – what makes a formative assessment formative – is whether I have a chance to get and use feedback in a later version of the ‘same’ performance. It’s only formative if it is ongoing; it’s only summative if it is the final chance, the ‘summing up’ of student performance.
What’s really more irksome for me in these kinds of matters is when people utterly abuse the idea of formative by describing any non-end-of-course assessment as formative. It is completely bogus to declare that ‘homework’ and quizzes are ‘formative’ simply because they are different from quizzes and because they occur throughout the year. If the specific demand only occurs once and you can’t use the feedback from them to do better next time, i.e. if the homework and journals are unique one-time events, then that individual homework assignment is summative. Just because it is not at the end of the year or semester doesn’t make it formative. It’s onlyformative if it recurs as a task in which I can learn from feedback to improve at the ‘same’ task.
Now, a critic may say – c’mon, Grant: they learn from doing homework how to do homework – so it’s formative and not 1-shot. And we grade it because we want them to be accountable for homework. Fair enough, I suppose (if that is really true). Calling homework assignments formative seems like a stretch to me. The content is unique and we grade the content not just the doing of it in most cases. If the grade is just for turning it in then I might acquiesce. On the other hand, calling a journal ‘formative’ and grading it for development over time seems reasonable.
However, we can say for sure that any truly one-shot assessment – homework, journal, quiz, paper – is summative, no matter when it occurs.
Call me a cynic, but my hunch here is that the makers of this rule are doing it unthinkingly in terms of the pure ideas. They seem to be just throwing trendy language around (at least from what you sent me; that’s all I can refer to). It sounds like they care less about the true meaning of ‘formative’ assessment than they care about making sure that kids do their work and are held accountable for it. FINE! But don’t call homework and journals ‘formative’ then.
Suppose I am wrong; suppose that the point of the plan is to make sure kids do their work AND to provide more truly formative assessment opportunities for kids, AND to expand the different types of assessments to give kids more options for showing what they know and can do. Fine! But then the policy would probably be different than the one proposed, I think.
A policy sensitive to these issues and designed to give kids more opportunities to get good feedback and use it would make that more clear in the policy statement. It would make clear when you should and when you should not include a grade in a kid’s average, what kinds of ‘fair chances’ to learn they need, and that journals and homework are only ‘formative’ if kids can improve at those tasks over time and have the later grades count more than the earlier grades (or some other reasonable rule).
What is really wanted here is a full discussion of the assumption that lurks below consciousness: the longstanding thoughtlessness of averaging grades to compute interim and final grades. This is just a dumb habit that penalizes growth and over-rewards wild swings of performance – though steady growth is what we presumably value in learners.
Another day I’ll tackle that beast!
This article first appeared on Grant’s personal blog; follow Grant on twitter; image attribution flickr user anthonypbruce; Using Homework As Formative Assessment