Losing it: The Influence of Losses on Individuals' Normalized Gains

Abstract:

Researchers and practitioners routinely use the normalized gain (Hake, 1998) to evaluate the effectiveness of instruction. Normalized gain (g) has been useful in distinguishing active engagement from traditional instruction. Recently, concerns were raised about normalized gain because it implicitly neglects retention (or, equivalently, "losses"). That is to say, g assumes no right answers become wrong after instruction. We analyze individual standardized gain (G) and loss (L) in data collected at Harvard University during the first five years that Peer Instruction was developed. We find that losses are non-zero, and that losses are larger among students with lower pre-test performances. These preliminary results warrant further research, particularly with different student populations, to establish whether the failure to address loss changes the conclusions drawn from g.