Metacognition and Learning

A Framework for Evaluating and Enhancing Alignment in Self-Regulated Learning Research
Dent AL and Hoyle RH
We discuss the articles of this special issue with reference to an important yet previously only implicit dimension of study quality: alignment across the theoretical and methodological decisions that collectively define an approach to self-regulated learning. Integrating and extending work by leaders in the field, we propose a framework for evaluating alignment in the way self-regulated learning research is both conducted and reported. Within this framework, the special issue articles provide a springboard for discussing methodological promises and pitfalls of increasingly sophisticated research on the dynamic, contingent, and contextualized features of self-regulated learning.
Assessing Comprehension During Reading with the Reading Strategy Assessment Tool (RSAT)
Magliano JP, Millis KK, and Levinstein I
Comprehension emerges as the results of inference and strategic processes that support the construction of a coherent mental model for a text. However, the vast majority of comprehension skills tests adopt a format that does not afford an assessment of these processes as they operate during reading. This study assessed the viability of the Reading Strategy Assessment Tool (RSAT), which is an automated computer-based reading assessment designed to measure readers' comprehension and spontaneous use of reading strategies while reading texts. In the tool, readers comprehend passages one sentence at a time, and are asked either an indirect ("What are your thoughts regarding your understanding of the sentence in the context of the passage?") or direct (e.g., why X?) question after reading each pre-selected target sentence. The answers to the indirect questions are analyzed on the extent that they contain words associated with comprehension processes. The answers to direct questions are coded for the number of content words in common with an ideal answer, which is intended to be an assessment of emerging comprehension. In the study, the RSAT approach was shown to predict measures of comprehension comparable to standardized tests. The RSAT variables were also shown to correlate with human ratings. The results of this study constitute a "proof of concept" and demonstrate that it is possible to develop a comprehension skills assessment tool that assesses both comprehension and comprehension strategies.
Prior failures, laboring in vain, and knowing when to give up: Incremental versus entity theories
Bae J, Hong SS and Son LK
Against intuition, a set of "desirable difficulties" has been touted as a way in which to improve learning and lengthen retention. This includes, for instance, varying the conditions of learning to allow for more active, effortful, or challenging, contexts. In the current paper, we introduce data that show that, on the contrary, learning to know when to take the road may be crucial when it comes to avoiding "laboring in vain." We presented participants with prior problems - either easy or difficult - followed by choices of selecting an easy or a difficult current problem. Our primary goal was to examine the notion that past failures (which are more likely on the difficult prior items) may be a basis for allowing learners to then choose the easy rather than the difficult current problem. In other words, if one has labored in vain already, the easier items may now be more desirable. In addition, we compare the selections that are made between and perspectives, given their fundamentally opposing views on effort. Our results showed that, interestingly, incremental theorists, who generally are proponents of effort, were more likely to select the easy problems, but only when they had experienced failure on prior, and similar, difficult tasks. We interpret these data to suggest that those holding an incremental view may be more in tune with their past efforts, resulting in a Metacognition-by-Experience, or ME strategy, and also hint at its generalizability through cross-cultural comparisons.
The MAPS model of self-regulation: Integrating metacognition, agency, and possible selves
Frazier LD, Schwartz BL and Metcalfe J
Self-regulation, a social-cognitive process at the intersection of metacognition, motivation, and behavior, encompasses how people conceptualize, strive for, and accomplish their goals. Self-regulation is critical for behavioral change regardless of the context. Research indicates that self-regulation is learned. Integral to successful self-regulation of behavior are: (a) an articulated concept of one's possible selves, (b) metacognitive knowledge and effective strategies, and (c) a sense of one's own agency. We present the theoretical linkages, research evidence, and applied utility for these three components in promoting self-regulation of behavior, specifically in the domain of learning. We propose the MAPS model to account for the pathways of influence that lead to behavioral change. This model illustrates the dynamic and feed-forward processes that derive from the interactions among possible selves, metacognition, and agency to provide the context for developing self-regulated and effective learning that promotes student success, the transfer of knowledge, and the foundation for life-long learning.
Simulating the dynamics of self-regulation, emotion, grit, and student performance in cyber-learning environments
Kooken JW, Zaini R and Arroyo I
This research presents the results of development and validation of the Cyclical Self-Regulated Learning (SRL) Simulation Model, a model of student cognitive and metacognitive experiences learning mathematics within an intelligent tutoring system (ITS). Patterned after Zimmerman and Moylan's (2009) Cyclical SRL Model, the Simulation Model depicts a feedback cycle connecting forethought, performance and self-reflection, with emotion hypothesized as a key determinant of student learning. A mathematical model was developed in steps, using data collected from students during their sessions within the ITS, developing solutions using structural equation modeling, and using these coefficients to calibrate a System Dynamics (SD) Simulation model. Results provide validation of the Cyclical SRL Model, confirming the interplay of grit, emotion, and performance in the ITS. The Simulation Model enables mathematical simulations depicting a variety of student background types and intervention styles and supporting deeper future explorations of dimensions of student learning.
The effect of delayed judgments of learning on retention
Tekin E and Roediger HL
Evidence is mixed concerning whether delayed judgments of learning (JOLs) enhance learning and if so, whether their benefit is similar to retrieval practice. One potential explanation for the mixed findings is the truncated search hypothesis, which states that not all delayed JOLs lead to a full-blown covert retrieval attempt. In three paired-associate learning experiments, we examined the effect of delayed JOLs on later recall by comparing them to conditions of restudy, overt retrieval, and various other delayed JOL conditions. In Experiment 1, after an initial study phase, subjects either restudied word pairs, practiced overt retrieval, or made cue-only or cue-target delayed JOLs. In Experiments 2a and 2b, where conditions were manipulated within-subjects, subjects either restudied word pairs, practiced overt retrieval, made cue-only delayed JOLs, made cue-only delayed JOLs followed by a retrieval question or, in another condition, by an overt retrieval prompt. The final cued recall tests were delayed by two days. In Experiment 1, recall after cue-only delayed JOLs did not reliably differ from recall after overt retrieval or restudy. In Experiments 2a and 2b, delayed JOLs consistently produced poorer recall relative to overt retrieval. Furthermore, reaction times for delayed JOLs were shorter relative to delayed JOLs paired with overt retrieval prompts. We conclude that only some delayed JOLs elicit covert retrieval attempts, a pattern supporting the truncated search hypothesis.
Validation of metacognitive academic writing strategies and the predictive effects on academic writing performance in a foreign language context
Teng MF, Qin C and Wang C
This empirical study serves two purposes. The first purpose is to validate a newly developed instrument, the Metacognitive Academic Writing Strategies Questionnaire (MAWSQ), which represents the multifaceted structure of metacognition in an English as a Foreign Language (EFL) academic writing setting. The second purpose is to delineate the predictive effects of different metacognitive strategies on EFL academic writing performance. Data were collected from 664 students at a university in mainland China. Confirmatory factor analyses (CFA) provided evidence for the fit for two hypothesized models, i.e., an eight-factor correlated model and a one-factor second-order model. Model comparisons documented that the one-factor second-order model was a better model, through which metacognition functions as a higher order construct that can account for the correlations of the eight metacognitive strategies, pertaining to declarative knowledge, procedural knowledge, conditional knowledge, planning, monitoring, evaluating, information management, and debugging strategies. Results also provided evidence for the significant predicting effects of the eight strategies on EFL academic writing performance. The empirical evidence supports the transfer of metacognition theory from educational psychology to interpreting EFL academic writing.
The self-regulation for learning online (SRL-O) questionnaire
Broadbent J, Panadero E, Lodge JM and Fuller-Tyszkiewicz M
The Self-Regulation for Learning Online (SRL-O) questionnaire was developed to encompass the breadth of motivational beliefs and learning strategies that are often used in online and/or blended learning contexts. No current measure meets all these needs. This study used two non-duplicate samples to provide evidence of the psychometric properties of SRL-O using exploratory factor analyses (sample 1,  = 313), and confirmatory factor analyses, convergent and content validity and reliability (sample 2,  = 321). The SRL-O has a 10-factor structure, made up of (1) online self-efficacy, (2) online intrinsic motivation, (3) online extrinsic motivation, (4) online negative achievement emotion, (5) planning and time management, (6) metacognition, (7) study environment, (8) online effort regulation, (9) online social support, and (10) online task strategies. The SRL-O was also found to have two superordinate factors (motivational beliefs and learning strategies). The SRL-O was demonstrated to be a psychometrically sound measure of online SRL for learners studying in online and blended learning contexts. There is no other online self-regulated learning questionnaire that currently covers such a wide range of motivational beliefs and learning strategies.
Connecting teachers' classroom instructions with children's metacognition and learning in elementary school
van Loon MH, Bayard NS, Steiner M and Roebers CM
Many children have difficulties with accurate self-monitoring and effective regulation of study, and this may cause them to miss learning opportunities. In the classroom, teachers play a key role in supporting children with metacognition and learning. The present study aimed to acquire insights into how teachers' cognitive and metacognitive strategy instruction, as well as teacher-directed and child-centered instructional practices are related to children's self-monitoring accuracy, regulation of study, and learning performance. Twenty-one teachers and 308 children (2nd and 4th grade elementary school) participated. Teachers instructed a secret code task, children had to learn the match between letters of the alphabet and corresponding symbols. Teachers were observed and audio-recordings were made of their instructions. Then, children were asked to (a) make restudy selections, (b) complete a test, and (c) self-monitor test performance. Although teachers both addressed cognitive and metacognitive strategies, they more often instructed children about cognitive strategies. Further, teaching practices were more often teacher-directed than child-centered. Although there were no relations between teachers' instructions for metacognitive strategies and children's outcome measures, teaching cognitive strategies was positively associated with children's performance and self-monitoring accuracy. However, teaching cognitive strategies did not predict effective restudy selections. Rather, child-centered instructions (i.e., giving children autonomy to regulate their own learning) positively predicted children's restudy, and further, children's self-monitoring was more accurate in classrooms where teachers more often used child-centered instructional practices. This seems to imply that not only the content of the instructions itself, but particularly the way these are given, affects children's metacognition.
Comparing metacognitive monitoring between native and non-native speaking primary school students
Buehler FJ, van Loon MH, Bayard NS, Steiner M and Roebers CM
Metacognitive monitoring is a significant predictor of academic achievement and is assumed to be related to language competencies. Hence, it may explain academic performance differences between native and non-native speaking students. We compared metacognitive monitoring (in terms of resolution) between native and non-native speaking fourth graders (~ 10 year olds) in two studies. In Study 1, we matched 30 native and 30 non-native speakers and assessed their monitoring in the context of a paired-associates task, including a recognition test and confidence judgements. Study 1 revealed that recognition and monitoring did not differ between native and non-native speaking children. In Study 2, we matched 36 native and 36 non-native speakers and assessed their monitoring with the same paired-associates task. Additionally, we included a text comprehension task with open-ended questions and confidence judgments. We replicated the findings of Study 1, suggesting that recognition and monitoring do not necessarily differ between native and non-native speakers. However, native speaking students answered more open-ended questions correctly than non-native speaking students did. Nevertheless, the two groups did not differ in monitoring their answers to open-ended questions. Our results indicate that native and non-native speaking children may monitor their metacognitive resolution equally, independent of task performance and characteristics. In conclusion, metacognitive monitoring deficits may not be the primary source of the academic performance differences between native and non-native speaking students.
Association and dissociation between judgments of learning and memory: A Meta-analysis of the font size effect
Chang M and Brainerd CJ
The font size effect is a metamemory illusion in which larger-font items produce higher judgments of learning (JOLs) but not better memory, relative to smaller-font items. We conducted meta-analyses to determine what is currently known about how font size affects JOLs and memory accuracy. In addition, we implemented both univariate and multivariate meta-regressions to isolate the moderators of JOL effects and memory effects. The results revealed a small-to-moderate effect of font size on JOLs. There was also a small but significant effect of font size on memory. This suggests that JOLs and memory accuracy both increase with font size, rather than being completely dissociated. Moreover, JOL-memory dissociation only occurred when font size ranged between very small and intermediate. Our working explanation is that the memory effects of font size are tied to (dis)fluency, but its JOL effects are not. Some boundary conditions were identified for font size effects on both JOLs and memory. Specifically, larger font sizes only reliably increased both JOLs and memory accuracy (a) when font sizes ranged from intermediate to very large, (b) when study materials were unrelated word lists, (c) when JOLs were solicited immediately after encoding, and (d) when study time was relatively brief.
Reactivity from judgments of learning is not only due to memory forecasting: evidence from associative memory and frequency judgments
Maxwell NP and Huff MJ
Research has shown that judgments of learning (JOLs) often produce a reactive effect on the learning of cue-target pairs in which target recall differs between participants who provide item-based JOLs at study versus those who do not. Positive reactivity, or the memory improvement found when JOLs are provided, is typically observed on related pairs, while no reactivity is commonly found on unrelated pairs. In four experiments, we examined JOL reactivity effects by comparing JOL and no-JOL groups to other groups who engaged in relational-type encoding/judgment tasks. Experiment 1 replicated positive JOL reactivity effects with related pairs with an extension to symmetrically related pairs. Next, Experiment 2 found that providing judgments of associative memory-a task that does not involve memory predictions-yielded equivalent reactivity patterns as JOLs. Experiment 3 replicated this reactivity pattern using a frequency of co-occurrence judgment task. Finally, In Experiment 4, a similar positive reactivity pattern was found using a relational encoding task when compared to a standard JOL. Collectively, our results suggest that previous JOL reactivity patterns are not solely due to memory forecasting processes via JOLs and likely reflect relational encoding that is strategically applied towards related, but not unrelated pairs.
Confidence in COVID problem solving: What factors predict adults' item-level metacognitive judgments on health-related math problems before and after an educational intervention?
Scheibe DA, Fitzsimmons CJ, Mielicki MK, Taber JM, Sidney PG, Coifman K and Thompson CA
The advent of COVID-19 highlighted widespread misconceptions regarding people's accuracy in interpreting quantitative health information. How do people judge whether they accurately answered health-related math problems? Which individual differences predict these item-by-item metacognitive monitoring judgments? How does a brief intervention targeting math skills-which increased problem-solving accuracy-affect people's monitoring judgments? We investigated these pre-registered questions in a secondary analysis of data from a large Qualtrics panel of adults ( = 1,297). Pretest performance accuracy, math self-efficacy, gender, and math anxiety were associated with pretest item-level monitoring judgments. Participants randomly assigned to the intervention condition, relative to the control condition, made higher monitoring judgments post intervention. That is, these participants they were more accurate when answering problems. Regardless of experimental condition, those who actually were correct on health-related math problems made higher monitoring judgments than those who answered incorrectly. Finally, consistent with prior research, math anxiety explained additional variance in monitoring judgments beyond trait anxiety. Together, findings indicated the importance of considering both objective (e.g., problem accuracy) and subjective factors (e.g., math self-efficacy, math anxiety) to better understand adults' metacognitive monitoring.
Remedying the Metamemory Expectancy Illusion in Source Monitoring: Are there Effects on Restudy Choices and Source Memory?
Schaper ML, Bayen UJ and Hey CV
Metamemory monitoring, study behavior, and memory are presumably causally connected. When people misjudge their memory, their study behavior should be biased accordingly. Remedying should debias study behavior and improve memory. One metamemory illusion concerns source memory, a critical aspect of episodic memory. People predict better source memory for items that originated from an expected source (e.g., toothbrush in a bathroom) rather than an unexpected source (e.g., shampoo in a kitchen), whereas actual source memory shows the opposite: an . This biases restudy choices: Participants restudy more unexpected than expected source-item pairs. The authors tested the causal relationships between metamemory and source memory with a delay and a source-retrieval attempt between study and metamemory judgment to remedy the expectancy illusion and debias restudy choices. Debiased restudy choices should enhance source memory for expected items, thereby reducing the inconsistency effect. Two groups studied expected and unexpected source-item pairs. They made metamemory judgments and restudy choices immediately at study or after delay, restudied the selected pairs, and completed a source-monitoring test. After immediate judgments, participants predicted better source memory for expected pairs and selected more unexpected pairs for restudy. After delayed judgments, participants predicted a null effect of expectancy on source memory and selected equal numbers of expected and unexpected pairs. Thus, the expectancy illusion was partially remedied and restudy choices were debiased. Nevertheless, source memory was only weakly affected. The results challenge the presumed causal relationships between metamemory monitoring, study behavior, and source memory.
Development of Children's monitoring and control when learning from texts: effects of age and test format
Steiner M, van Loon MH, Bayard NS and Roebers CM
This study investigated elementary school children's development of monitoring and control when learning from texts. Second ( = 138) and fourth ( = 164) graders were tested in the middle (T) and end (T) of the school year. The study focused on the cross-sectional and longitudinal development of monitoring and control, and aimed to investigate the development of metacognition for two test formats. After reading expository texts, children completed a comprehension test consisting of open-ended and true-false questions. They monitored their test performance by making confidence judgments, and controlled performance by deciding whether to maintain or withdraw their given answers. Overall, monitoring and control accuracy was higher for open-ended questions than for true-false questions. For open-ended questions, results indicated higher metacognitive accuracy for fourth graders than second graders. No such age effects were found for monitoring and control for true-false questions. Longitudinally, children of both age groups improved their monitoring and control accuracy from T to T, for open-ended and true-false questions. For both test types, improvement mainly occurred for the monitoring and controlling of incorrect, rather than correct answers. Additionally, the results indicated inter-individual stability of performance, but no stability of monitoring and control accuracy over time. The findings indicate that developmental as well as task-related factors affect children's metacognitive accuracy.
Teachers' monitoring of students' text comprehension: can students' keywords and summaries improve teachers' judgment accuracy?
Engelen JAA, Camp G, van de Pol J and de Bruin ABH
We investigated intra-individual monitoring and regulation in learning from text in sixth-grade students and their teachers. In Experiment 1, students provided judgments of learning (JOLs) for six texts in one of three cue-prompt conditions (after writing delayed keywords or summaries or without a cue prompt) and then selected texts for restudy. Teachers also judged their students' learning for each text, while seeing - if present - the keywords or summaries each student had written for each text, and also selected texts for restudy. Overall, monitoring accuracy was low (.10 for students, -.02 for teachers) and did not differ between cue-prompt conditions. Regulation, indexed by the correlation between JOLs and restudy selections, was significant (-.38 for students, -.60 for teachers), but was also not affected by cue-prompt condition. In Experiment 2, teachers judged students' comprehension of six texts without knowing the students' names, so that only the keywords and summaries, not prior impressions, could inform judgments. Again, monitoring accuracy was generally low (.06), but higher for keywords (.23) than for summaries (-.10). These results suggest that monitoring intra-individual differences in students' learning is challenging for teachers. Analyses of the diagnosticity and utilization of keywords suggest that these may contain insufficient cues for improving teacher judgments at this level of specificity.
Introduction to special issue
Dignath C and Mevarech Z
Are children's judgments of another's accuracy linked to their metacognitive confidence judgments?
Baer C, Malik P and Odic D
The world can be a confusing place, which leads to a significant challenge: how do we figure out what is true? To accomplish this, children possess two relevant skills: reasoning about the likelihood of their own accuracy (metacognitive confidence) and reasoning about the likelihood of others' accuracy (mindreading). Guided by Signal Detection Theory and Simulation Theory, we examine whether these two self- and other-oriented skills are one in the same, relying on a single cognitive process. Specifically, Signal Detection Theory proposes that confidence in a decision is purely derived from the imprecision of that decision, predicting a tight correlation between decision accuracy and confidence. Simulation Theory further proposes that children attribute their own cognitive experience to others when reasoning socially. Together, these theories predict that children's self and other reasoning should be highly correlated and dependent on decision accuracy. In four studies ( = 374), children aged 4-7 completed a confidence reasoning task and selective social learning task each designed to eliminate confounding language and response biases, enabling us to isolate the unique correlation between self and other reasoning. However, in three of the four studies, we did not find that individual differences on the two tasks correlated, nor that decision accuracy explained performance. These findings suggest self and other reasoning are either independent in childhood, or the result of a single process that operates differently for self and others.
Metacognitive errors in the classroom: The role of variability of past performance on exam prediction accuracy
Geraci L, Kurpad N, Tirso R, Gray KN and Wang Y
Students often make incorrect predictions about their exam performance, with the lowest-performing students showing the greatest inaccuracies in their predictions. The reasons why low-performing students make inaccurate predictions are not fully understood. In two studies, we tested the hypothesis that low-performing students erroneously predict their exam performance in part because their past performance varies considerably, yielding unreliable data from which to make their predictions. In contrast, high-performing students tend to have consistently high past performance that they can rely on to make relatively accurate predictions of future test performance. Results showed that across different exams (Study 1) and different courses (Study 2), low-performing students had more variable past performance than high-performing students. Further, results from Study 2 showed that variability in past course performance (but not past exam performance) was associated with poor calibration. Results suggest that variability in past performance may be one factor that contributes to low-performing students' erroneous performance predictions.