Agentic Engagement in Massive Open Online Courses Scale

Abstract

The Agentic Engagement in Massive Open Online Courses Scale (Kim & Song, 2023) is an empirical psychometric instrument developed to operationalize and evaluate the proactive, intentional, and constructive contributions of online learners within self-paced Massive Open Online Courses (MOOCs). Traditionally conceptualized in conventional classroom environments as a four-component architecture—spanning behavioral, emotional, cognitive, and agentic engagement—agentic engagement specifically captures the self-initiated transactional behaviors through which students enrich, personalize, and modify their educational circumstances. Built upon the classical scale development framework established by Hinkin (1995), the instrument originated from a comprehensive literature synthesis drawing heavily upon the foundational works of Reeve and colleagues (Reeve & Tseng, 2011; Reeve, 2013). An initial item pool was subjected to a rigorous two-round Delphi survey involving field specialists who evaluated both English and Korean item variants, culminating in a refined 7-item multidimensional scale after strict Content Validity Index (CVI) threshold application and professional back-translation.

The scale captures three structural dimensions: Agentic Support Request (ASR), Agentic Learning Strategy (ALS), and Agentic Learning Construction (ALC). Across exploratory structural equation modeling (ESEM) and confirmatory factor analysis (CFA) conducted on adult learners enrolled in the Korean Massive Open Online Course (K-MOOC) platform, the 7-item, 3-factor structure demonstrated robust psychometric adequacy (χ² = 30.972, df = 10, TLI = 0.938, CFI = 0.970, SRMR = 0.048). Reliability evaluations revealed solid internal consistency across subscales: EFA Cronbach's alpha ranged from 0.86 to 0.91 (total scale α = 0.87), CFA Cronbach's alpha ranged from 0.68 to 0.91, and composite reliability (CR) indices ranged between 0.75 and 0.91. Discriminant validity was fully sustained, with square root Average Variance Extracted (AVE) values (0.71–0.91) outpacing inter-factor correlations, alongside Heterotrait-Monotrait (HTMT) ratios falling between 0.56 and 0.78. This brief, targeted questionnaire provides educational psychologists, instructional designers, and higher education researchers with an empirically validated diagnostic tool to investigate self-regulatory dynamics and alleviate attrition in autonomous digital learning ecosystems.

Keywords

Student Engagement, Agentic Engagement, Self-Paced Massive Open Online Courses, MOOCs, Agentic Support Request, Agentic Learning Strategy, Agentic Learning Construction, Psychometrics, Scale Validation, Self-Regulated Learning

Authors

The Agentic Engagement in Massive Open Online Courses Scale was developed by scholars at Chung-Ang University in Seoul, Republic of Korea:

• Rang Kim, Ph.D.
  Affiliation: Center for Teaching and Learning, Chung-Ang University, Seoul, Republic of Korea.
  ORCID: 0000-0002-4657-1170
• Hae-Deok Song, Ph.D. (Corresponding Author)
  Affiliation: Department of Education, College of Education, Chung-Ang University, Seoul, Republic of Korea.
  ORCID: 0000-0003-0879-9999
  Contact / Email: [email protected]

Purpose

The primary purpose of the Agentic Engagement in Massive Open Online Courses Scale is to diagnose, measure, and analyze the specific behavioral manifestations of learner agency within self-paced online environments. Massive Open Online Courses represent an educational paradigm shift characterized by global accessibility, asynchronous modular design, and open registration. However, these systems are chronically undermined by high attrition, passive non-completion, and transactional learner isolation. While classic models of academic engagement adequately measure behavioral compliance (e.g., watching assigned videos, completing quizzes), emotional affinity (e.g., enjoying the course subject), and cognitive investment (e.g., memorization or deep processing), they routinely treat the learner as a reactive entity responding passively to pre-existing instructional curricula.

In contrast, Reeve (2013) posited that optimal academic development relies on agentic engagement: the constructive, proactive, and intentional process by which students actively contribute to the flow of instruction, personalize their educational trajectory, and solicit environmental affordances to satisfy their personal psychological needs. In self-paced MOOCs, where real-time face-to-face feedback is fundamentally absent, agentic engagement is vital for sustained persistence. Learners cannot simply rely on instructor prompts; they must autonomously orchestrate pedagogical interactions, bridge disconnected modular topics, and mobilize available asynchronous resources.

From an applied research and clinical-educational perspective, this instrument addresses several major gaps:

• Diagnostic Assessment of Attrition Vulnerability: Enabling academic researchers and learning analytics teams to detect deficits in proactive engagement early in a course schedule, allowing for automated, timely nudges or instructional interventions.
• Instructional Design Evaluation: Permitting educational developers to evaluate whether specific online course designs (e.g., branching video structures, peer assessment networks, integrated discussion boards) foster or constrain authentic proactive agency.
• Empirical Investigation of Self-Regulated Learning: Providing educational psychologists with a lean, reliable metric to explore the nomological network connecting autonomous motivation, self-efficacy, transactional distance, and deep learning strategies in adult distance education.

Psychological Construct

Agentic engagement represents a dialectical conceptualization of the learner-environment relationship. Rather than viewing the student as an instructional consumer whose engagement depends strictly on instructional delivery, the agentic paradigm recognizes the learner as an active collaborator who actively reshapes the learning environment to align with their learning goals. In the self-paced MOOC context, Kim and Song (2023) demonstrated that agentic engagement departs from traditional face-to-face conceptualizations—which often emphasize verbal questions asked during live lectures—and instead manifests across three distinct behavioral dimensions:

1. Agentic Support Request (ASR)

Agentic Support Request reflects a learner’s intentional effort to seek out external human scaffolding, instructional assistance, or curated external materials within an asynchronous digital ecosystem. In typical offline settings, support requests occur spontaneously via hand-raising or informal post-lecture dialogues. In self-paced MOOCs, seeking assistance demands substantial proactive initiative, as learners must navigate discussion forums, submit helpdesk inquiries, or reach out to distant peers across disparate time zones. ASR captures the student’s conscious willingness to combat isolated cognitive bottlenecks by asking instructors or fellow learners for targeted clarifications, supplementary readings, and pedagogical help.

2. Agentic Learning Strategy (ALS)

Agentic Learning Strategy denotes the meta-motivational and metacognitive orchestrations that learners utilize to align digital course structures with their internal motivational states and explicit objectives. Because self-paced MOOC environments contain vast amounts of asynchronous digital material without rigid deadlines, learners must actively generate and sustain intrinsic motivation. Within this subscale, learners monitor their interest levels, establish personal goal relevance, curate complementary materials that serve their professional ambitions, and systematically structure their cognitive ideas to master complex modular themes.

3. Agentic Learning Construction (ALC)

Agentic Learning Construction represents the direct behavioral restructuring and personalization of the learning pathway. Unlike static educational programs where students follow a linear, prescribed path through textbooks and lectures, MOOCs allow non-linear navigation. ALC captures the intentional decisions by which learners disregard default course sequences to optimize their personal learning. For instance, learners may choose to attempt end-of-module assessment quizzes before watching didactic videos to evaluate prior knowledge, or selectively alternate between discussion boards, supplementary transcripts, and video lectures. Furthermore, it measures the deliberate linking of external reference materials to current course units, thereby personalizing the educational experience.

Theoretical Framework

The conceptual framework of the Agentic Engagement in Massive Open Online Courses Scale is grounded in the intersection of Self-Determination Theory (SDT), Social Cognitive Theory, and contemporary models of Self-Regulated Learning (SRL).

Self-Determination Theory and Dialectical Motivation

Pioneered by Ryan and Deci (2000) and further operationalized within educational settings by Reeve (2013), the dialectical model of motivation posits that students possess innate psychological needs for autonomy, competence, and relatedness. Engagement is not simply an outcome of an environment that supports autonomy; it is an active contribution to that environment. When individuals exercise agency, they initiate reciprocal feedback loops: by expressing personal goals, asking clarifying questions, and requesting customized materials, they alter the teacher’s instructional style and prompt richer contextual support. In self-paced MOOCs, where the instructor is largely present only asynchronously, learners must self-initiate this dialectic by transforming static course materials into dynamic, personalized environments.

Bandura’s Triadic Reciprocal Causation

Albert Bandura's (1986, 2001) agentic perspective within Social Cognitive Theory emphasizes that human beings are neither entirely driven by inner forces nor automatically shaped by environmental stimuli. Instead, human adaptation operates through triadic reciprocal causation, wherein internal personal factors (cognitive, affective, and biological events), behavioral patterns, and environmental influences operate as interacting determinants that influence one another bidirectionally. Bandura distinguished between individual agency, proxy agency, and collective agency. The Agentic Engagement in MOOCs scale focuses squarely on individual agency and proxy support: the intentionality, forethought, and self-reactiveness by which an individual learner intervenes in the pre-packaged online delivery architecture to achieve intended educational outcomes.

Metacognitive Self-Regulation in Distance Education

Classical models of self-regulated learning (e.g., Pintrich, 2004; Zimmerman, 2008) delineate forethought, performance, and self-reflection phases. While traditional cognitive engagement inventories measure internal processes (such as rehearsal, elaboration, and organization), agentic engagement explicitly captures action-oriented behaviors that modify the external instructional conditions. In digital distance education, where transactional distance (Moore, 1993) often leads to psychological disconnection, agentic behaviors serve as an active bridge, transforming isolated, passive video viewing into an interactive, self-directed learning experience.

Validity

The psychometric validation of the Agentic Engagement in Massive Open Online Courses Scale adhered to the three-stage scale development framework proposed by Hinkin (1995): item generation, scale administration, and item evaluation.

Content Validity and Expert Review

Following item extraction from theoretical engagement frameworks (notably Reeve & Tseng, 2011; Reeve, 2013; Bong et al., 2012), a preliminary 14-item pool was evaluated through a formal Delphi survey with expert panels in educational technology and psychometrics. Experts evaluated items across English and Korean formats for conceptual relevance, linguistic clarity, and contextual fit for self-paced MOOCs. Using the Content Validity Index (CVI), items that exhibited semantic ambiguity, double-barreled structures, or poor contextual alignment with asynchronous learning environments were removed. Seven core items were retained and subsequently back-translated into English by three bilingual medical doctors to verify that the cross-linguistic and semantic integrity of the items remained intact.

Construct and Structural Validity

Construct validity was established through both Exploratory Structural Equation Modeling (ESEM) and Confirmatory Factor Analysis (CFA) using a large sample of registered adult students within the Korean Massive Open Online Course (K-MOOC) ecosystem. The CFA analysis confirmed the hypothesized three-factor architecture. The goodness-of-fit indices satisfied standard structural equation modeling cutoffs:

• Chi-Square / Degrees of Freedom: χ² = 30.972, df = 10 (χ²/df = 3.10)
• Comparative Fit Index (CFI): 0.970 (exceeding the standard 0.95 criterion for exemplary model fit)
• Tucker-Lewis Index (TLI): 0.938 (exceeding the acceptable 0.90 cutoff)
• Standardized Root Mean Square Residual (SRMR): 0.048 (well below the conservative 0.08 ceiling)

Convergent Validity

According to empirical criteria established by Hair et al. (2018), convergent validity requires standardized factor loadings to exceed 0.50 (ideally > 0.70) and reach statistical significance. All seven standardized factor loadings of the scale comfortably exceeded the 0.50 cutoff, achieving a mean factor loading magnitude of 0.77 (all p < .001). This confirms that each latent factor accounts for a substantial share of variance in its corresponding observable indicators.

Discriminant Validity

Discriminant validity was established via two complementary methodological tests:

1. Fornell-Larcker Criterion: The square root of the Average Variance Extracted (AVE) for each of the three dimensions was calculated and compared with the inter-construct correlations. The square root of the AVE ranged from 0.71 to 0.91 across the three dimensions, consistently exceeding the bivariate correlations observed between constructs.
2. Heterotrait-Monotrait (HTMT) Ratio of Correlations: Following guidelines established by Henseler et al. (2015), HTMT values should fall well below the conservative threshold of 0.85 to confirm that distinct concepts are not redundant. The observed HTMT ratios ranged between 0.56 and 0.78, demonstrating clear differentiation between Agentic Support Request, Agentic Learning Strategy, and Agentic Learning Construction.

Reliability

The scale's internal consistency has been systematically evaluated through traditional Cronbach's alpha coefficients and Composite Reliability (CR) metrics across both exploratory and confirmatory phases.

Internal Consistency Across Phases

During the Exploratory Factor Analysis phase, the composite 7-item scale showed high overall internal consistency with a full-scale Cronbach's alpha of 0.87, well above the traditional 0.70 standard (Nunnally & Bernstein, 1994). The individual dimensions demonstrated the following alpha coefficients during EFA:

• Agentic Support Request (ASR): α = 0.86
• Agentic Learning Strategy (ALS): α = 0.87
• Agentic Learning Construction (ALC): α = 0.91

In the subsequent Confirmatory Factor Analysis phase, the reliability coefficients remained robust, confirming consistent item performance across samples:

• Agentic Support Request (ASR): α = 0.91; Composite Reliability (CR) = 0.91
• Agentic Learning Strategy (ALS): α = 0.68; Composite Reliability (CR) = 0.75
• Agentic Learning Construction (ALC): α = 0.76; Composite Reliability (CR) = 0.76

Because Cronbach's alpha can underestimate reliability in brief scales with few items per factor, Composite Reliability (CR)—which draws on structural factor loadings rather than raw item covariances—offers a more accurate assessment. All CR estimates exceeded the classical 0.60 to 0.70 thresholds established by Fornell and Larcker (1981), confirming that each latent construct is measured reliably despite the scale's brief, 7-item length.

Factor Analysis

To determine and confirm the dimensional stability of the scale, both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were conducted on empirical data collected from active adult MOOC learners.

Exploratory Factor Analysis & Parallel Analysis

Exploratory Structural Equation Modeling (ESEM) was implemented to evaluate competing latent factor solutions. Initial evaluations demonstrated that 1-factor (unidimensional engagement) and 2-factor structural configurations failed to achieve satisfactory psychometric fit. Conversely, a 3-factor model aligned closely with the hypothesized theoretical dimensions.

To guard against over-factoring, an empirical Parallel Analysis was conducted, contrasting the observed sample eigenvalues against those derived from 1,000 synthetic random datasets. The parallel analysis demonstrated that the 3-factor solution was optimal: through the third factor, the observed empirical eigenvalues remained substantially higher than the synthetic random eigenvalues. At the fourth factor, the eigenvalue for random data (0.99) surpassed the empirical eigenvalue of the actual dataset (0.50), confirming the extraction of three substantive latent dimensions.

Confirmatory Factor Analysis and Factor Loadings

The subsequent CFA on an independent evaluation sample confirmed that the 3-factor first-order specification fit the observed data well:

Construct Dimension	Standardized Factor Loading (λ)	Composite Reliability (CR)	Square Root of AVE
Agentic Support Request (ASR) (Items 1, 2)	0.88 – 0.94	0.91	0.91
Agentic Learning Strategy (ALS) (Items 3, 4, 5)	0.58 – 0.81	0.75	0.71
Agentic Learning Construction (ALC) (Items 6, 7)	0.71 – 0.84	0.76	0.78

Overall model fit indices were: χ²(10) = 30.972, CFI = 0.970, TLI = 0.938, SRMR = 0.048. Every item loaded significantly onto its targeted construct without cross-loadings, confirming structural stability across the subscales.

Instrument / Measurement Tool

The operational specifications of the Agentic Engagement in Massive Open Online Courses Scale are detailed below:

• Instrument Name: Agentic Engagement in Massive Open Online Courses Scale
• Test Type: Original self-report psychometric survey / inventory
• Target Population: Adult learners enrolled in self-paced digital distance education and Massive Open Online Courses (K-MOOC, Coursera, edX, Canvas Network). Validated across adult demographics including young adulthood (18–29), thirties (30–39), and middle adulthood (40–64).
• Administration Format: Digital online questionnaire or print-administered inventory
• Administration Time: Approximately 2 to 4 minutes
• Item Count: 7 items across 3 dimensions
• Structural Subscales:
  • Agentic Support Request (ASR): 2 items
  • Agentic Learning Strategy (ALS): 3 items
  • Agentic Learning Construction (ALC): 2 items
• Response Format: 5-point Likert-type scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree.
• Scoring Protocol: All items are framed positively (no reverse-coded items). Subscale scores are derived by calculating the mean of the items within that dimension: ASR (mean of items 1–2), ALS (mean of items 3–5), and ALC (mean of items 6–7). A total composite agentic engagement score can be calculated as the mean across all 7 items, with higher scores reflecting greater learner agency within the digital course.

Permissions & Fee and Test Year

• Year of Initial Publication: 2023
• Copyright & Ownership: Copyright © 2023 Open and Distance Learning Association of Australia, Inc. Published by Routledge / Taylor & Francis Group.
• Commercial Status: Non-commercial instrument. Developed for academic, diagnostic, and scholarly research.
• Fee: Free for non-commercial educational and scholarly research applications.
• Licensing / Permissions: For non-commercial academic research, the instrument may be utilized with proper citation of the source publication (Kim & Song, 2023). For formal republication, inclusion in commercial assessment batteries, or derivative instructional technology systems, formal permission should be requested via the publisher (Taylor & Francis) or by contacting the corresponding author ([email protected]).

References

• Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
• Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1–26. https://doi.org/10.1146/annurev.psych.52.1.1
• Bong, M., Cho, C., Ahn, H. S., & Kim, J. (2012). Comparison of self-efficacy for self-regulated learning and self-regulation across academic domains and grade levels. Korean Journal of Educational Research, 50(3), 259–288.
• Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
• Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2018). Multivariate data analysis (8th ed.). Cengage Learning.
• Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8
• Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21(5), 967–988. https://doi.org/10.1177/014920639502100509
• Kim, R., & Song, H.-D. (2023). Developing an agentic engagement scale in a self-paced MOOC. Distance Education, 44(1), 120–136. https://doi.org/10.1080/01587919.2022.2155619
• Moore, M. G. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22–38). Routledge.
• Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
• Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16(4), 385–407. https://doi.org/10.1007/s10648-004-0006-x
• Reeve, J. (2013). How students create motivationally supportive learning environments for themselves: The concept of agentic engagement. The Journal of Educational Research, 106(6), 479–495. https://doi.org/10.1080/00220671.2013.823318
• Reeve, J., & Tseng, C.-M. (2011). Agency as a fourth aspect of students' engagement during learning activities. Contemporary Educational Psychology, 36(4), 257–267. https://doi.org/10.1016/j.cedpsych.2011.02.002
• Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68
• Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological breakthroughs, and future directions. American Educational Research Journal, 45(1), 166–183. https://doi.org/10.3102/0002831207312909

Items of the Scale