Terms of Use:
Konten ini milik Karin Sari Saputra Official.
Anda boleh mempublikasikan sebagian atau seluruh konten ini dengan syarat:
Cukup dengan menyertakan link pada tulisan tersebut dan/atau mengutip alamat konten ini.
Copyright © 2026 Karin Sari Saputra. All rights reserved. 

---

From Understanding Educational Phenomena to Addressing Their Underlying Challenges

Introduction: Is Monitoring Enough if We Only Know What Is Happening?

Hi, I’m Dr. Karin Sari Saputra, the developer of the Standardized Monitoring Framework (SMF).

There is something interesting I would like to share with fellow academics, educators, researchers, and educational practitioners.

In educational settings, we regularly observe learning activities, collect assessment results, distribute questionnaires, conduct classroom observations, and prepare evaluation reports. These activities provide valuable information about what is happening in our educational environments.

However, I believe there is a more fundamental question we need to ask:

Are we monitoring education merely to understand what is happening, or are we also using the evidence to identify underlying problems and determine what can be addressed appropriately?

Consider a student who consistently achieves low assessment scores. We may identify the problem, but do we understand what contributes to it? Is it related to instructional materials, assessment strategies, feedback quality, learning engagement, or the student’s ability to regulate their own learning?

Similarly, when lecturers experience difficulties implementing English as a Medium of Instruction (EMI), should we immediately conclude that their English proficiency is the primary issue? Or should we also examine pedagogical readiness, student comprehension, institutional resources, classroom interaction, and the availability of appropriate support systems?

These questions illustrate why educational monitoring should go beyond observation and data collection.

Monitoring should help us understand educational phenomena, examine the conditions behind them, identify what needs attention, and determine appropriate actions to address the challenges.

This is the fundamental idea behind the Standardized Monitoring Framework (SMF).

How Do the Three Constructs Work Together?

The three constructs provide complementary perspectives.

• MOELAS provides assessment-related evidence about learning progress and achievement.
• FUSPOEL examines learners’ perceptions of feedback, usefulness, satisfaction, and learning experiences.
• OSEL-OSE examines dimensions of self-regulated learning and student engagement.

Within the empirical conceptual model developed for SMF, these constructs are examined through interconnected relationships. FUSPOEL is also considered as a potential mediating construct in the relationship between MOELAS and OSEL-OSE.

This allows researchers to explore not only what learners achieve, but also how feedback experiences and learning perceptions may be associated with engagement and self-regulated learning.

These relationships are research propositions to be examined empirically. They should not be interpreted as automatically established causal relationships in every educational context.

Together, MOELAS, FUSPOEL, and OSEL-OSE strengthen the analytical foundation of SMF by connecting assessment evidence, feedback experiences, and learner engagement.

From Observation to Explanation: Why Standardization Matters

One of the recurring challenges in educational evaluation is the tendency to move too quickly from observation to conclusion.

Consider these examples:

• Students appear passive during classroom discussions.
• Lecturers frequently switch between English and Indonesian.
• Students obtain lower scores than expected.
• Learning materials are not consistently used.
• An educational programme has not achieved its intended implementation targets.

These statements may describe actual observations. However, they do not necessarily explain the underlying issues.

Student passivity, for example, may be associated with limited confidence, unclear instructions, insufficient opportunities for interaction, or difficulties understanding disciplinary content.

Similarly, lecturers’ language switching during EMI instruction may reflect concerns about student comprehension, limited access to classroom language resources, or insufficient pedagogical support.

Without a structured monitoring approach, evaluators may rely too heavily on personal impressions or attribute a complex issue to a single factor.

Standardization can help address this challenge by establishing clearer monitoring dimensions, indicators, procedures, and sources of evidence.

It also encourages evaluators to distinguish four important elements:

• Evidence: What was observed, measured, documented, or reported?
• Interpretation: What might the available evidence indicate?
• Inference: What possible explanations can be considered, and what alternative explanations remain?
• Recommendation: What action can reasonably be proposed based on the findings?

This distinction is essential for producing more transparent and accountable evaluation reports.

Standardization does not automatically eliminate evaluator bias. Rather, it helps reduce dependence on unsupported assumptions by making the reasoning process more systematic and traceable.

SMF in Practice: An Institutional EMI Evaluation at Universitas Sugeng Hartono

An interesting example of applying a structured monitoring approach can be found in the evaluation of English as a Medium of Instruction (EMI) implementation at Universitas Sugeng Hartono (USH).

As part of the university’s internationalization direction, EMI implementation requires more than lecturers’ ability to deliver subject content in English. It also involves pedagogical readiness, student preparedness, instructional resources, classroom interaction, assessment practices, and institutional support.

The evaluation therefore examined classroom practices and several dimensions of institutional readiness.

Classroom observations indicated that English was used in certain instructional activities, particularly during opening instructions. However, lecturers also switched to Indonesian when explaining more complex disciplinary concepts. Student participation and confidence in English-language interaction were among the areas requiring attention.

These observations needed to be interpreted within a broader context.

For instance, a lecturer’s decision to switch languages should not automatically be interpreted as a lack of commitment to EMI. It may also reflect concerns about students’ comprehension, limited access to ready-to-use classroom language, or the absence of sufficient pedagogical support.

The evaluation considered three interconnected dimensions of readiness.

1. Technical Readiness

Technical readiness concerns the availability of resources and instructional tools needed to support EMI implementation.

Areas requiring further development included bilingual classroom language resources, presentation language support, EMI-oriented instructional materials, adjustments to course plans, and assessment rubrics that distinguish content understanding from language performance.

2. Operational Readiness

Operational readiness concerns the institutional arrangements that enable implementation.

The evaluation highlighted the importance of scheduled monitoring and mentoring, accessible language support, and clearly defined responsibilities among the relevant institutional teams and programme stakeholders.

3. Substantive Readiness

Substantive readiness concerns pedagogical and psychological conditions surrounding EMI implementation.

Issues identified included lecturers’ concerns about making language errors, students’ confidence in participating in English, and the misconception that EMI requires lecturers to demonstrate native-like English proficiency.

These findings suggest that EMI readiness is not merely a matter of language competence. It also involves pedagogical confidence, classroom interaction, instructional design, and a supportive learning environment.

The three dimensions provide a more comprehensive picture than an evaluation based solely on the frequency of English use in the classroom.

This example illustrates how a multidimensional monitoring approach can connect classroom observations with the technical, operational, and substantive conditions that influence institutional programme implementation.

Importantly, these findings represent evidence and interpretations documented in the institutional evaluation. They do not establish that each identified condition has a single cause, nor do they demonstrate that every proposed intervention has already produced measurable improvements. Continued monitoring is needed to examine implementation and outcomes.

From Evidence to Action: Identifying, Diagnosing, Explaining, and Mitigating Challenges

For me, one of the most important purposes of SMF is to connect monitoring findings with meaningful actions.

In practical applications, this can be explained through five connected activities.

1. Identify

Recognise conditions that require attention.

At the learner level, these may include declining participation, limited progress, low assessment performance, difficulties with learning materials, or signs of disengagement.

At the institutional level, they may include insufficient teaching resources, unclear implementation responsibilities, limited professional development, inconsistent assessment practices, or gaps in programme coordination.

2. Diagnose

Examine the dimensions associated with the identified condition.

For example, low learner performance should not immediately be attributed to a lack of motivation. Educators may need to examine instructional clarity, content accessibility, assessment alignment, feedback quality, and learner engagement.

Similarly, inconsistent EMI implementation should be examined in relation to lecturer readiness, student comprehension, institutional resources, and operational support.

3. Explain

Develop a reasoned interpretation of the findings.

At this stage, evaluators should distinguish observed evidence from possible explanations. Multiple sources of information can be compared to determine whether an interpretation is sufficiently supported.

The objective is not to claim certainty where the evidence does not permit it, but to develop a defensible explanation that can guide subsequent decisions.

4. Mitigate

Determine appropriate responses to the issues identified.

At the learner level, possible responses include additional instructional support, revised learning materials, more targeted formative feedback, alternative assessment opportunities, or structured opportunities for interaction.

At the institutional level, responses may include professional development, improved teaching resources, revised assessment procedures, clearer responsibilities, and more consistent monitoring arrangements.

The proposed response should be connected to the issue identified rather than selected simply because it is commonly used or appears desirable.

5. Monitor and Evaluate

Examine whether the proposed actions have been implemented and whether they are contributing to the intended improvements.

It is important to distinguish evidence that an action has taken place from evidence that the action has been effective.

For example, a record showing that lecturers attended EMI training demonstrates participation. It does not, by itself, establish that their classroom practices have improved.

Further observation and evaluation are needed to examine whether changes in practice and learning experiences have occurred.

These five activities offer a practical explanation of how SMF can inform educational decision-making. They complement, rather than replace, the four formal monitoring phases of SMF: Preparation, Exploration, Adjustment, and Mastery.

Why SMF Matters at Both Learner and Institutional Levels

Educational challenges rarely exist in isolation.

A learner’s difficulties may be influenced by instructional practices, assessment design, learning resources, or institutional support. Likewise, institutional programme performance ultimately depends on the experiences and outcomes of the people involved in its implementation.

This is why monitoring should consider both levels.

At the learner level, SMF can help educators examine learning progress, engagement, feedback, self-regulation, assessment performance, and emerging risks that may affect learning continuity.

At the institutional level, SMF can help programme managers examine implementation readiness, teaching practices, resource availability, assessment alignment, professional development, coordination, and the consistency of programme delivery.

Connecting these levels makes it possible to examine how institutional conditions may relate to classroom experiences and learner outcomes.

The three constructs MOELAS, FUSPOEL, and OSEL-OSE provide a foundation for the online English learning application of SMF. When the framework is applied in other educational settings, such as institutional EMI evaluation, the monitoring indicators and instruments may need to be adapted to the relevant context and unit of analysis.

Such adaptation should be guided by methodological considerations, including construct definitions, instrument validity, reliability, and the quality of the evidence collected.

In other words, the underlying monitoring logic can inform different educational contexts, but the measurement instruments should not be assumed to be interchangeable or automatically valid across settings.

What SMF Does—and Does Not—Claim

It is important to clarify the scope of SMF.

SMF is not simply a dashboard, a collection of performance indicators, or a mechanism for ranking educators and learners.

Nor does the use of a standardized framework automatically guarantee objective evaluation or successful educational interventions.

Its value lies in providing a coherent structure for examining evidence and connecting monitoring activities with educational decisions.

The quality of its application still depends on the relevance of the indicators, the validity and reliability of the instruments, the quality of observations, the credibility of data sources, the competence of evaluators, and the transparency of interpretation.

Triangulation is particularly important when findings are used to inform institutional decisions. Observations should be examined alongside other relevant evidence rather than treated as conclusive on their own.

Furthermore, recommendations need to be followed by subsequent monitoring. Without this step, an evaluation may produce a comprehensive report but remain disconnected from actual improvement.

SMF should therefore be understood as a framework that supports systematic inquiry and evidence-informed action, not as an automatic solution to every educational challenge.

Conclusion: Monitoring Should Lead to Understanding and Improvement

Educational institutions do not need more data simply for the sake of collecting it. They need meaningful evidence that can help educators and decision-makers understand what is happening, examine why it may be happening, and determine what should be done next.

The Standardized Monitoring Framework offers a structured approach to this challenge by integrating five pillars of educational practice, four monitoring phases, and three core constructs in its online English learning application: MOELAS, FUSPOEL, and OSEL-OSE.

The institutional EMI evaluation at Universitas Sugeng Hartono illustrates how a multidimensional monitoring approach can move evaluation beyond a narrow examination of classroom language use. It can help connect teaching practices with technical, operational, and substantive conditions that influence programme implementation.

Ultimately, the purpose of monitoring is not merely to document performance or identify shortcomings. It is to make educational decisions more transparent, evidence-informed, and responsive to the people and institutions involved.

A monitoring framework should do more than describe what happens. It should help us understand what needs attention, examine the evidence behind it, and determine what can reasonably be done next.

That is the direction SMF seeks to support: moving beyond data collection towards structured understanding, and from evaluation towards informed educational improvement.

An Invitation to Connect and Collaborate

I would like to invite fellow academics, researchers, educators, and institutional leaders to explore the potential of the Standardized Monitoring Framework.

How might we use SMF in different educational contexts? How can its monitoring dimensions and measurement approaches be developed further? What opportunities might emerge when we connect systematic monitoring with meaningful educational interventions?

I believe these are valuable questions to explore together.

Let’s connect, exchange ideas, share experiences, and discover what we can maximize through SMF.

NANTIKAN STANDARDIZED MONITORING FRAMEWORK (SMF) SERIES BERIKUTNYA

Temukan Solusi Permasalahan Belajar Bahasa Inggris 24/7

(*) Informasi Kendala Teknis
Silakan tanyakan pada kolom komentar untuk kendala teknis yang mungkin kalian temui.