The question is harder than it sounds
People often ask whether open educational resources are “as good as” conventional ones. The question hides several others. Good at what? For whom? Compared with which conventional material, used in which course, taught by whom? Research on this topic is a collection of partial answers to narrow versions of the question, and the broad claim that open education raises or lowers quality is not something any single study can settle.
This article does not summarize particular studies. Instead it explains how quality is usually defined, how comparisons are built, and what a careful reader should check. That is more durable than a list of findings, and it helps you judge any new claim you meet.
What “open” does and does not mean
Open materials are, in the usual definition, resources that carry a license permitting free access and some reuse, such as adaptation and redistribution. The license says nothing about the content. An openly licensed resource can be excellent or poor, just as a commercial one can. Any honest comparison therefore treats openness as a property of the license and cost, and quality as a separate property of the material and how it is used.
This matters because many claims blur the two. A study that finds “no difference” between an open text and a commercial one is not saying openness has no effect. It is saying that, in that setting, the particular texts performed similarly on the particular measure used.
Ways quality is defined
Researchers and institutions tend to describe quality along several dimensions. None is complete on its own.
- Content accuracy and currency. Is the material correct, clear and up to date? Subject experts or reviewers usually judge this.
- Pedagogical design. Are there clear learning objectives, worked examples, practice and feedback? Good design can matter as much as good content.
- Accessibility. Can learners with different needs, devices and connections use it? Accessible learning materials are part of quality, not an extra.
- Fit with the course. A well-written resource that does not match what is being taught and assessed is a poor fit regardless of its polish.
- Learner experience. Do students find it readable, usable and relevant? Surveys capture this, though opinions and learning are not the same thing.
- Learning results. Do students learn what the course intends? This is the dimension most people care about and the hardest to measure fairly.
A claim about “quality” should say which of these it means. Many disagreements in the literature come from people measuring different dimensions and calling all of them quality.
How comparisons are usually built
The most common design compares groups of students who used open materials with groups who used conventional ones. The groups might be different sections of the same course, different semesters, or different institutions. Each choice brings problems.
When students choose their own section, the groups may differ in motivation or prior preparation. When one semester is compared with another, the instructor, schedule or exam may have changed. When institutions are compared, almost everything differs. Studies that randomly assign students are rarer, and even they usually run in a single course, so their findings may not carry over elsewhere.
There is also a practical point. Switching to open materials often happens alongside other changes, such as a redesigned syllabus or new assessments. If outcomes shift, it is hard to say which change did it. A reasonable reader treats this as a reason for caution rather than as proof of either benefit or harm.
Measures and their limits
Outcome measures deserve scrutiny too. Course grades are convenient but depend on the instructor’s marking. Common exams are more comparable but only cover what the exam covers. Completion and pass rates are easy to count, yet they reflect many things besides material quality, including cost, workload and life circumstances.
One effect that appears in discussions is access. Where cost is a barrier, students may not obtain a required text at all, and textbook costs can change who has the material in the first week. If open materials improve outcomes because more students have the book on time, that is a real benefit, but it is a benefit of access rather than of better writing. Good studies try to separate the two, and a reader should note whether they did.
Evidence about teaching methods in general is mixed, and the same caution applies here. Effects, where reported, tend to be modest and to vary by subject, level and setting.
Reading a quality claim critically
A short checklist helps when you meet a statement such as “open resources match commercial ones” or “open materials are lower quality.”
- What was compared? Name the specific materials and the course, not just the category.
- How were groups formed? Random assignment, self-selection or different time periods?
- What was measured? Grades, a shared exam, satisfaction, completion?
- Who funded or wrote it? Advocacy groups and publishers both have interests, and neither is automatically wrong.
- How big is the effect, and how uncertain? A tiny difference in a small sample says little.
- Does it generalize? One course in one institution may not predict yours.
The same habits apply to any research. Learning to spot misleading charts helps, since an axis that starts above zero can make a small gap look large. So does knowing how to approach reading papers as a non-specialist, which starts with the abstract, the method and the limitations rather than the headline.
Quality assurance in practice
Beyond research, institutions and teachers judge quality directly. Checklists and rubrics often cover accuracy, clarity, alignment with course goals, accessibility and technical function. Peer review, instructor adoption notes and student feedback are common signals. Because open licenses allow adaptation, a teacher can also repair a weak section or update an out-of-date example, which is a quality advantage that fixed commercial editions do not always allow.
Licensing details need attention too. Before adapting anything, check the terms under copyright and licensing, since reuse conditions differ between licenses.
What a reasonable reader can conclude
On the available picture, it is sensible to say that openness by itself neither guarantees nor rules out good quality. Many comparisons report that learners using well-chosen open materials do about as well as those using conventional ones, but the studies vary in design and strength, and the finding should not be stretched beyond the settings examined. The more useful question for a teacher or student is usually narrower: is this particular resource accurate, well designed, accessible and a good fit for this course? That can be checked directly, and it does not depend on settling the broader debate.