Evaluation in Distance Education and E-Learning The Unfolding Model

With the rapid proliferation of distance education and e-learning courses, the need is growing for a comprehensive, professional approach to evaluating their effectiveness. This indispensable book offers a road map to guide evaluation practice in these innovative learning environments. Providing practical, step-by-step guidelines and tools for conducting evaluation studies?including how to deal with stakeholders, develop surveys and interview protocols, collect other scientific evidence, and analyze and blend mixed-methods data?the work also features a template for writing high-quality reports. The unfolding model developed by the authors draws on Messick's influential assessment framework and applies it to program evaluation. Two case studies of actual programs (a distance learning course and an e-learning course) demonstrate the unfolding model in action.

_x000D_ _x000D_ _x000D_

l. Why Do We Need a New Approach to Evaluation in Distance Education and E-Learning?

_x000D_

Distance Education versus E-Learning

_x000D_

The Rapid Expansion of Distance Education and E-Learning

_x000D_

What Is Evaluation?

_x000D_

Why Do We Need a Professional Approach to Evaluation?

_x000D_

What Does a Professional Approach to Evaluation Look Like?

_x000D_

Responding to the Call for a Professional Evaluation Approach: The Unfolding Model

_x000D_

Conclusions: Our Approach to Evaluation

_x000D_

2. The Theory and Practice of Program Evaluation

_x000D_

Why Are Program Evaluation Models Important?

_x000D_

Classification Frameworks for Program Evaluation Models

_x000D_

Alkin and Christie's Evaluation Tree: The Roots and Branches

_x000D_

Where Do We Diverge from Alkin and Christie?

_x000D_

Conclusions

_x000D_

3. Evaluation Theory and Practice in Distance Education and E-Learning

_x000D_

Evaluation Theory

_x000D_

Models with Scientific Evidence in the Foreground

_x000D_

Models Based on Evidence, Values, and Consequences

_x000D_

Models Based on Messick's (1989) Framework

_x000D_

A Summary of Evaluation Models in Distance Education

_x000D_

Evaluation Studies

_x000D_

Evaluation Practice

_x000D_

Do Unintended Consequences Emerge in Authentic Evaluation Studies?

_x000D_

Conclusions

_x000D_

4. Messick's Framework: What Do Evaluators Need to Know?

_x000D_

The Overlap between Test Validity and Program Evaluation

_x000D_

Messick's Contributions

_x000D_

Messick's Framework

_x000D_

The Overlap among the Four Facets

_x000D_

The Controversy over Unintended Consequences

_x000D_

Implications for Evaluation

_x000D_

Conclusions

_x000D_

5. Getting Started

_x000D_

Planning the Evaluation Study

_x000D_

The Ethics Review Process

_x000D_

The Political Context of Evaluation

_x000D_

Using the Unfolding Model as a “Road Map”

_x000D_

Mixed Methods: Blending Quantitative and Qualitative Data

_x000D_

What Is Essential to Our Approach?

_x000D_

Tailoring the Unfolding Model to Your Needs

_x000D_

Conclusions

_x000D_

6. The Unfolding Model: Scientific Evidence

_x000D_

Definition of “Scientific Evidence”

_x000D_

Scientific Evidence

_x000D_

How to Write Good Survey Questions

_x000D_

Using the Unfolding Model to Write Survey and Interview Questions

_x000D_

Administering Surveys

_x000D_

Analyzing Survey Data

_x000D_

Qualitative Data: Interviews, Focus Groups, and Online Ethnographies

_x000D_

Qualitative Data Analysis

_x000D_

Outcomes

_x000D_

The Evaluation of Environmental Quality

_x000D_

Relevance

_x000D_

Cost–Benefit Analysis

_x000D_

Bringing It All Together: Mixed Methods

_x000D_

Conclusions

_x000D_

7. The Unfolding Model: Values and Consequences

_x000D_

Underlying Values

_x000D_

How to Identify the Values Underlying Your Course

_x000D_

Course Goals and Objectives

_x000D_

Writing Survey/Interview Questions about Underlying Values

_x000D_

Analyzing Data on Underlying Values

_x000D_

Unintended Consequences

_x000D_

How Can You Identify Unintended Consequences?

_x000D_

Writing Survey/Interview Questions about Unintended Consequences

_x000D_

Analyzing Data on Unintended Consequences

_x000D_

How to Enhance the Validity of Your Findings

_x000D_

Recommendations for Course Improvement

_x000D_

Writing the Evaluation Report

_x000D_

Conclusions

_x000D_

8. Findings from Two Authentic Case Studies

_x000D_

Methods and Procedures for Both Studies

_x000D_

Distance Learning: Computing Science 200 (CPSC 200)

_x000D_

E-Learning: Professional Writing 110 (PWRIT 110)

_x000D_

Conclusions

_x000D_

9. Bringing It All Together

_x000D_

Using Messick's Framework to Evaluate Distance and E-Learning Courses

_x000D_

Using the Unfolding Model to Evaluate Your Courses

_x000D_

Conducting Your Evaluation Study

_x000D_

What Have We Learned from Two Case Studies?

_x000D_

E-Learning and Beyond: Is the Unfolding Model the Last Word?

_x000D_

The Future of Distance Education and E-Learning

_x000D_

Valerie Ruhe is an Evaluation Studies Specialist at the University of British Columbia. Previously, she was an Assessment and Evaluation Consultant at the Center for Teaching and Learning, University of Minnesota. She has 10 years of professional program evaluation experience in distance education, K-12, and higher education.

Bruno D. Zumbo is Professor of Measurement, Evaluation, and Research Methodology, and of Statistics at the University of British Columbia. He is widely published in research methodology, validity, and validation processes, as well as statistical science and program evaluation methodology. His work has had wide-ranging influence across many fields in the social, educational, and health sciences.