This document represents the first review and summary of existing research on student course evaluations from a Canadian perspective. The scholarship in this area is vast and of varying quality and scope. Our review is an attempt to capture and synthesize the key issues and findings regarding the validity and utility of student course evaluations. We have organized our research into the following seven sections:
Section 1: Introduction provides an overview of the scope, methodology and limitations of this study.
Section 2: Context identifies the current state of scholarship and interest in course evaluations and the evaluation of teaching more generally. It also reviews student, faculty and administrator perceptions of course evaluation systems.
Section 3: Current Policy and Practice in North America offers an overview of evaluation instruments, policies and processes from 22 post-secondary institutions in Canada and the United States as well as policies related to course evaluations from system-level and government agencies.
Section 4: Reliability, Validity and Interpretation of Course Evaluation Data summarizes and reviews the findings from previous studies conducted over the past 40 years with a particular emphasis on the last two decades.
Section 5: Implementing Effective Evaluation Measures: Recommendations from the Research synthesizes research findings and identifies recommendations for improved administration and interpretation of course evaluations.
Section 6: Emerging Trends, Existing Gaps and Suggestions for Further Research highlights issues currently being considered in the scholarship along with those that have been identified as areas requiring more in-depth analysis.
Section 7: Concluding Remarks provides a brief summary of our most important findings and recommendations.
Overall, our findings indicate that while course evaluation instruments generally provide reliable and valid data, significant barriers to the effective use of such evaluation systems continue to exist due to: Persistent myths and misconceptions about variables affecting evaluation results; Unclear concepts and definitions of effective teaching; Student Course Evaluations: Research, Models and Trends Insufficient education about the goals, uses and validity of course evaluations for students, faculty and administrators; Poor presentation and contextualization of evaluation data; and Inconsistent and inequitable policies and practices regarding the implementation and administration of course evaluations.
Our findings suggest that no matter the reliability and validity of the evaluation instruments themselves, the policies, processes and practices at an institution determine the degree to which evaluations are an effective measure of teaching quality.
Student pathways increasingly rely on transfer between postsecondary institutions as greater numbers of students move between institutions, pursue multiple credentials, or return to postsecondary education. In a 2011 survey of Ontario college students, 41% reported having some post-secondary experience; the same survey also found that 19% of respondents said their main goal in applying for their current program was to “prepare for further university or college study.” Transfer of credit for prior learning is clearly an increasingly mainstream educational activity, and institutions are under increasing pressure to improve the processes by which this occurs.
It’s exam time. Research suggests that while some students will be pleasantly surprised by how they did on exams, a larger group will falsely believe they did much better on their exams than they did.
At this time of year, university students across the country are preparing for exams. Some will happily get higher-than-expected marks. But a larger group instead will be surprised by lower scores.
Negative surprises are common partly because we humans tend to be overly optimistic. Look at how people buy lottery tickets, borrow money or invest in stocks.
Students also tend to be unduly optimistic about their learning and forthcoming grades. Less skilled students are especially likely to over-estimate. This may lead them to make poor choices. If they mistakenly believe they’re already doing well, they may not study enough.
I often see this problem among my undergraduate students. So, I’ve experimented by giving them extra feedback about their grades and then surveying their reactions. A Chancellor’s Chair for Teaching Excellence award from Brock University funded this research.
When you think of Canada, you probably think of its picture postcard beauty – wide-open spaces, dramatic mountains, pristine forests and majestic lakes. What might not come to mind, however, is that Canada is a modern, progressive, open and tolerant multi-cultural society with 2 official languages – English and French.
Living in Canada is similar in many respects to living in other Western countries, however there are some aspects of daily life that are unique to our nation. This section of the website will give you a high level overview of our country as well as some helpful tips to know before you arrive to study in Canada.
It is a common practice at universities to have students complete end-of-term questionnaires about their courses and instructors. Sometimes called student evaluations of teaching (SETs) or student questionnaires on courses and teaching (SQCTs), these are often used to make decisions about faculty tenure and promotion without an appreciation of their limitations. These uestionnaires could be good for capturing the student experience, but responses are inherently influenced by factors outside of the professor's control, including the subject being taught, class size, and the professor's gender, race, or accent. Further, the comment sections in these anonymous questionnaires can and have been vehicles of harassment.
Ontario’s faculty understand the value of student feedback, but the manner in which this feedback is sought, and the ends to which it is used are problematic. The goal of student questionnaires should be to inform the understanding of the teaching and learning experience, not to punish faculty for their class size, instructional innovations, gender, or skin colour.
To consider these issues, the Ontario Confederation of University Faculty Associations (OCUFA) has set up a working group with experts in methodology, research ethics, and human rights. The group has been tasked with developing a deeper understanding of how student questionnaires are currently being used at Ontario's universities, defining the limitations of these questionnaires, and developing proposals for ensuring that these questionnaires are used appropriately. The working group is expected to release its report and recommendations later this year. What follows is a summary of the group’s findings so far.
The present study used meta-analytic methodology to synthesize research on the relationship between student ratings of instruction and student achievement. The data for the meta-analysis came from 41 independent validity studies reporting on 68 separate multisection courses relating student ratings to student achievement. The average correlation between an overall instructor rating and student achievement was .43; the average correlation between an overall course rating and student achievement was .47. While large effect sizes were also found for more specific rating dimensions such as Skill and Structure, other dimensions showed more modest relationships with student achievement. A hierarchical multiple regression analysis showed that rating/achievement correlations were larger for full-time faculty when students knew their final grades before rating instructors and when an external evaluator graded students’ achievement tests. The results of the meta-analysis provide strong support for the validity of student ratings as measures of teaching effectiveness.
I remember the first time I tackled the controversial subject of students as customers. It was in an in-house newsletter, well before the advent of the Internet and e-mail. Even so, I had numerous phone calls, memos, encounters on campus, and discussions about it in every activity the teaching center sponsored for the next year. I hadn’t even taken a side; I had simply listed arguments for both sides. But, as far as the faculty were concerned then and pretty much since, there aren’t two sides. Students are not customers. Tuition dollars do not buy grades. Education does not come with a money-back guarantee. And students don’t get to choose what they learn—well, they do, but if they don’t choose to learn what we require, the consequences are costly.
Douglas Mulford worried when his lab course moved to remote instruction this past spring. Mulford, a senior lecturer of chemistry at Emory University, had worked out a system for giving in-person exams in large classes. But with his 440 students taking their final online, he feared, it would be much easier for them to cheat.
So Mulford set out to protect his test. He looked into lockdown browsers, which limit what students can do on their computers during a test, but concluded they were pointless: Most of his students had a smartphone, too, he figured, and could simply consult it instead. He thought about using a proctoring service, but wasn’t convinced it could handle this volume
of tests on such short notice. So he settled on what he calls “Zoom proctoring,” having students take their final in a Zoom room, with videos turned on, while a TA watched them and recorded the session.
The idea for sharing this post came from a session I recently conducted at the annual teaching conference organized by my university. A pedagogical conundrum was raised by a colleague whose enthusiasm and question stayed with me and inspired me to write this post. The question posed by this colleague is relevant to all instructors who have ever used group work to assess their students: How should one deal with the issues that arise when members of a group are not picking up their share of the responsibilities during a group work project?
The development of essential employability skills (EES) has become an increasingly critical component of the postsecondary curriculum. Today’s graduates must be able to demonstrate a range of skills such as communication, teamwork and problem solving, and use these skills within diverse employment contexts. Although essential skill learning outcomes have been a part of Ontario’s postsecondary college curriculum for more than two decades, there is a distinct gap in research on the development and assessment of these skills. Reports on skill gaps, employment trends and EES assessment in education have led some educators to speculate whether electronic portfolios (ePortfolios) can be used to address the development and assessment of EES — and help shine a spotlight on EES for postsecondary students.
There's been an increase in university students doing "contract cheating" — hiring out ghostwriters or someone to
take tests, warns a University of Calgary professor.
Both services are widely available on the internet, says Sarah Eaton, who is the acting associate dean of teaching
and learning at the Werklund School of Education.
On Wednesday, the second International Day of Action against Contract Cheating called for increased awareness
against firms that aggressively market contract cheating services to students on campus.
The Canadian Graduate and Professional Student Survey (CGPSS) is a national survey that was completed by over 51,000 students across 48 universities in 2013. This comprehensive survey includes questions covering a broad array of topics including students’ satisfaction with their departments, programs and advisors, availability of funding, use and quality of university services, and satisfaction with professional development supports (CAGS, 2010). This report uses data and opinions collected from graduate students through the CGPSS in an effort to contribute to the conversation on graduate student education in Canada.
The Canadian Graduate and Professional Student Survey (CGPSS) is a national survey that was completed by over 51,000 students across 48 universities in 2013. This comprehensive survey includes questions covering a broad array of topics including students’ satisfaction with their departments, programs and advisors, availability of funding, use and quality of university services, and satisfaction with professional development supports (CAGS, 2010). This report uses data and opinions collected from graduate students through the CGPSS in an effort to contribute to the conversation on graduate student education in Canada.
NEW YORK, NY, October 8, 2015—The JED Foundation, Partnership for Drug-Free Kids and The Jordan Porco Foundation today released the results of a national “First-Year College Experience” survey, exploring the challenges associated with young adults’ transition from high school to college. Results have significant implications for parents, educators and students alike, revealing important touch points for better communication, programming and meaningful intervention. Among the most critical findings, the Harris Poll of 1,502 U.S. first-year college students uncovered that emotional preparedness – defined by the organizations as the ability to take care of oneself, adapt to new environments, control negative emotions or behavior and build positive relationships – is a major factor to students’ success during their first year of college.
"Plan for the students you actually have, not those you wish you had, or think you used to have, or think you used to be like."
So John N. Gardner, the creator of the term "first-year experience," advised college officials charged with making sure that the experience is a good one. In other words, be realistic; don’t expect too much of students.
That mind-set contrasts with the one evoked by the New Yorker writer David Denby in his new book, Lit Up: One Reporter. Three Schools. Twenty-Four Books That Can Change Lives. The New York Times last week noted, "Lit Up is a refreshing lesson in what motivates students and why not to dumb down reading lists." Denby opens a window into the classrooms of several gifted high-school English teachers who assign Faulkner, Orwell, Frankl, Dostoevsky, Hemingway, Shakespeare, Poe, and Twain — and whose love of reading is contagious to their teen students.
"Plan for the students you actually have, not those you wish you had, or think you used to have, or think you used to be like."
So John N. Gardner, the creator of the term "first-year experience," advised college officials charged with making sure that the experience is a good one. In other words, be realistic; don’t expect too much of students.
That mind-set contrasts with the one evoked by the New Yorker writer David Denby in his new book, Lit Up: One Reporter. Three Schools. Twenty-Four Books That Can Change Lives. The New York Times last week noted, "Lit Up is a refreshing lesson in what motivates students and why not to dumb down reading lists." Denby opens a window into the classrooms of several gifted high-school English teachers who assign Faulkner, Orwell, Frankl, Dostoevsky, Hemingway, Shakespeare, Poe, and Twain — and whose love of reading is contagious to their teen students.
Students Will Rise When Colleges Challenge Them to Read Good Books
Study looks at the reasons why people pirate scholarly articles from peer-to-peer research sharing communities -- it's more convenience than ideology.
Boundary issues: should a student get a tattoo if it advances her research?
Would you consider getting a tattoo on the basis that it could help you to write your PhD? Or tolerate sexism in order to secure a strong case study for your research?
Such dilemmas faced by doctoral researchers are explored in a recent study that examines the challenges of
conducting qualitative research.
This study examines the TERTIARY educational needs in Kosovo. Review of past studies revealed limited information on specific needs or suggestions on how to solve the problem. A total of 24 experts from various industries, organizations, and government were interviewed. Gaps in fields of study as well as skills were identified. The fields identified as most needed are Accounting, IT, Marketing, Human Resource Management, and Agricultural Engineering and Management. While some of these fields of study exist already in the university system in Kosovo, the experts showed concern over lack of various business and entrepreneurial skills currently provided. Lack of education in International Relations was related to inability to create a sustainable global exporting system across all industries. This assessment leads to a further conclusion that several basic skills expected from college graduates are missing. The skills identified as most important included Organizational Management, Efficient Planning, Critical Thinking, Analytical skills, Positive Attitude, Work Ethic, Time Management, Writing Reports, Self-Discipline, Self-Motivation, Teamwork, and Team Motivation.