This document represents the first review and summary of existing research on student course evaluations from a Canadian perspective. The scholarship in this area is vast and of varying quality and scope. Our review is an attempt to capture and synthesize the key issues and findings regarding the validity and utility of student course evaluations. We have organized our research into the following seven sections:
Section 1: Introduction provides an overview of the scope, methodology and limitations of this study.
Section 2: Context identifies the current state of scholarship and interest in course evaluations and the evaluation of teaching more generally. It also reviews student, faculty and administrator perceptions of course evaluation systems.
Section 3: Current Policy and Practice in North America offers an overview of evaluation instruments, policies and processes from 22 post-secondary institutions in Canada and the United States as well as policies related to course evaluations from system-level and government agencies.
Section 4: Reliability, Validity and Interpretation of Course Evaluation Data summarizes and reviews the findings from previous studies conducted over the past 40 years with a particular emphasis on the last two decades.
Section 5: Implementing Effective Evaluation Measures: Recommendations from the Research synthesizes research findings and identifies recommendations for improved administration and interpretation of course evaluations.
Section 6: Emerging Trends, Existing Gaps and Suggestions for Further Research highlights issues currently being considered in the scholarship along with those that have been identified as areas requiring more in-depth analysis.
Section 7: Concluding Remarks provides a brief summary of our most important findings and recommendations.
Overall, our findings indicate that while course evaluation instruments generally provide reliable and valid data, significant barriers to the effective use of such evaluation systems continue to exist due to: Persistent myths and misconceptions about variables affecting evaluation results; Unclear concepts and definitions of effective teaching; Student Course Evaluations: Research, Models and Trends Insufficient education about the goals, uses and validity of course evaluations for students, faculty and administrators; Poor presentation and contextualization of evaluation data; and Inconsistent and inequitable policies and practices regarding the implementation and administration of course evaluations.
Our findings suggest that no matter the reliability and validity of the evaluation instruments themselves, the policies, processes and practices at an institution determine the degree to which evaluations are an effective measure of teaching quality.
This 2014 mba.com Prospective Students Survey Report explores the motivations, behaviors, program choices, and intended
career outcomes shared by more than 12,000 individuals who registered on mba.com from October 2012 through September
2013. Survey data collected in 2013 are compared with earlier data collected from more than 71,000 prospective business
school students who have responded to our mba.com registrants’ surveys over the past four years. With survey responses
available for all world regions as well as 15 specific countries, this is the largest data resource of its kind.
Getting students to take their reading assignments seriously is a constant battle. Even syllabus language just short of death threats, firmly stated admonitions regularly delivered in class, and the unannounced pop quiz slapped on desks when nobody answers questions about the reading don’t necessarily change student behaviors or attitudes. Despite the correlation between reading and course success, many students remain committed to trying to get by without doing the reading, or only doing it very superficially, or only doing it just prior to exam dates. In return, some exasperated instructors fall into the trap of using
valuable class time to summarize key points of the readings. It’s not a new problem, and clearly we can’t simply bemoan the fact that students don’t read. Furthermore, doing what we’ve been doing — the threats, the endless quizzes, the chapter summaries — has failed to solve the problem. The better solution involves designing courses so that students can’t do well without reading, and creating assignments that require students to do more than just passively read.
Featuring 11 articles from The Teaching Professor, this special report was created to give faculty new ways of attacking an age-old problem. Articles in the report include:
• Enhancing Students’ Readiness to Learn
• What Textbook Reading Teaches Students
• Helping Students Use Their Textbooks More Effectively
• Text Highlighting: Helping Students Understand What They Read
• When Students Don’t Do the Reading
• Pre-Reading Strategies: Connecting Expert Understanding and Novice Learning
Whether your students struggle with the material or simply lack the motivation to read what’s
assigned, this report will help ensure your students read and understand their assignments.
Maryellen Weimer
Editor
The Teaching Professor
The 2015 Campus Freedom Index is the fifth annual report released by the Justice Centre for Constitutional Freedoms (JCCF) to measure the state of free speech at Canada’s universities.
Starting with a survey of only 18 universities in 2011, this year’s edition has grown to include 55 publicly funded Canadian universities—the largest and most expansive Index released so far, with information relevant to the more than 750,000 students who attend these institutions. The 2015 Campus Freedom Index includes an individual report about each university and student union.
The Association of Registrars of the Universities and Colleges of Canada (ARUCC) and the Pan-Canadian Consortium on Admissions and Transfer (PCCAT) have collaborated to lead an extensive study to understand current transcript and transfer credit nomenclature practices in Canada. These findings will ultimately inform a comprehensive update and expansion of the 2003 ARUCC National Transcript Guide and potentially result in a searchable database of transcript practices and Canadian transfer credit nomenclature. The ultimate goal is to enhance the clarity, consistency and transparency of the academic transcript and transfer credit resources that support student mobility. The specific deliverable for this phase was to identify and summarize Canadian transcript and transfer credit nomenclature practices, review four international jurisdictions as a means to highlight promising practices related to these two areas and, finally, to provide both an overview of systems and an initial examination of emergent perspectives and themes. The report purposefully avoids suggesting prescriptive solutions or outcomes; however, the findings from this study will provide a solid foundation from which to move forward the standards and terminology discourse in Canada. This report collates the findings from the supporting research conducted from January through to April 2014.
About Ontario Centres of Excellence Inc.
The Ontario Centres of Excellence (OCE) not-for-profit program was formally established in 1987 with seven independent centres that evolved and amalgamated into Ontario Centres of Excellence Inc. in 2004.
Twenty-five years ago, the traditional economic foundation for the province, and for Canada, was shifting from a north american focused and commodities-based economy to one that is globally oriented and knowledge-based.
Prior to the creation of OCE, there was limited connection between universities, colleges, research hospitals and industry. Consensus was that these academic and research institutions were producing quality research that was not being utilized to its full potential by industry.
OCE was designed to bridge that gap and create productive working partnerships between university and college
research departments, research hospitals and Ontario industry.
In 2013, the national economy began to recover more earnestly. Some states even increased funding for higher education, although not by much.1 Performance-based funding, greater accountability, student completion rates and gainful employment became the often-heard buzz words of 2013. Not to be out done, most distance education programs are pressured to find ways to close the student achievement gap many online programs still experience as compared to face-to-face courses, or risk
seeing further budget and staff reductions. As the authors of the ITC survey have suggested for the past several years, the Great Recession has forced many states to undergo a paradigm shift in how they will make funding decisions for colleges and
universities in the years to come.
College prices have increased by 45 percemt on average over the past decade, while household income has declinded by 7 percent in the same period.
MyLivePD is a completely new model of
PD that focuses on delivering timely,
relevant and actionable coaching for math teachers through live online sessions with no appointments needed. For the first time, math teachers can connect with an experienced coach to ask a specific question about their teaching on their own schedule from any Internet-enabled computer. The service was created to be
completely driven by the teacher. It is also meant to be a continuous process where teachers get help throughout the school year. This level of personalization and privacy does not exist in any other PD model.
MyLivePD was implemented in three
districts and several Teach for America
regions in the fall of 2011. All districts and teachers have been granted access to the service through December 2012. This paper will provide further details on how MyLivePD works, the initial results from the pilot program to date and conclusions on how MyLivePD can be used by schools as part of their PD
programs.
While Scott Jeffrey, PhD, was getting his doctorate at the University of Chicago, he investigated which rewards would be the most effective in getting University staff members to improve speed and accuracyiii in the University’s incentive lab. In a controlled study he tested hard cold cash against a variety of non-monetary rewards, such as massages and tangible rewards. He used only a verbal “thank you” for the control group.
Every developed country is racing to keep up with profound and fundamental changes in the 21st century The new knowledge economy is creating unprecedented demands for higher levels of expertise and skills, while, at the same time, changing demographics will significantly reduce the numbers of qualified people available in the economy The cumulative impact presents great opportunities and great challenges to Ontario The province has an opportunity to implement meaningful and transformational changes that exploit the potential for growth in the new economy and drive Ontario’s prosperity to
unprecedented levels
But the threats to Ontario’s future are just as great Failing to move forward now with significant
measures could leave Ontario unprepared for the challenges ahead, and strand thousands of
people as permanently unemployable
In the emerging knowledge-based economy, employers are requiring new levels of skill from labour market entrants. As employers’ expectations of postsecondary graduates increase, Ontario’s publicly funded colleges and universities are working to provide students with much of the knowledge, skills, and training needed for success in the community and in the changing workplace. As a result, there has been a movement within the postsecondary education (PSE) sector to provide a closer integration of learning and work as a strategy for workforce skills development (Fisher, Rubenson, Jones, & Shanahan, 2009).
In particular, work-integrated learning (WIL) programs such as co-operative education, internship, and apprenticeship are frequently endorsed as educational modes of delivery to support such integration.
Offering work-integrated learning experiences for students requires a significant investment of human and financial resources to be effective. Faculty in particular play an important role in designing, supporting, and implementing WIL opportunities for students. Despite a growing recognition of the essential role played by faculty, very little is known about their perceptions of and experiences with WIL. To shed light on this issue, this report provides the results of the WIL Faculty Survey conducted by the Higher Education Quality Council of Ontario (HEQCO) in partnership with 13 Ontario postsecondary institutions.
The report is part of a broader multi-phase project being undertaken by HEQCO on WIL in Ontario’s PSE
sector.
The WIL Faculty Survey was designed to better understand faculty experiences with and perceptions of WIL as an element of postsecondary curriculum. Guided by a Working Group comprised of representatives from the 13 participating postsecondary institutions, the study sought to address four primary research questions:
1) How do faculty perceive the value and benefits of WIL to students, faculty members, and
postsecondary institutions?
2) Do faculty views about WIL differ by employment status, program, gender, years of teaching, previous employment experience, or their own past WIL experience?
3) How do faculty integrate students’ work experiences into the classroom?
4) What concerns do faculty have about introducing or expanding WIL opportunities in postsecondary institutions?
The survey instrument was developed in consultation with the Working Group and was pre-tested with 25 faculty members. The survey was administered online from March to May, 2011, with e-mail invitations to participate sent to 18,232 faculty from the 13 partner institutions (6,257 college faculty and 11,975 university faculty). In total, 1,707 college faculty and 1,917 university faculty completed the survey to an acceptable cut-off point, for an overall response rate of 19.9%.
Close to two-thirds of college faculty and roughly half of university faculty respondents reported having experience teaching in a program in which students participate in a co-op or apprenticeship. Fewer faculty had experience personally teaching a course with a WIL component, with 47.5% of college faculty and 28.9% of university faculty currently or previously having taught a course involving WIL. Among those who had taught a course with a WIL component, field placements were the most common type of WIL among college faculty, followed by mandatory professional practice (student placements required for licensure or professional designation). For university respondents, mandatory professional practice was the most common type of WIL taught, followed by applied research projects.
This report outlines a series of recommendations for the post-secondary sector arising from a research study carried out by researchers from Queen’s University and St. Lawrence College. Funding for this 30-month project, which began in January 2013, was provided by the Ontario Ministry of Training, Colleges and Universities under the Mental Health Innovation Fund. In the fall of 2012, each post-secondary institution in Ontario was invited to submit proposals for funding and this project was one of ten successful applications.
Between 1991 and 2011, the proportion of employed people aged 25 to 34 with a university degree rose from 19% to 40% among women, and from 17% to 27% among men. Given the increase in the proportion of university graduates, did the occupational profile of young workers change over the period? This article examines long-term changes in the occupation profiles of young men and women, for both those who did and did not have a university degree. Changes in the share of women employed in these occupations are also examined.
Check the backpack of any higher education student and you’re likely to find a smartphone. The handy handheld tool has long been a favorite of on-the-go college kids to remain on task through the use of calendaring; up-to-date with e-mail and Internet access; and ‘in the know’ by way of social media, IM and text messaging.
Mobile computing is mainstream. But despite its ubiquity in the personal lives of students — and the efficiencies it brings — mobile computing has not been utilized by the higher education community to enhance student learning and deliver content and resources with greater efficiency. Until now.
Identified as the No. 1 technology to watch for out of more than 110 technologies considered, the Horizon Report predicts that mobile computing will enter mainstream use for teaching and learning within the next 12 months. The set of teaching and learning activities that are well-suited to mobile devices continues to evolve rapidly as mobile devices and networks improve, educators and instructional designers develop innovative uses for those devices and networks as applied to education, and courses and curriculum are redesigned to take advantage of mobile computing as a delivery medium for blended and online programs.
Business programs in particular are poised to take advantage of the benefits mobile computing has to offer, with the following uses becoming commonplace in undergraduate business concentrations and MBA programs:
• Course registration and scheduling
Students can register for courses via mobile devices and view class schedules and calendars once enrolled. In addition, mobile devices provide the perfect platform for communicating last-minute changes to meeting times or places, as well as accessing other timely alerts.
• Access to assignments and course materials Students can access course content via learning management
systems, cloud computing solutions and shared portals.
Information and data can be uploaded, downloaded and revised.
• Collaboration on group projects
Group work is a substantial and critical component of business
school curriculum, and mobile computing enables teams of students to communicate and collaborate on projects across space and time.
• In-class polling
Some mobile device platforms are capable of running applications to support in-class polling, effectively eliminating the
need for standalone clicker systems in lecture halls.
The Council of Europe (since the mid-1960s), the European Commission (since the late 1980s) and many European states and civil society organisations (in the aftermath of the Second World War) have long fostered programmes and strategies to enhance the mobility of young people.
The prevailing notion of such programmes is that the process of economic and political integration in Europe will indeed remain fragmentary and unstable without accompanying social and educational measures. Instead of a Europe with non-transparent
bureaucratic institutions, a “Europe of Citizens” was meant to develop wherein people would get to know each other,
appreciate their mutual cultural differences and, at the same time, form a European identity by saying “yes” to core European values. As such, mobility is considered important for the personal development of young people, contributing as it does to their employability and thus their social inclusion.
Public colleges are the only academic institutions in Canada that deliver a robust range of career-focused programs and training to all segments of the population. The colleges’ labour-market programs, such as Second Career, employment counselling, academic upgrading and apprenticeship training serve more than 160,000 students
each year.
Ontario’s public college programs are affordable and reach students in all socioeconomic groups – from people who need upgrading in order to qualify for full-time college programs, to university graduates seeking marketable skills.
Graduates of Ontario’s 24 public colleges earn credentials that have met the province’s rigorous standards for post-secondary education and are valued by employers. College graduates continue to be in high demand.
In 2011, as part of a comprehensive research agenda on learning outcomes development and measurement, the Higher Education Quality Council of Ontario (HEQCO) began supporting eight Ontario institutions to assess the generic skills acquisition of their students. This report summarizes the activities and results of the eight institutions that piloted the Council for Aid to Education’s Collegiate Learning Assessment (CLA), a written examination designed to assess the critical thinking and problem solving skills of entering and graduating students. It reviews the rationale for the project, the challenges and issues encountered with CLA test administration and implementation, and the institutions’ impressions of the value of the resulting data. While there is significant interest from institutions and programs in measuring the generic skills of students and understanding the amount of learning that can be attributed to the institution, the experiences of the institutions that participated in this project highlight certain administrative and methodological challenges that arise in the move from theory to practice in large scale assessments.
Over the past few decades, Canada’s labour requirements have changed drastically—from a need for physical labourers to a need for knowledge workers—as a result of changes in economic and social conditions that have included advances in information and communication technologies, globalization of economic activity and shifting demographics. Consequently, employers and firms are increasingly seeking skilled workers with a more sophisticated array of capabilities. Of recent concern, the current global recession has led to the deterioration of labour-market conditions in Canada and worldwide, profoundly affecting—through increased vulnerability to unemployment—the economic and social well-being of families and communities across Canada. Canada’s economic strength, as in other countries, depends on its ability to develop a skilled and flexible workforce, capable of adapting to continuous change. While Canada’s formal education is of a high standard, it alone cannot provide the conditions needed to secure the development of Canada’s talent—its human infrastructure*—which is a necessary element of our country’s future prosperity. Against this backdrop, Securing Prosperity through Canada’s Human Infrastructure, CCL’s second†report on the state of adult learning and workplace training in Canada, demonstrates that investments in human infrastructure—both in times of economic uncertainty and relative prosperity—are critical to securing a strong economy and greater social equity.
The mission of America’s community colleges is focused on three areas of commitment: access, responsiveness to community need, and equity. The commitment to access is exemplified by the open admissions policies of community colleges and the multiple ways colleges remove financial, physical, and academic barriers to entry. That access has resulted in entry into higher education by first generation, low-income, minorities, dropouts, working adults, and others who lacked the financial, academic, time, or location means to participate in traditional higher education systems. Looking at multiple college mission statements as well as scholarly definitions, the essential core of agreement is that community colleges provide access to the education necessary for both a productive life for individuals and healthy and successful communities they serve. Access to higher education is essential to a democratic society and a strong middle class.