This document represents the first review and summary of existing research on student course evaluations from a Canadian perspective. The scholarship in this area is vast and of varying quality and scope. Our review is an attempt to capture and synthesize the key issues and findings regarding the validity and utility of student course evaluations. We have organized our research into the following seven sections:
Section 1: Introduction provides an overview of the scope, methodology and limitations of this study.
Section 2: Context identifies the current state of scholarship and interest in course evaluations and the evaluation of teaching more generally. It also reviews student, faculty and administrator perceptions of course evaluation systems.
Section 3: Current Policy and Practice in North America offers an overview of evaluation instruments, policies and processes from 22 post-secondary institutions in Canada and the United States as well as policies related to course evaluations from system-level and government agencies.
Section 4: Reliability, Validity and Interpretation of Course Evaluation Data summarizes and reviews the findings from previous studies conducted over the past 40 years with a particular emphasis on the last two decades.
Section 5: Implementing Effective Evaluation Measures: Recommendations from the Research synthesizes research findings and identifies recommendations for improved administration and interpretation of course evaluations.
Section 6: Emerging Trends, Existing Gaps and Suggestions for Further Research highlights issues currently being considered in the scholarship along with those that have been identified as areas requiring more in-depth analysis.
Section 7: Concluding Remarks provides a brief summary of our most important findings and recommendations.
Overall, our findings indicate that while course evaluation instruments generally provide reliable and valid data, significant barriers to the effective use of such evaluation systems continue to exist due to: Persistent myths and misconceptions about variables affecting evaluation results; Unclear concepts and definitions of effective teaching; Student Course Evaluations: Research, Models and Trends Insufficient education about the goals, uses and validity of course evaluations for students, faculty and administrators; Poor presentation and contextualization of evaluation data; and Inconsistent and inequitable policies and practices regarding the implementation and administration of course evaluations.
Our findings suggest that no matter the reliability and validity of the evaluation instruments themselves, the policies, processes and practices at an institution determine the degree to which evaluations are an effective measure of teaching quality.
The Association of Registrars of the Universities and Colleges of Canada (ARUCC) and the Pan-Canadian Consortium on Admissions and Transfer (PCCAT) have collaborated to lead an extensive study to understand current transcript and transfer credit nomenclature practices in Canada. These findings will ultimately inform a comprehensive update and expansion of the 2003 ARUCC National Transcript Guide and potentially result in a searchable database of transcript practices and Canadian transfer credit nomenclature. The ultimate goal is to enhance the clarity, consistency and transparency of the academic transcript and transfer credit resources that support student mobility. The specific deliverable for this phase was to identify and summarize Canadian transcript and transfer credit nomenclature practices, review four international jurisdictions as a means to highlight promising practices related to these two areas and, finally, to provide both an overview of systems and an initial examination of emergent perspectives and themes. The report purposefully avoids suggesting prescriptive solutions or outcomes; however, the findings from this study will provide a solid foundation from which to move forward the standards and terminology discourse in Canada. This report collates the findings from the supporting research conducted from January through to April 2014.
While Scott Jeffrey, PhD, was getting his doctorate at the University of Chicago, he investigated which rewards would be the most effective in getting University staff members to improve speed and accuracyiii in the University’s incentive lab. In a controlled study he tested hard cold cash against a variety of non-monetary rewards, such as massages and tangible rewards. He used only a verbal “thank you” for the control group.
In the emerging knowledge-based economy, employers are requiring new levels of skill from labour market entrants. As employers’ expectations of postsecondary graduates increase, Ontario’s publicly funded colleges and universities are working to provide students with much of the knowledge, skills, and training needed for success in the community and in the changing workplace. As a result, there has been a movement within the postsecondary education (PSE) sector to provide a closer integration of learning and work as a strategy for workforce skills development (Fisher, Rubenson, Jones, & Shanahan, 2009).
In particular, work-integrated learning (WIL) programs such as co-operative education, internship, and apprenticeship are frequently endorsed as educational modes of delivery to support such integration.
Offering work-integrated learning experiences for students requires a significant investment of human and financial resources to be effective. Faculty in particular play an important role in designing, supporting, and implementing WIL opportunities for students. Despite a growing recognition of the essential role played by faculty, very little is known about their perceptions of and experiences with WIL. To shed light on this issue, this report provides the results of the WIL Faculty Survey conducted by the Higher Education Quality Council of Ontario (HEQCO) in partnership with 13 Ontario postsecondary institutions.
The report is part of a broader multi-phase project being undertaken by HEQCO on WIL in Ontario’s PSE
sector.
The WIL Faculty Survey was designed to better understand faculty experiences with and perceptions of WIL as an element of postsecondary curriculum. Guided by a Working Group comprised of representatives from the 13 participating postsecondary institutions, the study sought to address four primary research questions:
1) How do faculty perceive the value and benefits of WIL to students, faculty members, and
postsecondary institutions?
2) Do faculty views about WIL differ by employment status, program, gender, years of teaching, previous employment experience, or their own past WIL experience?
3) How do faculty integrate students’ work experiences into the classroom?
4) What concerns do faculty have about introducing or expanding WIL opportunities in postsecondary institutions?
The survey instrument was developed in consultation with the Working Group and was pre-tested with 25 faculty members. The survey was administered online from March to May, 2011, with e-mail invitations to participate sent to 18,232 faculty from the 13 partner institutions (6,257 college faculty and 11,975 university faculty). In total, 1,707 college faculty and 1,917 university faculty completed the survey to an acceptable cut-off point, for an overall response rate of 19.9%.
Close to two-thirds of college faculty and roughly half of university faculty respondents reported having experience teaching in a program in which students participate in a co-op or apprenticeship. Fewer faculty had experience personally teaching a course with a WIL component, with 47.5% of college faculty and 28.9% of university faculty currently or previously having taught a course involving WIL. Among those who had taught a course with a WIL component, field placements were the most common type of WIL among college faculty, followed by mandatory professional practice (student placements required for licensure or professional designation). For university respondents, mandatory professional practice was the most common type of WIL taught, followed by applied research projects.
This 2014 mba.com Prospective Students Survey Report explores the motivations, behaviors, program choices, and intended
career outcomes shared by more than 12,000 individuals who registered on mba.com from October 2012 through September
2013. Survey data collected in 2013 are compared with earlier data collected from more than 71,000 prospective business
school students who have responded to our mba.com registrants’ surveys over the past four years. With survey responses
available for all world regions as well as 15 specific countries, this is the largest data resource of its kind.
MyLivePD is a completely new model of
PD that focuses on delivering timely,
relevant and actionable coaching for math teachers through live online sessions with no appointments needed. For the first time, math teachers can connect with an experienced coach to ask a specific question about their teaching on their own schedule from any Internet-enabled computer. The service was created to be
completely driven by the teacher. It is also meant to be a continuous process where teachers get help throughout the school year. This level of personalization and privacy does not exist in any other PD model.
MyLivePD was implemented in three
districts and several Teach for America
regions in the fall of 2011. All districts and teachers have been granted access to the service through December 2012. This paper will provide further details on how MyLivePD works, the initial results from the pilot program to date and conclusions on how MyLivePD can be used by schools as part of their PD
programs.
In 2013, the national economy began to recover more earnestly. Some states even increased funding for higher education, although not by much.1 Performance-based funding, greater accountability, student completion rates and gainful employment became the often-heard buzz words of 2013. Not to be out done, most distance education programs are pressured to find ways to close the student achievement gap many online programs still experience as compared to face-to-face courses, or risk
seeing further budget and staff reductions. As the authors of the ITC survey have suggested for the past several years, the Great Recession has forced many states to undergo a paradigm shift in how they will make funding decisions for colleges and
universities in the years to come.
Check the backpack of any higher education student and you’re likely to find a smartphone. The handy handheld tool has long been a favorite of on-the-go college kids to remain on task through the use of calendaring; up-to-date with e-mail and Internet access; and ‘in the know’ by way of social media, IM and text messaging.
Mobile computing is mainstream. But despite its ubiquity in the personal lives of students — and the efficiencies it brings — mobile computing has not been utilized by the higher education community to enhance student learning and deliver content and resources with greater efficiency. Until now.
Identified as the No. 1 technology to watch for out of more than 110 technologies considered, the Horizon Report predicts that mobile computing will enter mainstream use for teaching and learning within the next 12 months. The set of teaching and learning activities that are well-suited to mobile devices continues to evolve rapidly as mobile devices and networks improve, educators and instructional designers develop innovative uses for those devices and networks as applied to education, and courses and curriculum are redesigned to take advantage of mobile computing as a delivery medium for blended and online programs.
Business programs in particular are poised to take advantage of the benefits mobile computing has to offer, with the following uses becoming commonplace in undergraduate business concentrations and MBA programs:
• Course registration and scheduling
Students can register for courses via mobile devices and view class schedules and calendars once enrolled. In addition, mobile devices provide the perfect platform for communicating last-minute changes to meeting times or places, as well as accessing other timely alerts.
• Access to assignments and course materials Students can access course content via learning management
systems, cloud computing solutions and shared portals.
Information and data can be uploaded, downloaded and revised.
• Collaboration on group projects
Group work is a substantial and critical component of business
school curriculum, and mobile computing enables teams of students to communicate and collaborate on projects across space and time.
• In-class polling
Some mobile device platforms are capable of running applications to support in-class polling, effectively eliminating the
need for standalone clicker systems in lecture halls.
In 2011, as part of a comprehensive research agenda on learning outcomes development and measurement, the Higher Education Quality Council of Ontario (HEQCO) began supporting eight Ontario institutions to assess the generic skills acquisition of their students. This report summarizes the activities and results of the eight institutions that piloted the Council for Aid to Education’s Collegiate Learning Assessment (CLA), a written examination designed to assess the critical thinking and problem solving skills of entering and graduating students. It reviews the rationale for the project, the challenges and issues encountered with CLA test administration and implementation, and the institutions’ impressions of the value of the resulting data. While there is significant interest from institutions and programs in measuring the generic skills of students and understanding the amount of learning that can be attributed to the institution, the experiences of the institutions that participated in this project highlight certain administrative and methodological challenges that arise in the move from theory to practice in large scale assessments.
The 2015 Campus Freedom Index is the fifth annual report released by the Justice Centre for Constitutional Freedoms (JCCF) to measure the state of free speech at Canada’s universities.
Starting with a survey of only 18 universities in 2011, this year’s edition has grown to include 55 publicly funded Canadian universities—the largest and most expansive Index released so far, with information relevant to the more than 750,000 students who attend these institutions. The 2015 Campus Freedom Index includes an individual report about each university and student union.
Every developed country is racing to keep up with profound and fundamental changes in the 21st century The new knowledge economy is creating unprecedented demands for higher levels of expertise and skills, while, at the same time, changing demographics will significantly reduce the numbers of qualified people available in the economy The cumulative impact presents great opportunities and great challenges to Ontario The province has an opportunity to implement meaningful and transformational changes that exploit the potential for growth in the new economy and drive Ontario’s prosperity to
unprecedented levels
But the threats to Ontario’s future are just as great Failing to move forward now with significant
measures could leave Ontario unprepared for the challenges ahead, and strand thousands of
people as permanently unemployable
Between 1991 and 2011, the proportion of employed people aged 25 to 34 with a university degree rose from 19% to 40% among women, and from 17% to 27% among men. Given the increase in the proportion of university graduates, did the occupational profile of young workers change over the period? This article examines long-term changes in the occupation profiles of young men and women, for both those who did and did not have a university degree. Changes in the share of women employed in these occupations are also examined.
About Ontario Centres of Excellence Inc.
The Ontario Centres of Excellence (OCE) not-for-profit program was formally established in 1987 with seven independent centres that evolved and amalgamated into Ontario Centres of Excellence Inc. in 2004.
Twenty-five years ago, the traditional economic foundation for the province, and for Canada, was shifting from a north american focused and commodities-based economy to one that is globally oriented and knowledge-based.
Prior to the creation of OCE, there was limited connection between universities, colleges, research hospitals and industry. Consensus was that these academic and research institutions were producing quality research that was not being utilized to its full potential by industry.
OCE was designed to bridge that gap and create productive working partnerships between university and college
research departments, research hospitals and Ontario industry.
The mission of America’s community colleges is focused on three areas of commitment: access, responsiveness to community need, and equity. The commitment to access is exemplified by the open admissions policies of community colleges and the multiple ways colleges remove financial, physical, and academic barriers to entry. That access has resulted in entry into higher education by first generation, low-income, minorities, dropouts, working adults, and others who lacked the financial, academic, time, or location means to participate in traditional higher education systems. Looking at multiple college mission statements as well as scholarly definitions, the essential core of agreement is that community colleges provide access to the education necessary for both a productive life for individuals and healthy and successful communities they serve. Access to higher education is essential to a democratic society and a strong middle class.
Public colleges are the only academic institutions in Canada that deliver a robust range of career-focused programs and training to all segments of the population. The colleges’ labour-market programs, such as Second Career, employment counselling, academic upgrading and apprenticeship training serve more than 160,000 students
each year.
Ontario’s public college programs are affordable and reach students in all socioeconomic groups – from people who need upgrading in order to qualify for full-time college programs, to university graduates seeking marketable skills.
Graduates of Ontario’s 24 public colleges earn credentials that have met the province’s rigorous standards for post-secondary education and are valued by employers. College graduates continue to be in high demand.
College prices have increased by 45 percemt on average over the past decade, while household income has declinded by 7 percent in the same period.
For many years now, people have been touting the arrival of the “digital native,” or students that were “born digital”. These terms were meant to describe members of a generation who, according to the more fevered sections of the technorati at least, actually have a different set of neural pathways – who, having been exposed since birth to the Internet and hypertext, “think and process information differently” from previous generations.1 In some quarters this has led to calls – on the basis of evidence that can sometimes be alarmingly thin – that curricula and instructional technologies be radically overhauled in order to cater to the “new learner.”
At the same time, much has been made about the quality-enhancing – and cost-reducing – potential of using the Internet for learning purposes in universities. The National Center for Academic Transformation in the United States, in particular, has been a leading voice in using course redesign as a means to improve both learning outcomes and resource allocation.2 This has not really been about moving whole courses online – the “disruptive technology” that some commentators suggest is about to change universities completely3 – but rather it has been about deploying e-learning resources in such a way as to complement and amplify what is being done in more traditional courses. The entwining of these kinds of resources into courses that
remain primarily physical and class-based is commonly referred to as “blended learning.”
Growing enrollments, shrinking budgets and unprecedented diversity in student populations are just a few of the challenges community colleges around the country are facing today. And there are no signs that the situation will change anytime soon.
The American Association of Community Colleges estimates that U.S. enrollment in two-year colleges increased 17 percent from 2007 to 2009, from 6.8 million students to 8 million. Anecdotal evidence says this trend will continue.
During an economic downturn, community colleges feel an even greater strain with enrollment. People go back to school to learn new skills or get certificates or degrees that help their careers. Many must learn new jobs because their previous ones have gone away. While it’s good to have more students, the growth has been so rapid that it has put pressure on the institutions. How do they handle more students every semester? How do they grow despite less funding from federal, state and county governments?
“Because community colleges are growing so fast, and because they’re relatively new as institutions, they don’t have
the infrastructure that the big universities have. And yet they are being asked to do more,” said John Halpin, Vice President of SLED Strategy and Programs at the Center for Digital Education (CDE), a national research and advisory institute focused on IT
policy and best practices in education.
A New Course Community colleges now have a terrific opportunity to evolve thanks to technology, Halpin said. Numerous technologies — wireless, broadband, cloud computing and others — have greatly matured in recent years. They’ve been proven in the real world, and they’ve become more efficient and less expensive.
At community colleges, whether it’s for teaching and learning or for financial aid or other back-end systems, technology is making a huge impact on productivity. Students are learning in exciting new ways. E-mentoring, e-advising, online tutoring and even educational gaming are effectively engaging students and enhancing the educational experience. Professors are incorporating audio/video content to deliver learning in a manner that grabs the student’s interest. Schools are processing incoming students more efficiently and less expensively by putting administrative functions, such as application, orientation and registration, online.
Online learning, or e-learning, is booming. “Students value distance learning,” said Wilton Agatstein, Senior Fellow with the CDE. “It is very convenient for them, as they can learn from any place and at any time. Schools value distance learning because they can serve more students and a larger student demographic without having to build new classrooms and campuses. Distance learning serves everyone well, which is why its adoption is accelerating.”
Technology expectations are sky high. Students step onto campus expecting to incorporate their own communications tools — phones, music players, e-book readers, laptops/netbooks and other devices — into the learning experience. They want wireless access from any point on campus. And they want the ability to connect to school resources even when off campus.
Teachers and staff want the best technology too, because the right tools help everyone.
Over the past few decades, Canada’s labour requirements have changed drastically—from a need for physical labourers to a need for knowledge workers—as a result of changes in economic and social conditions that have included advances in information and communication technologies, globalization of economic activity and shifting demographics. Consequently, employers and firms are increasingly seeking skilled workers with a more sophisticated array of capabilities. Of recent concern, the current global recession has led to the deterioration of labour-market conditions in Canada and worldwide, profoundly affecting—through increased vulnerability to unemployment—the economic and social well-being of families and communities across Canada. Canada’s economic strength, as in other countries, depends on its ability to develop a skilled and flexible workforce, capable of adapting to continuous change. While Canada’s formal education is of a high standard, it alone cannot provide the conditions needed to secure the development of Canada’s talent—its human infrastructure*—which is a necessary element of our country’s future prosperity. Against this backdrop, Securing Prosperity through Canada’s Human Infrastructure, CCL’s second†report on the state of adult learning and workplace training in Canada, demonstrates that investments in human infrastructure—both in times of economic uncertainty and relative prosperity—are critical to securing a strong economy and greater social equity.
The idea of “productivity” in higher education is becoming a concern for some policymakers and observers of Ontario’s universities. This interest is fuelled by the province’s challenging deficit situation, which has put a premium on “doing more with less”. Productivity is featured in the Government of Ontario’s recent discussion paper, Strengthening Ontario’s Centres of Creativity, Innovation, and Knowledge, and was a prominent focus of the Ministry of Training, Colleges, and Universities
strategic mandate agreement process.