This document represents the first review and summary of existing research on student course evaluations from a Canadian perspective. The scholarship in this area is vast and of varying quality and scope. Our review is an attempt to capture and synthesize the key issues and findings regarding the validity and utility of student course evaluations. We have organized our research into the following seven sections:
Section 1: Introduction provides an overview of the scope, methodology and limitations of this study.
Section 2: Context identifies the current state of scholarship and interest in course evaluations and the evaluation of teaching more generally. It also reviews student, faculty and administrator perceptions of course evaluation systems.
Section 3: Current Policy and Practice in North America offers an overview of evaluation instruments, policies and processes from 22 post-secondary institutions in Canada and the United States as well as policies related to course evaluations from system-level and government agencies.
Section 4: Reliability, Validity and Interpretation of Course Evaluation Data summarizes and reviews the findings from previous studies conducted over the past 40 years with a particular emphasis on the last two decades.
Section 5: Implementing Effective Evaluation Measures: Recommendations from the Research synthesizes research findings and identifies recommendations for improved administration and interpretation of course evaluations.
Section 6: Emerging Trends, Existing Gaps and Suggestions for Further Research highlights issues currently being considered in the scholarship along with those that have been identified as areas requiring more in-depth analysis.
Section 7: Concluding Remarks provides a brief summary of our most important findings and recommendations.
Overall, our findings indicate that while course evaluation instruments generally provide reliable and valid data, significant barriers to the effective use of such evaluation systems continue to exist due to: Persistent myths and misconceptions about variables affecting evaluation results; Unclear concepts and definitions of effective teaching; Student Course Evaluations: Research, Models and Trends Insufficient education about the goals, uses and validity of course evaluations for students, faculty and administrators; Poor presentation and contextualization of evaluation data; and Inconsistent and inequitable policies and practices regarding the implementation and administration of course evaluations.
Our findings suggest that no matter the reliability and validity of the evaluation instruments themselves, the policies, processes and practices at an institution determine the degree to which evaluations are an effective measure of teaching quality.
Nunavut Tunngavik Inc.’s (NTI) 2010/11 Annual Report on the State of Inuit Culture and Society focuses on the status of Inuit children and youth in Nunavut, with a focus on ages 2 to 18. In 2008, NTI reported on the overall health of Inuit, with
an emphasis on health service availability and delivery, and in many ways this report complements that document by focusing on the concept of wellness as it applies to Inuit children and youth, and the specific opportunities, challenges and priority areas associated with this rapidly growing demographic. Young people make up a larger proportion of Nunavut’s population than in any other Canadian jurisdiction (see Figure 1).
Children and youth are the most vulnerable people in society, relying on parents, guardians, and extended family members for food, shelter, nurturing, support, and protection. Factors impacting the well-being of Inuit children and youth, such as the availability of nutritious foods and reliable child, youth, and family services, adequate housing, and quality, early childhood and kindergarten to Grade 12 (K-12) education are beyond their influence or control. The high incidence of violent crime, sexual assault, and substance abuse in Nunavut can compound these challenges, making sustained political advocacy for this population all the more urgent.
About Ontario Centres of Excellence Inc.
The Ontario Centres of Excellence (OCE) not-for-profit program was formally established in 1987 with seven independent centres that evolved and amalgamated into Ontario Centres of Excellence Inc. in 2004.
Twenty-five years ago, the traditional economic foundation for the province, and for Canada, was shifting from a north american focused and commodities-based economy to one that is globally oriented and knowledge-based.
Prior to the creation of OCE, there was limited connection between universities, colleges, research hospitals and industry. Consensus was that these academic and research institutions were producing quality research that was not being utilized to its full potential by industry.
OCE was designed to bridge that gap and create productive working partnerships between university and college
research departments, research hospitals and Ontario industry.
While Scott Jeffrey, PhD, was getting his doctorate at the University of Chicago, he investigated which rewards would be the most effective in getting University staff members to improve speed and accuracyiii in the University’s incentive lab. In a controlled study he tested hard cold cash against a variety of non-monetary rewards, such as massages and tangible rewards. He used only a verbal “thank you” for the control group.
MyLivePD is a completely new model of
PD that focuses on delivering timely,
relevant and actionable coaching for math teachers through live online sessions with no appointments needed. For the first time, math teachers can connect with an experienced coach to ask a specific question about their teaching on their own schedule from any Internet-enabled computer. The service was created to be
completely driven by the teacher. It is also meant to be a continuous process where teachers get help throughout the school year. This level of personalization and privacy does not exist in any other PD model.
MyLivePD was implemented in three
districts and several Teach for America
regions in the fall of 2011. All districts and teachers have been granted access to the service through December 2012. This paper will provide further details on how MyLivePD works, the initial results from the pilot program to date and conclusions on how MyLivePD can be used by schools as part of their PD
programs.
The Canadian labour market suffered a severe blow during the last recession, with more than 430,000 persons losing their jobs and the unemployment rate reaching levels unseen since the latter half of the 1990s.
Subsequently, the labour market has shown great resilience, and there are now 900,000 more Canadians employed since the beginning of the recovery. Important weaknesses remain, however: long-term and youth unemploymentstill stand at obstinately high levels – despite a recent growth in job vacancies.
This E-Brief argues the best way to further support the Canadian labour market would be through policies that enhance labour mobility and emphasize skills training to help ensure unemployed Canadians have the right skill sets to
integrate into the workforce.
Check the backpack of any higher education student and you’re likely to find a smartphone. The handy handheld tool has long been a favorite of on-the-go college kids to remain on task through the use of calendaring; up-to-date with e-mail and Internet access; and ‘in the know’ by way of social media, IM and text messaging.
Mobile computing is mainstream. But despite its ubiquity in the personal lives of students — and the efficiencies it brings — mobile computing has not been utilized by the higher education community to enhance student learning and deliver content and resources with greater efficiency. Until now.
Identified as the No. 1 technology to watch for out of more than 110 technologies considered, the Horizon Report predicts that mobile computing will enter mainstream use for teaching and learning within the next 12 months. The set of teaching and learning activities that are well-suited to mobile devices continues to evolve rapidly as mobile devices and networks improve, educators and instructional designers develop innovative uses for those devices and networks as applied to education, and courses and curriculum are redesigned to take advantage of mobile computing as a delivery medium for blended and online programs.
Business programs in particular are poised to take advantage of the benefits mobile computing has to offer, with the following uses becoming commonplace in undergraduate business concentrations and MBA programs:
• Course registration and scheduling
Students can register for courses via mobile devices and view class schedules and calendars once enrolled. In addition, mobile devices provide the perfect platform for communicating last-minute changes to meeting times or places, as well as accessing other timely alerts.
• Access to assignments and course materials Students can access course content via learning management
systems, cloud computing solutions and shared portals.
Information and data can be uploaded, downloaded and revised.
• Collaboration on group projects
Group work is a substantial and critical component of business
school curriculum, and mobile computing enables teams of students to communicate and collaborate on projects across space and time.
• In-class polling
Some mobile device platforms are capable of running applications to support in-class polling, effectively eliminating the
need for standalone clicker systems in lecture halls.
Between 1991 and 2011, the proportion of employed people aged 25 to 34 with a university degree rose from 19% to 40% among women, and from 17% to 27% among men. Given the increase in the proportion of university graduates, did the occupational profile of young workers change over the period? This article examines long-term changes in the occupation profiles of young men and women, for both those who did and did not have a university degree. Changes in the share of women employed in these occupations are also examined.
Public colleges are the only academic institutions in Canada that deliver a robust range of career-focused programs and training to all segments of the population. The colleges’ labour-market programs, such as Second Career, employment counselling, academic upgrading and apprenticeship training serve more than 160,000 students
each year.
Ontario’s public college programs are affordable and reach students in all socioeconomic groups – from people who need upgrading in order to qualify for full-time college programs, to university graduates seeking marketable skills.
Graduates of Ontario’s 24 public colleges earn credentials that have met the province’s rigorous standards for post-secondary education and are valued by employers. College graduates continue to be in high demand.
In 2011, as part of a comprehensive research agenda on learning outcomes development and measurement, the Higher Education Quality Council of Ontario (HEQCO) began supporting eight Ontario institutions to assess the generic skills acquisition of their students. This report summarizes the activities and results of the eight institutions that piloted the Council for Aid to Education’s Collegiate Learning Assessment (CLA), a written examination designed to assess the critical thinking and problem solving skills of entering and graduating students. It reviews the rationale for the project, the challenges and issues encountered with CLA test administration and implementation, and the institutions’ impressions of the value of the resulting data. While there is significant interest from institutions and programs in measuring the generic skills of students and understanding the amount of learning that can be attributed to the institution, the experiences of the institutions that participated in this project highlight certain administrative and methodological challenges that arise in the move from theory to practice in large scale assessments.
The 2015 Campus Freedom Index is the fifth annual report released by the Justice Centre for Constitutional Freedoms (JCCF) to measure the state of free speech at Canada’s universities.
Starting with a survey of only 18 universities in 2011, this year’s edition has grown to include 55 publicly funded Canadian universities—the largest and most expansive Index released so far, with information relevant to the more than 750,000 students who attend these institutions. The 2015 Campus Freedom Index includes an individual report about each university and student union.
For many years now, people have been touting the arrival of the “digital native,” or students that were “born digital”. These terms were meant to describe members of a generation who, according to the more fevered sections of the technorati at least, actually have a different set of neural pathways – who, having been exposed since birth to the Internet and hypertext, “think and process information differently” from previous generations.1 In some quarters this has led to calls – on the basis of evidence that can sometimes be alarmingly thin – that curricula and instructional technologies be radically overhauled in order to cater to the “new learner.”
At the same time, much has been made about the quality-enhancing – and cost-reducing – potential of using the Internet for learning purposes in universities. The National Center for Academic Transformation in the United States, in particular, has been a leading voice in using course redesign as a means to improve both learning outcomes and resource allocation.2 This has not really been about moving whole courses online – the “disruptive technology” that some commentators suggest is about to change universities completely3 – but rather it has been about deploying e-learning resources in such a way as to complement and amplify what is being done in more traditional courses. The entwining of these kinds of resources into courses that
remain primarily physical and class-based is commonly referred to as “blended learning.”
In 2013, the national economy began to recover more earnestly. Some states even increased funding for higher education, although not by much.1 Performance-based funding, greater accountability, student completion rates and gainful employment became the often-heard buzz words of 2013. Not to be out done, most distance education programs are pressured to find ways to close the student achievement gap many online programs still experience as compared to face-to-face courses, or risk
seeing further budget and staff reductions. As the authors of the ITC survey have suggested for the past several years, the Great Recession has forced many states to undergo a paradigm shift in how they will make funding decisions for colleges and
universities in the years to come.
This 2014 mba.com Prospective Students Survey Report explores the motivations, behaviors, program choices, and intended
career outcomes shared by more than 12,000 individuals who registered on mba.com from October 2012 through September
2013. Survey data collected in 2013 are compared with earlier data collected from more than 71,000 prospective business
school students who have responded to our mba.com registrants’ surveys over the past four years. With survey responses
available for all world regions as well as 15 specific countries, this is the largest data resource of its kind.
Over the past few decades, Canada’s labour requirements have changed drastically—from a need for physical labourers to a need for knowledge workers—as a result of changes in economic and social conditions that have included advances in information and communication technologies, globalization of economic activity and shifting demographics. Consequently, employers and firms are increasingly seeking skilled workers with a more sophisticated array of capabilities. Of recent concern, the current global recession has led to the deterioration of labour-market conditions in Canada and worldwide, profoundly affecting—through increased vulnerability to unemployment—the economic and social well-being of families and communities across Canada. Canada’s economic strength, as in other countries, depends on its ability to develop a skilled and flexible workforce, capable of adapting to continuous change. While Canada’s formal education is of a high standard, it alone cannot provide the conditions needed to secure the development of Canada’s talent—its human infrastructure*—which is a necessary element of our country’s future prosperity. Against this backdrop, Securing Prosperity through Canada’s Human Infrastructure, CCL’s second†report on the state of adult learning and workplace training in Canada, demonstrates that investments in human infrastructure—both in times of economic uncertainty and relative prosperity—are critical to securing a strong economy and greater social equity.
The idea of “productivity” in higher education is becoming a concern for some policymakers and observers of Ontario’s universities. This interest is fuelled by the province’s challenging deficit situation, which has put a premium on “doing more with less”. Productivity is featured in the Government of Ontario’s recent discussion paper, Strengthening Ontario’s Centres of Creativity, Innovation, and Knowledge, and was a prominent focus of the Ministry of Training, Colleges, and Universities
strategic mandate agreement process.
College prices have increased by 45 percemt on average over the past decade, while household income has declinded by 7 percent in the same period.
The mission of America’s community colleges is focused on three areas of commitment: access, responsiveness to community need, and equity. The commitment to access is exemplified by the open admissions policies of community colleges and the multiple ways colleges remove financial, physical, and academic barriers to entry. That access has resulted in entry into higher education by first generation, low-income, minorities, dropouts, working adults, and others who lacked the financial, academic, time, or location means to participate in traditional higher education systems. Looking at multiple college mission statements as well as scholarly definitions, the essential core of agreement is that community colleges provide access to the education necessary for both a productive life for individuals and healthy and successful communities they serve. Access to higher education is essential to a democratic society and a strong middle class.
This report outlines a series of recommendations for the post-secondary sector arising from a research study carried out by researchers from Queen’s University and St. Lawrence College. Funding for this 30-month project, which began in January 2013, was provided by the Ontario Ministry of Training, Colleges and Universities under the Mental Health Innovation Fund. In the fall of 2012, each post-secondary institution in Ontario was invited to submit proposals for funding and this project was one of ten successful applications.
The third stream agenda is a critical strategy in the pursuit of enriched learning, enhancing student employability and much needed revenues. Voices of support of the third stream agenda resonate across political parties, the business community and universities. Academic journals have also reflected a scenario in which the academic community of practice have transformed its rational into ‘can do’ mission statements and strategic policies with a clear focus to source, convert and embed third stream activities. In return, universities seek quarries such as more marketable programmes of studies, committed and commercially
aware academics, improved business interchange and in light of the economic recession and subsequent austere measures, the replenishment of new revenue streams.