This document represents the first review and summary of existing research on student course evaluations from a Canadian perspective. The scholarship in this area is vast and of varying quality and scope. Our review is an attempt to capture and synthesize the key issues and findings regarding the validity and utility of student course evaluations. We have organized our research into the following seven sections:
Section 1: Introduction provides an overview of the scope, methodology and limitations of this study.
Section 2: Context identifies the current state of scholarship and interest in course evaluations and the evaluation of teaching more generally. It also reviews student, faculty and administrator perceptions of course evaluation systems.
Section 3: Current Policy and Practice in North America offers an overview of evaluation instruments, policies and processes from 22 post-secondary institutions in Canada and the United States as well as policies related to course evaluations from system-level and government agencies.
Section 4: Reliability, Validity and Interpretation of Course Evaluation Data summarizes and reviews the findings from previous studies conducted over the past 40 years with a particular emphasis on the last two decades.
Section 5: Implementing Effective Evaluation Measures: Recommendations from the Research synthesizes research findings and identifies recommendations for improved administration and interpretation of course evaluations.
Section 6: Emerging Trends, Existing Gaps and Suggestions for Further Research highlights issues currently being considered in the scholarship along with those that have been identified as areas requiring more in-depth analysis.
Section 7: Concluding Remarks provides a brief summary of our most important findings and recommendations.
Overall, our findings indicate that while course evaluation instruments generally provide reliable and valid data, significant barriers to the effective use of such evaluation systems continue to exist due to: Persistent myths and misconceptions about variables affecting evaluation results; Unclear concepts and definitions of effective teaching; Student Course Evaluations: Research, Models and Trends Insufficient education about the goals, uses and validity of course evaluations for students, faculty and administrators; Poor presentation and contextualization of evaluation data; and Inconsistent and inequitable policies and practices regarding the implementation and administration of course evaluations.
Our findings suggest that no matter the reliability and validity of the evaluation instruments themselves, the policies, processes and practices at an institution determine the degree to which evaluations are an effective measure of teaching quality.
Many higher education institutions use student satisfaction surveys given at the end of a course to measure course and instructor quality. But is that really a true measure of quality? All things being equal, an instructor who teaches a rigorous course will likely score much lower than an instructor whose course is a little less demanding. Then there’s the whole timing of the satisfaction surveys. For the most part, students are simply glad the course is over (even if they liked it) and put little thought or time into completing the survey. Unless of course they know they failed, in which case you will get a detailed assessment of how you are boring, inflexible, out of touch, or otherwise unfit to teach.
No wonder surveys get such a bad rap. If end-of-course evaluations are the only surveys you use, there’s a lot more you can, and should, be doing. Done correctly, surveys can deliver tremendous insight into what’s working, what’s not, and why. This special report features 10articles from Online Classroom, including a three-part and a five-part series that provides stepby-
step guidance on how to use surveys and evaluations to improve online courses, programs, and instruction. You’ll learn when to use surveys, how to design effective survey questions, why it’s important to ensure anonymity, and the advantages and disadvantages of Web-based surveys.
Articles in Online Course Quality Assurance: Using Evaluations and Surveys to Improve Online Teaching and Learning include:
• Online Teaching Fundamentals: What to Evaluate, parts 1-3
• Course and Instructor Evaluation: If It’s So Good, Why Does It Feel So Bad?
• Getting Evaluation Data through Surveys: What to Consider before Getting Started
• Using Surveys to Improve Courses, Programs, and Instruction, parts 1-5
If you’re dedicated to continuous improvement, this special report is loaded with practical advice that will help you create more effective surveys before, during, and after your course ends.
Every developed country is racing to keep up with profound and fundamental changes in the 21st century The new knowledge economy is creating unprecedented demands for higher levels of expertise and skills, while, at the same time, changing demographics will significantly reduce the numbers of qualified people available in the economy The cumulative impact presents great opportunities and great challenges to Ontario The province has an opportunity to implement meaningful and transformational changes that exploit the potential for growth in the new economy and drive Ontario’s prosperity to
unprecedented levels
But the threats to Ontario’s future are just as great Failing to move forward now with significant
measures could leave Ontario unprepared for the challenges ahead, and strand thousands of
people as permanently unemployable
The iconic image of the Baby Boom generation is a 1960s-era snapshot of an exuberant, long-haired, rebellious young adult. That portrait wasn’t entirely accurate even then, but it’s hopelessly out of date now. This famously huge cohort of Americans finds itself in a funk as it approaches old age. On January 1, 2011, the oldest Baby Boomers will turn 65. Every day for the next 19 years, about 10,000 more will cross that threshold. By 2030, when all Baby Boomers will have turned 65, fully 18% of the nation’s population will be at least that age, according to Pew Research Center population projections. Today, just 13% of Americans are ages 65 and older.
Perched on the front stoop of old age, Baby Boomers are more downbeat than other age groups about the trajectory of their own lives and about the direction of the nation as a whole.
Some of this pessimism is related to life cycle – for most people, middle age is the most demanding and stressful time of life. 1 Some of the gloominess, however, appears to be particular to Boomers, who bounded onto the national stage in the 1960s with high hopes for remaking society, but who’ve spent most of their adulthood trailing other age cohorts in overall life satisfaction.
At the moment, the Baby Boomers are pretty glum. Fully 80% say they are dissatisfied with the way things are going in the country today, compared with 60% of those ages 18 to 29 (Millennials); 69% of those ages 30 to 45 (Generation Xers) and 76% of those 65 and older (the Silent and Greatest Generations), according to a Pew Research Center survey taken earlier this month.
Background/Context: The rapid pace of technological change, undergirded by near ubiquitous access to the web, is producing a new learning ecology—a new ecology of information, of knowledge, of reading, of teaching, and of thinking. This instant availability of digital resources frees both time and cognitive energy that may be used to facilitate higher order thinking. This article provides a framework through which to better understand, evaluate, and scaffold the generative synthesis of knowledge in a web-mediated world.
Purpose/Objective: The purpose of this article is to describe a theory that can stimulate additional scholarly work examining higher order, or generative, thinking in web-mediated environments.
While Canada leads other OECD member countries in postsecondary education (PSE) participation rates, there still remain underrepresented segments of the population which are less likely to pursue PSE. Ontarians who come from low-income households, have parents with no PSE, live in a rural area, identify as an Aboriginal person, and/or have a disability are less likely to enrol in PSE (Norrie & Zhao, 2011).
Youth from some ethnic and racial groups are also less likely to pursue PSE, particularly university (Abada, Hou, & Ram, 2008).
This paper focuses on early intervention programs as one approach to support underrepresented youth to complete secondary school and make the transition to PSE. These programs are intended to provide youth with the resources, support and information necessary to avoid dropping out of school and to increase their chances of participating in PSE (Chambers & Deller, 2011). Early intervention programs can originate within the elementary and secondary school systems, colleges, universities, community centres or other community-based organizations.
The first section of this report is a literature review summarizing the key thinking on the role of early
intervention programs in supporting access. Much of what we know in Canada about these programs is
drawn from the American context, where research on the topic has been extensive. As a result, the
literature review draws heavily from American sources, making links to the Canadian context where
possible.
This paper reports the results of a study of provincial level arrangements for coordination of planning and operations between university and college sectors in Canada. The data are drawn from a survey of senior government and sector officials in which respondents were asked to describe existing arrangements for coordination and to comment upon the importance attached to, and priority issues for, coordination; characteristics of effective structures for coordination; and their satisfaction with existing arrangements. The findings indicate that inter-sector coordination is perceived as an important issue; that coordination structures are most developed in the provinces in which there is the strongest mandate for articulation between sectors; and that efforts are under way in most provinces to refine and improve structures for inter-sector coordination.
Cet article prisente Les risultats d'une itude sur Les modes de coordination, d l'ichelle provinciale, de la planification et du fonctionnement intersectoriels des universitis et des colleges au Canada. Les informations utilisies pour les fins de cette analyse ont iti obtenues d partir d'une enquete effectuie aupres des hauts fonctionnaires des gouvernements provinciaux et aupres des institutions d'enseignement postsecondaire. L'objet de cette enquete a porte sur Les modes de coordination en place, sur /'evaluation de /'importance attribuee a ces activites, sur Les questions prioritaires necessitant la coordination, sur Les caracteristiques des structures de coordination qui s'averent Les plus efficaces, et en.fin sur le niveau de satisfaction en regard des structures existantes. Les resultats de l'enquete indiquent qu 'on attache generalement une grande importance aux structures de coordination intersectorielles; que Les provinces possedant Les structures Les plus developpees sont celles ayant etabli un mandat clair de coordination; et en.fin, que toutes Les provinces sont deja engagees dans un processus qui vise a developper et d ameliorer Les structures existantes.
Established in 2004, the Industry Training Authority (ITA) is the provincial Crown agency that governs and manages the industry training system. Working in close collaboration with industry, it keeps occupational standards current and relevant, assesses skills, manages the apprenticeship training pathway, and works to align the profile and number of newly credentialed workers with labour market needs. It provides career development opportunities for individuals and a skilled workforce for
industry.
An annual Government’s Letter of Expectation (GLE) is ITA’s primary source of guidance in setting its strategic direction. In 2008 – consistent with the relevant GLE’s emphasis on expanding access to training for groups that are traditionally under-represented or face barriers to labour force participation – ITA began to establish a distinct suite of Aboriginal Initiatives. The overarching objective was to increase the representation of Aboriginal participants in the trades.
Welcome to our fi rst issue of IQ – McMaster’s research newsmagazine. We’re excited to share a few research highlights and tell you about some of the country’s most dynamic, creative and innovative research that’s happening right here in your community.
In this issue, our focus is on clean technologies – whether they are related to water, automotive or solar research. Our researchers are doing their part to develop the technologies and innovations that will lead to a greener and cleaner Canada for future generations. They are indeed on an Innovation Quest to see that this happens.
I hope you enjoy the fi rst issue and I welcome your comments on what you’ve read here and what you’d like to see in future issues.
The members of the Principal’s Commission on Mental Health are pleased to submit their final report to Principal Daniel Woolf.
This report is the result of a year-long process embedded in comprehensive input from the Queen’s and broader communities. Commissioners Lynann Clapham, Roy Jahchan, Jennifer Medves, Ann Tierney and David Walker (Chair) heard from students, faculty, staff, parents, alumni, mental health professionals and community members, all of whom generously gave their time to provide valuable insight and expertise.
Following the release of a discussion paper in June 2012 extensive feedback was received, for which commission members were most grateful. This input has been integrated into this final report.
ABSTRACT
Community college systems were established across North America from the early 1960s through the early 1970s. The new systems had two principal models: in one model, the college combined lower-division, university-level general education with technical education programs; in the other, most or all of the colleges were intended to concentrate on technical education. Ontario was the largest of the provinces and states in North America that opted for the second model. Many of the issues that planners confronted when designing these college systems have either persisted or re-emerged in recent years. This
article re-examines the debate on the design of Ontario’s colleges that took place when they were founded and considers its implications for the present.
RÉSUMÉ
Depuis le début des années 1960 et jusqu’au début des années 1970, lorsqu’on créait des réseaux de collèges communautaires partout en Amérique du Nord, deux modèles majeurs étaient proposés pour ces nouveaux réseaux. Dans un des modèles, le collège combinait l’enseignement général universitaire de division inférieure avec les programmes d’enseignement technique ; dans l’autre, la plupart des collèges, sinon tous, se concentraient sur l’enseignement technique. L’Ontario était la plus importante parmi les provinces et les États en Amérique du Nord qui ait opté pour le deuxième modèle. Beaucoup des défis auxquels les planifi cateurs ont été confrontés lorsqu’ils ont conçu le réseau des collèges sont encore présents ou sont réapparus au cours des dernières années. Cet article réexamine l’ancien débat sur la conception des collèges de l’Ontario et considère ses implications actuelles.
Between 1991 and 2011, the proportion of employed people aged 25 to 34 with a university degree rose from 19% to 40% among women, and from 17% to 27% among men. Given the increase in the proportion of university graduates, did the occupational profile of young workers change over the period? This article examines long-term changes in the occupation profiles of young men and women, for both those who did and did not have a university degree. Changes in the share of women employed in these occupations are also examined.
ABSTRACT
As Canadian universities seek to attract more international students, there is a need to recognize and respond to the diversity within this group and to question the binary categories of domestic students and international students.
Relying primarily on 116 qualitative interviews with international undergraduates at the University of British Columbia, we utilize American students as a case study from which to explore the complex and blurred boundaries between these two categories. Americans resemble domestic students in some respects and international students in others, yet they are often less prepared to meet adaptational challenges because they have low expectations of cultural and institutional differences. We compare the experiences of Americans and international students from other countries, as well as other groups of students who fall between the cracks of the domestic and international student classifications. We argue that, by targeting services on the basis of these broad administrative categories, categories that were created for financial purposes, the university reduces the take-up of the very services students need.
RÉSUMÉ
À une époque où les universités canadiennes cherchent à attirer de plus en plus d’étudiants internationaux, il est nécessaire de reconnaître la diversité de ce groupe et d’agir en fonction de celle-ci. Cela demande de s’interroger sur la division binaire des étudiants entre les catégories « canadien » et « international ». En nous appuyant sur 116 entrevues qualitatives avec des étudiants internationaux en études de premier cycle à l’Université de la
Colombie-Britannique, nous entreprenons une étude de cas des étudiants américains, afin d’explorer la complexité et l’imprécision des frontières entre ces deux catégories. Sur certains points, le profil des étudiants américains est semblable à celui des étudiants canadiens, mais sur d’autres, il s’apparente plutôt à celui des étudiants internationaux. Pourtant, ces étudiants américains sont souvent moins prêts à faire face à des difficultés d’adaptation, car ils ne s’attendent pas à être confrontés à des différences culturelles et institutionnelles. Nous comparons les expériences des étudiants américains avec celles d’étudiants internationaux provenant d’autres pays, ainsi qu’avec celles d’autres groupes d’étudiants dont la situation ne correspond pas aux classifications « canadien » ou « international ». Nous soutenons que, quand les services d’aide ciblent les étudiants sur la base de vastes catégories administratives conçues pour des raisons financières, l’université contribue à limiter l’utilisation des services dont les étudiants ont précisément besoin.
Abstract
Along with the amount of time spent learning (or time-on-task), the quality of learning time has a real influence on learning performance. Quality of time in online learning depends on students’ time availability and their willingness to devote quality cognitive time to learning activities. However, the quantity and quality of the time spent by adult e-learners on
learning activities can be reduced by professional, family, and social commitments. Considering that the main time pattern followed by most adult e-learners is a professional one, it may be beneficial for online education programs to offer a certain degree of flexibility in instructional time that might allow adult learners to adjust their learning times to their professional constraints. However, using the time left over once professional and family requirements have been fulfilled could lead to a reduction in quality time for learning. This paper starts by introducing the concept of quality of learning time from an online student centred perspective. The impact of students’ time-related variables (working hours, timeon-task engagement, time flexibility, time of day, day of week) is then analyzed according to individual and collaborative grades achieved during an online master’s degree program. The data show that both students’ time flexibility (r = .98) and especially their availability to learn in the morning are related to better grades in individual (r = .93) and collaborative activities (r = .46).
Keywords: E-learning; computer-supported collaborative learning; academic performance;
e-learning quality; time flexibility; time-on-task; time quality; learner time
THE REPORT
Immigrants will represent nearly 100 per cent of net labour market growth in Canada by the year 2011.1 More than ever, employers recognize the need to effectively integrate immigrants into the workplace and they seek solutions to leverage the talents and contributions immigrants bring to the Canadian economy.
From January to March 2009, Colleges Ontario and 12 colleges consulted with employers, ethno-cultural business organizations, business associations and unions to find out their views on employing immigrants and how colleges can support the transition of immigrants to the province’s workforce. Input was obtained through a variety of formats including facilitated round-table discussions, one-on-one dialogues, and an online questionnaire. The purpose of these consultations was to obtain advice from employers on how colleges can better address language needs for the workplace and support immigrant integration.
Colleges engaged in discussions with 218 organizations. These organizations represented a wide cross-section of large, medium and small businesses in five industry sectors that included health care, hospitality, science and technology, construction and manufacturing. Many of these organizations were interested in participating because they understand the valuable role of immigrants in helping companies respond to current labour and consumer market realities.
This report presents the findings from these consultations, offering a snapshot of the experiences of the participants, and outlining some suggestions on how colleges can play an even greater role in effectively integrating immigrants into the workplace.
CONSULTING WITH EMPLOYERS
As part of the Language Skills for the Workplace2 project funded by the federal government, colleges had an opportunity to hold discussions with employers on language needs and immigrant integration. Participants were asked about:
• their experiences in the recruitment, hiring, retention and promotion of immigrants
. training, education and development priorities in the workplace • occupation-specific and workplace-specific language needs
. ways that colleges can effectively help employers in the integration of immigrants
into employment.
Colleges held discussions with their local employer community and Colleges Ontario contacted larger provincewide employers and associations. There were 218 unique organizations that participated: 198 employers, 17 associations and three unions (See Appendix for list of participants). Employers from a broad range of sectors were invited to participate. Approximately 60 per cent of participants were from small- and medium-sized businesses and 40 per cent were large employers (employers with more than 500 employees).
The Council of Ministers of Education, Canada (CMEC), affirmed their commitment to improving outcomes for Aboriginal students and identified the gaps in academic achievement and graduation rates between Aboriginal and non-Aboriginal students as a key area for attention. One of the strategies articulated in the CMEC Aboriginal Education Action Plan for addressing these gaps in outcomes is “strengthening the capacity for evidence-based decision making.” Toward that goal, CMEC commissioned a report to consider how better data and evidence can be developed to support jurisdictions’ efforts to improve the academic achievement and attainment of Aboriginal students in provincial and territorial elementary and secondary schools.
With a population of 13 million people, the province of Ontario covers a significant geographic distribution of 917,741 square kilometres (Statistics Canada, 2005). Fourteen per cent of the population is categorized as living in a rural, remote or northern area (Statistics Canada, 2011). Within this land mass is a rich diversity of people, systems and institutions that are privileged to call it home - including Francophone persons and First Nations, Inuit and Métis people. There are unique challenges that exist within these communities that affect access to health services: geographic distance, socioeconomic status, availability of health human resources and infrastructure. These factors have an impact on health status, wellness and the ability to
offer person-centred health care.
One thing is becoming increasingly clear in the area of workplace training: the standard approaches applied are not adapted to individual needs and the knowledge society as a whole. Offering something more than a standard one-size-fits-all product involves “personalizing” learning. What do we mean by personalizing learning? We mean considering the diversity of learners (learning characteristics) in order to better adapt their learning to their needs (current and target skills), by offering them customized online solutions (synchronous, asynchronous and mixed) and by optimizing learning situations (alternate teaching methods) to reflect work-related requirements (e.g., adapted to their time constraints, work environment and job demands) and each learner’s skills.
Background Web technologies are developing at an unprecedented pace and constitute an excellent tool for improving the flexibility and effectiveness of learning. An increasing number of studies demonstrate that an adult can learn more – and faster – with an online course than face to face in a classroom. What about teachers, who must teach themselves on the job how to use these technologies and effectively integrate them into their teaching? A number of obstacles and a certain resistance hinder this training and integration, the most significant being the time available and the motivation to learn.
Increasingly, teachers are seeking à la carte training solutions that can be split up and accessed at different times in the workplace or close to home. The Internet offers more and more courses that successfully bring the knowledge conveyed in line with learners’ actual needs, regardless of where they may be on the planet, or where their workplace is located. But what do we really know about the impact of these online solutions on workplace learning? There is little literature on the subject, thus the relevance of research to analyze these types of intervention and document the success factors of online training in the workplace.
Goal
Given that little formal research has been conducted on the use of Web technologies for developing the technological and pedagogical skills of teachers in the workplace, and even less on operating training programs that provide a personalized approach to learning, the goal of this study was to test a mixed online learning model that provides a personalized mix of synchronous classroom instruction and asynchronous distance learning to suit the learning characteristics of adult learners in the workplace. With this educational approach in mind, the Form@tion program was launched online to offer professional development opportunities to teachers in the workplace who wished to develop their skills in online teaching. More specifically, the objectives of this study are to understand working teachers’ resistance to change and the obstacles facing them in terms of information and communications technologies Study on the impact of mixed online training (synchronous and asynchronous) on the skills development of teachers in the workplace (ICT); to test a mixed training program that offers a personalized training plan to meet the training needs and learning characteristics of practising teachers; and to measure the changing attitudes of teachers towards the need for lifelong learning.
It is due to the courage and determination of former students—the Survivors of Canada’s residential school system—that the Truth and Reconciliation Commission of Canada (trc) was established. They worked for decades to place the issue of the abusive treatment that students were subjected to at residential schools on the national agenda. Their persever- ance led to the
reaching of the historic Indian Residential Schools Settlement Agreement. All Canadians must now demonstrate the same level of courage and determination, as we commit to an ongoing process of reconciliation. By establishing a new and respect- ful relationship between Aboriginal and non-Aboriginal Canadians, we will restore what must be restored, repair what must be repaired, and return what must be returned.
Inpreparationforthereleaseofitsfinalreport, the Truthand Reconciliation Commission of Canada has developed a definition of reconciliation and a guiding set of principles for truth and reconciliation. This definition has informed the Commission’s work and the principles have shaped the calls to action we will issue in the final report.
There has been substantial discussion, research, and debate about the role of academic freedom within higher education, primarily centered on the university model. Not as well documented or understood is the issue of academic freedom within colleges and institutes in Canada. In this paper, we exam- ine the current state of academic freedom in colleges and institutes using a historical analysis of two Canadian provinces, British Columbia and Ontario. Beginning with an overview of academic freedom within universities, we then examine the development and evolution of colleges and institutes and discuss how or if academic freedom applies to them. We consider issues of collegial- ity, faculty engagement, and governance as they impact the concept and practice of academic freedom within these institutions. We also discuss the different origins, intents, roles, and governance models of universities in contrast to colleges and institutes, which are generally representative of the broader Canadian higher education landscape.