As online education moves from the fringes to the mainstream, one question still persists: “How do I know what my online students have learned?” There are no simple answers, just as there aren’t in face-to-face courses, but with a little creativity and flexibility, you soon discover that the online learning environment opens up a host of new student assessment possibilities. And, just as with traditional courses, the trick is finding the right combination that works best for your particular course.
This special report features 12 articles from Online Classroom that will cause you to examine your current methods of online assessment, and perhaps add something new to your assessment toolbox. It even talks about some of the common assessment mistakes you’ll want to avoid.
Take a look at some of the articles you will find in Assessing Online Learning: Strategies, Challenges and Opportunities:
• Authentic Experiences, Assessment Develop Online Students’ Marketable Skills
• Four Typical Online Learning Assessment Mistakes
• Assessing Whether Online Learners Can DO: Aligning Learning Objectives with
Real-world Applications
• Strategies for Creating Better Multiple-Choice Tests
• Assessing Student Learning Online: It’s More Than Multiple Choice
• Using Self-Check Exercises to Assess Online Learning
• Measuring the Effectiveness of an Online Learning Community
• Ongoing Student Evaluation Essential to Course Improvement
Online courses enable a strong student-centered approach to learning and, as a result, assessment. We hope this report helps you design and develop online assessment strategies that take full advantage of the many formal and informal assessment tools now at your fingertips.
While discussions on the value of education often focus on economic gains, the social returns to education are vast and can be reaped at both the individual level (e.g., better health) and societal level (e.g., lower crime rates).
Based on a combination of new and existing analyses, this paper explores the individual benefits and disadvantages associated with education, focusing on civic engagement; health/happiness; crime; and welfare/unemployment. The findings clearly suggest that investing in education has both individual and social benefits. While no causal link can be made between level of education and the returns examined, it is evident that those with some form of postsecondary education (PSE) often fare better than those with no more than a high school education.
For example, in terms of civic engagement, university graduates are more likely than high school graduates to volunteer and donate money. Higher levels of education also increase the likelihood of voting and other forms of political participation. In terms of health and happiness, university graduates tend to rate their physical and mental health higher than those with fewer years of education and are also less likely to smoke. Finally, happiness and life satisfaction also tend to increase with education.
Educated individuals are less likely to be incarcerated, most notably when comparing high school graduates with those who did not graduate. With that said, certain types of crime are more prevalent among certain populations and individuals with higher levels of education are more likely to commit white collar crimes. Finally, those with more education have lower unemployment rates and fared better during the most recent economic recession. They were less likely to require social assistance and had shorter welfare spells, especially for women.
The issue of the “boy gap” or “boy crisis” in education has been the subject ofincreasing attention across a number of OECD countries. The issue has also captured the attention of the Canadian media. As the Globe and Mail recently emphasized in their six-part series on ‘failing boys’:
“data suggests that boys, as a group, rank behind girls by nearly every measure of scholastic achievement. They earn lower grades overall in elementary school and high school. They trail in reading and writing, and 30 per cent of them land in the bottom quarter of standardized tests, compared with 19 per cent of girls. Boys are also more likely to be picked out for behavioural problems, more likely to repeat a grade and to drop out of school altogether”. (Globe and Mail, October 15, 2010)
Given the importance of this issue and the need to better understand the situation in boys' education, this report draws on material and data from a review of websites, research reports and relevant data sources, as well as informal consultations with some official and expert sources, to scope out four main questions:
1. What is the situation regarding education and training participation and
results for boys and men throughout the OECD, including post-secondary
education and trades?
2. Are there policies and practices in place to attenuate unfavourable trends?
3. What are Canadian jurisdictions doing?
4. What do we know about the success and failure of various models OECDwide
with a focus on Germany, the United States, Australia and the United
Kingdom?
It should be noted that there is a substantial disconnect between public policy commentary on issues in the “developed” and “non-developed” worlds. In the latter, priority attention continues to be centred on the barriers and obstacles faced by females in education and the labour market. Access to education in all its forms is still significantly more available to males in such countries. The UN Girls’ Education Initiative (UNGEI) is focused on ensuring that a gender equity
and equality perspective is brought to bear within the broad context of the UN’s Education for All (EFA) initiative, and is reflected in the Global Monitoring Reports issued by the EFA.
In is also the case that attention within OECD countries continues to be paid to the traditional barriers faced by women in many areas of education and
employment. A “question scan” done by CCL for the British Columbia Ministry of
Advanced Education only a few years ago identified a number of studies and
reports on the issue of gender in PSE access; all of them focused on the
question of female participation and access, none on the “boy crisis”.
It is also the case that attention in several OECD jurisdictions has shifted in some circles in the past number of years to the phenomenon of a substantial shortfall of the percentage of males, compared to females, who complete secondary schooling, and who are enrolled in and graduate from PSE. The implications of this “boy gap” are increasingly being pondered in such countries as Canada, the U.S., the U.K. and Australia. The statistical picture in terms of this gender gap, as
shown in literacy rates, school achievement in literacy, and participation and success in university studies, has been quite clear in such jurisdictions for two decades and more; the implications of this gap, however, are not at all a matter of consensus. Nor are the public policy and program responses either clear or consistent.
Question 1: What is the situation for boys and men throughout the OECD,
including PSE and the trades?
The purpose of this section is to present general statistics on performance and participation in education and training for both boys/young men and girls/young women across OECD countries. The data have been selected to provide a preliminary overview that can be used to direct further research and analysis.
Given the parameters of this project, it is not possible to complete a comprehensive survey of data. For the purposes of this paper, the focus is on Australia, Canada, Germany, the United Kingdom and the United States, using a limited number of variables.
This first section focuses entirely on statistics and trends. It becomes obvious early in any consideration of this issue that its complexity and multifaceted nature present challenges. For clarity, the findings below are presented by educational sector.
1. Overview—general trends over time The relationship between education and skill development has been a well explored topic over the last decade, with many countries concluding that highly skilled and educated citizens are essential to meet the challenges of globalization and the knowledge economy. In an effort to help understand the complex network and inter-relationship of factors that influence individuals to participate and succeed in education and training, researchers have undertaken detailed research on educational outcomes and the influences on motivation, participation and completion of education.
Over the last couple of decades there has been increasing emphasis on maximizing the participation of under-represented groups such as immigrants, women and other minorities in education. Along the way, an interesting trend has emerged that is now clearly illustrated by the statistics—the statistics indicate that, overall, girls and women tend to do better in school environments, outperforming males. This is evident in both the secondary- and higher-education sectors. Research shows that girls/young women and boys/young men have distinctly different experiences in the various educational sectors.
2. K–12
For many years, gender-related research in the K–12 sector was focused on dropout rates in secondary schools. These rates were usually significantly higher for boys than girls, a trend which held across OECD countries.
PISA
The OECD PISA (Programme for International Student Assessment), designed to explore “the educational performance and attitudes of adolescent males and females”, provides data to answer questions related to why female and male students perform differently. Ironically, one of the main rationales for PISA was to determine why females appeared disinterested in, and tended to be less successful in, mathematics and the physical sciences. However, PISA findings that demonstrated that boys had difficulty in the area of reading spurred further research into literacy among boys and, eventually, the design of specific
interventions to address related issues.
Statistical evidence about gender differences among young boys and girls is quite detailed. The OECD report, “Equally Prepared for Life?”, provides a summary of gender issues from early childhood based on results from PISA,
PIRLS (Progress in International Reading Literacy Study), TIMSS (Trends in International Mathematics and Science Study) and other statistics that are gathered regularly.
Some of the main findings across OECD countries include1:
• Gender differences appear at early stages of education but they are small.
Females show better performance in reading in primary schools.
• Females showed significantly higher reading achievement than males in all (except two) countries by Grade 4. (2004 data)
• At Grade 4, the results for mathematics and science were mixed. Males had significantly higher scores for math in 12 countries while females had significantly higher scores in eight countries. In science, the scores for males and females were somewhat similar in more than half the countries
(2007).
• By Grade 8, on average, females had higher achievement than males in mathematics, although there were country variations. (2007) The same was true for science.
• Although PISA 2006 showed no significant differences between males and females in the overall performance in science, females were better identifying scientific issues while males were better at explaining phenomena scientifically.
- In the PISA 2009 reading assessment, girls outperform boys in every participating country by an average, among OECD countries, of 39 PISA score points—equivalent to more than half a proficiency level or one year
of schooling.
- On average across OECD countries, boys outperform girls in mathematics by 12 score points while gender differences in science performance tend to be small, both in absolute terms and when compared with the large 1 OECD, “Equally Prepared for Life?” 2009, pp. 3; 10–12;16–19; 2–24 and 27.
The choice of whether and where to attend college is among the most important investment decisions individuals and families make, yet people know little about how institutions of higher learning compare along important dimensions of quality. This is especially true for the nearly 5,000 colleges granting credentials of two years or fewer, which together graduate nearly 2 million students annually, or about 39 percent of all postsecondary graduates. Moreover, popular rankings of college quality, such as those produced by U.S. News, Forbes, and Money, focus only on a small fraction of the nation’s four-year colleges and tend to reward highly selective institutions over those that contribute the most to student success.
Drawing on a variety of government and private data sources, this report presents a provisional analysis of college value-added with respect to the economic success of the college’s graduates, measured by the incomes graduates earn, the occupations in which they work, and their loan repayment rates. This is not an attempt to measure how much alumni earnings increase compared to forgoing a postsecondary education. Rather, as defined here, a college’s value-added measures the difference
between actual alumni outcomes (like salaries) and predicted outcomes for institutions with similar characteristics and students. Value-added, in this sense, captures the benefits that accrue from both measurable aspects of college quality, such as graduation rates and the market value of the skills a college teaches, as well as unmeasurable “x factors,” like exceptional leadership or teaching, that contribute to student success.
Over the last decade, Ontario has had great success increasing high school graduation rates (5 year rates have improved from approximately 68% to 82%1), and sending more graduates on to university, college, or apprenticeships. But some students—Aboriginal, low-income, disabled, and those from the English-speaking Caribbean and Central and South America—still do not share equally in educational success.
Improved graduation rates stem largely from the province’s Student Success Strategy, which has created more caring,
motivating environments for students in grades 7 to 12, with focused support for at-risk learners and at key transitions.
Ontario has also expanded co-operative education, developed ways to make up components of failed courses through a
system known as “credit recovery,” created focused programs called Specialist High Skills Majors, and designed programs
that let students earn dual credits that count toward both their high-school diploma and a post-secondary diploma, degree or
certification.
Students enter college hoping it will be a major step up from what they were doing before, writes Roger Martin, but
they are often disappointed.
Student Debt and the Class of 2013 is our ninth annual report on the cumulative student loan debt of recent graduates from four-year colleges. Our analysis of available data finds debt levels continue to rise, with considerable variation among states as well as colleges.
About seven in 10 (69%) college seniors who graduated from public and private nonprofit colleges in 2013 had student loan debt. These borrowers owed an average of $28,400, up two percent compared to $27,850 for public and nonprofit graduates in 2012. About one-fifth (19%) of the Class of 2013’s debt was comprised of private loans, which are typically more costly and provide fewer consumer protections and repayment options than safer federal loans.
Since the Higher Education Quality Council of Ontario (HEQCO) was launched, it has completed and published more than 140 research studies – and funded dozens more that are currently underway – that explore a wide range of trends and issues involving Ontario's postsecondary system. Drawing mainly from HEQCO's own research, this @Issue paper:
. Describes how the definition of student success has gradually broadened at Ontario colleges and universities;
. Summarizes some of the underlying institutional and student population factors that also impact on most current measures of student success;
. Provides broad observations about some recent findings as they relate to the awareness, utilization and impact of various student service, course-based and other initiatives designed to promote student success;
. Recommends what can be measured – as well as how and what outcomes can be expected – when it comes to initiatives and interventions designed to improve student success.
Some readers will be looking for the "silver bullet" within this paper. They will want to be told about a best practice that has been proven to be most effective at improving academic achievement, retention or engagement at an Ontario college or university, and that can be replicated to equal effect elsewhere. This @Issue paper does not identify “silver bullets.†As explained in the pages that follow, the scope and scale of an intervention may make it difficult to measure – or even expect – considerable impacts on student success, especially in the short term.
This paper does provide broad lessons, however, that are likely to be applicable across a wide range of student service, course-based and other interventions currently offered at Ontario colleges and universities.
Defining “Student Successâ€
For several decades, both governments and colleges/universities in Ontario and across Canada have tried to broaden access to postsecondary education (PSE) In particular, it was believed that a wide variety of barriers – family and social background, financial resources, information about options, etc. – needed to be overcome to encourage broader PSE participation, especially by those from traditionally under-represented groups (low income, first-generation, Aboriginal, visible
minority, rural, etc.).
Abstract
The HANDS (Helping Autism-diagnosed teenagers Navigate and Develop Socially) research project involves the creation of an e-learning toolset that can be used to develop individualized tools to support the social development of teenagers with an autism diagnosis. The e-learning toolset is based on ideas from persuasive technology. This paper addresses the system design of the HANDS toolset as seen from the user’s perspective. The results of the evaluation of prototype 1 of the toolset and the needs for further development are discussed. In addition, questions regarding credibility and reflections on ethical issues related to the project are considered.
Keywords: E-learning; autism; mobile learning; persuasive technology
Peter Øhrstrøm
Aalborg University, Denmark
Over the past few decades, Canada’s labour requirements have changed drastically—from a need for physical labourers to a need for knowledge workers—as a result of changes in economic and social conditions that have included advances in information and communication technologies, globalization of economic activity and shifting demographics. Consequently, employers and firms are increasingly seeking skilled workers with a more sophisticated array of capabilities. Of recent concern, the current global recession has led to the deterioration of labour-market conditions in Canada and worldwide, profoundly affecting—through increased vulnerability to unemployment—the economic and social well-being of families and communities across Canada. Canada’s economic strength, as in other countries, depends on its ability to develop a skilled and flexible workforce, capable of adapting to continuous change. While Canada’s formal education is of a high standard, it alone cannot provide the conditions needed to secure the development of Canada’s talent—its human infrastructure*—which is a necessary element of our country’s future prosperity. Against this backdrop, Securing Prosperity through Canada’s Human Infrastructure, CCL’s second†report on the state of adult learning and workplace training in Canada, demonstrates that investments in human infrastructure—both in times of economic uncertainty and relative prosperity—are critical to securing a strong economy and greater social equity.
Public colleges are the only academic institutions in Canada that deliver a robust range of career-focused programs and training to all segments of the population. The colleges’ labour-market programs, such as Second Career, employment counselling, academic upgrading and apprenticeship training serve more than 160,000 students
each year.
Ontario’s public college programs are affordable and reach students in all socioeconomic groups – from people who need upgrading in order to qualify for full-time college programs, to university graduates seeking marketable skills.
Graduates of Ontario’s 24 public colleges earn credentials that have met the province’s rigorous standards for post-secondary education and are valued by employers. College graduates continue to be in high demand.
In 2008, the OECD launched the AHELO feasibility study, an initiative with the objective to assess whether it is possible to develop international measures of learning outcomes in higher education.
Learning outcomes are indeed key to a meaningful education, and focusing on learning outcomes is essential to inform diagnosis and improve teaching processes and student learning. While there is a long tradition of learning outcomes’ assessment within institutions’ courses and programmes, emphasis on learning outcomes has become more important in
recent years. Interest in developing comparative measures of learning outcomes has increased in response to a range of higher education trends, challenges and paradigm shifts.
AHELO aims to complement institution-based assessments by providing a direct evaluation of student learning outcomes at the global level and to enable institutions to benchmark the performance of their students against their peers as part of their improvement efforts. Given AHELO’s global scope, it is essential that measures of learning outcomes are valid across
diverse cultures and languages as well as different types of higher education institutions (HEIs).
So much of what determines the overall success or failure of a course takes place well in advance of the first day of class. It’s the thoughtful contemplation of your vision for the course — from what you want your students to learn, to selecting the instructional activities, assignments, and materials that will fuel that learning, to determining how you will measure
learning outcomes. Course Design and Development Ideas That Work examines this multifaceted issue from a variety of fronts to bring you proven course design alternatives implemented in courses of varying sizes and disciplines. Featuring 12 articles pulled from the pages of The Teaching Professor, the report will inspire you to rethink some components of your course.
For example, in the article titled A Large Course with a Small Course Option, we learn about an innovative course design for a large 300-level course. Essentially, the instructor created two options: in one, students attend lectures and take four exams. In the second option, students are responsible for those same lectures and exams, but they also participate in small group
discussions and complete a set of writing assignments. The second option was most valued by students who are not very good test-takers or who have a keen interest in the subject.
In the article The Placement of Those Steppingstones, the University of Richmond’s Joe Ben Hoyle compares the placement of steppingstones in a koi pond to the educational processes teachers use to help their students get from point A to point B. Hoyle theorizes that “education stumbles when either the learning points are not sequenced in a clearly logical order or they are not placed at a proper distance from each other.”
Other articles in Course Design and Development Ideas That Work include:
• A Course Redesign that Contributed to Student Success
• Pairing vs. Small Groups: A Model for Analytical Collaboration
• How Blended Learning Works
• Should Students Have a Role in Setting Course Goals?
• In-Class Writing: A Technique That Promotes Learning and Diagnoses Misconceptions
If you’re looking to update an existing course, this report will give you sound strategies to
consider.
Maryellen Weimer
Editor
The Teaching Professor
About Ontario Centres of Excellence Inc.
The Ontario Centres of Excellence (OCE) not-for-profit program was formally established in 1987 with seven independent centres that evolved and amalgamated into Ontario Centres of Excellence Inc. in 2004.
Twenty-five years ago, the traditional economic foundation for the province, and for Canada, was shifting from a north american focused and commodities-based economy to one that is globally oriented and knowledge-based.
Prior to the creation of OCE, there was limited connection between universities, colleges, research hospitals and industry. Consensus was that these academic and research institutions were producing quality research that was not being utilized to its full potential by industry.
OCE was designed to bridge that gap and create productive working partnerships between university and college
research departments, research hospitals and Ontario industry.
In a traditional face-to-face class, students have many opportunities to interact with their instructor and fellow students. Whether it’s an informal chat before or after class, or participating in the classroom discussion, interaction can be an important factor in student success.
Creating similar opportunities for participation and collaboration in an online course is one of the biggest challenges of teaching online. Yet, opportunities for meaningful interaction online are plentiful, provided you design and facilitate your course in the correct manner and with the proper tools.
The digital revolution is transforming our work, our organisations and our daily lives. Driverless cars are now legal in three American states. One third of payments in Kenya are made via mobile phones. Wearable computing will soon mean that your jacket will monitor your heart rate (should you want it to). I have seen a violin - played beautifully - that was 3-D printed.
This revolution is already in homes across the developed world and increasingly in the developing world too. And there, it is transforming the way children and young people play, access information, communicate with each other and learn. But, so far, this revolution has not transformed most schools or most teaching and learning in classrooms.
In this workshop, key strategies that integrate quality ideas with quality change processes will be presented as they apply to concrete change situations.
Participants will learn about effective approaches to each of the following levels: within school success; success across schools and regions; and how to relate to the state and federal levels. Specific examples will be examined at each level. Next generation reform will be identified related to factors that will deepen and accelerate learning required for future societies through powerful
new pedagogies linked to digital resources.
The Strategic Mandate Agreement (SMA) exercise was intended to address at least three desired
outcomes:
1. To promote the government’s stated goal1 of increasing the differentiation of the Ontario postsecondary system by asking each Ontario postsecondary institution to articulate an institutional mandate statement identifying its distinctive strengths or aspirations and to identify key objectives aligned with that aspiration.
2. To advance and inform the discussion about how the Ontario system could increase its productivity to deliver a quality education to more students within the financial constraints expected in the public sector.
3. To elicit the best thinking from institutions about innovations and reforms that would support higher quality learning and, in its most ambitious form, transform Ontario’s public postsecondary system.
To assist with the evaluation of the SMAs, the Ministry of Training, Colleges and Universities (MTCU)
“…instructed the Higher Education Quality Council of Ontario (HEQCO) to establish a peer review panel to evaluate…mandate submissions … for their ability to achieve significant improvements in productivity, quality and affordability through both innovation and differentiation.” The members of the Expert Panel are listed in Appendix 1.
61% of parents have more than one type of debt, with a mean number of debt types at 2.25
▪ 28% of parents have either type of student loan debt (for parents’ or kids’ education), and 5%
have student loan debt for both parents’ and kids’ education
▪ Parents with student loan debt (from parents’ education) are significantly more likely to have
credit card debt (67% vs. 54%) and payday loan debt (19% vs. 7%)
▪ Parents with student loan debt (from kid’s education) are significantly more likely to have
credit card debt (75% vs. 54%) and payday loan debt (38% vs. 5%)
▪ Parents with student loan debt (from parents’ education) are significantly more likely to say
they lose sleep worrying about college costs for their kids (49% vs. 40%)
The 2015 Sustainable Campus Index highlights top-performing colleges and universities in 17 areas, as measured by STARS. Data submitted by top performers has been reviewed by AASHE staff to ensure that content meets credit criteria (see page 51 for a detailed methodology). The report also includes trends and best practices from over 50 institutions that submitted STARS reports in the last 12 months (July 1, 2014 - June 30, 2015).