GLC Summit 2014
The Inaugural Meeting and Symposium of the Global Learning Council
September 4-5, 2014 | Pittsburgh, PA
The GLC is made up of thought leaders working to develop metrics, best practices and standards for learning systems that enhance learning outcomes. The September inaugural meeting brought together GLC members and guests to address these key issues in symposia-like sessions that included formal and informal talks, panel sessions, and breakout sessions. Sessions included panels on: The Role of Learning Science in Learning Gain: Best Practices and How to Practice them Big Data in the Service of Better Learning K-12 TEL: Readiness for Life and Learning
Support for this event was provided by the U.S. National Science Foundation.
- Anant Agarwal, CEO, edX
- Joan Ferrini-Mundy, Assistant Director of the National Science Foundation for Education and Human Resources
- Patrick Gallagher, Chancellor, University of Pittsburgh
- Anoop Gupta, Distinguished Scientist, Microsoft Research
- Tom Kalil, Deputy Director for Technology and Innovation, Office of Science and Technology Policy, The White House
- Matthias Kleiner, President, Leibniz Association
- Peter McPherson, President, Association of Public and Land-grant Universities (APLU)
- Hunter Rawlings, President, American Association of Universities (AAU)
- Andrew Rosen, Chairman, Kaplan, Inc.
- James H. Shelton III, Deputy Secretary, U.S. Department of Education
- Alfred Spector, Vice President of Research and Special Initiatives, Google
- Luis von Ahn, CEO and founder of Duolingo, language-learning software that was “App of the Year” in 2013
- Subra Suresh, President, Carnegie Mellon University (GLC Chair)
- Suzanne Walsh, Deputy Director, Postsecondary Success, Bill & Melinda Gates Foundation
Videos & Highlights
Watch videos from the September 2014 meeting below. The full playlist is available on YouTube.
Thursday, September 4, 2014
Introductions and Welcome Remarks
Subra Suresh, President, Carnegie Mellon University; Chair, Global Learning Council
Pat Gallagher, Chancellor, University of Pittsburgh
President, Carnegie Mellon University; Chair, Global Learning Council
The rapid transformation in communications technology over the past 30 years has given rise to new opportunities for teaching and learning, especially with a tech-savvy and knowledge hungry generation. New systems, MOOCs, and games are being created at rapid speed: What is missing is a systematic and evidence-based perspective on how much knowledge is being absorbed.
The Global Learning Council was formed to focus on how to achieve strong learning outcomes. Suresh noted that no one institution or sector can or should do this alone—but who is in charge? Who sets standards? The GLC brings together thought leaders from academia, industry, and nonprofits who all have much at stake to start a conversation. This is a collective effort to develop mechanisms to share information and move forward to expand K-grave learning opportunities around the world.
Chancellor, University of Pittsburgh; Member, Global Learning Council
The world is at the crossroads of three very powerful currents in education: the growing capacities of technology (and it ability to measure and record data), the development of the science of learning, and the increasing importance of knowledge-dependent enterprises who seek talent. All of these make the goals of the GLC of concern to The GLC is convening in Pittsburgh to begin to take advantage of these three currents, and Dr. Gallagher argued that the GLC collaboration promises to have very productive results.
Tom Kalil, Deputy Director, White House Office of Science & Technology Policy
Office of Science and Technology Policy, The White House
Much attention has been paid to the “supply” side of learning technology innovations—new courses, software, games—but the “demand” side of learning science innovation gets less attention. There is a gap between the potential of learning technologies ae a much larger impact learners. Most studies of the effectiveness of new educational technologies are showing that almost none are effective in actually improving learning outcomes. Kalil addressed how this imbalance could be addressed: How do we pay for outcomes and not inputs? How do we establish “pull” mechanisms (such as prizes, advance market contracts, paying for success) to move this field forward? How can we come to greater agreements on goals and incentives? How can we more effectively collect “before” and “After” data? How can teachers and other stakeholders be involved?
Panel 1: The Role of Learning SCIENCE in Learning GAIN: TEL Best Practices & How to Practice Them
How do we improve learning outcomes for all through advancing the science and technology of learning?
Lead in & Co-Moderator: Marsha Lovett, Director, Eberly Center for Teaching Excellence & Educational Innovation
Discussant & Co-Moderator: Andrew Rosen, Chairman, Kaplan Inc.
- Frank DiGiovanni, Director, Force Readiness and Training, U.S. Office of the
Undersecretary of Defense
- Anoop Gupta, Distinguished Scientist, Microsoft Research
- Peter McPherson, President, Association of Public and Land-Grant Universities (APLU)
- Charles A. Perfetti, Director, Learning Research and Development Center; Distinguished University Professor, University of Pittsburgh
Lead in & Co-Moderator: Marsha Lovett
Director, Eberly Center for Teaching Excellence & Educational Innovation
Lovett defined learning science as a blend of cognitive psychology, computation, and data science to measure the impact of practices, technologies on actual learning outcomes. Scientists have over the past 30 years identifies some core principles of how learning works, and have attempted to use that knowledge to shape how learning environments are created and used. The emphasis is on learning—in whatever setting it takes place—and that means the emphasis is on the thinking and action of every learner. Learning is complex—there are many factors involved (in person or technology enhanced learning; the role of peers and instructors, etc.) Learning is also familiar—we are all tempted to think we know more about it than we do. For example, spaced practice produces more durable results than last minute cramming, but we still cram.) This panel is focused on looking at specific aspects of linking evidence based learning science principles in all their complexity and ongoing development, and connect these to the actual activities online and in classrooms.
Discussant & Co-Moderator: Andrew Rosen
Chairman, Kaplan Inc.; Member, Global Learning Council
How can the GLC have major impact on learning at scale? And why do we not use what we already know? Kaplan uses a course design and delivery checklist, to make sure that courses reflect current learning science: all courses have objectives; activities and assessments are connected to those objectives; there are opportunities for practice and review. Etc. This is analogous to what doctors do: not every GP has to understand the details of every procedure, but just proceeds along with recognized “standard of care.” That is what is needed in all education settings, and the GLC could help to create this kind of tool.
Technology is not the solution to learning outcomes—it is a tool to help us achieve them, the panelists agree: Looking at social context; motivation, blended learning, the need for human connection. But what is technology’s role? It can accelerate learning when well designed and blended with human instruction.
Director, Training Readiness and Strategy, Office of the Deputy Assistant Secretary of Defense
The Army has done a lot of work on personal tutors over the past three years, and DiGiovanni outlined some of the issues and observations that have emerged from DoD’s work on technology enhanced learning: 1. Technology cannot become a substitute for human thought (e.g., dependence on calculators has negatively affected the ability to do basic math), so regression is a concern. 2. Technology can lead to a loss of humanity, with people focusing on devices not on the environment or people around them; 3. We seek an elegant user interface; 4. We must account for social impacts (and unintended social consequences) of personalized learning. 5. We believe learning needs to engage all five senses, which technology cannot usually do; 6. We want to leverage social media, as a means for learners to connecting to and get help from other learners.
DiGiovanni recounted the experience with a DARPA course called Education Dominance, a 16 week course on IT (vs 35 weeks in conventional classrooms) for the US Navy. The results were spectacular. They then offered a course to 100 students through the VA with 97 out of 100 students graduating and demonstrating more robust learning, and getting well paid jobs.
There are a number of differentiating features to education in a military setting, which are different from other settings: first, in the military, there is a high level of personal discipline and accountability among students; second, what they are learning is truly a matter of life and death, which heightens motivation; and third, there is a strong emphasis on competency-based assessment, where knowledge is tested and skills had to be demonstrated.
Distinguished Scientist, Microsoft Research; Member, Global Learning Council
Gupta began by noting the learning technology so far has touched only a small number of highly motivated students. Without a much larger and more diverse group of students, we cannot tell yet how effective this will be. In particular, motivation for learning is a key factor: students need first a sense of “why does this matter to me?” and second a sense of their own self-efficacy as learners. Often this motivation is inspired by teachers, parents, coaches—not so much by mere “content creators.” We need to design learning systems that keep the human being in the loop—blended learning models, that offer technology to support faculty members (versus purely online MOOCs). What we need are “massively empowered classrooms” and tools for teachers to make technology work in their own local contexts. Teachers are using video lectures and Khan Academy modules stopping and discussing key points along the way or allowing individual students or small groups to watch, review, or segment sections, which help delivery more successful outcomes.
President, Association of Public and Land-Grant Universities (APLU); Member, Global Learning Council
Public universities were established to provide higher education for a large number of students—enrollments are up 28% even as in most states support is dropping. We need to do more to reduce costs and improve outcomes at public institutions, and technology-enhanced learning is clearly pat of the answer. Blended classrooms are key: this allows faculty to handle larger classes and students with very diverse backgrounds and levels of preparation, but it also allows students to have personal interaction with a faculty expert in that subject area, which is vital. Technology and the data it generates can support student services as well: e.g., when to intervene with a struggling student; how to advise them; how to design gateway courses at the right level for the students at a given institution, etc. Technology is a political issue for most public institutions: faculty are leery of it as a job-destroyer, especially for non-tenured and adjunct professors. Marshalling data on how technology can build more effective learning outcomes will be an essential tool in persuading faculty to accept technology in public university classrooms.
Charles A. Perfetti
Director, Learning Research and Design Center; Distinguished University Professor, University of Pittsburgh
Learning science has so far focused on cognitive measures—the way an individual mind interacts with content. Increasingly this is expanding: it is now including neurocognitive studies, which focus on how the new studies of the biology of the brain will affect our understanding of learning. Also there is interest in the “techno-cognitive,” the way individuals interact with a technology or course delivery system. Perfetti welcomed this expansion and suggested that the most important area of current and future inquiry is recognizing the role of social and environmental contexts of learning. Schools will continue to be relevant, he said, and understanding how this affects learning outcomes will be important. Social context matters, and Dr. Perfetti calls for greater attention among learning scientists to this.
Luis von Ahn, CEO, DuolingoSummary of Remarks
Luis von Ahn
CEO and Co-founder, DuoLingo
Professor of Computer Science, Carnegie Mellon University
Duo Lingo is the most popular language learning tutoring system in the world, approaching 40 million users, on both Android and iOS platforms. About 1.2 billion people are learning a foreign language around the world—mostly learning English to get a better job. There are more people using DuoLingo in the United States than there are people studying in classrooms. Our ultimate goal is to use language learning to help translate the web into many more languages.
Seven tenets or principles guide us in thinking about education and creating DuoLingo:
1. The best education should be free, not a luxury.
2. Should support itself, not lean on others for funding.
3. It should be in your pocket, not in some building.
4. Learning should be fun and not a drag.
5. Based on data, not opinions.
6. Data should be used to personalize education not just personalize ads.
7. We design for students not the system.
Von Ahn also talks about DuoLingo TestCenter, an English language proficiency test that will make it much easier and more affordable for people to establish qualifications for universities and the job market.
Panel 2: Big Data in the Service of Better Learning
What are the challenges and opportunities in establishing an evidence-based culture for improving technology-enhanced learning?
Lead in & Co-Moderator: Ken Koedinger, Director, Pittsburgh Science of Learning Center and Professor,
Discussant & Co-Moderator: Alfred Spector, Vice President, Research and Special Initiatives, Google
- Tom Brock, Commissioner, National Center for Education Research (NCER), Institute of Education Sciences (IES)
- Matthias Kleiner, President, Leibniz Association
- Anant Agarwal, CEO, edX
- Ryan Baker, President, International Educational Data Mining Society; Associate Professor, Teachers College, Columbia University
- Heidi Wachs, Research Director, Gartner
- Carl Wieman, Professor, Physics and Graduate School of Education, Stanford University
Lead in & Co-Moderator: Ken Koedinger
Director, LearnLab; Professor of Psychology and Human Computer Interaction, Carnegie Mellon University
The need for an evidence-based culture in education has never been more apparent, as the power of data in other fields is widely recognized: there is more data available, and there is more pressure to demonstrate improved learning outcomes. It should be the goal that all students see improvement over the course of the year, by measuring interaction data—measuring progress, identifying barriers and designing ways to overcome them (including motivation and affect). Another challenge is involving teachers in using data to design courses and assess student progress.
Discussant & Co-Moderator: Alfred Spector
Vice President, Research and Special Initiatives, Google; Member, Global Learning Council
Different meanings and purposes for learning data were highlighted by each speaker. Matthias Kleiner talked about the progress on national scale to track learning outcomes. Anant Agarwal has lots of data, but faces barriers to sharing it. Privacy is very area specific, and Heidi Wachs talked about the parameters of a policy to protect such privacy. Baker talked about the balance between privacy protection, which if violated could affect our careers and lives) and the good it could do (identifying learning difficulties). There are also policy uses for data. We need to be guided by the intended use, and trust by the user community and regulators that data will only be used in the proscribed fashion. Carl Wieman noted that the data is useless unless we understand the conditions and purposes that shaped the collection.
CEO, edX; Member, Global Learning Council
edX is awash in big data, with 3 million students around the world; collecting data and conducting research is a key part of our mission from Day 1. We have about 4 billion records tied to what we know about each student, and we can constantly learn about improvements to courses. For example, we studied the optimal length for instructional videos: the answer turned out to be 6 minutes, long enough to transmit an idea but short enough for students to sustain attention. Privacy is a huge barrier to sharing data; our users’ privacy must be protected and deidentification is a big challenge. The GLC could usefully establish protocols for data sharing partnerships, create ways to appropriately protect privacy, and make companies like edX more comfortable about releasing data for learning research.
President, International Educational Data Mining Society, Associate Professor, Teachers College, Columbia University
We can infer lots of information from educational data; it can be really useful. Prediction can be made about, for example, at–risk students, that could improve educational outcomes. It is complex to design experiments to be sure we are collecting what we need to make the proper inferences and predictions; this mix of data mining and educational theory is challenging to get right. We need many more learning analytics professionals, and the existence of new graduate programs (e.g., at Columbia and CMU) is a good start. The GLC can help share resources and create the cohort of skilled analysts. Privacy is a challenge; the development of trusted clearinghouses for learning data can serve valuable roles. We need to balance the needs for privacy with the potential for great impact that such information can have for learners.
Commissioner, National Center for Education Research(NCER), Institute of Education Sciences (IES)
Different groups need data for different things; using it in classrooms is different than using it in course design, which is different than using it in large scientific studies. For teachers, the challenge is finding usable information about children in real time; Course designers need it to make improvements. Learning scientists want to detect large patterns in experiments, building theory to guide future practice. Some of this is aggregated, looking at large patterns over time; some is personal to a particular child. Good training is key: we need collaborative models to do this. But the funding is siloed—it is hard to find funding that can support learning scientists, practitioners and teachers, developers, yet these challenges are affecting all of these together. The opportunity to use data to replace achievement testing is appealing, but it is unlikely that testing will be replaced anytime soon, because there are legitimate policy needs for performance assessment at certain levels. But we do need to move away from high stakes testing as much as possible, and these technological systems may help to accomplish that.
President, Leibniz Association
Leibniz Institute, a German association of leading universities, is studying learning outcomes in science with several large-scale, long term studies, following 10,000 student projects for science education, looking at the consequences of learning paths, the changes in learning environments, and changing in work habits. The Leibniz Educational Research Alliance is looking infrastructure for education, looking at resources, open source materials, and the role of libraries. The GLC can offer a platform for sharing definitions, policy ideas, best practices, so that we can all improve, avoid mistakes, and advocate effectively for funding.
Research Director, Gartner
Privacy is a huge issue—and we have very little collective experience to help us move forward. Best practices usually come from mistakes and missteps—we simply haven’t enough experience to create the protocols edX and others need. Privacy must involve notice and awareness; real choice and consent; access to records; transparency of uses; and some reasonable form of redress. There must be a clear “opt in” so that learners know what is being collected and how it will be used, and how any errors can be corrected. Social science and medicine have developed possible models here. GLC is in a position to work with university associations and others to move this forward.
Professor, Physics and Graduate School of Education, Stanford University; Member, Global Learning Council
Data depend on who wants them. Giving university instructors a little data has a big impact on how they teach. The key for effective evidence in universities is not big data, but useful data, especially for the classroom instructor. We have to collect data that is meaningful for the objectives we have, in the settings we are in, and in the conditions we face.
Joan Ferrini-Mundy, Assistant Director, U.S. National Science Foundation, Directorate for Education and Human ResourcesSummary of Remarks
Assistant Director, U.S. National Science Foundation, Directorate for Education and Human Resources
Realizing the promise of a data-driven science of learning has been an NSF priority for many years, especially in its impact on how STEM fields are taught; how more students could be engaged in STEM learning; and in terms of impact on the future STEM labor force. Ferrini Mundy provided some indicators of progress and current challenges from NSF’s perspective. She emphasized the need for a virtuous cycle where research findings lead to systematic development of instructional approaches and model building, which then in turn feeds back new questions for new research.
Friday, September 5, 2014
James H. Shelton III, Deputy Secretary, U.S. Department of EducationSummary of Remarks
James H. Shelton III
Deputy Secretary, U.S. Department of Education
Unlocking Human Potential: Providing Breakthroughs and learning outcomes at scale
Secretary Shelton reviewed the potential for R&D to move forward to achieve breakthroughs in learning that could be rapidly scaled up. Inventions change possibilities, but scaled innovations change lives. There are ways we could do better. Shelton reviewed areas where improvements could be made: knowledge gaps, Design failures, adoption and use. What infrastructure do we need now to expand the research possibilities, a kind of innovation cluster to take this on? He proposed five steps for new ecosystems:
- Design and build a global network of learning research and innovation clusters
- Develop a shared research agenda driven by potential impact
- Launch a focus on a new method, protocol, ethical /legal framework, enabling
- Engage the for-profit sector—government-private partnerships.
- Advocate policies that reward efficacy and evidence.
If our general orientation assumed achievement by every learner, instead of a bell curve assumption, how far could our children go? We can afford it: not acting on what we know is unacceptable. There is a special place in hell for those who would let this stand.
Closing Remarks by Subra Suresh
Subra Suresh, President, Carnegie Mellon University; Chair, Global Learning Council
Subra Suresh reports to the gathering on the GLC business meeting, where two areas of focus in TEL were identified for action in the year ahead. Dr. Suresh also noted that because the center of gravity for TEL has so far been in the U.S., this group is U.S.-centric; the GLC intentionally wants to change this, since the impact and opportunity for TEL will also be felt globally.
Problem Statement #1: Current practitioners do not have a trusted and understandable information source to help them promote evidence-based Technology enhanced learning in higher education.
Action Item: The GLC will develop and provide understandable criteria and evidence for what works in terms of best practices in the design of learning environments; best practices in continuous improvement; and best practices in adoption of these tools.
Problem Statement #2: Many organizations gather large amounts of learning data and are willing to share these data for education and research. However, currently there is a lack of widely accepted principles and criteria for sharing data.
Action Item: GLC will create a framework for principles and criteria for sharing of learning data.
Dr. Suresh noted that GLC cannot tell anyone what to do, but hopes to provide useful input and insight on these problems, given the different backgrounds, interests, and perspectives of GLC members. We will create documents on these two action items, with engagement of many stakeholders between now and April; the goal is to present these to AAU and APLU presidents (most of the world’s top universities) next spring and early summer.
The location of the next meeting of the GLC will be announced in a few months; it may very well be held outside the United States.
Panel 3: K-12 TEL: Readiness for life and learning
How can a cross-sector effort in technology-enhanced learning meet the technical, political, economic, and scientific challenges of improving learning in the primary and secondary school classroom so that students are better prepared for learning and for life?
Lead in & Co-Moderator: Justine Cassell, Associate Vice-Provost, Technology Strategy and Impact and Professor,
Discussant & Co-Moderator: Hunter Rawlings, President, Association of American Universities (AAU)
- Pierre Dillenbourg, Director, Center for Digital Education; Professor, École
Polytechnique Fédérale de Lausanne (EPFL)
- William (Brit) Kirwan, Chancellor University System of Maryland
- Nichole Pinkard, Founder, Digital Youth Network and Remix Learning; Associate
Professor, DePaul University
- Tan Eng Chye, Deputy President & Provost, National University of Singapore
- Suzanne Walsh, Deputy Director, Postsecondary Success, Bill & Melinda Gates Foundation
- Rebecca Winthrop, Senior Fellow and Director, Center for Universal Education, The Brookings Institution
Lead in & Co-Moderator: Justine Cassell
Associate Vice Provost, Technology Strategy and Impact and Professor of Human Computer Interaction, Carnegie Mellon University
Early learning (K-12) has always been regarded differently than higher education; while higher education is dominated by faculty with expertise in subject areas, K-12 is more the domain of philosophers and psychologists. The question for K-12 is less “how can we best teach math” and more “how can we best teach the child?”
Technology is particularly controversial in K-12, and generates a series of tensions, including:
- Fear of TEL vs TEL as panacea
- Play vs learning
- Cognition vs metacognition
- Focus vs multimodality
- Children as knowledge consumer vs knowledge producers
- K-12 as preparation for life and citizenship vs K-12 as preparation for college
These complicated issues form the contentious ground today’s panel will cover—even the GLC debated about what impact the group could have on K-12, given its multiple stakeholders and complex politics.
Discussant & Co-Moderator: Hunter Rawlings
President, Association of American Universities (AAU); Member, Global Learning Council
Dr. Rawlings summarized the panels’ views and asked a general question: Can we scale up technology for K-12 and preserve the quality of schools? He also discussed his own experience working with a DC area prep school, where technology was not involved yet learning outcomes were very impressive. There is no single answer, and many things must be attempted. Despite a difficult policy environment at a time when government funding is stagnant or declining, It is too important not to pursue possible improvements in this space.
Director, Center for Digital Education; Professor, École Polytechnique Fédérale de Lausanne (EPFL)
Dr. Dillenbourg believes there are important gaps in learning scientists’ thinking about technology and education, and outlined three of these gaps:
- We focus too much on higher ed—we need to focus on vocational education and training. How can we use TEL to create greater connection between school and employers? Technology can help bridge the gaps and bring their aims together.
- Teachers do not care about learning gains, pre/test-post/test—they care about discipline, time, space, constraints that affect their real life in the classroom. Researchers must start to address how teachers orchestrate classes.
- Learning science analytics are not ready—we do not have good models for what we do. There is gap between models and the desire to collect data.
William (Brit) Kirwan
Chancellor University System of Maryland
Our greatest challenge as a nation is the undereducation of the U.S. population. This is creating two long term challenges: a decline in competitiveness and a rise in social inequity. Higher ed is the gatekeeper for good jobs, so we are recreating an economic caste system that our ancestors came here to escape. To fix this, we need to make many more students college ready. We need a lower cost way of delivering a high quality education, and technology-enhanced learning could be the way forward. He spoke of the College Park Academy, a charter school connected to the University of in Maryland, and what impressive results they have achieved.
Founder, Digital Youth Network and Remix Learning; Associate Professor, DePaul University
Dr. Pinkard advocated that importance of seeing students and schools in the context of communities, and to understand learning infrastructure at the city level, involving school, afterschool, informal spaces (libraries, museums), and virtual spaces. Now these are fragmented, but much potential in making learning pathways more visible to kids so they could see the many ways to pursue their ideas and interests. She described some models in Chicago that provide these connections, and see schools as just one part of a larger learning system that allows g\kids to move forward at their own pace.
Tan Eng Chye
Deputy President & Provost, National University of Singapore
Noting that this is National Teachers Day in Singapore, he discussed the many experiments of information and communications technology in the 600000 student Singapore school systems. He described the history of the project and the implementation. He noted that some success had been achieved in providing teacher training, but that teachers are confused about their roles and how to define their purpose in an ICT-driven learning environment.
Deputy Director, Postsecondary Success, Bill & Melinda Gates Foundation; Member, Global Learning Council
Walsh spoke of the need to create more connections between higher end and K-12, in particular around acceptance of the Common Core. The Common Core is not about assessment or about particular curriculum content—it is about a way of thinking about skills. Higher ed needs these skills (e.g., find evidence for an argument in a text). TEL can help to create a continuum of skill building between K-12 and higher ed. This could change the conversation between K-12 and higher ed, and really start thinking of lifelong learning, how we support people at different points of life.
Senior Fellow and Director, Center for Universal Education, The Brookings Institution
Dr. Winthrop moved the discussion to a global context, and to education justice for the very poorest is the focus of her work at Brookings. Expectations about education have changed: parents around the world now see the importance of education and schools Access has improved, but outcomes have not. There are 130 million kids not in school at all, and 120 million more in primary school in large classes—the GLC can help with that. Specifically, spreading basic information on learning and how schools can improve learning outcomes, and expanding well designed TEL systems, especially using mobile phones could both be very powerful. Working in developing countries is easier than you might think, and the potential is huge.