As an Alternative to the Tech Transfer Approach, Carnegie Mellon Will Open-Source Dozens of Internally Developed EdTech Tools
Henry Kronk | IBL News
In March, Carnegie Mellon University (CMU) announced an unprecedented initiative. Over the course of the year, they plan to release dozens of digital learning tools they have developed over the past decade on an open-source license. These include the learning analytics platform LearnSphere and their pioneering adaptive learning project the Open Learning Initiative (OLI). In all, CMU estimates $100 million in grants and university funding went into these efforts. The effort was spearheaded by the Simon Initiative, which continues the legacy of Nobel Laureate, Turing Award recipient, and CMU professor Herbert Simon.
Simon made numerous contributions in various fields throughout his lifetime. But in education, he is remembered for the concept of learning engineering. As Simon wrote on the subject, “Most of us were trained as teachers by serving as TA’s in a couple of classes when we were graduate students; and our students, with rare exceptions, have never received any systematic and consistent instruction in how to learn. Yet that is the skill they have been exercising every working day of their lives for more years than they would like to remember. So students don’t study the skills of learning, and university teachers don’t study the skills of teaching.”
By promoting learning engineering, Simon hoped to both study the process of teaching and learning as academics do their own fields and disciplines, and then apply what they’ve learned to improve their own classrooms.
Norman Bier is the director of both the Simon Initiative and the OLI. He has a unique perspective on university-created edtech products. Besides being on campus while other CMU edtech efforts such as Acrobatiq were developed and brought to market, he used to work for iCarnegie. While not strictly a software company, iCarnegie sought to train software developers much like coding bootcamps do today.
IBL News reached Bier to hear more about CMU’s massive open-source effort.
Henry Kronk: Naturally, the main thing I’d like to know is why do this open-source initiative in the first place?
Norman Bier: So Carnegie Mellon doesn’t have a college of education. At the same time, we’ve got a really rich tradition of interesting work at the place where psychology, cognitive science, human-computer interaction, and computer science intersect. We’ve got a rich tradition of building sort of weird cross-discipline teams that you don’t always see in other institutions. Particularly from Herb Simon’s influence, we’ve got this deep tradition of thinking seriously about how to improve learning.
So no, we don’t have a college of education. But what we’ve had from these collaborations are some really exciting and impactful individual projects that range from things like our work in cognitive tutors (in the algebra space, we’ve seen use of the tutors can help K-12 students get an additional year’s worth of training compared to their traditional peers) There’s also the OLI. The OLI is probably best known for its work in statistics where we saw roughly twice the learning outcomes in half the time. But there are a whole host of other things as well from virtual chemistry labs to new tutoring approaches in second language acquisition to educational data mining—lots and lots of work in this space that we’re really proud of.
Henry Kronk: How is CMU education research applied?
Norman Bier: We don’t always see this work being used here on campus for CMU students’ benefit. Despite some of the impressive numbers I’ve just cited, we don’t always see this work getting out into the world in a way that has a real impact in terms of improving learning. Although I do think we collaborate as well as anyone, even here, we often see that projects don’t always integrate well. Sometimes we see replication of effort. So the Simon Initiative was launched to better interconnect and accelerate the work that is happening in this space to position it in a way so that we can really use these tools but also these techniques and approaches we have been developing to transform the learning experience for Carnegie Mellon students, to also get it out into the world in ways that have a broader impact and improve learning outcomes more globally.
In a lot of ways, we follow Herb Simon’s challenge to his colleagues at the end of his career—that if we want to improve learning, we need to stop thinking about it as teaching and start creating it as a community-based research activity … In some ways, our charge is if we don’t have a college of education, we should treat the entire university as a college of education and make the classroom a learning laboratory.
With this diversity of approaches and tools, we need to think about what is common across the work. This is where we really emphasize the ‘learning engineering approach.’
Over the years, we’ve been thinking about how to position these tools and approaches in a way that will have the greatest amount of impact. How do we get more people to engage with them?
Henry Kronk: Why take this approach compared to a more traditional, for-profit go-to-market strategy?
Norman Bier: Particularly in the educational technology space, there are some real challenges in the traditional tech transfer approach. I think that, broadly speaking, the edtech market is not one that broadly emphasizes or rewards effectiveness. Too often, we see these really effective approaches get pushed out into the market and then they end up needing to pivot to focus on the things that are going to help them sell, rather than the things that made the product effective in the first place.
Most of this work still needs to be embedded in its research. We’re not at a point where any of this is mature and the research is finished. But rather, when we think about this as part of an engineering effort, we see this real need to staying close to this ongoing research, recognizing where we have new questions that we’re asking, and if we start to really push on this notion that every new learning experience that we’re designing represents a hypothesis, staying close to the toolset, staying close to the approach becomes really important.
As we’ve investigated new ways towards getting this learning engineering approach broadly used, more broadly accepted, it became increasingly clear that we didn’t have a perfect existing model to tie into.
The tech transfer approach wasn’t going to work. We started saying, ‘Maybe we’ll think about CMU just building this stuff for the world. We can build the world’s best statistics course, everyone can use that.’
Henry Kronk: So then, what specifically is attractive about the open-sourcing model?
Norman Bier: Our ongoing research keeps showing us that 1) there is no perfect statistics course for the world and 2) cultural context is so important for learning that what we really need to do (if we think these tools and approaches are important) is position them in ways such that folks can deploy them in their own context with their own learners’ needs in mind. And if we were going to learn from that approach, we need to position this work in a way that the results coming back from those efforts can come back in a way that we and the rest of the community can see them.
This really started pushing us to more of a community-based approach and that in turn led us to this notion that we need to be more open about our work.
By opening up these tools and opening up the software, I’m not sure that I’m trying to encourage every institution out there to spin up their own instance of the DataShop platform. That’s 1) not very efficient and 2) for a lot of the folks we’re trying to see impacted, they don’t have the resources or expertise to do this. But in order to encourage this use of some of the centralized tools and centralized approaches, that code base was sort of table stakes. It was a way of saying, ‘Look, you can count on the fact that this stuff won’t disappear because it’s out here for you.’
Henry Kronk: A lot of existing coverage of OLI’s open-source initiative is saying something like, ‘Carnegie Mellon is giving these digital tools away.’ Now, this is a purposefully imperfect metaphor, but let’s say I developed a tool to fix my bike. I could hand that tool to somebody else so that they could fix their bike. But that’s not exactly what’s going on here, is it?
Norman Bier: No, I think it’s different in two ways. What we are providing is a garage where you can come, bring your bike, and, to push the metaphor, you need to bring your own data or your own grease or your own brake pads. But what we have is a community-supported bike garage where you can go in and take advantage of some of the tools there.
But if you have the resources, we’re also handing you the tools to go and open your own bike garage. If engaging with this centrally-supported garage isn’t going to work for you, then you have the opportunity to go home and build your own. Take the case of LearnSphere (an iteration of an earlier version of DataShop, which was intended as that one central educational data mining and warehousing infrastructure for the community. A lot of folks had been consistent users over the years, but we were consistently hearing that, though they were comfortable with the notion of sharing their data, and though they were comfortable with the notion of sharing techniques, they weren’t necessarily comfortable with taking their data and sticking it on CMU servers.
So one piece of the LearnSphere project has been to create a more distributed infrastructure so that, if the University of Memphis stands up their own LearnSphere data, they’re now completely in control of their own data. They’re able to expose that so that if others need to use that data for secondary analysis, they can still get access. They participate in some of the larger sharing of analytic methods and some of the visualizations.
Henry Kronk: And then, there’s also the data-sharing aspect.
Norman Bier: Yes. This is starting to take the bike shop metaphor in weird places, but if you want to go off and stand up your own workspace, it can still connect with and share some of the resources with the original bike shop or any other one in a way that should be mutually beneficial to everyone. That’s a really important piece in this. When we talk about the learning engineering approach, we also talk about, in the aggregate, that data feeding fresh discoveries in the learning sciences. A model where everyone goes off, stands up their own DataShop, stands up their own OLI, and only uses that information internally really limits our ability to learn from what is happening, to identify what is working, and to see that pushed out to a larger group of students.
Some piece of this is saying, ‘You don’t need to become dependent on our bike shop and worry that we’re going to close the doors or start charging for it. You could go build your own if you need it. But there is a benefit with engaging and contributing to this work as part of a larger community.’
Henry Kronk: Based on your time spent with other edtech tools and services that have gone the for-profit proprietary route, are you moving forward with this open-source initiative with any specific lessons learned?
Norman Bier: I think when I combine my experience at iCarnegie with some more recent work here at OLI, it’s become increasingly important to find ways to let individuals revise and re-contextualize the learning materials that are getting pushed out. If we are serious about leveraging learning data to improve that learning experience, and we’re serious about faculty needing to be able to make these changes, then we really need to focus on putting tools in their hands that exposes them to the data, makes that information actionable, and gives them that chance to engage in their own closed-loop cycle of review, analysis, design, development, and deployment.
That was a piece that we didn’t have at all at iCarnegie. Again, this was prior to the existence of the concept of open education. Maybe that’s not fair. I have some colleagues at the Open University who would hunt me down and beat me up for saying that. This is prior to the Creative Commons license existing before we were talking about OER. We never got faculty to a place where we were asking, ‘How can we equip faculty to make changes to this?’ let alone, ‘How can we more deeply involve them in this improvement process?’
That’s been a lesson that stretched into OLI. We have a process that has historically produced some really remarkable and effective courseware. But at the same time, to get faculty to engage with the larger course development team … for a lot of faculty, this notion that I don’t need to build a full course, I’ve got this one specific learning challenge I need to address, and I’ve got some ideas about what can be effective—it seemed pretty important to include that population in this work. We’ve been spending a lot of attention and making some deep investments in the kind of authoring and improvement tools that really should allow anyone to come in and make some changes to an existing course or test out their hypothesis but also making sure that there’s going to be data coming back.
