West’s PrairieLearn making waves of change across campus

MechSE Associate Professor Matt West’s educational programs PrairieLearn and the Computer-Based Testing Facility (CBTF) have been improving courses in the College of Engineering since their inception. Now, their influence has spread beyond the Bardeen Quad. 

In 2012, West teamed up with Associate Professor Elif Ertekin and Professor Daniel Tortorelli (now Professor Emeritus), for a Strategic Instructional Innovations Program (SIIP) project to restructure fundamental Theoretical and Applied Mechanics (TAM) courses. The trio worked on many aspects of the classes, reforming the discussion sections, the worksheets, and the structure of the lectures—but even with these changes there was still progress to be made. 

“Our initial motivation was about getting people to practice more problems, get immediate feedback on how well they were doing on the problems, and free up the TAs to actually talk about the material and help the students,” West said. 

He released the first version of PrairieLearn in late 2013. The site provides online homework that uses mastery learning theory to adapt to a student’s performance throughout an assignment and determine his/her proficiency in the material. This system is made to ensure that each student gets the practice they individually require for each problem. Although a few similar efforts have been made at other universities, as of this past academic year, PrairieLearn is the first of its kind implemented among Illinois’ peers.

To further increase instructor-student interaction, in 2014 a repurposed Engineering Work Station (EWS) lab became the first CBTF. A computerized examination center, the CBTF tests students, providing multiple tries at a problem, giving instant feedback, and opening up class time.

Initially only testing the TAM 251 sections, the CBTF has since expanded dramatically. It is now located in the Grainger Library, where it holds 80-85 seats and runs 12 hours per day, seven days a week. In the fall of 2017 the CBTF ran 50,000 exams for 6,000 students across multiple engineering departments. Universities around the world have experimented with computer-based testing, but West said that the CBTF at Illinois is the largest and most successful of its type.

Educational psychology states that more frequent and less high-stakes testing is better for students’ long-term retention of information. In the first implementation of the CBTF, for TAM 251 students, West saw the number of A-grades on the final exam double and the number of failing grades decrease by 60 percent. CS 421 students experienced an average increase of 10 percent on their final grade after a semester of using the CBTF. 

For faculty, the creation of PrairieLearn-compatible material opens up an opportunity for reevaluation of homework and exam questions. As they develop the material, they can turn a critical eye to how the course was previously tested to ensure that new questions are representing the desired takeaways from the course.

“I think that’s one of the impacts—that it’s made faculty rethink assessment in general,” said Laura Hahn, Director of the Academy for Excellence in Engineering Education (AE3) in the College of Engineering. 

An unforeseen and critical benefit of PrairieLearn and the CBTF is that it creates an educational paradigm shift away from anecdote-based curriculum assessment. Both programs provide massive amounts of data about how students handle different questions on homework and quizzes.

“Once we saw the data, we could then actually prove that something really is different,” West said. “We could change the way we were teaching that topic and see if it made a difference to the way students learn.”

The success of West’s programs has rippled throughout the College of Engineering, with passionate supporters stepping up. Professor Tim Bretl in the Department of Aerospace Engineering led another SIIP project called, “Growing the PrairieLearn Community,” which aspires to create an arsenal of material to make the programs faculty-friendly and easier to integrate. 

One unique component of the project is a group dubbed the “Knitting Circle,” where faculty of different levels of expertise come together twice weekly to develop their course material side-by-side. This establishes a community of faculty learning from each other’s experiences and ideas—a surprising rarity in academia today. 

“It grew in a collaborative way and I think that’s what draws people in,” Hahn said. “This whole idea of collaborative ownership is pretty unique.”

Beyond engineering, PrairieLearn is used in a statistics course, and instructors of General Chemistry II (104) classes are developing PrairieLearn material to be used in the future. West said he hopes it spreads even further across campus.  

His work has proven to cause multidimensional educational change. Computerizing the testing process has actually re-humanized the learning process, opening up instructor and TA time for student interaction. Implementing mastery, incentivizing practice, and frequent testing has deepened student understanding of material. The use of online programs allows instructors to evaluate teaching strategies using actual data instead of solely anecdotal evidence. Now more than ever, faculty are collaborating to create the best learning experience for their students and optimize the effectiveness of their courses. 

“This is all happening because of professors coming together to work cooperatively,” West said. “It’s not something where this was a company that came in and we paid millions of dollars to the company to do something. I think that’s been really effective because it means the way things work is really being tailored to Illinois and to the capabilities of our students.”