“My supervisor, Jim Portegies, teaches the course Analysis 1, in which first-year students learn to formulate mathematical proofs,” Wemmenhove says. When Portegies noticed that many students were having trouble writing proofs, back in 2019, he started a project to develop an educational tool that would guide them through the process. The program, aptly named Waterproof, instantly detects errors and provides students with detailed feedback to help them progress.

Proof assistants

The program is based on existing accounting tools for mathematicians and computer scientists, known as ‘proof assistants’. “These professional tools are very technical and focused on efficiently formulating complex proofs, but this often makes them too complicated for novice students,” Wemmenhove explains. The goal, therefore, was to develop a user-friendly version that would be especially accessible to first-year math students.

Wemmenhove’s PhD research focused on evaluating the first version of Waterproof and its further development. Based on feedback from the Analysis 1 course and interdisciplinary, applied research, the researcher worked on improvements that would allow the program to increasingly meet the needs of both students and teachers.

Extreme precision

“I did the Bachelor Applied Mathematics at TU/e myself, so I knew it was a difficult subject,” Wemmenhove says. But what exactly does it entail to write mathematical proofs, and why is it so difficult?

“In high school, you barely have to deal with that, so for first-year students it’s really something new,” Wemmenhove explains. “It requires very precise and logical reasoning as to why certain assumptions lead to certain conclusions. Within mathematics, that extreme precision is crucial – otherwise you run the risk of drawing wrong conclusions.”

In addition, a proof’s form is something that takes a lot of getting used to for students. “There are a lot of formal rules for how you build a proof logically and write it down. This is the only way a teacher can properly follow your reasoning and check that you have actually proven what you wanted to prove.”

Bike with training wheels

“Using the program, we want to give students an understanding of the steps involved in proving a theorem. What logical symbols should you use? What steps are involved?” the researcher explains. “To prove that a property holds for all elements in a set, the first step is to introduce an arbitrary element. And if you need to prove that A or B is true, you have to consciously choose and state which of the two you’re going to prove.”

The program helps students practice when to use certain steps and what steps are available at different stages of a proof. “For beginners, this is often still unclear,” the PhD candidate says.

A major advantage that the tool offers is that students receive immediate feedback as they work on an assignment. “Without this tool, they would often not receive feedback from the teacher, in the form of graded homework, until a week later, when they might already be working on another assignment.”

Although the program supports students with homework assignments, during exams they must work without aids, only with pen and paper. “The program is meant to help them at the beginning, to master the necessary skills – a bit like learning to ride a bike with training wheels. But eventually they really have to be able to do it themselves,” says the PhD candidate.

Concrete improvements

To get a broad overview of how students used the program, Wemmenhove evaluated the program in collaboration with Human-Technology Interaction colleagues. This included examining the role of teachers and the program’s embedding in the Analysis 1 course. One improvement that was made on the basis of this concerned the installation process, enabling the students to download the program more easily. Another was the modification of the feedback students receive from the program when they make mistakes, which made it easier to read and follow.

Together with Eindhoven School of Education, Wemmenhove studied the educational effect of the tool. They compared exams of students who did work with Waterproof and students who didn’t, to see if any differences could be observed. The patterns discovered provided new insights. For example, they found that a proof step that was expected on paper wasn’t built into Waterproof as a separate step. “So we added it to the program,” Wemmenhove explains. “By collaborating with different departments and using their expertise, we were able to make concrete improvements to the program and its embedding in the course.”

Practical and accessible

Feedback from students and teachers played a crucial role in the development process, Wemmenhove says. “As they worked with a new version of the program every year, we were able to see how they responded to the improvements and how the tool functioned in practice.” Based on these observations and evaluations, Waterproof was continuously optimized, resulting in the current version.

The intention is to further improve the program and widen its use, so that more and more students receive support from a program that’s both practical and accessible. “We’re currently working with Utrecht University; they want to start using the program for a first-year course as well.” Wemmenhove found it particularly valuable to contribute to a useful tool that really helps students: “I loved seeing how they worked with it.”