Automating Grading of Assignments in a MATLAB Programming Course

When I began teaching Optimal Control and Dynamic Programming with MATLAB^® at Technische Universiteit Eindhoven (TU/e), I anticipated a class size of about 40. To my surprise, 150 students enrolled. I was initially pleased, but I soon realized that the popularity of the course presented a significant problem: I had only two teaching assistants. That semester, one assistant spent virtually all his time grading; it took him three days to test the students’ MATLAB code and record the grades for a single assignment.

The next time I taught the course, we used Cody Coursework™ to grade the assignments automatically. This approach significantly improved learning outcomes. It enabled the teaching assistants to spend more time working directly with students, motivated the students by providing immediate feedback on their work, and showed me what concepts students were learning well and where they needed help.

Teaching Optimal Control and Dynamic Programming with MATLAB

Some instructors teach optimal control and dynamic programming with a heavy emphasis on mathematics—specifically on the Pontryagin’s maximum principle. Others bypass the math and proceed directly to applications. I try to find a balance, but I mostly favor the latter approach because TU/e is a technical university and our students typically prefer practical applications over abstract theory.

MATLAB enabled me to strike a balance between theory and practice. Students learn concepts in the lectures and then implement and test in MATLAB. For example, they learn how to apply the Pontryagin’s maximum principle by solving a two-point boundary value problem using numerical methods in MATLAB. Many of the other functions that the students need are built into MATLAB; when they want to use a linear quadratic regulator (LQR) or Kalman filter, for example, they can simply invoke it from their script instead of having to implement it themselves. Dynamic programming and decision-making problems, such as generating an optimal strategy for selling an asset, are easily coded in MATLAB.

Although my students come from a wide range of backgrounds, most entered the class already familiar with MATLAB. The few transfer students who did not know MATLAB learned it easily as the course progressed; their lack of previous experience was never a hindrance.

Grading Assignments Automatically with Cody Coursework

Introducing Cody Coursework to the class was straightforward. I provided the students with some tips on how to submit their solutions; no lengthy training sessions were required.

For each of the three course assignments, students program MATLAB functions to solve specific problems. When the students think that the functions they have written are correct, they submit them to Cody Coursework for grading. Cody Coursework automatically grades the assignments using test suites that my teaching assistants and I created, and then provides immediate feedback on the correctness of the solution. Students are free to resubmit solutions until they are satisfied with their grade.

The first assignment, on discrete optimization, proved relatively easy. The second, on approximate dynamic programming, was more challenging because there was no optimal solution. This meant students had to try a number of suboptimal strategies, including model predictive control, rollout, and certainty equivalent control. On this assignment, the students came to appreciate the value of instant grading. If they submitted an incorrect solution, they could return to their code to look for ways to improve it without waiting to consult me or the teaching assistants.

For the final assignment, the students were asked to tune the matrices of an LQR controller to satisfy a specific set of constraints. Once again, they used Cody Coursework to check their solutions immediately and continue refining them until they met the requirements. The teaching assistants and I relied on Cody Coursework to grade the assignments and provide a summary of the results, which we used to monitor progress (Figure 1).

Unexpected Benefits of Automated Grading

My primary objective in using Cody Coursework was to minimize the time teaching assistants spent grading assignments. I discovered that Cody Coursework has other benefits.

The instant feedback on assignments proved to be a significant motivator. Seeing one’s classmates successfully complete an assignment shows that it can be done, and inspires students who are struggling to persevere. At the other end of the spectrum, students who grasped the material more easily compete with each other to be the first to find a correct solution.

My teaching assistants and I monitored the results in Cody Coursework to track correct and incorrect solutions, when the solutions were submitted, and the size of the submissions, measured in lines of code (Figure 2). We used this information to identify concepts that the students were finding difficult so that we could later review those concepts in lecture.

We designed the tests used by Cody Coursework to check progressively more sophisticated aspects of the students’ work. For example, on problems that incorporated a cost function, we created one test without a terminal cost and one with a terminal cost. Student solutions that took terminal cost into account would pass both tests, while those that didn’t would pass only the first. We also used both visible tests (in which the correct output to a test shown) and hidden tests (in which the test only reveals whether the code was correct or incorrect).This level of granularity provided a fairer way to assign grades and helped us better understand how well the students were absorbing the material.

Expanding the Use of Cody Coursework

For future versions of the course, I am considering using Cody Coursework to automate exam grading. I am also open to making the course more interactive. For example, if students are not grasping a particular concept, I may add an extra, previously unplanned assignment. Without automated grading, adding another assignment on the fly like this would not be feasible because of the time demands it would place on the teaching assistants.

Meanwhile, I am advocating for expanded use of Cody Coursework at TU/e. As some of my colleagues have already discovered, for any course that uses MATLAB, Cody Coursework is a highly effective tool that saves time and work and fully engages students in the course assignments.

Note: Cody Coursework became MATLAB Grader in July 2018.