SJSU data on MOOC “experiments” found dubious

MOOC  "Experiments"

A blistering critique of recent data on results of San José State’s MOOC experiments, which are done in collaboration with the online course provider Udacity, calls into question the veracity and transparency of the claimed outcomes.

Phil Hill, a big name in advocating e-learning, picked over the data and found it too limited to draw good conclusions. Even so, the data that was released suggests the alleged MOOC progress had more to do with changing the types of students taking the classes (“student population”) than with making the MOOCs work better for students.

Looking at the numbers released regarding the spring MOOC experiment, the summer MOOC experiment, and SJSU on-campus classes, Hill writes, “The student populations between these three groups are completely different, to the point where other comparisons, such as passing rates or completion rates, should not be made.”

He points out the differences were extreme. “That’s right – the summer pilot includes 53% of students already having a college degree, 48% with a bachelor’s or higher. In the spring, none of the students had a college degree,” he found.

Hill is especially critical of SJSU administrators, people he argues should know how to work with data correctly.

Hill’s comments are circulating among online teaching advocates including EdSurge and others. Check out his critique at e-Literate.

To compare this with a different point of view, check out Udacity head Sebastian Thrun’s blog on the summer results of his MOOCs at San José.

For the overview of what’s going on, check out “Boost for Udacity Project” where Inside Higher Ed explains that two months after San José State announced it would put its partnership with Udacity on “pause,” university officials touted the summer results as “significantly better.”

CFA President Lillian Taiz argued in the news story that the sluggish pass rate among students in the remedial math course suggests the SJSU Plus (MOOC) project is a poor match for students in the California State University System.