Software that helps to keep first-year students on track

Students may find their first few months at college a shock to the system, having been minded by parents and teachers through school.

With newfound freedom and no one to check their homework, make them go to lectures or point them in the general direction of the library, it’s no surprise students might find themselves overwhelmed with all the autonomy.

It's also no surprise that dropout rates among first years are high. The most recent figures from the Higher Education Authority showed 16 per cent of first-years drop out. That means more than one in six first year students will not make it to year two.

In releasing the report, HEA chairman John Hennessy said while Ireland’s participation and progression rates in higher education were good, we need to understand why many students don’t make it.

What colleges do now

Most colleges make an effort to support new students during the sometimes bumpy transition to third level education.

Orientation programmes for first years are commonplace and offer both college-wide and course-specific information to students. Colleges often have designated staff members to help new students, in the form of tutors and advisors or, in UCC’s case, a “first year experience coordinator”, whom students can approach.

University of Limerick’s “first seven weeks” initiative highlights a different theme each week to help students settle in and thrive. In week one, students learn how to navigate the campus. This is followed by study skills and time management, and health and wellness, with fitness sessions and stress management workshops.

College supports are available year-round. Most universities have maths and writing support centres, along with study skills workshops. At Trinity College, some workshops are virtual. DCU has a centre for academic or personal guidance and a student learning centre. Most colleges have fairly extensive supports and introductory programmes in place, though students are not always aware of them.

A novel idea

Prof Alan Smeaton, director of the Insight Centre for Data Analytics at DCU, has developed a new system to support first years adjusting to college life during their first semester.

“Everyone’s end goal in this is student retention or progression. It’s all about improving a student’s performance in a module so they stay in college,” he says. “A lot of first-year dropouts are not because the student is a mismatch to the course. A lot of them are because they’re lost and haven’t found their feet yet. They come from school, where everything is managed for them.”

Smeaton and his team work in the field of learning analytics. They collect and analyse data about learners and the factors that influence them in an effort to improve learning outcomes. It’s a growing area around the world.

“Universities all across the world are using student data footprints to analyse past performance in order to change future performance. That’s looking through the rear-view mirror. PredictED is one of the only examples of looking forward through the windscreen,” he says.

Insight’s solution, the PredictED system, uses academic data to predict how a student is doing in a particular module, that is, whether they are on the path towards passing or failing. Here’s how it works: students sign up for PredictED in a module. The system then monitors their usage of Moodle, the university’s online learning platform. It collects data about how often they log on to Moodle and what they do while they’re there.

It then compares the student’s patterns on Moodle with students that took the same module in previous years, to predict, on a weekly basis, whether the student is likely to pass or fail.

Every week, PredictED sends the student an email, to let them know how they’re doing. If they are not on the right track, the email will contain suggestions, such as who they should approach in student services. It also tells the student how much they have used Moodle that week compared with the rest of the class.

Trial of the system

At the start of the 2014 academic year, Smeaton and his team tested PredictED in 10 first-year modules. The modules ranged from politics to biology to maths. The things they had in common were large class sizes and a history of high failure rates.

The system is designed only for first-years in their first semester. “By the time you reach second semester, you’re a bit more streetwise. Students have already learned how to navigate,” Smeaton says.

He and his team visited each of the modules during the first week of lectures to explain the system. The project had the approval of lecturers, DCU’s data protection officer, Student Support Services and the university’s research ethics committee. Students could choose whether to try PredictED or not. It was completely voluntary, and they had to sign a release form that allowed Insight to use their Moodle data.

Of all the students taking the 10 modules, 75 per cent signed up for PredictED.

Both groups of students – those who signed up and those who didn’t – were identical in terms of academic profile, according to Smeaton.

The trial was a success. Students who signed up to PredictED scored higher than those who didn’t, even after the research team controlled for factors such as how well they did in the Leaving Cert.

“Everyone who opted in had a boost of 3 per cent in their overall mark in that exam.”

The comparison between the control group and the group getting the alerts was unimpeachable,” Smeaton says.

Students have their say

Afterwards, some of the students completed a survey about using PredictED. “The theme that emerged was it was almost like a comfort blanket,” Smeaton says. It was the equivalent of someone watching over the student and caring for them by telling them, every Monday afternoon, whether they were doing okay or needed improving.

“Even though this is a completely data-driven process, the fact that it was feedback of any kind is what they were taking solace from,” he says.

“If your boss says you’re doing a great job, the result of that is the next time you’ll do an even better job because you’ll feel better. You’ll feel empowered. The feedback was causing the students to engage more with the material.

“If you’re not doing a good job, your boss is going to say there are ways in which you can improve . . . To improve people’s productivity, you never do it by pointing out how bad they are, you do it by pointing out how good they are and how they could improve.”

STUDENTS ON PREDICTED: 'IT WAS GOOD TO GET A REMINDER TO REFOCUS'

Killian O’Donnell and Aisling McCormick are two DCU students who tested out the PredictED system last year.

KILLIAN O’DONNELL (19), from Athlone, has just finished his first year studying economics, politics and law. He says it took him a couple of months to get used to college life.

“I was living away from home for the first time. Parents weren’t there to tell you what you needed to be doing. You come home from college, and you could be sitting around watching TV all day if you wanted, and there’d be nobody saying, ‘Will you go and do some work?’ ”

He signed up for PredictED in his business maths module, which he found “hard enough”.

The weekly email that told him how much time he was spending on Moodle compared with the rest of his class was useful, he says. Some weeks he thought he was studying much more than he actually was.

“Some weeks I slacked up a bit, and it was good to get the reminder to help refocus. Sometimes I’d leave one subject for a week or two and then would get the email, and I’d know I had to do more work for it.”

He says the reminders didn’t stress him out, except during exam time. “But I never got too worried about it.”

When asked whether there were any other supports available to him in first year, he mentions the Maths Learning Centre in the library, which he used once or twice. He says there is also a mentoring programme in the Law Society, but those were the only other resources he knew about.

O’Donnell ended up getting a 2.1 in the course.

“I think it was beneficial. I don’t think I would have done as well without it, to be honest,” he said.

AISLING McCORMICK (20) just finished her first year of business studies at DCU. It’s the second time she has been in first year, as she studied law first and decided to switch courses.

She noticed the college newbies in her class leaving assignments until the last minute. “In my second year of college, when I went back to do business, I didn’t do that because I learned the first time. But I think everyone does that at some point.”

McCormick says she wasn’t aware of any supports, other than the Maths Learning Centre, that were available to her in first year.

She signed up for PredictED in her maths module because there wasn’t a good reason not to.

She said the most interesting thing was realising how much everyone else was studying compared with her.

“You have no idea what anyone else is doing in college. When you ask them, they say they’re not doing anything. But the system allows you to see how much they’re studying compared to you.”

McCormick “regularly” got emails saying she was in the bottom 2 per cent of Moodle usage in her module, which she found a bit stressful, but also surprising. “No one wants to be at the bottom. Once you get the email, you want to work harder and do better.”

She received an overall mark of 90 per cent in maths in the end. While she wouldn’t say that was all because of PredictED, “it definitely helped. It was the module I did the best in, even though it was one of the hardest ones. There’s something to be said about it,” she says.

GET WITH THE PROGRAMME: OTHER COLLEGE ANALYTICS PROJECTS

• Purdue University, Indiana: Data scientists at this US university have developed Course Signals, which helps to predict academic and behavioural issues and notifies teachers and students when action is required.
• University of Limerick: Researchers at the University of Limerick are studying different kinds of student data to find methods of improving retention rates for students in transition. One programme focuses on helping students through the first seven weeks of college.
• Open University UK: This university has several ongoing projects in the field of learning analytics. One such is Lace (learning analytics community exchange), which integrates communities in learning analytics and educational data-mining by sharing solutions to real problems, including problems in higher education.
• Desire2learn: This learning management system company has a team of PhDs building algorithms and models to predict student performance. The algorithms make predictions beginning with the first week of term.