Hackathons replace lectures

To the uninitiated, the word ‘hackathon’ may invoke images of a cybercrime spree. But for those in the know, it actually describes a marathon collaborative programming event that uses big data to solve real-world problems. UTS has teamed up with Transport NSW to run in-class hackathons that help students learn what it really means to be a data scientist and enable Transport NSW to improve their services too.                                           

28 August 2015: four Master of Data Science and Innovation (MDSI) students find themselves crammed around computers, balanced on small wooden tables, sifting through gigabytes of data.

The Unaccompanied Miners, as the team was then known, were on a mission: use coal survey data to develop a way to automate the detection of coal seams and reduce the delays and costs associated with using geologists.

“Our mission was part of Unearthed Sydney – a hackathon aimed at accelerating innovation in the resources sector. It was our very first hackathon, but it wasn’t our last,” says MDSI student and former Ausgrid electrical engineer Perry Stephenson.

The Unaccompanied Miners during Unearthed Sydney. Photo by: Theresa Anderson

Unbeknownst to Stephenson, Passiona Cottee, Daniel Booth and Greg Paul (just four of six members of the MDSI team), their upcoming success would change the way UTS MDSI students learn.

Today, UTS’s Connected Intelligence Centre (CIC) has established a partnership with the NSW Government to embed real-life hackathon challenges into the curriculum of three different MDSI subjects – Data Driven Decision Making, Data Visualisation and Narratives, and Project Management for Data Driven Solutions.

So how did it come about?

Hackathons, also known as ‘Data Challenges’ and ‘Dataslams’, are competitions held by organisations as a way to harness innovative ideas that use big data to help make their processes more efficient and cost-effective.

The participants are given the data and a time limit, after which they present their findings to a panel of judges who determine the viability of their solutions.

At Unearthed Sydney, the Unaccompanied Miners placed 5th out of the 20 competing teams.

"It was good because we spent two-and-a-half days just learning,” explains Stephenson. “It wasn’t just lecturers giving us materials, it was like, ‘What do I need to solve this problem? I'm going to try this, this and this. Oh, this kind of worked, let's go forward from here’.”

Senior Lecturer at CIC and Course Director of the MDSI Dr Theresa Anderson was the driving force behind the Unaccompanied Miners’ appearance at Unearthed Sydney.

She says, although the team went into the hackathon with very little expectation or experience, they did have high levels of enthusiasm.

"They were up against really well-equipped computer science specialists from various programs," says Anderson. "What I like to point out is that they lost out to four PhDs in machine learning. The group that won at the Unearthed challenge had been working as a team for a few years.”

Stephenson agrees. "We were expecting to be better at it. There were teams that actually produced real things at the end. We developed a really solid pitch based on the machine learning system we were trying to build, but the system itself was completely unsuccessful. So, of course we should've lost, but it was a fairly good effort for the first time.”

But the experience did give Anderson an idea – keep the team together, develop their partnerships and skill sets, then send them out to battle again in a new hackathon. This time for Transport NSW.

The team, now renamed Team Gosling (an homage to their first training room for the Transport NSW hackathon) were confident. Their new mission was to use advanced machine learning techniques to deliver new insights from Opal card data to determine what drives patronage on the City Rail network.

"If you want to increase patronage on the network, then look at what drives it," explains Stephenson. "If people aren't catching a train from the station when it’s raining, then maybe go and have a look at that station and see why. That's a useful bit of information you can use to make business decisions in the planning sphere.”

This time, Team Gosling were awarded first place. "It was pretty good,” recalls Stephenson. “It’s a fairly lucrative sort of activity; the prize money for that hackathon was $16,000 split within the team.

"Considering we didn't even know we were going to be doing it two months beforehand, it was a fairly good windfall."

Following Team Gosling’s success, Anderson began working with Transport NSW to develop a way to introduce hackathons into the MDSI curriculum. Today, the three MDSI subjects that include hackathon assessments all focus on a real-life business problem identified by Transport NSW.

“The challenge from Transport NSW has moved from one subject to another during the semester,” explains Anderson.

Before one class hands the project over to the next, Transport NSW meets with the MDSI subject coordinators to make sure the business problem will be clear to the next round of students and meet that subject’s core learning objectives.

"Each round is with a different class,” adds Anderson. “But what holds it together is that in each one of those classes I know I have a core group of four students, who are full-time students, who were taking every single one of those subjects. They became the carriers of knowledge from one round to the next."

Paul, a Team Gosling member and former electrical engineer for Schneider Electric, believes hackathons are vital for educating prospective data analysts.

In a hackathon, says Paul, "You have to make a lot of compromises to get to the end point because of time constraints. For example, you can’t do a lot of heavy processing that you’d normally like to do.

"But what it does expose you to is the way it would work in a real job. In a real job there's less handling. It's more like, ‘Look, we want a solution to this problem, here's some information, go and get started’."

For Paul, such activities develop skills you can’t obtain through ordinary assessments. "In an assessment there's an existing end point; there is a right answer. In the real world though, there really aren’t right answers, but there are good answers. And this teaches you how to get there.

"It gives you perspective – what does the company want? What does the client want? What are they looking out for? And with better questions, you get better answers."