Australasian Science: Australia's authority on science since 1938

Time for a New Measure of Research Impact

By Margaret Hartley

We need to measure industry engagement as well as publications.

Industry Minister Ian Macfarlane set off a small storm recently when he suggested that university researchers should receive research grants based on the number of patents they register, rather than the number of papers they publish.

Shadow Minister Kim Carr attacked his comments, saying that Macfarlane’s approach would have deprived the world of inventions such as WiFi, which were produced as a result of basic, curiosity-driven Australian research.

Macfarlane is on the right track. Ultimately, as a nation, we hope that the research conducted in our universities leads to new technologies and applications, even if that wasn’t the original intent of the research.

Additionally, it has been demonstrated that the establishment of the ongoing collaborative relationships between researchers and business help drive innovation within companies. Collaboration also helps introduce new products to our markets. To encourage this, our researchers need incentives to engage and collaborate with business, as it is the commercial sector that translates the results of research into applications.

Nobel Laureate Joseph Stiglitz, who recently visited Australia, tells us that “what you measure affects what you do. If you don’t measure the right thing, you don’t do the right thing.” Perhaps nowhere is this netter illustrated than in Australia’s research sector.

But Senator Carr is worth listening to as well. Simply measuring patent numbers is too limited. There is a fear that relying on patent numbers would encourage the filing of “junk” patents that produce no tangible product or application.

Research “worth” is presently measured through a process known as Excellence in Research Australia (ERA), which primarily uses publication records and citations in academic journals to determine “excellence”. Professor Aidan Byrne, the CEO of the Australian Research Council, which administers ERA, has pointed out that using a single indicator – be it patents, papers, or something else – would create perverse incentives for researchers.

Unfortunately, this is already the case with the ERA, where the ERA ratings affect research funding and student demand. In doing so, the ERA influences academic behaviour – causing academics to focus on academic publications and citations and, in great part, to ignore engagement and collaboration with industry.

Our studies confirm that the current ERA-based system of recognising and rewarding excellence discourages researchers from collaborating with business. We need a way to ensure that industry engagement by universities and researchers is appropriately recognised and rewarded.

The Academy of Technological Sciences and Engineering (ATSE) – which numbers among its Fellows some of Australia’s leading and most respected applied scientists, technologists and engineers – has proposed a parallel system to ERA called Impact and Engagement for Australia. This metric would take data already collected from universities as part of ERA to provide an indication of how effective researchers are at collaborating with business.

Rather than focusing just on patent numbers, ATSE proposes looking at the value of intellectual property, measured through the amount of non-government income brought into an institution. This could be from sources such as patent royalties or licensing (hence indicating that the IP was actually doing something useful), or research funding from industry (major companies aim to make sure their money is invested in researchers doing work they consider worthwhile).

In the end, our approach to rating science and technology research needs to be bipartisan or we will miss the boat. Australia rates last in the OECD for collaboration between researchers and industry. As the Chief Scientist, Professor Ian Chubb, has continually highlighted, we are the only OECD country without a national science and technology strategy.

Helping to incentivise our researchers to engage with business – by measuring the right things – would be a positive move towards a prosperous and healthy future.

Dr Margaret Hartley FTSE is Chief Executive of the Australian Academy of Technological Sciences and Engineering.