How do you measure a university’s interaction with society?

In Sweden, universities have three official mandates – research, education and the so-called ‘third assignment’ “samverkan” – interaction with society. The third assignment stipulates that universities should collaborate with the surrounding society, inform people about their research and work to ensure that their research is of use to society. In the 2012 Research and Innovation Bill, the Swedish government tasked VINNOVA, Sweden’s innovation agency, with developing a model that can be used to evaluate the quality and performance of a university’s interaction with society with the ultimate aim of using this model for resource allocation.  But how do you measure this type of engagement and which indicators should be used?

A seminar held in Stockholm on 5 March 2014, jointly organised by the Swedish Association of Graduate Engineers (Sveriges Ingenjörer), Hanaholmen (Swedish-Finnish Cultural Centre) and VA (Public & Science), took a look at progress to date and discussed some of the challenges and issues. Here is a summary of the event:

THE MODEL UNDER DEVELOPMENT

VINNOVA, Sweden’s innovation agency, is currently half way through a four year project to develop an evaluation model. The first two years have been focused on consultation with universities to identify good examples of science–society interaction and gathering suggestions on models that could be used to evaluate a university’s interaction with society. They have also been studying other countries’ knowledge and experiences, although no international models exist that could be replicated directly.

According to Maria Landgren, Chief Strategist at VINNOVA, the model should be based on principles such as being goal-orientated; take into consideration the management’s approach to society interaction issues; be able to be used for business development purposes; and be used to identify patterns of successful interaction. Such patterns occur, for example, when research students have direct involvement with industry as part of their degree projects or when strategic advice is shared between academia and industry. Maria Landgren stressed that it is important to consider science–society interaction in the broadest sense of the term.

VINNOVA has now started the second part of the project, developing and outlining the first prototype of the model, which includes both quantitative and qualitative indicators. The first version will go out for consultation in Spring 2014.

PRINCIPLES FOR QUALITY

The Swedish Association of Graduate Engineers (Sveriges Ingenjörer) is publishing a report on 28 March based on a year-long investigation into the principles for quality in science–society interaction. The aim of the report is to strengthen the view of interaction as an important activity for all parties involved. According to Dan Brändström, who chairs the Association’s expert group, the starting point should be that interaction with society increases the quality of research and education at universities.  He identified three components of successful interaction: the existence of meeting places (such as Uppsala University’s AIM day that provides a meeting place for industry to meet researchers), mobility of people between academia and society (such as adjunct professors teaching at research schools), and a structure that supports long-term relationships for collaboration (e.g. joint courses and programmes). Dan Brändström believes that interaction should also be measured using a peer review process, not just indicators. He also stressed the importance of not trying to micromanage Swedish universities but trust that they are striving to fulfil the three missions of research, education and societal interaction.

THE NEED FOR QUALITATIVE AND QUANTATIVE INDICATORS

Vetenskap & Allmänhet, VA, (Public & Science) has long argued that resources should be allocated to support interaction between universities and society and that indicators on public engagement should be used for resource allocation to universities.

Maria Lindholm, Research Director at VA, said that despite the ‘third assignment’ being formalised in the University Act, in practice it often isn’t prioritised and there is a lack of support for researchers who want to undertake public engagement activities.

VA has been looking at indicators to measure public engagement and has developed a comprehensive list of possible indicators to measure interaction, including popular scientific publications, public lectures, study visits and research projects carried out in collaboration with external organisations.  According to Maria Lindholm, the limitation with many indicators is their focus on quantity and not quality; they only measure what is requested and is measurable; they are often based on data that is difficult to capture; and they favour universities that have more of a natural connection to external organisations. In addition, they only measure the one-way flow of knowledge from academia to society. Maria Lindholm believes it is important to capture the reciprocal nature of interaction between academia and society and that this calls for a broad perspective on science–society interaction. She also advocates the use of peer review to complement indicators and stressed the importance of testing the models.

Increasing the status of societal interaction within a university’s culture is something that Maria Lindholm sees as a long-term strategic goal, which will take time. Indicators alone will not change attitudes – it must be rewarding for researchers to be involved in public engagement. Both time and resources need to be allocated to the ‘third assignment’.

THE FINNISH PERSPECTIVE

The Chancellor of the University of Helsinki, Thomas Wilhelmsson, gave a Finnish perspective on the issue. In Finland, society interaction indicators are not used directly for resource allocation to universities. However, Finnish universities may use their own indicators for a small part of their resource allocation; for example, active alumni can be used as a measure of interaction, which Thomas Wilhelmsson also sees as a reciprocal concept. Helsinki University’s 2020 strategy has the strapline “To the Top and Out to Society”, which illustrates an understanding of the importance of its surrounding society to the university. Interaction with society must be a natural part of the culture of universities.

Internally, Finnish universities use indicators such as patents, partnership contracts, alumni and external funding to measure interaction with society. Following an internal assessment and discussions, interaction is sometimes rewarded with additional resources. It can also be used a criteria for wage setting.

Thomas Wilhelmsson also sees it as important to recognise the diversity of universities and that each subject area, such as law, physics, the humanities, has a slightly different way of interacting with society. You cannot have the same indicators for all.

USEFULNESS VERSUS UTILISATION

The Deputy Vice Chancellor of Uppsala University, Anders Malmberg highlighted the difference in the wording of the assignments that were given to VINNOVA and the Swedish Research Council (VR) by the Swedish government. Whereas VINNOVA has been asked to propose a model for the distribution of funding based on an evaluation of the ’quality and performance’ of a university’s interaction with its surrounding society, the Swedish Research Council has been asked to develop a model to measure the ’relevance and usefulness’ of research to society. In Anders Malmberg’s view, funding should not be allocated on the basis of what is considered to be ’useful’ research, as it is impossible to determine when research becomes ’useful’ to society. Certain research that is not considered particularly ’useful’ today, could lay the groundwork for important advancements in the future. However, he believes that you can strive for ’utilisation’ i.e. utilising the skills and knowledge that are developed by universities in the best way possible. Interaction with society can be an important tool for the utilisation of research-based knowledge and increasing the quality of both research and education.

Anders Malmberg agreed that in terms of developing a model for assessing and rewarding societal interaction, it must be based upon a broad perspective of collaboration, and be field-normalised i.e. a literary scholar must be compared against other literary scholars. An overall qualitative assessment is needed, otherwise quantitative indicators could be so controlling that universities would spend all their time trying to achieve the indicators rather than interacting with society.

As an example of a successful interaction between academia and industry, he spoke about Uppsala University’s AIM day – a concept where company representatives and researchers meet with each other in group discussions. The aim is to match the needs of companies with experts from academia during a mutual exchange of experience. The secret behind the success of the concept, according to Anders Malmberg, is that the starting point is a company’s problem rather than existing research results.

Sustained commitment from university management is required to bring about a culture change as well as prizes and awards which give positive attention to interaction with society.

CONCLUSIONS

Interaction with society can be measured but it is important to consider what it is that is being measured and how it is measured. Indicators are important but should not be followed blindly and peer review is central to evaluating quality.

Everyone – universities, society, industry and civil society – gain from increased interaction. It should be reciprocal and viewed from a broad perspective. To raise the status of societal interaction within universities will require a culture change, which takes time. So it is good that the work has begun already!


Lägg till en kommentar