University institutional performance: HEFCE, UK universities and the media

deem11 This entry has been kindly prepared by Rosemary Deem, Professor of Sociology of Education, University of Bristol, UK. Rosemary’s expertise and research interests are in the area of higher education, managerialism, governance, globalization, and organizational cultures (student and staff).

Prior to her appointment at Bristol, Rosemary was Dean of Social Sciences at the University of Lancaster. Rosemary has served as a member of ESRC Grants Board 1999-2003, and Panel Member of the Education Research Assessment Exercise 1996, 2001, 2008.

GlobalHigherEd invited Rosemary to respond to one of the themes (understanding institutional performance) in the UK’s Higher Education Debate aired by the Department for Innovation, Universities and Skills  (DIUS) over 2008.

~~~~~~~~~~~~~~

Institutional performance of universities and their academic staff and students is a very topical issue in many countries, for potential students and their families and sponsors, governments and businesses. As well as numerous national rankings, two annual international league tables in particular, the Shanghai Jiao Tong,  developed for the Chinese government to benchmark its own universities and the commercial Times Higher top international universities listings, are the focus of much government and institutional  interest,  as  universities vie with each other to appear in the top rankings of so-called world-class universities, even though the quest for world-class status has negative as well as positive consequences for national higher education systems (see here).

International league tables often build on metrics that are themselves international (e.g publication citation indexes) or use proxies for quality such as the proportions of international students or staff/student ratios, whereas national league tables tend to develop their own criteria, as the UK Research Assessment Exercise (RAE) has done and as its planned replacement, the Research Excellence Framework is intended to do. deem2

In March 2008, John Denham, Secretary of State for (the Department of) Innovation, Universities and Skills (or DIUS) commissioned the Higher Education Funding Council for England (HEFCE) to give some advice on measuring institutional performance. Other themes  on which the Minister commissioned advice, and which will be reviewed on GlobalHigherEd over the next few months, were On-Line Higher Education Learning, Intellectual Property and research benefits; Demographic challenge facing higher education; Research Careers; Teaching and the Student Experience; Part-time studies and Higher Education; Academia and public policy making; and International issues in Higher Education.

Denham identified five policy areas for the report on ‘measuring institutional performance’ that is the concern of this entry, namely: research, enabling business to innovate and engagement in knowledge transfer activity, high quality teaching, improving work force skills and widening participation.

This list could be seen as a predictable one since it relates to current UK government policies on universities and strongly emphasizes the role of higher education in producing employable graduates and relating its research and teaching to business and the ‘knowledge economy’.

Additionally, HEFCE already has quality and success measures and also surveys, such as the National Student Survey of all final year undergraduates for everything except workforce development.  The five areas are a powerful indicator of what government thinks the purposes of universities are, which is part of a much wider debate (see here and here).

On the other hand, the list is interesting for what it leaves out – higher education institutions and their local communities (which is not just about servicing business), or universities’ provision for supporting the learning of their own staff (since they are major employers in their localities) or the relationship between teaching and research

The report makes clear that HEFCE wants to “add value whilst minimising the unintended consequences”, (p. 2), would like to introduce a code of practice for the use of performance measures and does not want to introduce more official league tables in the five policy areas.  There is also a discussion about why performance is measured: it may be for funding purposes, to evaluate new policies, inform universities so they can make decisions about their strategic direction, improve performance or to inform the operation of markets. The disadvantages of performance measures, the tendency for some measures to be proxies (which will be a significant issue if plans to use metrics and bibliometrics  as proxies for research quality in  the new Research Excellence Framework are adopted) and the tendency to measure activity and volume but not impact are also considered in the report.

However, what is not emphasized enough are that the consequences once a performance measure is made public are not within anyone’s control.  Both the internet and the media ensure that this is a significant challenge.  It is no good saying that “Newspaper league tables do not provide an accurate picture of the higher education sector” (p 7) but then taking action which invalidates this point.

Thus in the RAE 2008, detailed cross-institutional results were made available by HEFCE to the media before they are available to the universities themselves last week, just so that newspaper league tables can be constructed.

Now isn’t this an example of the tail wagging the dog, and being helped by HEFCE to do so? Furthermore, market and policy incentives may conflict with each other.  If an institution’s student market is led by middle-class students with excellent exam grades, then urging them to engage in widening participation can fall on deaf ears.   Also, whilst UK universities are still in receipt of significant public funding, many also generate substantial private funding too and some institutional heads are increasingly irritated by tight government controls over what they do and how they do it.

Two other significant issues are considered in the report. One is value-added measures, which HEFCE feels it is not yet ready to pronounce on.  Constructing these for schools has been controversial and the question of over what period should value added measures be collected is problematic, since HEFCE measures would look only at what is added to recent graduates, not what happens to them over the life course as a whole.

The other issue is about whether understanding and measuring different dimensions of institutional performance could help to support diversity in the sector.  It is not clear how this would work for the following three reasons:

  1. Institutions will tend to do what they think is valued and has money attached, so if the quality of research is more highly valued and better funded than quality of teaching, then every institution will want to do research.
  2. University missions and ‘brands’ are driven by a whole multitude of factors and importantly by articulating the values and visions of staff and students and possibly very little by ‘performance’ measures; they are often appealing to an international as well as a national audience and perfect markets with detailed reliable consumer knowledge do not exist in higher education.
  3. As the HEFCE report points out, there is a complex relationship between research, knowledge transfer, teaching, CPD and workforce development in terms of economic impact (and surely social and cultural impact too?). Given that this is the case, it is not evident that encouraging HEIs to focus on only one or two policy areas would be helpful.

There is a suggestion in the report that web-based spidergrams based on an seemingly agreed (set of performance indicators might be developed which would allow users to drill down into more detail if they wished). Whilst this might well be useful, it will not replace or address the media’s current dominance in compiling league tables based on a whole variety of official and unofficial performance measures and proxies. Nor will it really address the ways in which the “high value of the UK higher education ‘brand’ nationally and internationally” is sustained.

Internationally, the web and word of mouth are more critical than what now look like rather old-fashioned performance measures and indicators.  In addition, the economic downturn and the state of the UK’s economy and sterling are likely to be far more influential in this than anything HEFCE does about institutional performance.

The report, whilst making some important points, is essentially introspective, fails to sufficiently grasp how some of its own measures and activities are distorted by the media, does not really engage with the kinds of new technologies students and potential students are now using (mobile devices, blogs, wikis, social networking sites, etc) and focuses far more on national understandings of institutional performance than on how to improve the global impact and understanding of UK higher education.

Rosemary Deem

McDonalds to offer its own nationally-recognised qualifications in the UK

While GlobalHigherEd‘s focus is on higher education, there are major developments afoot in the UK’s post-16 sector that are worth drawing attention to because of the knock-on implications for the UK’s higher education system.

Today a BBC report stated that the Qualification and Curriculum Authority (QCA) in the UK had just announced that the fast food giant – McDonalds, along with Airline Flybe and Network Rail – would become one of the first set of private firms in the UK to offer their own nationally-recognized qualifications – up to A-Level standard. The BBC reported that McDonalds regarded this as “an important and exciting new step for the company” whilst the Minister in charge of this initiative – John Denham – stated that:

…this is an important step toward ending the old divisions between company training schemes and national qualifications…

For those of you who don’t know the UK system well, A-levels subjects are those studied for university entrance. To date, schools and colleges have been the only providers of A-level courses, and these are regulated through systems of professional training, inspection and assessment.

There is little detail yet available on how these new education providers – McDonald’s, Network Rail and Airline Flybe – will be regulated or how the training itself will be funded (government?). What we do know, however, is that the course content will be tailored to the training needs of the firm, and that the focus will be on workplace skills rather than broader competencies that might develop the whole person.

These new qualifications will face some hostility from the university sector. The BBC report notes (following interviews with 10 university admissions tutors from leading universities) that some universities are already stating they would not accept students who have taken these diplomas.

Nor are the  teaching unions happy about this, according to the Financial Times – with Chris Keates, general secretary of NASUWT arguing:

It is a huge mistake to allow McDonald’s, with its poor track record of employment practice and anti-trade union attitude, to pioneer private sector provision of training.”

There are also industry-based training issues facing this new qualifications venture, as this Financial Times report pointed out:

One independent industry analyst acknowledged national accreditation would increase recognition of McDonald’s qualifications among other employers in the “hospitality industry”. But he said a McDonald’s qualification would not be useful throughout the sector. The skills required in “a high-class restaurant”, including “the etiquette and serving techniques, are quite different.

Today’s announcement by QCA – which essentially amounts to opening up ‘the education system’ to new players in the field – should be seen as part of wider deregulatory trends in the UK education scene. In 2007 GlobalHigherEd reported on a Privy Council decision to give BPP Holdings plc (BPP), Europe’s leading provider of professional education, degree awarding powers. As we noted at the time, not only did this give a for-profit private sector company permission to enter a very substantial and profitable market, but it had the right to award a qualification that has been the preserve of so called proper ‘universities’ . It is also worth noting that this initiative is in alignment with changes to the regulatory system in other countries including Australia. As the Financial Times noted today:

A skills expert in Whitehall said that by giving awarding power to large employers, England was moving into line with other rich countries such as Australia – where Coles, the huge supermarket chain, has a large nationally accredited programme.

The QCA’s policy decision – to let firms like McDonalds award an A-level qualification – will surely present employers and universities with some interesting challenges about how these are to be treated. Like the Privy Council’s decision, it may well be looked back upon as a watershed moment in the governance of the ‘education system’ in the UK.

Susan Robertson