AHELO

by

On being seduced by The World University Rankings (2011-12)

Well, it’s ranking season again, and the Times Higher Education/Thomson Reuters World University Rankings (2011-2012) has just been released. The outcome is available here, and a screen grab of the Top 25 universities is available to the right. Link here for a pre-programmed Google News search for stories about the topic, and link here for Twitter-related items (caught via the #THEWUR hash tag).

Polished up further after some unfortunate fall-outs from last year, this year’s outcome promises to give us an all improved, shiny and clean result. But is it?

Like many people in the higher education sector, we too are interested in the ranking outcomes, not that there are many surprises, to be honest.

Rather, what we’d like to ask our readers to reflect on is how the world university rankings debate is configured. Configuration elements include:

  • Ranking outcomes: Where is my university, or the universities of country X, Y, and Z, positioned in a relative sense (to other universities/countries; to peer universities/countries; in comparison to last year; in comparison to an alternative ranking scheme)?
  • Methods: Is the adopted methodology appropriate and effective? How has it changed? Why has it changed?
  • Reactions: How are key university leaders, or ministers (and equivalents) reacting to the outcomes?
  • Temporality: Why do world university rankers choose to release the rankings on an annual basis when once every four or five years is more appropriate (given the actual pace of change within universities)? How did they manage to normalize this pace?
  • Power and politics: Who is producing the rankings, and how do they benefit from doing so? How transparent are they themselves about their operations, their relations (including joint ventures), their biases, their capabilities?
  • Knowledge production: As is patently evident in our recent entry ‘Visualizing the uneven geographies of knowledge production and circulation,’ there is an incredibly uneven structure to the production of knowledge, including dynamics related to language and the publishing business.  Given this, how do world university rankings (which factor in bibliometrics in a significant way) reflect this structural condition?
  • Governance matters: Who is governing whom? Who is being held to account, in which ways, and how frequently? Are the ranked capable of doing more than acting as mere providers of information (for free) to the rankers? Is an effective mechanism needed for regulating rankers and the emerging ranking industry? Do university leaders have any capability (none shown so far!) to collaborate on ranking governance matters?
  • Context(s): How do schemes like the THE’s World University Rankings, the Academic Ranking of World Universities (ARWU), and the QS World University Rankings, relate to broader attempts to benchmark higher education systems, institutions, and educational and research practices or outcomes? And here we flag the EU’s new U-Multirank scheme, and the OECD’s numerous initiatives (e.g., AHELO) to evaluate university performance globally, as well as engender debate about benchmarking too. In short, are rankings like the ones just released ‘fit for purpose’ in genuinely shed light on the quality, relevance and efficiency of higher education in a rapidly-evolving global context?

The Top 400 outcomes will and should be debated, and people will be curious about the relative place of their universities in the ranked list, as well as about the welcome improvements evident in the THE/Thomson Reuters methodology. But don’t be invited into distraction and only focus on some of these questions, especially those dealing with outcomes, methods, and reactions.

Rather, we also need to ask more hard questions about power, governance, and context, not to mention interests, outcomes, and potential collateral damage to the sector (when these rankings are released and then circulate into national media outlets, and ministerial desktops). There is a political economy to world university rankings, and these schemes (all of them, not just the THE World University Rankings) are laden with power and generative of substantial impacts; impacts that the rankers themselves often do not hear about, nor feel (e.g., via the reallocation of resources).

Is it not time to think more broadly, and critically, about the big issues related to the great ranking seduction?

Kris Olds & Susan Robertson

by

Global regionalism, interregionalism, and higher education

The development of linkages between higher education systems in a variety of ‘world regions’ continues apace. Developments in Europe, Asia, Africa, the Gulf, and Latin America, albeit uneven in nature, point to the desire to frame and construct regional agendas and architectures. Regionalism -– a state-led initiative to enhance integration to boost trade and security — is now being broadened out such that higher education, and research in some cases, is being uplifted into the regionalism impulse/dynamic.

The incorporation of higher education and research into the regionalism agenda is starting to generate various forms of interregionalisms as well.  What I mean by this is that once a regional higher education area or research area has been established, at least partially, relations between that region, and other regions (i.e. partners), then come to be sought after. These may take the form of relations between (a) regions (e.g., Europe and Asia), (b) a region and components of another region (e.g., Europe and Brazil; Latin America and the United States; Southeast Asia and Australia). The dynamics of how interregional relations are formed are best examined via case studies for, suffice it to say, not all regions are equals, and nor do regions (or indeed countries) speak with singular and stable voices. Moreover some interregional relations can be practice-oriented, and involve informal sharing of best practices that might not formally be ‘on the books.’

Let me outline two examples of the regionalism/interregionalism dynamic below.

ALFA PUENTES

The first example comes straight from an 8 July 2011 newsletter from the European University Association (EUA), one of the most active and effective higher education institutions forging interregional relations of various sorts.

In their newsletter article, the EUA states (and I quote at length):

The harmonisation agenda in Central America: ALFA PUENTES sub-regional project launch (July 07, 2011)

 EUA, OBREAL, HRK and university association partners from Costa Rica, Guatemala, Honduras, Panama, and Mexico gathered in Guatemala City on 27-28 June both to discuss and formally launch the sub-regional project ‘Towards a qualifications framework for MesoAmerica’, one of the three pillars of the European Commission supported structural project ‘ALFA PUENTES’ which EUA is coordinating.

Hosted by sub-regional project coordinator CSUCA (Consejo Universitario CentroAmericana), and further attended by the sub-regional coordinators of the Andean Community (ASCUN), Mercosur (Grupo Montevideo), partners discussed current higher education initiatives in Central America and how the ALFA PUENTES project can both support and build upon them.

CSUCA, created in 1948 with a mission to further integration in Central America and improve the quality of higher education in the region, has accelerated its agenda over the past 10 years and recently established a regional accreditation body. This endeavour has been facilitated by project partner and EUA member HRK (in conjunction with DAAD) as well as several other donors. The association, which represents around 20 public universities in Central America, has an ambitious agenda to create better transparency and harmonisation of degrees, and has already agreed to a common definition of credit points and a template for a diploma supplement.

Secretary General Dr Juan Alfonso Fuentes Soria stated in a public presentation of the project that ALFA PUENTES will be utilised to generate a discussion on qualifications frameworks and how this may accelerate the Central America objectives of degree convergence. European experience via the Bologna Process will be shared and European project partners as well as Latin American (LA) partners from other regions will contribute expertise and good practice.

ALFA PUENTES is a three-year project aimed at both supporting Latin American higher education convergence processes and creating deeper working relationships between European and Latin American university associations. Thematic sub-regional projects (MesoAmerica, Andean Community and Mercosur) will be connected with a series of transversal activities including a pan-Latin American survey on change forces in higher education, as well as two large Europe-LA University Association Conferences (2012 and 2014).

This lengthy quote captures a fascinating array of patterns and processes that are unfolding right now; some unique to Europe, some unique to Latin America, and some reflective of synergy and complementarities between these two world regions.

TUNING the Americas

The second example, one more visual in nature, consists of a recent map we created about the export of the TUNING phenomenon. As we have noted in two previous GlobalHigherEd entries:

TUNING is a process launched in Europe to help build the European Higher Education Area (EHEA). As noted on the key TUNING website, TUNING is designed to:

Contribute significantly to the elaboration of a framework of comparable and compatible qualifications in each of the (potential) signatory countries of the Bologna process, which should be described in terms of workload, level, learning outcomes, competences and profile.

The TUNING logic is captured nicely by this graphic from page 15 of the TUNING General Brochure.

Over time, lessons learned about integration and educational reform via these types of mechanisms/technologies of governance have come to be viewed with considerable interest in other parts of the world, including Africa, North America, and Latin America. In short, the TUNING approach, an element of the building of the EHEA, has come to receive considerable attention in non-European regions that are also seeking to guide their higher educational reform processes, and as well as (in many cases) region-building processes.

As is evident in one of several ‘TUNING Americas’ maps we (Susan Robertson, Thomas Muhr, and myself) are working on with the support of the UW-Madison Cartography Lab and the WUN, the TUNING approach is being taken up in other world regions, sometimes with the direct support of the European Commission (e.g., in Latin America or Africa). The map below is based on data regarding the institutional take-up of TUNING as of late 2010.


Please note that this particular map only focuses on Europe and the Americas, and it leaves out other countries and world regions. However, the image pasted in below, which was extracted from a publicly available presentation by Robert Wagenaar of the University of Groningen, captures aspects of TUNING’s evolving global geography.

Despite the importance of EU largesse and support, it would be inaccurate to suggest that the EU is foisting TUNING on world regions; this is the post-colonial era, after all, and regions are voluntarily working with this European-originated reform mechanism and Europe-based actors. TUNING also only works when faculty/staff members in higher education institutions outside of Europe drive and then implement the process (a point Robert Wagenaar emphasizes). Or look, for example, at the role of the US-based Lumina Foundation in its TUNING USA initiative. Instead, what we seem to have is capacity building, mutual interests in the ‘competencies’ and ‘learning outcomes’ agenda, and aspects of the best practices phenomenon (all of which help explain the ongoing building of synergy between the OECD’s AHELO initiative with the European/EU-enabled TUNING initiative). This said, there are some ongoing debates about the possible alignment implications associated with the TUNING initiative.

These are but two examples of many emerging regionalisms/interregionalisms in the global higher education landscape; a complicated multiscalar phenomenon of educational reform and ‘modernization,’ and region building, mixed in with some fascinating cases of relational identity formation at the regional scale.

Kris Olds (with thanks to Susan Robertson & Thomas Muhr)

by

The OECD’s AHELO: a PISA for higher education?

Editor’s note: greetings from Paris, one of the ‘calculative centres’ associated with the globalization of higher education.  One of the key institutions associated with this development process is the Paris-based Organisation for Economic Co-operation and Development/Organisation de coopération et de développement économiques (OECD/OCDE) given its work on higher education, as well as on related issues such as innovation, science and technology, and so on.

See below for a recent presentation about the OECD’s Assessment of Higher Education Learning Outcomes (AHELO) initiative. This presentation is courtesy of Diane Lalancette, an Analyst with the AHELO initiative, OECD – Directorate for Education.

In ‘tweeting‘ about this presentation a few weeks ago, I detected that a few people sent it on while calling AHELO “a PISA for higher education”. PISA, for those of you who don’t know, is the OECD’s Programme for International Student Assessment, hence the PISA acronym. As the OECD puts it:

PISA assesses how far students near the end of compulsory education have acquired some of the knowledge and skills that are essential for full participation in society. In all cycles, the domains of reading, mathematical and scientific literacy are covered not merely in terms of mastery of the school curriculum, but in terms of important knowledge and skills needed in adult life.

Yet as Diane Lalancette put it (in a note to me):

While AHELO takes a similar approach to PISA in that it will assess student knowledge and skills directly, it is a feasibility study and will not provide information at national or system level like PISA does.

In short, the focus of the AHELO learning outcomes measures will be at the level of institutions and will not allow for comparisons at national levels, one of the key elements that can put national governments on edge (depending on how well their compulsory education systems do in a relative sense).

Our thanks to the Diane Lalancette and Richard Yelland of the OECD’s Directorate of Education for permission to post the presentation below.

Kris Olds

~~~~~~~~~~~~

by

OECD launches first global assessment of higher education learning outcomes

Editor’s note: the slideshow below about the Assessment of Higher Education Learning Outcomes (AHELO) initiative, and the associated press release, were kindly provided to GlobalHigherEd by Richard Yelland, Head of the Education Management and Infrastructure Division (Directorate for Education), OECD. Coverage of the AHELO launch yesterday, at the Council for Higher Education Accreditation’s 2010 Annual Conference (January 25-28, Washington, D.C.), was evident in today’s Chronicle of Higher Education (‘OECD Project Seeks International Measures for Assessing Educational Quality‘), Inside Higher Ed (‘Measuring Student Learning, Globally‘) and Lloyd Armstrong’s weblog Changing Higher Education.

Today’s guest entry (via slideshow) in GlobalHigherEd is designed to shed light on the nature of AHELO, an initiative that reflects the OECD’s ‘collective learning machinery’ role; a role that numerous stakeholders (e.g., state and provincial governments; non-profit foundations, ministries) link into in a myriad of ways. AHELO is emerging at an historical moment when the clamoring for a better understanding of learning outcomes, and associated processes of quality assurance, is evident around the world. In this context it is important to understand what AHELO is, as perceived by the OECD itself, but also why select agencies and institutions (e.g., the US-based ones noted in the press release) value the OECD’s work.

~~~~~~~~~~~~~

~~~~~~~~~~~~~

OECD launches first global assessment of higher education learning outcomes

1/27/2010

The OECD today announced the launch of the Assessment of Higher Education Learning Outcomes (AHELO) initiative. The AHELO generic assessment component will look at skills such as problem solving and critical thinking. A US$1.2 million contract has been awarded to the Council for Aid to Education based in New York City to develop an international version of the Collegiate Learning Assessment (CLA).

Speaking at the Council for Higher Education Accreditation conference in Washington, DC, Richard Yelland, who is leading the OECD’s AHELO initiative said: “AHELO is a pioneering international attempt to assess the quality of higher education by focussing on what students have learned during their studies and what skills they have acquired. Success will provide higher education systems and institutions with diagnostic tools for improvement that go far beyond anything currently available”.

This ground-breaking project aims to demonstrate that reliable and useful comparisons of learning outcomes can be made on a global scale and will point the way for future improvements.

Welcoming this announcement, US Under-Secretary for Education, Martha Kanter, said: “We appreciate OECD’s leadership to assess student performance on an international scale. The AHELO initiative provides the US with an exciting opportunity to collaborate with other countries to assess higher education learning outcomes in our global society.”

Council for Aid to Education  (CAE)  President Roger Benjamin commented: “Because of its success in important international assessments, the OECD is the right venue for creating AHELO and its generic strand which will focus on the skills thought to be critical for human capital development and citizenship in the 21st century. We are pleased that the CLA has been chosen for this purpose.

Funding for this work comes from participating countries and from the Lumina Foundation for Education which has made a USD750 000 grant to the OECD.

“With Lumina’s investments focused heavily on increasing the number and quality of postsecondary degrees and  credentials, the work of AHELO is essential and will help to ensure that these credentials are learning outcome-based and relevant in the United States as well as internationally,” said Jamie P. Merisotis, president and chief executive officer of Lumina Foundation.

Other components of AHELO will measure student knowledge in two discipline areas – economics and engineering. Contextual analysis will factor in student background and national differences. In time a value-added strand will look at learning gains over time.

Higher education is an increasingly strategic investment for countries and for individuals. It is estimated that some 135 million students study worldwide in more than 17 000 universities and other institutions of post-secondary education.

At least thirteen culturally diverse countries across the globe are joining the US as participants in this groundbreaking project, including Finland, Italy, Mexico, Japan, and Kuwait. AHELO will test a sample of students in a cross-section of institutions in each country. Institutions in four states (Connecticut, Massachusetts, Missouri, and Pennsylvania) will be working together, and with the State Higher Education Executive Officers (SHEEO) association to participate on behalf of the United States.

SHEEO President Paul Lingenfelter said: “This is a real opportunity for institutions in the four states to engage in improving knowledge and practice with respect to college learning outcomes. U.S participation is essential, and we will all benefit from their efforts.”

For information, journalists are invited to contact: Susan Fridy(202) 822-3869 at the OECD Washington Center, or Angela Howard at OECD in Paris +33 1 45 24 80 99. For more information on the AHELO, go to: www.oecd.org/edu/ahelo.