Field-specific cultures of international research collaboration

Editors’ note: how can we better understand and map out the phenomenon of international research collaboration, especially in a context where bibliometrics does a patchy job with respect to registering the activities and output of some fields/disciplines? This is one of the questions Dr. Heike Jöns (Department of Geography, Loughborough University, UK) grapples with in this informative guest entry in GlobalHigherEd. The entry draws from Dr. Jöns’ considerable experience studying forms of mobility associated with the globalization of higher education and research.

Dr. Jöns (pictured above) received her PhD at the University of Heidelberg (Germany) and spent two years as a Feodor Lynen Postdoctoral Research Fellow of the Alexander von Humboldt Foundation at the University of Nottingham (UK). She is interested in the geographies of science and higher education, with particular emphasis on transnational academic mobility.

Further responses to ‘Understanding international research collaboration in the social sciences and humanities’, and Heike Jöns’ response below, are welcome at any time.

Kris Olds & Susan Robertson

~~~~~~~~~~~~~~~~~~~~~

The evaluation of research performance at European universities increasingly draws upon quantitative measurements of publication output and citation counts based on databases such as ISI Web of Knowledge, Scopus and Google Scholar (UNESCO 2010). Bibliometric indicators also inform annually published world university rankings such as the Shanghai and Times Higher Education rankings that have become powerful agents in contemporary audit culture despite their methodological limitations. Both league tables introduced field-specific rankings in 2007, differentiating between the natural, life, engineering and social sciences (both rankings), medicine (Shanghai) and the arts and humanities (Times Higher).

But to what extent do bibliometric indicators represent research output and collaborative cultures in different academic fields? This blog entry responds to this important question raised by Kris Olds (2010) in his GlobalHigherEd entry titled ‘Understanding international research collaboration in the social sciences and humanities‘ by discussing recent findings on field-specific research cultures from the perspective of transnational academic mobility and collaboration.

The inadequacy of bibliometric data for capturing research output in the arts and humanities has, for example, been demonstrated by Anssi Paasi’s (2005) study of international publishing spaces. Decisions about the journals that enter the respective databases, their bias towards English-language journals and their neglect of monographs and anthologies that dominate in fields dominated by individual authorship are just a few examples for the reasons of why citation indexes are not able to capture the complexity, place- and language-specificity of scholarship in the arts and humanities. Mapping the international publishing spaces in the sciences, the social sciences and the arts and humanities using ISI Web of Science data in fact suggests that the arts and humanities are less international and even more centred on the United States and Europe than the sciences (Paasi 2005: 781). Based on the analysis of survey data provided by 1,893 visiting researchers in Germany in the period 1954 to 2000, this GlobalHigherEd entry aims to challenge this partial view by revealing the hidden dimensions of international collaboration in the arts and humanities and elaborating on why research output and collaborative cultures vary not only between disciplines but also between different types of research work (for details, see Jöns 2007; 2009).

The visiting researchers under study were funded by the Humboldt Research Fellowship Programme run by the Alexander von Humboldt Foundation (Bonn, Germany). They came to Germany in order to pursue a specific research project at one or more host institutions for about a year. Striking differences in collaborative cultures by academic field and type of research work are revealed by the following three questions:

1. Could the visiting researchers have done their research project also at home or in any other country?

2. To what extent did the visiting researchers write joint publications with colleagues in Germany as a result of their research stay?

3. In which ways did the collaboration between visiting researchers and German colleagues continue after the research stay?

On question 1.

Research projects in the arts and humanities, and particularly those that involved empirical work, were most often tied to the research context in Germany. They were followed by experimental and theoretical projects in engineering and in the natural sciences, which were much more frequently possible in other countries as well (Figure 1).

Figure 1 — Possibility of doing the Humboldt research project in another country than Germany, 1981–2000 (Source: Jöns 2007: 106)

These differences in place-specificity are closely linked to different possibilities for mobilizing visiting researchers on a global scale. For example, the establishment of new research infrastructure in the physical, biological and technical sciences can easily raise scientific interest in a host country, whereas the mobilisation of new visiting researchers in the arts and humanities remains difficult as language skills and cultural knowledge are often necessary for conducting research projects in these fields. This is one reason for why the natural and technical sciences appear to be more international than the arts and humanities.

On question 2.

Joint publications with colleagues in Germany were most frequently written in physics, chemistry, medicine, engineering and the biological sciences that are all dominated by multi-authorship. Individual authorship was more frequent in mathematics and the earth sciences and most popular – but with considerable variations between different subfields – in the arts and humanities. The spectrum ranged from every second economist and social scientist, who wrote joint publications with colleagues in Germany, via roughly one third in language and cultural studies and history and every fifth in law to only every sixth in philosophy. Researchers in the arts and humanities had much more often than their colleagues from the sciences stayed in Germany for study and research prior to the Humboldt research stay (over 95% in the empirical arts and humanities compared to less than 40% in the theoretical technical sciences) as their area of specialisation often required learning the language and studying original sources or local research subjects. They therefore engaged much more closely with German language and culture than natural and technical scientists but due to the great individuality of their work, they produced not only considerably less joint publications than their apparently more international colleagues but their share of joint publications with German colleagues before and after the research stay was fairly similar (Figure 2).

Figure 2 — Joint publications of Humboldt research fellows and colleagues in Germany, 1981–2000 (Source: Jöns 2007: 107)

For these reasons, internationally co-authored publications are not suitable for evaluating the international attractiveness and orientation of different academic fields, particularly because the complexity of different types of research practices in one and the same discipline makes it difficult to establish typical collaborative cultures against which research output and collaborative linkages could be judged.

On question 3.

This is confirmed when examining continued collaboration with colleagues in Germany after the research stay. The frequency of continued collaboration did not vary significantly between disciplines but the nature of these collaborations differed substantially. Whereas regular collaboration in the natural and technical sciences almost certainly implied the publication of multi-authored articles in internationally peer-reviewed journals, continued interaction in the arts and humanities, and to a lesser extent in the social sciences, often involved activities beyond the co-authorship of journal articles. Table 1 documents some of these less well-documented dimensions of international research collaboration, including contributions to German-language scientific journals and book series as well as refereeing for German students, researchers and the funding agencies themselves.



Table 1 — Activities of visiting researchers in Germany after their research stay (in % of Humboldt research fellows 1954-2000; Source: Jöns 2009: 327)

The differences in both place-specificity and potential for co-authorship in different research practices can be explained by their particular spatial ontology. First, different degrees of materiality and immateriality imply varying spatial relations that result in typical patterns of place-specificity and ubiquity of research practices as well as of individual and collective authorship. Due to the corporeality of researchers, all research practices are to some extent physically embedded and localised. However, researchers working with physically embedded material research objects that might not be moved easily, such as archival material, field sites, certain technical equipment, groups of people and events, may be dependent on accessing a particular site or local research context at least once. Those scientists and scholars, who primarily deal with theories and thoughts, are in turn as mobile as the embodiment of these immaterialities (e.g., collaborators, computers, books) allows them to be. Theoretical work in the natural sciences, including, for example, many types of mathematical research, thus appears to be the most ‘ubiquitous’ subject: Its high share of immaterial thought processes compared to relatively few material resources involved in the process of knowledge production (sometimes only pen and paper) would often make it possible, from the perspective of the researchers, to work in a number of different places (Figure 1, above).

Second, the constitutive elements of research vary according to their degree of standardisation. Standardisation results from the work and agreement previously invested in the classification and transformation of research objects. A high degree of standardisation would mean that the research practice relies on many uniform terms, criteria, formulas and data, components and materials, methods, processes and practices that are generally accepted in the particular field of academic work. Field sites, for example, might initially show no signs of standardisation, whereas laboratory equipment such as test tubes may have been manufactured on the basis of previous – and then standardised – considerations and practices. The field site may be unique, highly standardised laboratory equipment may be found at several sites to which the networks of science have been extended, thereby offering greater flexibility in the choice of the research location. In regard to research practices with a higher degree of immateriality, theoretical practices in the natural and technical sciences show a higher degree of standardisation (e.g., in terms of language) when compared to theoretical and argumentative-interpretative work in the arts and humanities and thus are less place-specific and offer more potential for co-authorship (Figures 1 and 2).

The resulting two dimensional matrix on the spatial relations of different research practices accommodates the empirically observed differences of both the place-specificity of the visiting researchers’ projects and their resulting joint publications with colleagues in Germany (Figure 3):

Figure 3 — A two-dimensional matrix on varying spatial relations of different research practices (Source: Jöns 2007: 109)

Empirical work, showing a high degree of materiality and a low degree of standardisation, is most often dependent on one particular site, followed by argumentative-interpretative work, which is characterised by a similar low degree of standardisation but a higher degree of immateriality. Experimental (laboratory) work, showing a high degree of both materiality and standardisation, can often be conducted in several (laboratory) sites, while theoretical work in the natural sciences, involving both a high degree of immateriality and standardisation is most rarely tied to one particular site. The fewest joint publications were written in argumentative-interpretative work, where a large internal (immaterial) research context and a great variety of arguments from different authors in possibly different languages complicate collaboration on a specific topic. Involving an external (material) and highly standardised research context, the highest frequency of co- and multi-authorship was to be found in experimental (laboratory) work. In short, the more immaterial and standardised the research practice, the lower is the place-specificity of one’s work and the easier it would be to work at home or elsewhere; and the more material and standardised the research practice, the more likely is collaboration through co- and multi-authorship.

Based on this work, it can be concluded – in response to two of Kris Olds’ (2010) key questions – that international research collaboration on a global scale can be mapped – if only roughly – for research practices characterised by co- and multi-authorship in internationally peer-reviewed English language journals as the required data is provided by citation databases (e.g., Wagner and Leydesdorff 2005; Adams et al. 2007; Leydesdorff and Persson 2010; Matthiessen et al. 2010; UNESCO 2010). When interpreting such mapping exercises, however, one needs to keep in mind that the data included in ISI Web of Knowledge, Scopus and Google Scholar do itself vary considerably.

Other research practices require different research methods such as surveys and interviews and thus can only be mapped from specific perspectives such as individual institutions or groups of researchers (for the application of bibliometrics to individual journals in the arts and humanities, see Leydesdorff and Salah 2010). It might be possible to create baseline studies that help to judge the type and volume of research output and international collaboration against typical patterns in a field of research but the presented case study has shown that the significance of specific research locations, of individual and collective authorship, and of different types of transnational collaboration varies not only between academic fields but also between research practices that crisscross conventional disciplinary boundaries.

In the everyday reality of departmental research evaluation this means that in fields such as geography, a possible benchmark of three research papers per year may be easily produced in most fields of physical geography and some fields of human geography (e.g. economic and social) whereas the nature of research practices in historical and cultural geography, for example, might make it difficult to maintain such a high research output over a number of subsequent years. Applying standardised criteria of research evaluation to the great diversity of publication and collaboration cultures inevitably bears the danger of leading to a standardisation of academic knowledge production.

Heike Jöns

References

Adams J, Gurney K and Marshall S 2007 Patterns of international collaboration for the UK and leading partners Evidence Ltd., Leeds

Jöns H 2007 Transnational mobility and the spaces of knowledge production: a comparison of global patterns, motivations and collaborations in different academic fields Social Geography 2 97-114  Accessed 23 September 2010

Jöns H 2009 ‘Brain circulation’ and transnational knowledge networks: studying long-term effects of academic mobility to Germany, 1954–2000 Global Networks 9 315-38

Leydesdorff L and Persson O 2010 Mapping the geography of science: distribution patterns and networks of relations among cities and institutes Journal of the American Society for Information Science & Technology 6 1622-1634

Leydesdorff L and Salah A A A 2010 Maps on the basis of the Arts &Humanities Citation Index: the journals Leonardo and Art Journal, and “Digital Humanities” as a topic Journal of the American Society for Information Science and Technology 61 787-801

Matthiessen C W, Schwarz A W and Find S 2010 World cities of scientific knowledge: systems, networks and potential dynamics. An analysis based on bibliometric indicators Urban Studies 47 1879-97

Olds K 2010 Understanding international research collaboration in the social sciences and humanities GlobalHigherEd 20 July 2010  Accessed 23 September 2010

Paasi A 2005 Globalisation, academic capitalism, and the uneven geographies of international journal publishing spaces Environment and Planning A 37 769-89

UNESCO 2010 World Social Science Report: Knowledge Divides UNESCO, Paris

Wagner C S and Leydesdorff L 2005 Mapping the network of global science: comparing international co-authorships from 1990 to 2000 International Journal of Technology and Globalization 1 185–208


Understanding international research collaboration in the social sciences and humanities

How can we map out and make sense of the changing nature of research collaboration at a global scale? This is an issue many people and institutions are grappling with, with no easy solutions.

As noted in several previous GlobalHigherEd entries:

collaboration between researchers across space is clearly increasing, as well being increasingly sought after. From a sense that ‘global challenges’ like climate change demand collaboration, through to a sense that international collaboration generates higher impact (in a citation impact factor sense) output, there are signs that the pressure to facilitate collaboration will only increase.

At the same time, however, government ministries, funding councils, higher education associations, and universities themselves, are all having a challenging time making sense of the changing nature of research collaboration across space. Common questions include:

  • Can this phenomenon be mapped out, and if so how and at what scales?
  • Can baseline studies be created such that the effects of new international collaborative research programs can be measured?
  • What happens to research practices and collaborative relations when universities join international consortia of universities?

One option is the use of bibliometric technologies to map out the changing nature of research collaboration across space. For example, the international linkages of the Australian Group of Eight (Go8) universities were mapped out (see some sample images below from the report Thomson ISI Go8 NCR dataset: Go8 International Collaborations, available via this University of Sydney website).

Other reports like Science-Metrix’s Scientific Collaboration between Canada and California: A Bibliometric Study (2008) used similar forms of data to understand collaboration between a country and a foreign state. I’ve also seen similar types of bibliometric-reliant reports while participating in discussions at Worldwide University Network (WUN) meetings, as well as on Thomson Reuters’ own website.

Another option is to take an institutionally-specific perspective, though via the acquisition and analysis of a broader array of forms of data. This type of mapping can be developed via bibliometric technologies, researcher surveys, an analysis of travel expense claim data, an analysis of media ‘expertise’ data bases maintained by universities, and so on. This is an oft-desired form of analysis; one designed to feed into central repositories of knowledge (e.g., the University of Notre Dame is developing such a site, tentatively called Global ND). Yet such an approach is challenging and resource consuming to implement.

In the end, for a range of reasons, bibliometrics are often the fallback tool to map out international collaboration. Bibliometrics have their important uses, of course, but they are not effective in capturing the research practices of all research scholars, especially those in the humanities and some arms of the social sciences.

Why? Well the main reason is different disciplines have different publishing practices, an issue funding councils like the Social Sciences and Humanities Research Council of Canada (SSHRC), or European agencies (including DFG, ESRC, AHRC, NWO, ANR and ESF) have recently been exploring. See for example, this March 2010 ESF report (Towards a Bibliometric Database for the Social Sciences and Humanities – A European Scoping Project), or Bibliometric Analysis of Research Supported by SSHRC: Design Study (March 2009) – a report for SSHRC by Science-Metrix.

If we go down the mapping route and rely too heavily upon bibliometrics, do we risk of letting the limitations of Thomson Reuters’ ISI Web of Knowledge, or the Scopus database, slowly and subtly create understandings of international collaboration that erase from view some very important researcher-to-researcher collaborations in the humanities, as well as some of the social sciences? Perhaps so, perhaps not!

In this context I am in search of some assistance.

If you or your colleagues have developed some insightful ways to map out international research collaboration patterns and trends in the social sciences and humanities, whatever the scale, please fill me in via <kolds@wisc.edu> or via the comments section below. Or one alternative response is to reject the whole idea of mapping, bibliometrics, and so on, and its associated managerialism. In any case, following a compilation of responses, and some additional research, I’ll share the findings via a follow-up entry in late August.

Thank you!

Kris Olds

Developments in world institutional rankings; SCImago joins the club

Editor’s note: this guest entry was kindly written by Gavin Moodie, principal policy adviser of Griffith University in Australia.  Gavin (pictured to the right) is most interested in the relations between vocational and higher education. His book From Vocational to Higher Education: An International Perspective was published by McGraw-Hill last year. Gavin’s entry sheds light on a new ranking initiative that needs to be situated within the broad wave of contemporary rankings – and bibliometrics more generally – that are being used to analyze, legitimize, critique, promote, not to mention extract revenue from.  Our thanks to Gavin for the illuminating contribution below.

~~~~~~~~~~~~~~~~~~~~~~~~

It has been a busy time for world institutional rankings watchers recently. Shanghai Jiao Tong University’s Institute of Higher Education published its academic ranking of world universities (ARWU) for 2009. The institute’s 2009 rankings include its by now familiar ranking of 500 institutions’ overall performance and the top 100 institutions in each of five broad fields: natural sciences and mathematics, engineering/technology and computer sciences, life and agriculture sciences, clinical medicine and pharmacy, and social sciences. This year Dr. Liu and his colleagues have added rankings of the top 100 institutions in each of five subjects: mathematics, physics, chemistry, computer science and economics/business.

Times Higher Education announced that over the next few months it will develop a new method for its world university rankings which in future will be produced with Thomson Reuters. Thomson Reuters’ contribution will be guided by Jonathan Adams (Adams’ firm, Evidence Ltd, was recently acquired by Thomson Reuters).

And a new ranking has been published, SCImago institutions rankings: 2009 world report. This is a league table of research institutions by various factors derived from Scopus, the database of the huge multinational publisher Elsevier. SCImago’s institutional research rank is distinctive in including with higher education institutions government research organisations such as France’s Centre National de la Recherche Scientifique, health organisations such as hospitals, and private and other organisations. Only higher education institutions are considered here. The ranking was produced by the SCImago Research Group, a Spain-based research network “dedicated to information analysis, representation and retrieval by means of visualisation techniques”.

SCImago’s rank is very useful in not cutting off at the top 200 or 500 universities, but in including all organisations with more than 100 publications indexed in Scopus in 2007. It therefore includes 1,527 higher education institutions in 83 countries. But even so, it is highly selective, including only 16% of the world’s estimated 9,760 universities, 76% of US doctoral granting universities, 65% of UK universities and 45% of Canada’s universities. In contrast all of New Zealand’s universities and 92% of Australia’s universities are listed in SCImago’s rank. Some 38 countries have seven or more universities in the rank.

SCImago derives five measures from the Scopus database: total outputs, cites per document (which are heavily influenced by field of research as well as research quality), international collaboration, normalised Scimago journal rank and normalised citations per output. This discussion will concentrate on total outputs and normalised citations per output.

Together these measures show that countries have been following two broad paths to supporting their research universities. One group of countries in northern continental Europe around Germany have supported a reasonably even development of their research universities, while another group of countries influenced by the UK and the US have developed their research universities much more unevenly. Both seem to be successful in support research volume and quality, at least as measured by publications and citations.

Volume of publications

Because a reasonable number of countries have several higher education institutions listed in SCImago’s rank it is possible to consider countries’ performance rather than concentrate on individual institutions as the smaller ranks encourage. I do this by taking the average of the performance of each country’s universities. The first measure of interest is the number of publications each university has indexed in Scopus over the five years from 2003 to 2007, which is an indicator of the volume of research. The graph in figure 1 shows the mean number of outputs for each country’s higher education research institutions. It shows only countries which have more than six universities included in SCImago’s rank, which leaves out 44 countries and thus much of the tail in institutions’ performance.

Figure 1: mean of universities’ outputs for each country with > 6 universities ranked


These data are given in table 1. The first column gives the number of higher education institutions each country has ranked in SCImago institutions rankings (SIR): 2009 world report. The second column shows the mean number of outputs indexed in Scopus for each country’s higher education research institutions from 2003 to 2007. The next column shows the standard deviation of the number of outputs for each country’s research university.

The third column in table 1 shows the coefficient of variation, which is the standard deviation divided by the mean and multiplied by 100. This is a measure of the evenness of the distribution of outputs amongst each country’s universities. Thus, the five countries whose universities had the highest average number of outputs indexed in Scopus from 2003 to 2007 – the Netherlands, Israel, Belgium, Denmark and Sweden – also had a reasonably low coefficient of variation below 80. This indicates that research volume is spread reasonably evenly amongst those countries’ universities. In contrast, Canada which had the sixth highest average number of outputs also has a reasonably high coefficient of variation of 120, indicating an uneven distribution of outputs amongst Canada’s research universities.

The final column in table 1 shows the mean of SCImago’s international collaboration score, which is a score of the proportions of the institution’s outputs jointly authored with someone from another country. The US’ international collaboration is rather low because US authors collaborate more often with authors in other institutions within the country.

Table 1: countries with > 6 institutions ranked by institutions’ mean outputs, 2007

Source: SCImago Research Group (2009) SCImago institutions rankings (SIR): 2009 world report.

Citations per paper by field

We next examine citations per paper by field of research, which is an indicator of the quality of research. This is the ratio between the average citations per publication of an institution and the world number of citations per publication over the same time frame and subject area. SCImago says it computed this ratio using the method established by Sweden’s Karolinska Intitutet which it called the ‘Item oriented field normalized citation score average’. A score of 0.8 means the institution is cited 20% below average and 1.3 means the institution is cited 30% above average.

Figure 2 shows mean normalised citations per paper for each country’s higher education research institutions from 2003 to 2007, again showing only countries which have more than six universities included in SCImago’s rank. The graph for an indicator of research quality in figure 2 is similar in shape to the graph of research volume in figure 1.

Figure 2: mean of universities’ normalised citations per paper for each country with > 6 universities ranked

Table 2 shows countries with more than six higher education research institutions ranked by their institutions’ mean normalised citations. This measure distinguishes more sharply between institutions than volume of outputs – the coefficient of variations for countries’ mean institutions normalised citations are higher than for number of publications. Nonetheless, several countries with high mean normalised citations have an even performance amongst their universities on this measure – Switzerland, Netherlands, Sweden, Germany, Austria, France, Finland and New Zealand.

Finally, I wondered whether countries which had a reasonably even performance of their research universities by volume and quality of publications reflected a more equal society. To test this I obtained from the Central Intelligence Agency’s (2009) World Factbook the Gini index of the distribution of family income within a country. A country with a Gini index of 0 would have perfect equality in the distribution of family income whereas a country with perfect inequality in its distribution of family would have a Gini index of 100. There is a modest correlation of 0.37 between a country’s Gini index and its coefficient of variation for both publications and citations.

Table 2: countries with > 6 institutions ranked by institutions’ normalised citations per output

Sources: SCImago Research Group (2009) SCImago institutions rankings (SIR): 2009 world report; Central Intelligence Agency (2009) The world factbook.

Conclusion

SCImago’s institutions research rank is sufficiently comprehensive to support comparisons between countries’ research higher education institutions. It finds two patterns amongst countries whose research universities have a high average volume and quality of research publications. One group of countries has a fairly even performance of their research universities, presumably because they have had fairly even levels of government support. This group is in northern continental Europe and includes Switzerland, Germany, Sweden, the Netherlands, Austria, Denmark and Finland. The other group of countries also has a high average volume and quality of research publications, but spread much more unevenly between universities. This group includes the US, the UK and Canada.

This finding is influenced by the measure I chose to examine countries’ performance, the average of their research universities’ performance. Other results may have been found using another measure of countries’ performance, such as the number of universities a country has in the top 100 or 500 of research universities normalised by gross domestic product. But such a measure would not reflect a country’s overall performance of their research universities, but only the performance of its champions. Whether one is interested in a country’s overall performance or just the performance of its champions depends on whether one believes more benefit is gained from a few outstanding performers or several excellent performers. That would usefully be the subject of another study.

Gavin Moodie

References

Central Intelligence Agency (2009) The world factbook (accessed 29 October 2009).

SCImago institutions rankings (SIR): 2009 world report (revised edition accessed 20 October 2009).