data friction and precarious knowledge: IEA and the movement of data in the 1960s and 1970s

In today’s increasingly data-driven world, we constantly encounter numbers that seem to have fallen from heaven: neat, tidy, and impossible to question, imposing a sense of order on a chaotic world. In education, this is clearly demonstrated by the role of international large-scale assessments such as PISA, TIMSS, and PIRLS, where international league tables are repeatedly published and used as sufficient evidence to explain educational performance worldwide.

However, before standardised data can be presented to the world, it has to be assembled. Collecting information on educational performance involves a complex collaboration between many actors in different national contexts (Landahl, 2023). Together, they must ensure that information on educational achievement is collected in schools and then aggregated at national and then international levels. The history of international testing can shed light on this process, as the inner workings of the first tests have been documented quite extensively. The International Association of Educational Achievement (IEA), which pioneered the field of international large-scale assessments in the 1960s and 1970s, has left rich traces of its inner workings in its archives, not least in the form of correspondence between the scholars involved. Such correspondence makes it possible to focus on the process of creating data, rather than analysing the role of the final reports – to get insights into what has been called “social knowledge in the making” (Camic et al., 2011). One of the fundamental aspects of this data-producing process is to manage the movement of data.

Producing international data is tantamount to moving information around, within and across national borders. Drawing on examples from the 1960s and 1970s, this article discusses how the movement of data was dealt with by the IEA during its early years. It is clear that the IEA, in many ways, benefitted from the development of technologies and infrastructures that facilitated the circulation of people, things and information. The international postal system, the development of fast ships and jet travel, along with the development of computer technology were essential conditions for the creation of international data on educational achievement. Nevertheless, while there were many factors that facilitated transnational cooperation, there were also factors that complicated things. If we want to understand the conditions and characteristics of the first international assessments, we need to take these factors into account. In what follows, I will illustrate how transnational collaboration was a complex process that encountered many obstacles, where technologies that increased the speed of data processing also created various kinds of tensions and frictions.

The role of computers is a case in point. On the one hand, computers simplified large-scale research. As one of the scholars involved in IEA put it in 1971: “Needless to say, with approximately fifty million pieces of information, this study could never have been completed without the use of a computer” (Postlethwaite, 1971, p. 70). However, the relationship between computers and what needs to be calculated is not as straightforward as the quotation above might suggest. Acceleration is never infinite, and what counts as high speed is always relative. Moving data is not an easy task. Paul N. Edwards has coined the term ‘data friction’ to describe the ways in which the movement of data can be impeded. It takes time, energy, and attention to collect, check, store, move, receive and access data. When data moves, whether across space, between computers or between different media (e.g. from punch cards to magnetic tape), data friction slows the movement (Edwards, 2010, p. 84).

The IEA used a variety of media to transmit information on educational performance around the world. Punched cards, magnetic tape, and various types of student-completed answer cards or answer sheets were sent to different locations for further processing. Sending this information relied on the postal system, which could cause problems. One was the risk of damage to the material being sent. For example, there was a problem with punch cards being broken in transit, so there was a need for better boxes to send them in.¹ Another problem was related to the temporal dimension. No one could guarantee how long it would take for a parcel to reach its destination. One obstacle was that postal packages could get stuck in customs. The Chairman of the IEA, Torsten Husén, described the situation in a letter to the Director-General of UNESCO. He wrote that the IEA was constantly having to ship punched cards and test materials in and out of a large number of countries and claimed that the experience had been “one of considerable frustration.” Customs regulations could cause delays of several weeks before the materials were cleared for import. One of the participating countries, England, had managed to get an exemption from the rules, and Husén wondered if UNESCO could guide the IEA on how to get exemptions for all participating countries.² In short, sending data around the world introduced an element of uncertainty. As one representative from Italy wrote when data was sent to the US: “The punched cards were sent yesterday; we hope they arrive undamaged.”³

A relevant source of data friction can be found in the materiality of data, the fact that it was transferred across borders and between machines in the form of punch cards, answer cards and magnetic tapes. As such, it is a case of what Martin Mulsow calls precarious knowledge: knowledge that is at risk of being lost or destroyed. In his history of precarious knowledge, Mulsow points to various sources of precarious knowledge. One example is knowledge carriers that are easily destroyed or lost, such as when texts or images exist in unique copies or in only a few manuscript copies. Attending to such knowledge transmitters makes it possible to understand also the complexity and riskiness of circulating knowledge (Mulsow, 2022).

Given the inherent risk in transmitting data over long distances, strategies had to be developed to reduce data friction. One of the ways in which this was done is illustrated by the mathematics survey published in 1967. When analysing the results, an important meeting was held in Chicago in 1965. The site of Chicago was chosen since it was the place where the computer used in the project was located. In a letter, Torsten Husén explained that he was going to Chicago “to chair the group which is going to sit close to the computer, busy writing up the outcomes of the international mathematics study.”⁴ In an information brochure about the IEA, this process was described in terms of a dialogue between the scholars and the computer. “The work was organized so that those writing on hypotheses could actually hold a ‘dialogue’ between themselves and the computer. A writer was able to ask for a particular analysis one evening and receive it next morning.”⁵ The Chicago meeting illustrates the persistence of space and the importance of physical encounters in transnational collaborations. Such meetings were necessary for a number of reasons, but in this case, there was a technological factor. The IEA used a computer designed specifically for research, the IBM 7090. Sold in a few hundred units and also used by NASA, it was an exclusive computer that not all universities had access to. At $3 million each, it was a significant investment and a prominent status symbol. The restricted access to certain types of computers made some locations particularly important. Due to its computer facilities, Chicago became, in Thomas Gieryn’s terms, a ‘truth spot’, a place where data could be processed to a degree that was difficult elsewhere (Gieryn, 2018). Travelling to Chicago and its computer, rather than communicating results by phone or letter, can therefore be seen as an attempt to reduce data friction.

Over time, specific sites for managing large numbers lost their privileged status. In the 1970s, there was a debate about where to locate a new data centre. This coincided with the development of the more versatile IBM 360 computer, which became more widely available. As a result, the IEA decided to split its computing facilities between two locations – New York and Stockholm. This was a change that promised to reduce data friction in one sense, as some of the data processing would be done in close proximity to the organisation’s headquarters in Stockholm. However, it was still a solution that created problems in terms of collaboration between two data processing teams across the North Atlantic. When misunderstandings, errors or delays occurred, it was easy to blame the division of labour between the Swedish and the American teams. As one of the scholars in the project, John Schwille later recalled: “the two data processing units were blaming each other for various shortcomings of the study” (Schwille, 2011, p. 635).

In sum, the production of data is a task that involves the movement of data, which Beaulieu and Leonelli label data journeys (Beaulieu & Leonelli, 2022). They emphasise that the movement of data is part of what makes data valuable, as it is the journey that allows data to reach new audiences in society. Unlike Beaulieu and Leonelli, I have not analysed the process by which data is presented to a broader audience. Instead, I have focused on the initial process by which data is produced. By focusing only on this first step in a potentially longer data journey, I have highlighted the different ways in which data is at risk of being lost, even at the production stage.  More generally, this illustrates a complex relationship between data and its flows across borders. The creation of international large-scale assessments was inevitably a product of the movement of data. Large amounts of information about school performance circled the globe: first collected at individual schools, then aggregated at the national level, then sent to an international centre, and finally returned to individual countries for further national analysis and dissemination. Since the movement of data was the precondition for international testing, we can also see that there was an inherent risk in the way the work was necessarily organised. The movement of data carried with it the double possibility of creating and destroying knowledge.

 

References

Beaulieu, A., & Leonelli, S. (2022). Data and society: A critical introduction. Sage.

Camic, C., Gross, N., & Lamont, M. (2011). Social knowledge in the making. University of Chicago Press.

Edwards, P. N. (2010). A vast machine: Computer models, climate data, and the politics of global warming. MIT Press.

Gieryn, T. (2018). Truth-spots: How places make people believe. University of Chicago Press.

Landahl, J. (2023). ‘The punched cards were sent yesterday, we hope they arrive undamaged.’ Computers and international large-scale assessments during the 1960s and 1970s. Learning, Media and Technology. Advance online publication.
https://doi.org/10.1080/17439884.2023.2218644

Mulsow, M. (2022). Knowledge lost: A new view of early modern intellectual history (H. C. E. Midelfort, Trans.). Princeton University Press.

Postlethwaite, N. (1971). International association for the evaluation of educational achievement (IEA): The mathematics study. Journal for Research in Mathematics Education, 2(2), 69–103.
https://doi.org/10.2307/748638

Schwille, J. (2011). Experiencing innovation and capacity building in IEA research, 1963–2008. In C. Papanastasiou, T. Plomp, & E. C. Papanastasiou (Eds.), IEA 1958–2008: 50 Years of Experiences and Memories (pp. 627–707). Cultural Center of the Kykkos Monastery.

 

Recommended Citation

Landahl, J. (2023). Data friction and precarious knowledge: IEA and the movement of data in the 1960s and 1970s. On Education. Journal for Research and Debate, 6(18).
https://doi.org/10.17899/on_ed.2023.18.1

 

Do you want to comment on this article? Please send your reply to editors@oneducation.net. Replies will be processed like invited contributions. This means they will be assessed according to standard criteria of quality, relevance, and civility. Please make sure to follow editorial policies and formatting guidelines.