Recent attention to citational justice in the field of HCI [1, 2, 3] has led to workshops in India, Latin America, and elsewhere on the topic. While reflecting after the workshops, we often returned to the question: Are current practices in computer science citationally just for researchers contributing from the Global Souths^a as well as the North? We argue a stance must be taken in this public debate.

The workshops brought diverse perspectives on citational practices, with experiences shared by early-career researchers, and questions from undergraduate students in India and graduate students in the Global North. These questions concerned the criteria for choosing papers to cite. We learned students considered papers with high citation counts "good" papers to cite, or simply replicated the citational choices of their instructors. They had received no explicit instruction in their computing courses about citational practices.

We continued to ponder: How did we first learn who to cite and what papers to include? How do we even determine what research to read and what to exclude when there are hundreds of papers related to a topic? Few of us recall any discussion of citational practices in our graduate studies, other than those associated with university plagiarism policies or the publishing policies of our organizations. We were also left unequipped to perceive and criticize the research database biases that push us to read and cite the most cited research on the topic we work on. To extend the critical dialogue on citational justice, workshop organizers had frequent online meetings. This article summarizes our collective thoughts of these discussions. Our purpose is to begin a discipline-wide conversation among students and faculty on citational practices, what they are, who determines what and who ought to be cited, and who is there in our research organizations to look out for those from the Global Souths who are rarely—or never—cited in the Global North.

  Social Implications of Citation Clusters

Digging deeper in computing, we discovered a dire need for corpus-based studies to collect data on who is citing whom so that we might learn how certain papers accumulate thousands of citations and others rarely see the light of day. The problem is not limited to the neglect of knowledge production from the Global Souths. Take for example research related to disabled users of computing products: Do we know if our colleagues outside the accessibility research area read our findings? If they do, why do we have so many new products being introduced to the market failing basic accessibility standards? By reflecting on cases like these, we realized citing has social implications beyond forming citation clusters, reaching up to the question of social justice.

Discussing citational justice means questioning epistemology, or what our field considers as valid knowledge, and whose knowledge-making is valued and whose is not. These questions should not only be of concern to students but also to faculty because our discussions uncover significant curricular gaps in computing. In principle, research communities inherit the practice of tracking and measuring knowledge production through explicit citations. Citations allow readers to independently assess the veracity of authors' claims instead of taking the authors' assertions on faith. In doing so, citations invite authors and readers to review evidence collaboratively. While authors enact the community's scientific norms to accumulate consensus concerning the available evidence [4], readers directly inspect authors' claims through citations. Citations differentiate new scholarly work but also give credit to authors for their previous contributions [4]. Consequently, citations emphasize the historicity of scientific work or its changing characteristics. Citing and being cited is also a dispute to write the history of a field.

Because of such disputes, citation patterns—the way authors cite a scholarly work or not—are never neutral. They reveal collective decisions within a scientific community shaping its boundaries and pathways. The most common pattern noticed in bibliographic analysis is centrality i.e., a source or a cluster of sources that receives substantially more citations. Scholars frequently take centrality as a measure of research impact [5]. The greater centrality a paper has in a citation network, the more likely it is that the paper influenced other scholarship. Centrality becomes a more prominent pattern when a citation network is partitioned by topic, institutions, school of thoughts, and even geographies. Large clusters stand out from smaller clusters or from unclustered, less-cited, disconnected, ignored, or unpublished sources. The result is the marginalization of authors who do not appear in large clusters.

When an academic community follows this kind of citational pattern, researchers in the Global Souths, people of ethnicities and races other than white, women, people with disabilities, or people who work part-time, all receive far fewer citations of their work, unless the patterns are actively counteracted. Breaking the pattern and citing alternative sources requires an extra, conscious, careful, and concerted effort to uncover whom the scientific community has systematically excluded from the structures of authority. For example, if we read our journals more carefully and broadly, both the information included in the research and the author bios can reveal which works are undercited and overcited.

---

Scholars working at the margins do not have the power to shape their fields unless they leave their places and move to the center of knowledge and power.

---

Everyday citational practices that appear rational and fair can preserve dominant knowledge systems and continue to stifle contributions from underrepresented scholars through oppressive patterns. Citing authors who have centralized the citations in their fields is expected of any new research because it must both engage with and acknowledge the field's scholarly consensus. At the same time, these central authors usually come from western, educated, industrialized, rich, and democratic (WEIRD) societies since scholars from those societies have been historically admitted to privileged communities. In contrast, citing both lesser-known and less-cited authors poses a risk, as reviewers might not accept their claims. Marginalized scholars—and the evidence their research offers—face greater scrutiny because the work is often experimental, creative, and even poses a change for the discipline owing to its origins in these scholars' early stage of their career [6]. Scholars working at the margins do not have the power to shape their fields unless they leave their places and move to the center of knowledge and power. This displacement process goes hand in hand with learning to abide by the power dynamics at play, which usually requires reducing one's differences and identity markers to be accepted and integrated.

  Power Dynamics in Latin American Research Spheres

A panelist at CLIHC 2021 (the Latin American Conference on Human-Computer Interaction) described how Latin American researchers struggle with two types of power dynamics: 1. networking and becoming visible within the Global North research communities, and 2. internal structures within Latin American academia. Given how much fewer citations conferences in Latin America comparatively receive [7], Latin American researchers need to learn how to navigate the top-ranked conferences and journals that mainly publish research from the Global North. In doing so, they must find ways of integrating their work with that from the North. Successfully navigating these venues typically leads to greater recognition of the researcher's work and, accordingly, more citations. This process usually carries the dual obligation to publish both in the Global North and locally, and thus, researchers frequently have to write in English and in local languages. This dual obligation requires substantially more effort than for researchers based in the Global North, and it is often born with fewer financial resources. In the Latin American (LATAM) region, a researcher's connections are more relevant than their work, as evidenced by the phenomenon of compadrismo i.e., the social class and personal connections of an individual. In an academic context, this can play a decisive role in the researcher's success in the region. Another panelist described researchers who conduct large studies in Mexico and receive significant funding opportunities from the Federal government are not necessarily the most experienced in the field but rather better connected with key actors, and they may pay less attention to the ethical implications of the work.

---

In terms of scholarly knowledge production, researchers from the Global North dominate the agenda-setting processes for conferences, journals, and academic curricula.

---

Interestingly, navigating the first power dynamic can circumvent the second. Becoming successful in the Global North research community helps researchers to deal with LATAM academic structures because "making it" in the North confers a greater advantage than social class, even though it may not serve the local needs in any meaningful manner. However, critics argue putting internationally competitive research before locally-relevant research has made universities highly international and "not truly Latin American" [8]. Limited knowledge about the Global North's academic infrastructure in Latin America can also advantage those who wish to promote their work as successful in the Global North, whether or not they have achieved that, since their local peers may not know enough to assess their performance. This was evidenced in a recent case where government actors from Mexico City's Agency of Public Innovation "bragged" about publishing work of questionable rigor on open-source scientific platforms, which are not peer-reviewed, such as socArXiv [9]. However, given that the general public did not understand this, government actors were able to promote a false image of being renowned global scientists. In the case of Mexico City, the dynamic unfortunately helped the local government conduct large-scale experimental medical trials without people's consent.

This form of academic "legacy" can be more prevalent within academia than we imagine, as it is not unique to LATAM. Who your academic "parents" are—that is, who your graduate or postgraduate advisor was—as well as the lab you worked in are common proxies of prestige and legitimacy.

  Citational Justice and Computing

Along with the questions for justice is the question of technology within citations. When approached as a technology, citations may seem a mundane artifact performing the simple, yet critical, task of signaling authorship. However, citations in the context of knowledge production act as "technical mediators" [2]. In this sense, citations are not merely a neutral technical construct facilitating authoring signals; they actively contribute to the less advantageous end product of its use. Citations technically mediate some of the social systems on which the dominant paradigm of academic knowledge production currently rests.

From a technological point of view, this socio-technical arrangement leverages several artifacts that enhance some of its negative (and its positive) applications. Citations have their own syntax in different languages (e.g., APA, Chicago), are housed within centralized information systems (e.g., Scopus, Crossref), and use universal coding mechanisms (e.g., DOIs). This facilitates interoperability, aggregation, and standardization in the production of knowledge, all for the purpose of human interaction, while at the same time drastically refereeing and limiting access to numerous sets of knowledge. These features also allow to integrate citations into large data models through automated suggestion systems, directional graphs, and other computational manipulations embodied in user-friendly tools (e.g., ResearchRabbit, Elicit), again, for the purpose of human use and "comfort." This scaling of information over an already biased set of knowledge makes access to citations susceptible to augmented bias, further constraining access to sets of knowledge while rewarding (potentially unfairly) others.

This techno-centric approach to citations makes it prone to becoming an oppressive technology. For example, centralized information systems enable "citation rings," excessive self-citation practices, and legacy citations, which in turn concentrate academic value and prestige in the hands of a few. All while silencing voices and traditions of knowledge on the margins, historically located in the Global Souths.

  Biases Against Global Souths Knowledge

Panelists exemplified that the Global North often integrate knowledge produced in the Global Souths without giving proper credit. For example, Paulo Freire's work has inspired Scandinavian researchers to devise participatory design methodologies [10], but his contributions are rarely mentioned in contemporary works. While the Global North could learn more from the Global Souths to enhance its knowledge base, panelists emphasized that global HCI research circles largely include participants from the Global Souths as research subjects, which limits the different and novel ways that the Global Souths can provide to advance HCI. Their designated spot in the field of HCI is that of a user, never a designer or a respected scientist [12]. This abstract inclusion maintains the tradition of researchers coming to the Global Souths to study Indigenous, underdeveloped, or poor people. Kou et al., for instance, reported this perspective of seeing the Global Souths as "exotic" people to be researched [11].

Global Souths researchers have started establishing themselves in the HCI community, studying the contexts they belong to and are more familiar with. Panelists emphasized that promoting this change includes disengaging with the Global North's epistemologies and situating the Global Souths' epistemologies as "epistemologies of the South." As a point of comparison, a panelist pointed out "They would not support this divide between a research subject and an object in the research. Therefore, we can't find, for example, in Latin American universities, a department of American studies to study people that live in the U.S., for example." The panelists called for involving researchers from the Global Souths in the larger community and creating infrastructure to support their participation as a means to strengthen citational justice in the long run. The panelists agreed citational justice requires rethinking the way academia is organized in society.

In terms of scholarly knowledge production, researchers from the Global North dominate the agenda-setting processes for conferences, journals, and academic curricula. Even when the Global Souths are the focus, Global North scholars define goals, research directions, and methodologies with little or no input from Global Souths researchers. The outcome results in the Global North, comprising just 15% of the world population, directing the knowledge enterprise for the remaining 85%. When writing research for dissemination, these researchers' citational practices face no accountability. For example, in Global North journals, researchers can limit their citations to publications from the Global North with impunity, even though ACM and its constituent organizations have a significant presence in the Global Souths, such as by hosting conferences and collecting data. We have not come across any editorial guidelines in computing journals or proceedings that impress on authors to be inclusive of citations of research from the Global Souths. In contrast, researchers from the Global Souths heavily cite—or are made to cite—researchers from the Global North to protect their work from the scrutiny of reviewers normalized by the Global North practices.

  Toward Citational Justice

Citational justice is at the nexus of political theories of justice and epistemology. Do we have a society in which everyone can contribute to our shared knowledge and be recognized for their contribution? Western liberal political theorists, such as John Rawls, ask us to adopt principles for societal organization stripped of individual identities and unencumbered by a person's unique history and experience [13]. This epistemic stance is crafted to consider whether society is organized justly for those who turn out to be the least well off in terms of luck [13]. The organizing principles for the infrastructure should center around those who are the least well off to create fair opportunities for advancement despite whatever situation they happened to be born into, even if it were the most unfortunate. Nevertheless, capability theorists critique Rawls' approach by arguing not everyone can convert opportunities into equitable outcomes [14, 15]. For some, offering an equitable opportunity will not be enough without the possibility of an equitable outcome. This critique draws on Global Souths feminist and disability theories to emphasize that there are embodied experiences that interact with opportunities that stymy the ability to take advantage of available resources. For example, a bicycle does not offer a mode of transportation to a person with paraplegia. This critique highlights the limits of formulating political principles a priori, and further, doing so discriminates against the lived and embodied experience of many.

  Implications for Computing Research

Participants of the citational justice panel and workshops offered several actionable ideas to contribute to a more equitable and fair citation of work beyond traditional geographical regions. One of the most important steps researchers could take is to search proactively for research conducted in the Global Souths; for instance, attending conferences organized by or located within the Global Souths, publishing in Global Souths open access journals, or using search engine filters to look for research in particular regions. Researchers could also suggest relevant work to their peers while writing their reviews and feedback for research articles.

The advantages of doing such outreach and recognition of work beyond the Global North are two-fold. First, researchers from the Global North could enrich their own theoretical, methodological, and epistemological approaches to data and study participants. Second, the recognition researchers from the Global Souths receive from their colleagues from the Global North may give a push for that very work to be recognized, validated, and used in the regions in the Global Souths where the research originates. Global North journal editors can also regularly invite researchers from the Global Souths to write integrative and scoping reviews of the wide-ranging HCI work happening in the Global Souths. Such reviews can also include scholarship appearing in other languages and create an impetus for the translation of significant work into English. Such reviews could help Global North researchers catch up with the neglected work from the Global Souths without scouring for such work.

---

Discussing citational justice means questioning epistemology, or what our field considers as valid knowledge, and whose knowledge-making is valued and whose is not.

---

The citational justice movement has gathered steam across research communities [16] and the field of computing has a special responsibility in magnifying the benefits of the movement in widening contributions to knowledge. Bidwell previously hinted at our challenge when she wrote about scholarship in HCI: "Researchers who seek to cite African research or innovation, can struggle to identify relevant work using terms from the dominant discourse. Concomitantly, to be visible in the dominant discourse, Africans must explain themselves according to these terms and the specific formulations of HCI communication" [17].

Machine learning and automated decision-making reproduce existing constraints in discourses and the visibility of certain social groups and certain knowledge. This is not only a result of the legacy of biases in different scholarly archives and researchers' own search and retrieval patterns [18]; more importantly, it reflects representation within the tech industry itself. The field of artificial intelligence (AI) is particularly exclusionary, comprising less than 20% women and even fewer people who are Black or Latin or from the Global Souths [19], which directly shapes the algorithmic biases and the ethical frameworks they hinge upon. For instance, the recent trend toward "human-centered" explainability intends to contribute to AI ethics of understandability and transparency; yet the "who" involved in training and testing explanations are from WEIRD constituencies themselves [20].

Citation patterns are not a single oppressive system of scholarly rituals but get embedded in many infrastructures that shape research and academic life, and as computing professionals we can significantly contribute to shaping the infrastructures of the future. But we can do it.

  References

[1] Ahmed, S. I. et al. Citational justice and the politics of knowledge production. interactions 29, 5 (2022), 78–82.

[2] Latour, B. Pandora's Hope. Harvard University Press, Cambridge, 1999.

[3] Kumar, N. and Karusala, N. Braving citational justice in human-computer interaction. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2021.

[4] Small, H. Referencing as cooperation or competition. In Theories of Informetrics and Scholarly Communication: A Festschrift in Honor of Blaise Cronin. De Gruyter Saur, 2016, 49–71; https://doi.org/10.1515/9783110308464-006

[5] Diallo, S. Y., Lynch, C. J., Gore, R., and Padilla, J. J. Identifying key papers within a journal via network centrality measures. Scientometrics 107, 3 (2016), 1005–1020; https://doi.org/10.1007/s11192-016-1891-8

[6] Mott, C. and Cockayne, D. Citation matters: Mobilizing the politics of citation toward a practice of "conscientious engagement." Gender, Place & Culture 24, 7 (2017), 954–973.

[7] Wong-Villacres, M., Garcia, A. A., Badillo-Urquiola, K., Machuca, M. D. B., Felice, M. C., Gaytán-Lugo, L. S., Lemus, O., Reynolds-Cuéllar P., and Perusquía-Hernández, M. Lessons from Latin America: Embracing horizontality to reconstruct HCI as a pluriverse. interactions 28, 2 (2021), 56–63.

[8] Arocena, R. and Sutz, J. Changing knowledge production and Latin American universities. Research Policy 30, 8 (2001), 1221–1234; https://doi.org/10.1016/S0048-7333(00)00143-8

[9] Pardo-Guerra, J. Las mentiras disfrazadas de ciencia: el caso de la ivermectina en la Ciudad de México. Nexos. Feb. 5, 2022; https://redaccion.nexos.com.mx/las-mentiras-disfrazadas-de-ciencia-el-caso-de-la-ivermectina-en-la-ciudad-de-mexico

[10] Ehn, P. Learning in participatory design as I found it (1970–2015). In Participatory Design for Learning. Routledge, 2017, 7–21.

[11] Kou, Y., Gray, C. M., Toombs, A., and Nardi, B. The politics of titling: The representation of countries in CHI papers. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2018; https://doi.org/10.1145/3170427.3188409

[12] Gonzatto, R. F. and Van Amstel, F. M. C. User oppression in human-computer interaction: a dialectical-existential perspective. Aslib Journal of Information Management 74, 5 (2022), 758–781; https://doi.org/10.1108/AJIM-08-2021-0233

[13] Rawls, J. A theory of justice. In A Theory of Justice. Harvard University Press, Cambridge, 1971.

[14] Nussbaum, M. C. Women and Human Development: The capabilities approach. Cambridge University Press, Cambridge, 2000.

[15] Sen, A. Commodities and Capabilities. Oxford University Press, 1999.

[16] Kwon, D. The rise of citational justice: How scholars are making references fairer. Nature 603, 7902 (2002), 568–71.

[17] Bidwell, N. J. Decolonising HCI and interaction design discourse: Some considerations in planning AfriCHI. XRDS 22, 4 (2016), 22–27.

[18] Benjamin, R. Race After Technology: Abolitionist Tools for the New Jim Code. Polity, 2019.

[19] Cheong, M., Leins, K., and Coghlan, S. Computer science communities: Who is speaking, and who is listening to the women? Using an ethics of care to promote diverse voices. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. ACM, New York, 2021.

[20] Ehsan, U., Passi, S., Liao, Q. V., Chan, L., Lee, I., Muller, M., and Riedl, M. O. The who in explainable AI: How AI background shapes perceptions of AI explanations. arXiv:2107.13509. 2021.

  Authors

Amy Ogan is the Thomas and Lydia Moran Assistant Professor of Learning Science at Carnegie Mellon University, where she studies, designs, and builds educational technologies, with a focus on culturally aware design.

Frederick M. C. van Amstel is assistant professor of service design and experience design at Federal University of Technology - Paraná (UTFPR), Brazil. His recent research and education address designing against oppression.

Gabriela Molina León is a Ph.D. student at the University of Bremen, Germany, conducting research on how to design data visualizations for interactive surfaces.

Juan Fernando Maestre Dr. Fernando Maestre (he/him) is an Ecuadorian researcher and educator. Fernando conducts human-computer interaction (HCI) research with vulnerable and marginalized groups. He is currently a President's Postdoctoral Fellow in the Department of Computer Science and Engineering at the University of Minnesota.

Kristin Williams is an assistant professor in computer science at Emory University. Her research focuses on making IoT both socially and environmentally sustainable.

Nicola J Bidwell is a professor at the University of Melbourne, Australia, and at the International University of Management, Namibia, researching contexts marginalized by techno-geopolitics, including rural and indigenous communities in the Global Souths.

Pedro Reynolds-Cuéllar is a Ph.D. candidate at the Massachusetts Institute of Technology, conducting research on how to build technology differently inspired by ancestral technologies.

Saiph Savage is an assistant professor at Northeastern University and research collaborator of the Universidad Nacional Autonoma de Mexico (UNAM) in Mexico City. Her research focuses on designing culturally aware technologies for workers.

Sushil Oswal is a professor of human centered design at the University of Washington. His research focuses on HCI and user experience in the contexts of disability and cross-cultural design.

Vishal Sharma is a human-centered computing Ph.D. student at the Georgia Institute of Technology. His research focuses on investigating the design of sociotechnical systems to support the building of sustainable futures.

  Footnotes

a. We pluralize "South" intentionally to underscore the variations present in the Global South, a concept generally used to denote peoples who have being colonized or oppressed in other ways.

 

This work is licensed under Creative Commons Attribution International 4.0.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2023 ACM, Inc.