Peer review and gender bias: A study on 145 scholarly journals

Flaminio Squazzoni,Giangiacomo Bravo,Mike Farjam,Ana Marusic,Bahar Mehmani,Michael Willis,Aliaksandr Birukou,Pierpaolo Dondio,Francisco Grimaldo

DOI: https://doi.org/10.1126/sciadv.abd0299

IF: 13.6

2021-01-06

Science Advances

Abstract:No evidence that women are penalized in peer review and editorial processes, finds study on 145 journals in multiple disciplines.

multidisciplinary sciences

What problem does this paper attempt to address?

Peer review perpetuates barriers for historically excluded groups

Olivia M. Smith,Kayla L. Davis,Riley B. Pizza,Robin Waterman,Kara C. Dobson,Brianna Foster,Julie C. Jarvey,Leonard N. Jones,Wendy Leuenberger,Nan Nourn,Emily E. Conway,Cynthia M. Fiser,Zoe A. Hansen,Ani Hristova,Caitlin Mack,Alyssa N. Saunders,Olivia J. Utley,Moriah L. Young,Courtney L. Davis

DOI: https://doi.org/10.1038/s41559-023-01999-w

2023-03-14

Abstract:Peer review is central to the scientific process and scientists' career advancement, but bias at various stages of the review process disadvantages some authors. Here we use peer review data from 312,740 biological sciences manuscripts across 31 studies to (1) examine evidence for differential peer review outcomes based on author demographics, (2) evaluate the efficacy of solutions to reduce bias and (3) describe the current landscape of peer review policies for 541 ecology and evolution journals. We found notably worse review outcomes (for example, lower overall acceptance rates) for authors whose institutional affiliations were in Asia, for authors whose country's primary language is not English and in countries with relatively low Human Development Indices. We found few data evaluating efficacy of interventions outside of reducing gender bias through double-blind review or diversifying reviewer/editorial boards. Despite evidence for review outcome gaps based on author demographics, few journals currently implement policies intended to mitigate bias (for example, 15.9% of journals practised double-blind review and 2.03% had reviewer guidelines that mentioned social justice issues). The lack of demographic equity signals an urgent need to better understand and implement evidence-based bias mitigation strategies.

ecology,evolutionary biology
Personal attributes of authors and reviewers, social bias and the outcomes of peer review: a case study

Richard Walker,Beatriz Barros,Ricardo Conejo,Konrad Neumann,Martin Telefont

DOI: https://doi.org/10.12688/f1000research.6012.2

2015-01-22

Abstract:Peer review is the "gold standard" for evaluating journal and conference papers, research proposals, on-going projects and university departments. However, it is widely believed that current systems are expensive, conservative and prone to various forms of bias. One form of bias identified in the literature is "social bias" linked to the personal attributes of authors and reviewers. To quantify the importance of this form of bias in modern peer review, we analyze three datasets providing information on the attributes of authors and reviewers and review outcomes: one from Frontiers - an open access publishing house with a novel interactive review process, and two from Spanish and international computer science conferences, which use traditional peer review. We use a random intercept model in which review outcome is the dependent variable, author and reviewer attributes are the independent variables and bias is defined by the interaction between author and reviewer attributes. We find no evidence of bias in terms of gender, or the language or prestige of author and reviewer institutions in any of the three datasets, but some weak evidence of regional bias in all three. Reviewer gender and the language and prestige of reviewer institutions appear to have little effect on review outcomes, but author gender, and the characteristics of author institutions have moderate to large effects. The methodology used cannot determine whether these are due to objective differences in scientific merit or entrenched biases shared by all reviewers.
Gender differences in research areas and topics: An analysis of publications in 285 fields

Mike Thelwall,Carol Bailey,Catherine Tobin,Noel-Ann Bradshaw

DOI: https://doi.org/10.48550/arXiv.1809.01255

2018-09-04

Digital Libraries

Abstract:Although the gender gap in academia has narrowed, females are underrepresented within some fields in the USA. Prior research suggests that the imbalances between science, technology, engineering and mathematics fields may be partly due to greater male interest in things and greater female interest in people, or to off-putting masculine cultures in some disciplines. To seek more detailed insights across all subjects, this article compares practising US male and female researchers between and within 285 narrow Scopus fields inside 26 broad fields from their first-authored articles published in 2017. The comparison is based on publishing fields and the words used in article titles, abstracts, and keywords. The results cannot be fully explained by the people/thing dimensions. Exceptions include greater female interest in veterinary science and cell biology and greater male interest in abstraction, patients, and power/control fields, such as politics and law. These may be due to other factors, such as the ability of a career to provide status or social impact or the availability of alternative careers. As a possible side effect of the partial people/thing relationship, females are more likely to use exploratory and qualitative methods and males are more likely to use quantitative methods. The results suggest that the necessary steps of eliminating explicit and implicit gender bias in academia are insufficient and might be complemented by measures to make fields more attractive to minority genders.
Gender disparities in UK research publishing: Differences between fields, methods and topics

Mike Thelwall,Mahshid Abdoli,Anna Lebiedziewicz,Carol Bailey

DOI: https://doi.org/10.3145/epi.2020.jul.15

2020-07-21

El Profesional de la información

Abstract:Gender disparities persist in UK research, with female minorities in most science, technology, engineering and mathematics (STEM) subjects but female majorities in others. The nature of the gender disparity differences between subjects needs to be understood if effective remedial actions are to be targeted at STEM subjects suffering from a lack of women, in contrast to other subjects without shortages. Evidence from the USA suggests that women engage more in people-related subjects, qualitative methods, veterinary science and cell biology and men engage more in thing-related subjects, power/control fields, patient-related research, abstraction and quantitative methods, except surveys. This article investigates gender disparity differences in UK first authorship for journal articles in nearly all of science split into 26 broad and 308 narrow Scopus fields. The results largely replicate the USA but suggest that more life science topics may be female-associated in the UK and patient-related research might not be male-associated. UK STEM gender parity initiatives might therefore emphasise people-oriented, and perhaps socially positive, aspects of currently masculine STEM topics and approaches (e.g., abstraction, mathematical quantitative methods), and promote female-friendly topics, methods and goals within male-dominated fields in addition to tacking implicit and explicit sexism and providing a supportive working environment.
Exploring Gender Bias in Six Key Domains of Academic Science: An Adversarial Collaboration

Shulamit Kahn,Stephen J. Ceci,Wendy M. Williams

DOI: https://doi.org/10.1177/15291006231163179

2023-04-26

Psychological Science in the Public Interest

Abstract:We synthesized the vast, contradictory scholarly literature on gender bias in academic science from 2000 to 2020. In the most prestigious journals and media outlets, which influence many people’s opinions about sexism, bias is frequently portrayed as an omnipresent factor limiting women’s progress in the tenure-track academy. Claims and counterclaims regarding the presence or absence of sexism span a range of evaluation contexts. Our approach relied on a combination of meta-analysis and analytic dissection. We evaluated the empirical evidence for gender bias in six key contexts in the tenure-track academy: (a) tenure-track hiring, (b) grant funding, (c) teaching ratings, (d) journal acceptances, (e) salaries, and (f) recommendation letters. We also explored the gender gap in a seventh area, journal productivity, because it can moderate bias in other contexts. We focused on these specific domains, in which sexism has most often been alleged to be pervasive, because they represent important types of evaluation, and the extensive research corpus within these domains provides sufficient quantitative data for comprehensive analysis. Contrary to the omnipresent claims of sexism in these domains appearing in top journals and the media, our findings show that tenure-track women are at parity with tenure-track men in three domains (grant funding, journal acceptances, and recommendation letters) and are advantaged over men in a fourth domain (hiring). For teaching ratings and salaries, we found evidence of bias against women; although gender gaps in salary were much smaller than often claimed, they were nevertheless concerning. Even in the four domains in which we failed to find evidence of sexism disadvantaging women, we nevertheless acknowledge that broad societal structural factors may still impede women’s advancement in academic science. Given the substantial resources directed toward reducing gender bias in academic science, it is imperative to develop a clear understanding of when and where such efforts are justified and of how resources can best be directed to mitigate sexism when and where it exists.

psychology, multidisciplinary
Bias against research on gender bias

Aleksandra Cislak,Magdalena Formanowicz,Tamar Saguy

DOI: https://doi.org/10.1007/s11192-018-2667-0

IF: 3.801

2018-02-17

Scientometrics

Abstract:The bias against women in academia is a documented phenomenon that has had detrimental consequences, not only for women, but also for the quality of science. First, gender bias in academia affects female scientists, resulting in their underrepresentation in academic institutions, particularly in higher ranks. The second type of gender bias in science relates to some findings applying only to male participants, which produces biased knowledge. Here, we identify a third potentially powerful source of gender bias in academia: the bias against research on gender bias. In a bibliometric investigation covering a broad range of social sciences, we analyzed published articles on gender bias and race bias and established that articles on gender bias are funded less often and published in journals with a lower Impact Factor than articles on comparable instances of social discrimination. This result suggests the possibility of an underappreciation of the phenomenon of gender bias and related research within the academic community. Addressing this meta-bias is crucial for the further examination of gender inequality, which severely affects many women across the world.

information science & library science,computer science, interdisciplinary applications
Gender differences in peer reviewed grant applications, awards, and amounts: a systematic review and meta-analysis

Karen B. Schmaling,Stephen A. Gallo

DOI: https://doi.org/10.1186/s41073-023-00127-3

2023-05-03

Research Integrity and Peer Review

Abstract:Abstract Background Differential participation and success in grant applications may contribute to women’s lesser representation in the sciences. This study’s objective was to conduct a systematic review and meta-analysis to address the question of gender differences in grant award acceptance rates and reapplication award acceptance rates (potential bias in peer review outcomes) and other grant outcomes. Methods The review was registered on PROSPERO (CRD42021232153) and conducted in accordance with PRISMA 2020 standards. We searched Academic Search Complete, PubMed, and Web of Science for the timeframe 1 January 2005 to 31 December 2020, and forward and backward citations. Studies were included that reported data, by gender, on any of the following: grant applications or reapplications, awards, award amounts, award acceptance rates, or reapplication award acceptance rates. Studies that duplicated data reported in another study were excluded. Gender differences were investigated by meta-analyses and generalized linear mixed models. Doi plots and LFK indices were used to assess reporting bias. Results The searches identified 199 records, of which 13 were eligible. An additional 42 sources from forward and backward searches were eligible, for a total of 55 sources with data on one or more outcomes. The data from these studies ranged from 1975 to 2020: 49 sources were published papers and six were funders’ reports (the latter were identified by forwards and backwards searches). Twenty-nine studies reported person-level data, 25 reported application-level data, and one study reported both: person-level data were used in analyses. Award acceptance rates were 1% higher for men, which was not significantly different from women (95% CI 3% more for men to 1% more for women, k = 36, n = 303,795 awards and 1,277,442 applications, I 2 = 84%). Reapplication award acceptance rates were significantly higher for men (9%, 95% CI 18% to 1%, k = 7, n = 7319 applications and 3324 awards, I 2 = 63%). Women received smaller award amounts ( g = -2.28, 95% CI -4.92 to 0.36, k = 13, n = 212,935, I 2 = 100%). Conclusions The proportions of women that applied for grants, re-applied, accepted awards, and accepted awards after reapplication were less than the proportion of eligible women. However, the award acceptance rate was similar for women and men, implying no gender bias in this peer reviewed grant outcome. Women received smaller awards and fewer awards after re-applying, which may negatively affect continued scientific productivity. Greater transparency is needed to monitor and verify these data globally.
Overcoming the gender bias in ecology and evolution: is the double-anonymized peer review an effective pathway over time?

Cibele Cássia-Silva,Barbbara Silva Rocha,Luisa Fernanda Liévano-Latorre,Mariane Brom Sobreiro,Luisa Maria Diele-Viegas

DOI: https://doi.org/10.7717/peerj.15186

IF: 3.061

2023-04-10

PeerJ

Abstract:Male researchers dominate scientific production in science, technology, engineering, and mathematics (STEM). However, potential mechanisms to avoid this gender imbalance remain poorly explored in STEM, including ecology and evolution areas. In the last decades, changes in the peer-review process towards double-anonymized (DA) have increased among ecology and evolution (EcoEvo) journals. Using comprehensive data on articles from 18 selected EcoEvo journals with an impact factor >1, we tested the effect of the DA peer-review process in female-leading ( i.e ., first and senior authors) articles. We tested whether the representation of female-leading authors differs between double and single-anonymized (SA) peer-reviewed journals. Also, we tested if the adoption of the DA by previous SA journals has increased the representativeness of female-leading authors over time. We found that publications led by female authors did not differ between DA and SA journals. Moreover, female-leading articles did not increase after changes from SA to DA peer-review. Tackling female underrepresentation in science is a complex task requiring many interventions. Still, our results highlight that adopting the DA peer-review system alone could be insufficient in fostering gender equality in EcoEvo scientific publications. Ecologists and evolutionists understand how diversity is important to ecosystems’ resilience in facing environmental changes. The question remaining is: why is it so difficult to promote and keep this “diversity” in addition to equity and inclusion in the academic environment? We thus argue that all scientists, mentors, and research centers must be engaged in promoting solutions to gender bias by fostering diversity, inclusion, and affirmative measures.

multidisciplinary sciences
A gender equality paradox in academic publishing: Countries with a higher proportion of female first-authored journal articles have larger first author gender disparities between fields

Mike Thelwall,Amalia Mas-Bleda

DOI: https://doi.org/10.48550/arXiv.2004.12159

2020-04-25

Abstract:Current attempts to address the shortfall of female researchers in Science, Technology, Engineering and Mathematics (STEM) have not yet succeeded despite other academic subjects having female majorities. This article investigates the extent to which gender disparities are subject-wide or nation-specific by a first author gender comparison of 30 million articles from all 27 Scopus broad fields within the 31 countries with the most Scopus-indexed articles 2014-18. The results show overall and geocultural patterns as well as individual national differences. Almost half of the subjects were always more male (7; e.g., Mathematics) or always more female (6; e.g., Immunology & Microbiology) than the national average. A strong overall trend (Spearman correlation 0.546) is for countries with a higher proportion of female first-authored research to also have larger differences in gender disparities between fields (correlation 0.314 for gender ratios). This confirms the international gender equality paradox previously found for degree subject choices: increased gender equality overall associates with moderately greater gender differentiation between subjects. This is consistent with previous USA-based claims that gender differences in academic careers are partly due to (socially constrained) gender differences in personal preferences. Radical solutions may therefore be needed for some STEM subjects to overcome gender disparities.

Digital Libraries
Assessing Gender Bias in the Information Systems Field: An Analysis of the Impact on Citations

Silvia Masiero,Aleksi Aaltonen

DOI: https://doi.org/10.48550/arXiv.2108.12255

2021-08-23

Abstract:Gender bias, a systemic and unfair difference in how men and women are treated in a given domain, is widely studied across different academic fields. Yet, there are barely any studies of the phenomenon in the field of academic information systems (IS), which is surprising especially in the light of the proliferation of such studies in the Science, Technology, Mathematics and Technology (STEM) disciplines. To assess potential gender bias in the IS field, this paper outlines a study to estimate the impact of scholarly citations that female IS academics accumulate vis-à-vis their male colleagues. Drawing on a scientometric study of the 7,260 papers published in the most prestigious IS journals (known as the AIS Basket of Eight), our analysis aims to unveil potential bias in the accumulation of citations between genders in the field. We use panel regression to estimate the gendered citations accumulation in the field. By doing so we propose to contribute knowledge on a core dimension of gender bias in academia, which is, so far, almost completely unexplored in the IS field.

Computers and Society
Cross-sectional study of the relationship between women’s representation among editors and peer reviewers in journals of the British Medical Journal Publishing Group

Ana-Catarina Pinho-Gomes,Amy Vassallo,Mark Woodward,Sanne Peters

DOI: https://doi.org/10.1136/bmjopen-2022-061054

IF: 3.006

2022-05-01

BMJ Open

Abstract:Objectives To investigate whether there is an association between women’s representation as peer reviewers and editors of medical journals. Methods In this cross-sectional study, the gender of editors and peer reviewers of journals of the British Medical Journal Publishing Group (BMJ-PG) in 2020 was determined based on given names. Trends over time were analysed for the BMJ between 2009 and 2017. Results Overall, this study included 47 of the 74 journals in the BMJ-PG. Women accounted for 30.2% of the 42 539 peer reviewers, with marked variation from 8% to 50%. Women represented 33.4% of the 555 editors, including 19.2% of the 52 editors-in-chief. There was a moderate positive correlation between the percentage of women as editors and as reviewers (Spearman correlation coefficient 0.590; p<0.0001). The percentage of women as editors, excluding editors-in-chief, was higher when the editor-in-chief was a woman than a man (53.3% vs 29.2%, respectively; p<0.0001). Likewise, the percentage of women as peer reviewers was higher in journals that had a woman as editor-in-chief in comparison with a man (32.0% vs 26.4%, respectively; p<0.0001). There was a slight increase in the percentage of women as peer reviewers from 27.3% in 2009 to 29.7% in 2017 in the BMJ . Conclusions Women account for less than one in three peer reviewers of medical journals. Women’s representation as peer reviewers is higher in journals with higher percentage of women as editors or with a woman as editor-in-chief. It is, thus, imperative to address the persisting gender gap at all levels of the publishing system.

medicine, general & internal
Quality of evidence revealing subtle gender biases in science is in the eye of the beholder

Ian M. Handley,Elizabeth R. Brown,Corinne A. Moss-Racusin,Jessi L. Smith

DOI: https://doi.org/10.1073/pnas.1510649112

IF: 11.1

2015-10-12

Proceedings of the National Academy of Sciences

Abstract:Significance Ever-growing empirical evidence documents a gender bias against women and their research—and favoring men—in science, technology, engineering, and mathematics (STEM) fields. Our research examined how receptive the scientific and public communities are to experimental evidence demonstrating this gender bias, which may contribute to women’s underrepresentation within STEM. Results from our three experiments, using general-public and university faculty samples, demonstrated that men evaluate the quality of research unveiling this bias as less meritorious than do women. These findings may inform and fuel self-correction efforts within STEM to reduce gender bias, bolster objectivity and diversity in STEM workforces, and enhance discovery, education, and achievement.
The distribution of authors and reviewers in EPS

Daniel Stockemer,Alasdair Blair,Ekaterina Rashkova

DOI: https://doi.org/10.1057/s41304-020-00251-4

2020-04-14

European Political Science

Abstract:Gender inequality as a phenomenon is also present in academic writing and publishing. In this article, we review the gender imbalance in the percentage of authors and reviewers in EPS from 2015 to 2019. At the submissions stage, male authors submit approximately twice as many manuscripts compared to female authors. At the publication stage, there is less of a gender difference due to a higher success rate for female authors. For reviewers, however, the gender discrepancies are even wider. At the invitation stage, we invited only roughly four women to review for every ten men. When it comes to completed reviews, the gap widens to roughly three women for ten men. Our findings show that we still have a long way to go to achieve parity in the review process. We suggest that parity in the review process is not independent of more women scholars being promoted to higher level academic positions.

political science
Gender bias in academic recruitment

Giovanni Abramo,Ciriaco Andrea D'Angelo,Francesco Rosati

DOI: https://doi.org/10.1007/s11192-015-1783-3

2018-12-21

Abstract:It is well known that women are underrepresented in the academic systems of many countries. Gender discrimination is one of the factors that could contribute to this phenomenon. This study considers a recent national academic recruitment campaign in Italy, examining whether women are subject to more or less bias than men. The findings show that no gender-related differences occur among the candidates who benefit from positive bias, while among those candidates affected by negative bias, the incidence of women is lower than that of men. Among the factors that determine success in a competition for an academic position, the number of the applicant's career years in the same university as the committee members assumes greater weight for male candidates than for females. Being of the same gender as the committee president is also a factor that assumes greater weight for male applicants. On the other hand, for female applicants, the presence of a full professor in the same university with the same family name as the candidate assumes greater weight than for male candidates.

Digital Libraries,Physics and Society
The Myth of Meritocracy and the Matilda Effect in STEM: Paper Acceptance and Paper Citation

Joana Fonseca

2023-06-19

Abstract:Biases against women in the workplace have been documented in various studies. There is also a growing body of literature on biases within academia. But particularly in STEM, due to the heavily male-dominated field, studies suggest that if one's gender is identifiable, women are more likely to get their papers rejected and not cited as often as men. We propose two simple modifications to tackle gender bias in STEM that can be applied to (but not only) IEEE conferences and journals. Regarding paper acceptance, we propose a double-blind review, and regarding paper citation, we propose one single letter to identify the authors' first names, followed by their family names. We also propose other modifications regarding gender bias in STEM and academia and encourage further reforms supported by current research on this topic with gender-segregated data.

Digital Libraries,Physics and Society
Global evidence of gender equity in academic health research: a scoping review

Andrea C Tricco,Vera Nincic,Nazia Darvesh,Patricia Rios,Paul A Khan,Marco M Ghassemi,Heather MacDonald,Fatemeh Yazdi,Yonda Lai,Rachel Warren,Alyssa Austin,Olga Cleary,Nancy N Baxter,Karen E A Burns,Douglas Coyle,Janet A Curran,Ian D Graham,Gillian Hawker,France Légaré,Jennifer Watt,Holly O Witteman,Jocalyn P Clark,Ivy L Bourgeault,Jeanna Parsons Leigh,Sofia B Ahmed,Karen Lawford,Alice B Aiken,Etienne V Langlois,Christopher McCabe,Sasha Shepperd,Becky Skidmore,Reena Pattani,Natalie Leon,Jamie Lundine,Évèhouénou Lionel Adisso,Wafa El-Adhami,Sharon E Straus

DOI: https://doi.org/10.1136/bmjopen-2022-067771

IF: 3.006

2023-02-17

BMJ Open

Abstract:Objectives To chart the global literature on gender equity in academic health research. Design Scoping review. Participants Quantitative studies were eligible if they examined gender equity within academic institutions including health researchers. Primary and secondary outcome measures Outcomes related to equity across gender and other social identities in academia: (1) faculty workforce: representation of all genders in university/faculty departments, academic rank or position and salary; (2) service: teaching obligations and administrative/non-teaching activities; (3) recruitment and hiring data: number of applicants by gender, interviews and new hires for various rank; (4) promotion: opportunities for promotion and time to progress through academic ranks; (5) academic leadership: type of leadership positions, opportunities for leadership promotion or training, opportunities to supervise/mentor and support for leadership bids; (6) scholarly output or productivity: number/type of publications and presentations, position of authorship, number/value of grants or awards and intellectual property ownership; (7) contextual factors of universities; (8) infrastructure; (9) knowledge and technology translation activities; (10) availability of maternity/paternity/parental/family leave; (11) collaboration activities/opportunities for collaboration; (12) qualitative considerations: perceptions around promotion, finances and support. Results Literature search yielded 94 798 citations; 4753 full-text articles were screened, and 562 studies were included. Most studies originated from North America (462/562, 82.2%). Few studies (27/562, 4.8%) reported race and fewer reported sex/gender (which were used interchangeably in most studies) other than male/female (11/562, 2.0%). Only one study provided data on religion. No other PROGRESS-PLUS variables were reported. A total of 2996 outcomes were reported, with most studies examining academic output (371/562, 66.0%). Conclusions Reviewed literature suggest a lack in analytic approaches that consider genders beyond the binary categories of man and woman, additional social identities (race, religion, social capital and disability) and an intersectionality lens examining the interconnection of multiple social identities in understanding discrimination and disadvantage. All of these are necessary to tailor strategies that promote gender equity. Trial registration number Open Science Framework: https://osf.io/8wk7e/.

medicine, general & internal
Do bibliometrics introduce gender, institutional or interdisciplinary biases into research evaluations?

Mike Thelwall,Kayvan Kousha,Emma Stuart,Meiko Makita,Mahshid Abdoli,Paul Wilson,Jonathan Levitt

DOI: https://doi.org/10.1016/j.respol.2023.104829

IF: 7.2

2023-06-14

Research Policy

Abstract:Systematic evaluations of publicly funded research sometimes use bibliometrics alone or bibliometric-informed peer review, but it is not known whether bibliometrics introduce biases when supporting or replacing peer review. This article assesses this by comparing three alternative mechanisms for scoring 73,612 UK Research Excellence Framework (REF) journal articles from all 34 field-based Units of Assessment (UoAs) 2014–17: REF peer review scores, field normalised citations, and journal average field normalised citation impact. The results suggest that in almost all academic fields, bibliometric scoring can disadvantage departments publishing high quality research, as judged by peer review, with the main exception of article citation rates in chemistry. Thus, introducing journal or article level citation information into peer review exercises may have a regression to the mean effect. Bibliometric scoring slightly advantaged women compared to men, but this varied between UoAs and was most evident in the physical sciences, engineering, and social sciences. In contrast, interdisciplinary research gained from bibliometric scoring in about half of the UoAs, but relatively substantially in two. In conclusion, out of the three potential sources of bibliometric bias examined, the most serious seems to be the tendency for bibliometric scores to work against high quality departments, assuming that the peer review scores are correct. This is almost a paradox: although high quality departments tend to get the highest bibliometric scores, bibliometrics conceal the full extent of departmental quality advantages, as judged by peer review. This should be considered when using bibliometrics or bibliometric informed peer review.

management
Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries

Mike Thelwall

DOI: https://doi.org/10.48550/arXiv.1808.03296

2018-08-10

Abstract:There are known gender imbalances in participation in scientific fields, from female dominance of nursing to male dominance of mathematics. It is not clear whether there is also a citation imbalance, with some claiming that male-authored research tends to be more cited. No previous study has assessed gender differences in the readers of academic research on a large scale, however. In response, this article assesses whether there are gender differences in the average citations and/or Mendeley readers of academic publications. Field normalised logged Scopus citations and Mendeley readers from mid-2018 for articles published in 2014 were investigated for articles with first authors from India, Spain, Turkey, the UK and the USA in up to 251 fields with at least 50 male and female authors. Although female-authored research is less cited in Turkey (-4.0%) and India (-3.6%), it is marginally more cited in Spain (0.4%), the UK (0.4%), and the USA (0.2%). Female-authored research has fewer Mendeley readers in India (-1.1%) but more in Spain (1.4%), Turkey (1.1%), the UK (2.7%) and the USA (3.0%). Thus, whilst there may be little practical gender difference in citation impact in countries with mature science systems, the higher female readership impact suggests a wider audience for female-authored research. The results show that the conclusions from a gender analysis depend on the field normalisation method. A theoretically informed decision must therefore be made about which normalisation to use. The results also suggest that arithmetic mean-based field normalisation is favourable to males.

Digital Libraries
Data-Driven Analysis of Gender Fairness in the Software Engineering Academic Landscape

Giordano d'Aloisio,Andrea D'Angelo,Francesca Marzi,Diana Di Marco,Giovanni Stilo,Antinisca Di Marco

2023-09-20

Abstract:Gender bias in education gained considerable relevance in the literature over the years. However, while the problem of gender bias in education has been widely addressed from a student perspective, it is still not fully analysed from an academic point of view. In this work, we study the problem of gender bias in academic promotions (i.e., from Researcher to Associated Professor and from Associated to Full Professor) in the informatics (INF) and software engineering (SE) Italian communities. In particular, we first conduct a literature review to assess how the problem of gender bias in academia has been addressed so far. Next, we describe a process to collect and preprocess the INF and SE data needed to analyse gender bias in Italian academic promotions. Subsequently, we apply a formal bias metric to these data to assess the amount of bias and look at its variation over time. From the conducted analysis, we observe how the SE community presents a higher bias in promotions to Associate Professors and a smaller bias in promotions to Full Professors compared to the overall INF community.

Computers and Society
ChatGPT identifies gender disparities in scientific peer review

Jeroen P H Verharen

DOI: https://doi.org/10.7554/eLife.90230

IF: 7.7

2023-11-03

eLife

Abstract:The peer review process is a critical step in ensuring the quality of scientific research. However, its subjectivity has raised concerns. To investigate this issue, I examined over 500 publicly available peer review reports from 200 published neuroscience papers in 2022-2023. OpenAI's generative artificial intelligence ChatGPT was used to analyze language use in these reports, which demonstrated superior performance compared to traditional lexicon- and rule-based language models. As expected, most reviews for these published papers were seen as favorable by ChatGPT (89.8% of reviews), and language use was mostly polite (99.8% of reviews). However, this analysis also demonstrated high levels of variability in how each reviewer scored the same paper, indicating the presence of subjectivity in the peer review process. The results further revealed that female first authors received less polite reviews than their male peers, indicating a gender bias in reviewing. In addition, published papers with a female senior author received more favorable reviews than papers with a male senior author, for which I discuss potential causes. Together, this study highlights the potential of generative artificial intelligence in performing natural language processing of specialized scientific texts. As a proof of concept, I show that ChatGPT can identify areas of concern in scientific peer review, underscoring the importance of transparent peer review in studying equitability in scientific publishing.

Peer review and gender bias: A study on 145 scholarly journals

Peer review perpetuates barriers for historically excluded groups

Personal attributes of authors and reviewers, social bias and the outcomes of peer review: a case study

Gender differences in research areas and topics: An analysis of publications in 285 fields

Gender disparities in UK research publishing: Differences between fields, methods and topics

Exploring Gender Bias in Six Key Domains of Academic Science: An Adversarial Collaboration

Bias against research on gender bias

Gender differences in peer reviewed grant applications, awards, and amounts: a systematic review and meta-analysis

Overcoming the gender bias in ecology and evolution: is the double-anonymized peer review an effective pathway over time?

A gender equality paradox in academic publishing: Countries with a higher proportion of female first-authored journal articles have larger first author gender disparities between fields

Assessing Gender Bias in the Information Systems Field: An Analysis of the Impact on Citations

Cross-sectional study of the relationship between women’s representation among editors and peer reviewers in journals of the British Medical Journal Publishing Group

Quality of evidence revealing subtle gender biases in science is in the eye of the beholder

The distribution of authors and reviewers in EPS

Gender bias in academic recruitment

The Myth of Meritocracy and the Matilda Effect in STEM: Paper Acceptance and Paper Citation

Global evidence of gender equity in academic health research: a scoping review

Do bibliometrics introduce gender, institutional or interdisciplinary biases into research evaluations?

Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries

Data-Driven Analysis of Gender Fairness in the Software Engineering Academic Landscape

ChatGPT identifies gender disparities in scientific peer review