| الوصف: |
Aims - To examine eligibility and assessment criteria of international individual research and publication recognition awards relevant to early and mid-career researchers in ecology and evolutionary biology. - To evaluate gender bias in the lists of past awardees of the above awards. - To communicate the need for improving awards policies, and how this can be done – ultimately making individual awards more equitable. Expected outcomes We will investigate scientific societies’ and journals’ commitment to addressing barriers and biases in academic awards by assessing whether they have inclusive and equitable practices and policies. As such, we will reveal which society/journal-run early/mid-career awards have equitable access and assessment, highlighting instances of good practice. We will also test whether such policies can be linked to improved gender diversity of awardees in recent years, where possible. Further, we will evaluate if Open Science practices are included in the assessment criteria. The resulting journal publication (JEB is the target journal) will hopefully act as a nudge for other award committees/societies/journals to shift from simple but non-equitable award policies towards strategies promoting inclusivity and diversity – benefiting whole research community, but especially these at early and mid-career stages. Collaboration overview The project team is diverse in terms of career stage, gender, location, and backgrounds (but all are working in the broad fields of ecology or evolution). The team will contribute to data extraction, checking, analyses and manuscript drafting. All collaborators contributing to data collection, checking or analyses will be offered co-authorship as a mid-author (likely in an alphabetic order, with first authorship reserved to the project lead) in the resultant manuscript if they read and approve the final manuscript. The manuscript will contain a CRediT statement detailing the roles of individual contributors. Data sources, selection and extraction We will focus on international academic societies/associations and journals focused on the disciplines of ecology and evolution. We will critically assess two categories of awards in evolutionary biology and ecology, with one category (“Best researcher” awards) focusing on the overall body of achievements of a researcher and the second category (“Best paper” awards) focusing more on a research contribution contained in a single publication, as described below. A1. “Best researcher” awards Search strategy: We will conduct Internet searches using DuckDuckGo.com platform running Google search algorithm with Region set to “All regions”, with the search string: “award|prize society|association ecology|evolution” (without quotation marks, note that the Google search algorithm will recognize different forms of these words, including plurals). We will screen top 100 search hits. We will perform supplementary manual searches by screening the websites with lists of links to other similar societies and lists of international learned societies (e.g. https://en.wikipedia.org/wiki/List_of_biology_awards). We will mainly use English in our searches, but will also endeavor to run search in other languages, within the abilities of the project team and expectations that regional societies may use these languages (e.g. Spanish/Portuguese for South America). Selection criteria: • We will include awards awarded by international societies to individual researchers for their overall achievements in scientific research. We will thus exclude institution-specific awards (e.g., department- or university-level) and country-specific awards, since these are usually restricted to students / staff / academics of a given institution / country (even if it is not explicitly specified in the society’s membership information; thus, if a name of the society includes a country/institution name, we will exclude it by default). • We will focus on societies and awards dedicated to broad studies in ecology and evolution. We will thus exclude societies and awards with narrow taxonomic scope (e.g., botany, ornithology, lichenology, human evolution), narrow habitat / geographical scope (e.g., coral reef), or narrow topic scope (e.g., systematics). • We will include awards targeting early and mid-career researchers, from undergraduate students to postdocs / lecturers or equivalent. We will thus exclude awards specifically targeting senior researchers or without any limitations on career stage / academic age. • We will exclude awards that are discontinued, awards that are restricted to applicants from underrepresented groups, e.g., women-only / minorities-only awards. • We will exclude travel awards, and awards where a project proposal is assessed, rather than only already completed work / achievements. • We will also exclude awards that focus solely on non-research achievements (e.g., outreach, teaching, mentoring), but will include awards where a multiple types of contributions to science are considered alongside research achievements. Preliminary list of potentially eligible “Best Researcher” awards: 1. John Maynard Smith Prize by the European Society for Evolutionary Biology 2. President’s Award by the European Society for Evolutionary Biology 3. T. Dobzhansky Prize by the Society for the Study of Evolution 4. T.H. Huxley Award by the Society for the Study of Evolution 5. Early Career Researcher Award by Australasian Evolution Society 6. Young Investigator Award by the Society for Molecular Biology and Evolution 7. International Ecology Award by Ecological Society of America A2. “Best paper” awards Search strategy: We will conduct Internet searches using DuckDuckGo.com platform running Google search algorithm with Region set to “All regions”, with search strings: “award|prize society|association|journal ecology|evolution paper/article/publication” (without quotation marks, note that the Google search algorithm will recognize different forms of these words, including plurals). We will screen top 100 search hits. We will also consider the top 50% journals from the Scimago journal ranking (https://www.scimagojr.com/journalrank.php?category=1105&area=1100&type=j&year=2020). In this search, we aim for a representative sample of high quality disciplinary journals. Selection criteria: • “Best paper” (or equivalent) awards, are usually associated with specific journals or publishers, and are often aimed at early career researchers (ECRs). We will thus include awards awarded by international societies or journals and awarded to individuals for a single published research contribution (theses / dissertation included if published as a journal article). • We will exclude conference-specific awards: “best talk”, “best poster”, etc., due to restricted space / time they allow for presentation of a given research study. • We will also exclude institutional awards (e.g., department- or university-level) and country-specific awards, since these are usually restricted to students / staff / academics of a given institution / country. • We will focus on awards focused on broad studies in ecology and evolution. We will thus exclude awards with narrow taxonomic scope (e.g. botany, ornithology, lichenology, human evolution), narrow habitat/geographical scope (e.g. coral reef), or narrow topic scope (e.g. systematics). • We will exclude awards that are discontinued, and awards that are restricted to applicants from underrepresented groups, e.g., women-only / minorities-only awards; also awards restricted to researchers from a single country. Preliminary list of potentially eligible “Best Paper” awards: 1. Stearns Graduate Student Prize by the European Society for Evolutionary Biology 2. Presidents’ Award for Outstanding Dissertation Paper in Evolution by the Society for the Study of Evolution 3. ECR Awards by “Ecology and Evolution” (journal) 4. ECR Awards from “Methods in Ecology and Evolution” (journal) 5. Best Student Paper by “Heredity” (journal) 6. S.J. O’Brien Award by “Journal of Heredity” (journal) Eligibilty screening We will collate a list of the societies and journals that meet our criteria. For each society/journal we will record any relevant awards for data extraction. Eligibility will be cross-checked by a second researcher. Data extraction We will extract relevant data on each award from the websites (e.g., societies or journals), as relevant, or other publicly documents available (e.g., instructions for applicants). We may attempt to contact the award committee / contact for clarifications, if essential. We will record the collected information using pre-piloted GoogleForms, which will be then converted into GoogleSheets for checking and filling in missing information, if necessary. Extracted data will be cross-checked by a second researcher. Gender will be assigned to awardee names using https://api.genderize.io, with probability threshold of >0.5. The extracted data will include information on the granting society / journal, award eligibility criteria, award assessment criteria, and past awardees (see the award-specific data listed below). “EDI” stands for “Equity, Diversity, Inclusion”. “NA” stands for “Not Applicable”. For each award type its eligibility for inclusion will be confirmed, and if deemed not eligible, no further data will be extracted. Additional variables will be extracted capturing potential barriers to EDI. Key data to be extracted for “Best Researcher” awards: General: o Full name of the award[singular variable: text]. o Full name of the awarding society [singular variable: text]. o Main source of information [singular variable: text]. o Society disciplinary focus [plural variable: ecology / evolution]. o Society geographical range [singular variable: global / regional / unclear]. o Commitment to EDI in the society policies [singular variable: yes / no / unclear]. o Commitment to EDI in the society structures [singular variable: yes / no / unclear / not applicable]. Eligibility: o Source of the information on the award eligibility criteria [singular variable: text – link to a webpage, file name, personal information, etc.]. o Target career stage of eligible applicants, as stated in the award information [plural variable: student / early / mid / unclear]. o Flexibility of the eligibility criteria – whether explicitly allowing for career interruptions in eligibility timeframes [singular variable: yes / no / unclear]. o Eligibility phrasing – wording of the eligibility criteria in relation to career stage in relevant documentation [singular variable: text]. o Inclusivity statement – whether underrepresented groups are encouraged to apply for the award (this does not mean that the award is restricted to underrepresented groups, e.g., women-only) or award information includes a statement of commitment to equity / diversity / inclusivity [singular variable: yes / no / unclear]. o Inclusivity phrasing – wording of the inclusivity statementin relevant documentation, if available [singular variable: text]. Assessment: o Assessors transparency – whether information is provided on who will be conducting assessments of researchers / papers [singular variable: yes / no / unclear]. o Assessors phrasing – wording of the information on who will be conducting the assessments, if available [singular variable: text]. o Process transparency – whether breakdown of the applicants / candidates by gender or geographic region is publicly available [singular variable: yes / no / unclear]. o Comment on process transparency [singular variable: text]. o Feedback availability – whether award information includes an offer of constructive feedback for unsuccessful applicants [singular variable: yes / no / unclear]. o Feedback phrasing – wording of the information on whether/how feedback will be provided, if available [singular variable: text]. o Criteria transparency – whether assessment criteria are detailed (usually more than one sentence) or vague (often stated as a single sentence, e.g., “assessed on innovation and novelty”) [singular variable: yes / no / unclear]. o Criteria phrasing – wording of the information on the assessment criteria, if available [singular variable: text]. o Assessment relatively to opportunity – whether assessment criteria explicitly state that assessment is performed relatively to opportunity, e.g., by considering career interruptions, caring responsibilities, access to resources [singular variable: yes / no / unclear / not applicable]. o Relative opportunity phrasing – wording of the information on the assessment criteria being applied relative to opportunity, if available [singular variable: text]. o Valuing diverse contributions – whether multiple dimensions of contributions to science and research excellence are considered, such as engagement in outreach, mentoring, reviewing, advocacy [singular variable: yes / no / unclear / not applicable]. o Valuing diverse contributions phrasing – wording of the information on the assessment criteria valuing diverse contributions, if available [singular variable: text]. o Valuing Open Science – whether any Open Science practices (data, code, materials sharing, preregistration, transparency of reporting, etc.) are explicitly included in the assessment criteria [singular variable: yes / no / unclear]. o Valuing Open Science phrasing – wording of the information on the assessment criteria valuing Open Science practices, if available [singular variable: text]. o Self-nomination allowed – candidates can self-nominate for the award [singular variable: yes / no / unclear]. o Letter required – candidates are required to provide nomination / recommendation letter/letters [singular variable: yes / no / unclear]. o Letter requirement phrasing – Wording of the information on the requirement for written nominations / reference letters, if available [singular variable: text]. Awardees: o Awardee list source - source of the information on the past awardees [singular variable: text]. o Awardee list number of years – for how many years information on past awardees is available [singular variable: number]. o Number of female awardees 2011-2020 [singular variable: number] (Note: gender of the past awardees will be assigned based on the first names, images or personal online profiles, as feasible. If main + finalists are provided for each year, include all these. Do not include shortlisted candidates). o Number of male awardees 2011-2020 [singular variable: number]. o Number of awardees with unassignable gender 2011-2020 [singular variable: number]. o Number of female awardees 2001-2010 [singular variable: number]. o Number of male awardees 2001-2010 [singular variable: number]. o Number of awardees with unassignable gender 2001-2010 [singular variable: number]. o Number of female awardees 1991-2000 [singular variable: number]. o Number of male awardees 1991-2000 [singular variable: number]. o Number of awardees with unassignable gender 1991-2000 [singular variable: number]. o Number of female awardees 1981-1990 [singular variable: number]. o Number of male awardees 1981-1990 [singular variable: number]. o Number of awardees with unassignable gender 1981-1990 [singular variable: number]. o Extractable awardee information available for prior to 1981? (To be extracted separately later, if available) [singular variable: yes / no / unclear]. Other: o Additional comment fields will be available for taking detailed notes and making comments on issues, assumptions, or seeking additional information [singular variable: text]. Other key data to be extracted for “Best Paper” awards: General: o Full name of the award [singular variable: text]. o Full name of the awarding journal [singular variable: text]. (Note: record society name if relevant). o Journal disciplinary focus [plural variable: ecology / evolution]. o Journal geographical range [singular variable: global / regional / unclear]. o Journal commitment to EDI in the journal policies - whether the website or policy documents mention commitment to EDI [singular variable: yes / no / unclear]. o Comment on journal commitment to EDI in the journal policies [singular variable: text]. Eligibility: o Source of the information on the award eligibility criteria [singular variable: text]. o Target career stage of eligible applicants, as stated in the award information [plural variable: student / early / mid / unclear]. o Flexibility of the eligibility criteria – whether explicitly allowing for career interruptions in eligibility timeframes [singular variable: yes / no / unclear]. o Eligibility phrasing – wording of the eligibility criteria in relation to career stage in relevant documentation [singular variable: text]. o Inclusivity statement – whether underrepresented groups are encouraged to apply for the award (this does not mean that the award is restricted to underrepresented groups, e.g., women-only) or award information includes a statement of commitment to equity / diversity / inclusivity [singular variable: yes / no / unclear]. o Inclusivity phrasing – wording of the inclusivity statement in the relevant documentation, if available [singular variable: text]. Assessment: o Assessors phrasing – wording of the information on who will be conducting the assessments, if available [singular variable: text]. o Process transparency – whether breakdown of the applicants / candidates by gender or geographic region is publicly available [singular variable: yes / no / unclear]. o Comment on process transparency [singular variable: text]. o Feedback availability – whether award information includes an offer of constructive feedback for unsuccessful applicants [singular variable: yes / no / unclear]. o Feedback phrasing – wording of the information on whether/how feedback will be provided, if available [singular variable: text]. o Criteria transparency – whether assessment criteria are detailed (usually more than one sentence) or vague (often stated as a single sentence, e.g., “assessed on innovation and novelty”) [singular variable: yes / no / unclear]. o Criteria phrasing – wording of the information on the assessment criteria, if available [singular variable: text]. o Valuing Open Science – whether any Open Science practices (data, code, materials sharing, preregistration, transparency of reporting, etc.) are explicitly included in the assessment criteria [singular variable: yes / no / unclear]. o Valuing Open Science phrasing – wording of the information on the assessment criteria valuing Open Science practices, if available [singular variable: text]. o Self-nomination allowed – candidates can self-nominate for the award [singular variable: yes / no / unclear]. o Letter required – candidates are required to provide nomination / recommendation letter/letters [singular variable: yes / no / unclear]. o Letter requirement phrasing – Wording of the information on the requirement for written nominations / reference letters, if available [singular variable: text]. Awardees: o Awardee list source - source of the information on the past awardees [singular variable: text – link to a webpage, file name, personal information, etc.]. o Awardee list number of years – for how many years information on past awardees is available [singular variable: number]. o Number of female awardees 2011-2020 [singular variable: number] (Note: gender of the past awardees will be assigned based on the first names, images or personal online profiles, as feasible. If main + finalists are provided for each year, include all these. Do not include shortlisted candidates). o Number of male awardees 2011-2020 [singular variable: number]. o Number of awardees with unassignable gender 2011-2020 [singular variable: number]. o Number of female awardees 2001-2010 [singular variable: number]. o Number of male awardees 2001-2010 [singular variable: number]. o Number of awardees with unassignable gender 2001-2010 [singular variable: number]. o Number of female awardees 1991-2000 [singular variable: number]. o Number of male awardees 1991-2000 [singular variable: number]. o Number of awardees with unassignable gender 1991-2000 [singular variable: number]. o Number of female awardees 1981-1990 [singular variable: number]. o Number of male awardees 1981-1990 [singular variable: number]. o Number of awardees with unassignable gender 1981-1990 [singular variable: number]. o Extractable awardee information available for prior to 1981? (To be extracted separately later, if available) [singular variable: yes / no / unclear]. Other: o Additional comment fields will be available for taking detailed notes and making comments on issues, assumptions, or seeking additional information [singular variable: text]. Data checking Extracted data will be independently cross-checked by second researcher. Any disagreements or unclear data will be resolved to consensus via discussion with the original data extractor, with input from additional researchers or clarifications sought from the award committee / contact, as needed. Data analyses We will summarize extracted information separately for “best researcher” and “best article” award categories. Specifically, we will: o Use R computational environment for final data cleaning, summaries, analyses, and visualizations, with code versioning via git and open collaboration via GitHub. o Summarize extracted data in tables and graphs. o Calculate proportions for each response option for the categorical variables and visualize results for key extracted variables as bar plots of proportions or counts. o Calculate awardee gender ratios for ten-year windows for each award with at least 10 years of awardee gender data (inferred from lists of awardees, see data extraction section). For awards with at least 20 years of awardee gender data available, we will plot temporal trends in gender ratios along ten-year periods (potentially analyze statistically if more than 10 data points available in each period or pool years if earlier categories are too small, e.g. to pre-2000). o Estimate pairwise association among the key categorical variables (for the “yes” and “no” response codes only), for the variables with at least 80% of responses coded as “yes” or “no” (it is not meaningful to analyze “unclear” or ”not applicable”, response codes). We will use Goodman and Kruskal’s τ measure of association between categorical predictor variables using function GKtauDataframe from the R package GoodmanKruskal. Conflict of Interests: All authors declare no conflict of interest. Funding: This project is supported by a grant from the European Society for Evolutionary Biology (ESEB) Equal Opportunities Initiative. |
