Citation Impact indicator

The bibliometrics Web of Science application from Thomson Reuters and Google Scholar application provide citation Impact indicators.

• definition & calculation

  One of the most commonly used bibliometric indicators for evaluating sets of documents is the citation Impact indicator (also called “citations per publication” or “average citation rate”).

  Citation Impact of a set of documents is calculated by dividing the total number of citations by the total number of publications. Citation Impact shows the average number of citations that a document has received.

  Citation Impact has been extensively used as a bibliometric indicator in research performance evaluation and can be applied at all organizational levels (author, institution, country/region, research field or journal). However, there are limitations to the indicator. For example, it ignores the total volume of research outputs.

While the citation Impact indicator is a better measure than the journal impact factor (JIF) to assess the intrinsic quality of a published work, the researcher should remain cautious and careful in interpreting these numbers. There may be strong differences in citation counts depending on the way they were calculated. Web of Science only includes citations originating from articles published in journals with official impact factor, while Google Scholar also considers citations from the grey literature (all types of magazines, thesis,…). In addition, the rate of citation will clearly depend on the size of the community working on similar research and is therefore dependent on the discipline in which the research is carried out. Thus, citation counts that normalize the citation rate of an item by type of discipline (Normalized Citation Impact) are more pertinent and should be used for an equitable evaluation process (The Leiden Manisfesto). Moreover, the citation impact of an article should be standardized according to the authorship position to more accurately assess the level of participation and involvement of the researcher. Finally, the citation Impact indicator also seems to depend on the prestige of the journal in which the work is published. Indeed, the study ‘Clinical Trial Registration — Looking Back and Moving Ahead‘ shows that the same clinical trial published in various journals received a number of citations per publication which varied according to the JIF of the journals ( https://stuartcantrill.com/2016/01/23/imperfect-impact/ ). Thus, an index that standardizes the citation rate of an article depending on the journal in which it is published (Journal Normalized Citation Impact) puts into perspective this influence and allows to appreciate the performance of the article in a given journal. Normalized citation impact (NCI)

Normalized citation impact (NCI)

The bibliometrics InCites 2.1 application from Thomson Reuters provides the normalized citation impact by discipline.

Citation rates vary across disciplines, citations grow over time, and different publication types have different citation behaviors. For accurate and fair research assessment, citation data should be normalized by discipline, year and publication type.

• calculation

  The Normalized Citation Impact (NCI) of a single publication is calculated by dividing the actual count of citing items by the expected citation rate (baseline) for publications with the same document type, year of publication and subject area. When a document is assigned to more than one subject area, an average of the ratios of the actual to expected citations is used. The NCI of a set of documents, for example, the collected works of an individual, institution or country, is the average of the NCI values for all the documents in the set.

NCI is a valuable and unbiased indicator of impact irrespective of age, subject focus or document type. Therefore, it allows comparisons between entities of different sizes and different subject mixes. An NCI value of one represents performance at par with world average, values above one are considered above average and values below one are considered below average. An NCI value of two is considered twice world average.

NCI is an ideal indicator for benchmarking at all organizational levels (author, institution, region etc). One can also use NCI to identify impactful sub-sets of documents and assess any research activity. For example, an institution may use the NCI to assess which collaborations are the most impactful or identify new potential collaboration opportunities. Or to identify the performance of up-and-coming researchers compared to established ones and to aid with faculty recruitment by assessing candidates. As a funding organization, one may use the NCI as a quantitative performance indicator to monitor the performance of funded projects, or assess the track record of a research teams applying for a new funding.

Journal normalized citation impact

The bibliometrics InCites 2.1 application from Thomson Reuters provides the journal normalized citation impact by discipline.

The Journal Normalized Citation Impact (JNCI) indicator is a similar indicator to the Normalized Citation Impact, but instead of normalizing per subject area or field, it normalizes the citation rate for the journal in which the document is published.

• calculation

  The Journal Normalized Citation Impact of a single publication is the ratio of the actual number of citing items to the average citation rate of publications in the same journal in the same year and with the same document type. The JNCI for a set of publications is the average of the JNCI for each publication.

The JNCI indicator can reveal information about the performance of a publication (or a set of publications) in relation to how other researchers perform when they publish their work in a given journal (or a set of journals). It can provide the answers to questions, such as “How do my papers perform in the journals I publish?” If the numerical value of the JNCI exceeds one, then the assessed research entity is performing above average. If it is less than one, then it is performing below the average.

h-index

The bibliometrics Web of Science application from Thomson Reuters and Google Scholar application provide the h-index indicator.

• definition & calculation

  The h-index (also known as Hirsch index) was introduced by J. Hirsch in 2005 and can be defined as follows: A researcher has an h-index, if he/she has at least h publications for which he/she has received at least h citations. For example, Researcher A has an h-index = 13 if he/she has published at least 13 documents for which he/she has received at least 13 citations. Its popularity as a bibliometric indicator has derived from the fact that it combines productivity (number of documents) and impact (number of citations) in one index.

The h-index can be applied to any level of aggregation (author, institution, journal, etc.) and it can reveal information about how the citations are distributed over a set of documents. At the author level, it is considered to be an indicator of a researcher’s lifetime scientific achievements. Some clear advantages of the h-index are that it is a mathematically simple index, it encourages large amounts of impactful research work while at the same time discourages publishing unimportant output and that single highly cited publications do not influence the h-index (unlike the Citation Impact).

It is important to keep in mind that the h-index is proportional to the duration of a researcher’s career. Therefore, early career researchers have a disadvantage when compared to more senior researchers since the latter would have more articles and would have received more citations per publication. Also, the h-index varies by field: life scientists top out at 200; physicists at 100 and social scientists at 20–30 (Hirsch, J. E. Proc. Natl Acad. Sci. USA 102, 16569–16572 -2005). Finally, the h-index varies depending of the database, with higher h-index in Google Scholar as compared to Web of Science.

Journal Impact Factor

To know the journal indicators for a journal of interest (JIF, Eigenfactor score,…), search for the journal using the “Go to Journal Profile” option in InCitesTM Journal Citation Reports. In the list of results, the JIFs of the journal are given for each year.

To compare journals:

• Ranking between known journals use the “select Journals” option.
• Ranking of journals by disciplines use the “select discipline” option.
• Ranking of journals by disciplines and JIF quartile use the “select discipline” and “JIF quartile”options.

Citations

To know the citation impact of an individual article, search for a publication using the ‘Search‘ or ‘Cited reference search‘ options in the WoS TM Core Collection. In the list of results, the citations per publication are given for each individual article ‘Times cited’. By clicking on the number, you can see all the details concerning the citing articles.

h-index

To know about individual author indicators (citation impact and h-index), search a name using the ‘Author search’ option in the WoS TM Core Collection. Enter the author name and add author Name Variants if required. Optionally, you can select the author’s Organization if you wish to limit your search. In the list of results, select the publication records corresponding to the author of interest by ticking the appropriate boxes. To view author indicators (citation impact and h-index) select ‘Create citation report’ at the top right.

ResearcherID helps you manage your author indicators and can easily be connected to your ORCID iD. After creating your ResearcherID in ‘My Tools’, you can regularly update your list of publication indexed by WoS or manually add your papers. For more details on how to create your ResearcherID click here.

Normalized citations: Incites 2.1

Video from Thomson reuters:

Advanced bibliometry is available through Thomson Reuters separate product Incites2, which  can help evaluate output, performance and trends at the individual, lab, department, institution and global levels. InCites 2.1 allows assessing authors performance in a specific discipline or in a specific journal, and evaluating the performance of your collaborative network.

To know more about Web of Science and Incites indicators calculation read the InCites_MetricsGuideBook_web

Citations

To know about the citation impact of an individual article, search for a publication using the ‘Search’ option in Google scholar. In the list of results, the citations per publication are given for each individual article ‘cited by’. By clicking on ‘cited by’ you can see all details concerning the citing articles.

h-index

To know about individual author indicators (citation impact, h-index and i10-index= number of publications with at least 10 citations), search a name using the ‘Search’ option in Google scholar. If a user profile for the author exists, it will show up at the top of the results list. Click on the user profile to visualize its indicators.

User profile in Google Scholar helps you manage your author indicators. For more details on how to create your Google Scholar profile click here.

Almetric

Article indicator

What does Altmetric do?

Altmetric is a free web service collecting and collating information across dozens of different platforms and multiple websites to provide researchers with a single visually informative view of the online activity surrounding their publications.

Altmetric begins collating the online mentions and shares of your research across several sources (blogs, social media, mainstream media, ….) as soon as it is published, giving you an immediate feedback on how it is being received that complements your article citation data.

The altmetric score and donut allow to easily identify how much and what type of attention a research output has received. Donuts are regularly used on publisher, institutional repositories, or laboratory publications pages. Clicking on the donut gives you the recording details of the publication outputs. Altmetric also offers the possibility to researchers to add a Bookmarklet for free on their webpage to get article level metrics for their publications.

Impacstory

Author indicators

Impactstory is an open-source, free web-based tool that helps scientists explore and share the diverse impacts of all their research products. By helping scientists tell data-driven stories about their altmetric impacts, they are building a new scholarly reward system.

Impacstory is committed to:

• open source
• free and open data, to the extent permitted by data providers
• radical transparency and open communication

Impactstory helps you discovering almetric indicators for your publications and can easily be connected to your ORCID iD.

Metric tide

We also strongly encourage you to read the independent review of the role of metrics in research assessment and management “the metric tide: executive summary”. This report recommends the involvement of peer reviews throughout the research assessment process and emits strong reserves on the abuse of metrics.

Metrics should support, not supplant, expert judgement. Peer review is not perfect, but it is the least worst form of academic governance we have, and should remain the primary basis for assessing research papers, proposals and individuals.

Responsible metrics are proposed as a way of framing appropriate uses of quantitative indicators in the governance, management and assessment of research and should englobe the following dimensions:

• Robustness: basing metrics on the best possible data in terms of accuracy and scope;
• Humility: recognising that quantitative evaluation should support – but not supplant – qualitative, expert assessment;
• Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results;
• Diversity: accounting for variation by field, and using a variety of indicators to support diversity across the research system;
• Reflexivity: recognising systemic and potential effects of indicators and updating them in response.

Call for Open Science

Importantly, in the Amsterdam Call for Action on Open Science (April 5-6 2016), Europe asks for new assessment, reward and evaluation systems to remove barriers to open science.

Open science presents the opportunity to radically change the way we evaluate, reward and incentivise science. Its goal is to accelerate scientific progress and enhance the impact of science for the benefit of society. By changing the way we share and evaluate science, we can provide credit for a wealth of research output and contributions that reflect the changing nature of science.

The assessment of research proposals, research performance and researchers serves different purposes, but often seems characterised by a heavy emphasis on publications, both in terms of the number of publications and the prestige of the journals in which the publications should appear (citation counts and impact factor). This emphasis does not correspond with our goals to achieve societal impact alongside scientific impact. The predominant focus on prestige fuels a race in which the participants compete on the number of publications in prestigious journals or monographs with leading publishers, at the expense of attention for high-risk research and a broad exchange of knowledge. Ultimately this inhibits the progress of science and innovation, and the optimal use of knowledge.

Concrete actions have to be taken to create new systems that really deal with the core of knowledge creation and account for the impact of scientific research on science and society at large, including the economy, and incentivise citizen science.

• National authorities, European Commission and research funders: reform reward systems, develop assessment and evaluation criteria, or decide on the selection of existing ones (e.g. DORA for evaluations and the Leiden Manifesto for research metrics), and make sure that evaluation panels adopt these new criteria.
• Research Performing Organisations, research funders and publishers: further facilitate and explore the use of so-called alternative metrics where they appear adequate to improve the assessment of aspects such as the impact of research results on society at large. Experiment with new approaches for rewarding scientific work.
• Research communities, research funders and publishers: develop and adopt citation principles for publications, data and code, and other research outputs, which include persistent identifiers, to ensure appropriate rewards and acknowledgment of the authors.
• Research communities and publishers: facilitate and develop new forms of scientific communication and the use of alternative metrics