A scientist’s performance is often measured by two factors: productivity, measured by the number of publications; and impact, as indicated by the number of times those articles are cited. It is widely believed that impact peaks early in a scientist’s career, while having a mature, established research program increases productivity. But in fact, little is known about how productivity and impact evolve over time.
In a recent study published in Sciencepeople have actually looked at how these measures change over the course of a scientist’s career, finding that our assumptions are not always well founded.
To understand how performance measures change over time, the publication profiles of scientists from multiple disciplines were reconstructed. The number of publications and the impact were registered for each scientist. The impact was expressed in the total number of citations 10 years after publication. The most influential work was identified as the single paper with the most citations at the time.
The team also grouped the scientists by their peak impact, regardless of when it occurred: high maximum impact (top 5 percent), low maximum impact (bottom 20 percent), and medium maximum impact (middle 75 percent).
Productivity and impact
The first question answered was how productivity changes over time. Productivity, as determined by the number of papers published, generally increases over the course of a scientist’s career. For high-impact scientists, productivity has almost tripled. For low-impact scientists, however, productivity growth was more modest. The team also randomized each scientist’s career by exchanging the impact of all their papers; this allows them to test whether trends or timing affect when the work with the greatest impact is done.
Next, the study looked at the times when a scientist’s most influential publications occurred. For each scientist, researchers determined the number of years that elapsed between the scientist’s first publication and their most impactful paper. This information was used to evaluate the likelihood that the highest impact article would occur in a given year after its initial publication. Analysis indicates that most scientists publish their paper with the greatest impact within the first 20 years of their careers; after 20 years that chance decreases.
To investigate the origin of this pattern, researchers randomized the order of publications and performed the same analysis. They found that the impact probability of the made-up career is indistinguishable from the original study. This suggests that probability depends on annual variations in productivity over an individual’s career.
Finally, the researchers measured the likelihood that the most cited work would appear sooner or later in the series of published articles. This analysis showed that impact is randomly distributed within a scientist’s oeuvre.
Neither publication time nor publication order influence this finding. The work with the highest impact can occur with equal probability anywhere in the series of publications, regardless of discipline, career length, career period, number of authors, and assigned author contributions. Further analysis found that growth in average impact during a scientific career can be attributed to growing productivity and that proficiency or excellence does not affect how impact develops over time.
Predict success
Using this information, the team created a model that uncovers underlying patterns that determine the emergence of scientific success. Since the impact varies greatly between scientists, a unique individual parameter Q was determined for each scientist to capture this aspect. The Q parameter is stable throughout an individual’s career and accurately predicts the evolution of a scientist’s impact, which is determined by factors such as independent recognition and cumulative citations.
This research suggests that it might be possible to conduct a standardized evaluation of an individual scientist’s performance in a quantitative manner. This news could be especially welcome for academic institutions that rely on metrics such as number of publications and number of citations to measure an academic’s success. However, we’re not sure what impact this predictive model might have if used to assess scientists in their early years.
Science2016. DOI: 10.1126/science.aaf5239 (About DOIs).