The research tested the accuracy of the VIVO Harvester software in identifying publications authored by faculty members affiliated with a National Institutes of Health Clinical and Translational Sciences Award (CTSA) site.

Health sciences librarians created “gold standard” lists of references for the years 2001 to 2011 from PubMed for twenty-five randomly selected investigators from one CTSA site. These gold standard lists were compared to the same twenty-five investigators' reference lists produced by VIVO Harvester. The authors subjected the discrepancies between the lists to sensitivity and specificity analyses.

The VIVO Harvester correctly identified only about 65% of the total eligible PubMed references for the years 2001–2011 for the CTSA-affiliated investigators. The identified references produced by VIVO Harvester were precise yet incomplete. The sensitivity rate was 0.65, and the specificity rate was 1.00.

While the references produced by VIVO Harvester could be confirmed in PubMed, the VIVO Harvester retrieved only two-thirds of the required references from PubMed. National Institutes of Health CTSA sites will need to supplement VIVO Harvester–produced references with the expert searching skills of health sciences librarians.