Ameet Doshi co-authors paper on use of National Academies consensus reports by the American public

Ameet Doshi, Head Librarian at Stokes Library, has co-authored the article “Widespread use of National Academies consensus reports by the American public,” published this week in The Proceedings of the National Academy of Sciences (PNAS), the official journal of the National Academy of Sciences, Engineering and Medicine.

Ameet Doshi, Head Librarian, Stokes Library.

Over the past two decades, the advent of open access has made a growing portion of the scholarly research universe available to anyone with an internet connection. Yet, there has been very little empirical analysis or even data available on how non-scientists and non-academics use this research. How scholarly information is used by the general public is of particular interest to Doshi. He is currently working on a dissertation at Georgia Institute of Technology (Georgia Tech) related to this—a secondary analysis of how non-scientists use open access research from university repositories. 

The published research was completed by Doshi and a team from Georgia Tech: lead author, Professor Diana Hicks of Georgia Institute of Technology’s School of Public Policy, Georgia Tech Professor Omar Asensio and Doshi’s fellow Georgia Tech PhD student Matteo Zullo.

Using BERT (Bidirectional Encoder Representations from Transformers), a state-of-the-art neural network natural language processor from Google, the team was able to analyze 6.6 million downloads and 1.6 million downloader comments between 2011 and 2020. However, a classification schema based on information science theory was still needed to draw sound conclusions, and a ground truth database was developed for training. In the end, human annotation and making the connection with information science theory still comprised a significant part of the effort.

The research found that over half of National Academies report downloads were for non-academic use. Despite the motivation and effort needed to parse scientific jargon and technical writing, and how easy it is to glean mis-and dis-information from social media, it seems clear that the American public still seeks open access to evidence-based, policy-oriented consensus science from a trusted institution. The PNAS paper describes an uplifting narrative of Americans accessing and integrating the best available science to improve their work, their institutions, and themselves through self-directed learning. 

Doshi joined Princeton University Library (PUL) as head of Stokes Library in October 2021. The research was largely conducted while he was a librarian and doctoral student at Georgia Tech, but at Princeton, Doshi was able to leverage PUL’s literature regarding human annotation in machine learning. PUL offers access to several of the journals at the intersection of computer science, linguistics, and philosophy. Doshi also attended sessions offered by the Center for Digital Humanities on machine language-driven classification of historic newspapers, as well as a Wintersession program about machine learning at Princeton. 

These conversations helped Doshi further explore prevailing notions regarding the accuracy of natural language processing with respect to social science domains.  

Additionally, interactions with PUL colleagues, especially Yuan Li and Wind Cowles, about the implications of open access for spreading scholarly knowledge more democratically were particularly productive. Indeed, there is a strong connection between the research findings and PUL's North Star statement, "We will meet patrons wherever they are, and democratize access to knowledge as a core component of the library experience." 

For Doshi, one of the most surprising aspects of the research was that 3,700 people used the phrase "for personal edification" as their reason for downloading a National Academies report. “Seeing that signal of innate curiosity and lifelong learning warmed my librarian heart,” said Doshi. 

He concluded, “I think the implications could also be used as hard evidence that investments in open science do pay identifiable dividends to broader society.” 

For more coverage about the research, read the article published by Georgia Tech’s Ivan Allen College of Liberal Arts.

Published February 24, 2022

Written by: Stephanie Oster, Publicity Manager

Media Contact: Barbara Valenza, Director of Library Communications