Abstract
This paper explores the growing need for more efficient and streamlined methods of retrieving academic data, especially when working with extensive datasets. It provides a detailed comparison of three retrieval techniques: manual searches, web scraping, and Application Programming Interfaces (APIs). While manual methods remain effective for small-scale searches, they quickly become impractical for large data volumes. In contrast, web scraping and APIs offer automation that significantly accelerates data collection. However, platforms like ResearchGate currently limit the use of these automated methods, forcing researchers to rely on manual processes. This paper advocates for ResearchGate to implement APIs, akin to those of Scopus and Web of Science, to provide controlled, secure, and efficient access to academic literature. Such advancements would be particularly beneficial to researchers from developing nations and institutions without access to subscription-based services. Furthermore, enabling automation would democratize access to scientific resources and foster a more inclusive global academic environment
Presenters
Mathias HaasJr. Researcher, Faculty of Social Sciences and Humanities, Escuela Superior Politécnica del Litoral (ESPOL), Guayas, Ecuador
Details
Presentation Type
Paper Presentation in a Themed Session
Theme
KEYWORDS
Web-scraping, Algorithms, Python, Literature review
