NCBI´s PubMed is a powerful literature retrieval tool widely utilised in many areas including science and regulatory affairs. In regulatory affairs, PubMed searches are employed to identify clinical evidence pertinent to product approval processes. To ensure traceability and reproducibility, a highly structured literature search strategy is advised, as laid out in numerous guidance documents issued by regulatory agencies such as the European Commission and the International Medical Device Regulators Forum. Recently, a new version of PubMed was deployed, including a new user interface and, less visibly, potential changes to search algorithms, which may affect the results delivered by search strings. To unravel potential differences among the legacy and new version of PubMed, head-to-head comparisons with increasing search complexity were performed. For the new version of PubMed, the user interface was redesigned and allows feature customisation. Importantly, as compared to the legacy version of PubMed, the new PubMed delivered diverging numbers of search hits. Of note, the PubMed inherent result sorting methods produced alternating search hit numbers only in the legacy version. Intriguingly, each version identified literature that was not found by the respective other version, although these publications were considered relevant in the search context. Technically, translation of entered search strings into detailed search strings varied between interfaces. Differences between the legacy (online at
least until September 30, 2020) and the new version were found, affecting the traceability, reproducibility, and reliability of PubMed data used for approval processes.