| アイテムタイプ |
学術雑誌論文 / Journal Article(1) |
| 公開日 |
2025-09-30 |
| タイトル |
|
|
タイトル |
Improving Systematic Review Updates With Natural Language Processing Through Abstract Component Classification and Selection: Algorithm Development and Validation |
| 言語 |
|
|
言語 |
eng |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
systematic review |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
natural language processing |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
guideline updates |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
bidirectional encoder representations from transformer |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
screening model |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
literature |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
efficiency |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
updating systematic reviews |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
language model |
| 資源タイプ |
|
|
資源タイプ |
journal article |
| アクセス権 |
|
|
アクセス権 |
open access |
| 著者 |
Hasegawa, Tatsuki
Kizaki, Hayato
Ikegami, Keisho
Imai, Shungo
Yanagisawa, Yuki
矢田, 竣太郎
荒牧, 英治
Hori, Satoko
|
| 抄録 |
|
|
内容記述タイプ |
Abstract |
|
内容記述 |
Background: A challenge in updating systematic reviews is the workload in screening the articles. Many screening models using natural language processing technology have been implemented to scrutinize articles based on titles and abstracts. While these approaches show promise, traditional models typically treat abstracts as uniform text. We hypothesize that selective training on specific abstract components could enhance model performance for systematic review screening. Objective: We evaluated the efficacy of a novel screening model that selects specific components from abstracts to improve performance and developed an automatic systematic review update model using an abstract component classifier to categorize abstracts based on their components. Methods: A screening model was created based on the included and excluded articles in the existing systematic review and used as the scheme for the automatic update of the systematic review. A prior publication was selected for the systematic review, and articles included or excluded in the articles screening process were used as training data. The titles and abstracts were classified into 5 categories (Title, Introduction, Methods, Results, and Conclusion). Thirty-one component-composition datasets were created by combining 5 component datasets. We implemented 31 screening models using the component-composition datasets and compared their performances. Comparisons were conducted using 3 pretrained models: Bidirectional Encoder Representations from Transformer (BERT), BioLinkBERT, and BioM- Efficiently Learning an Encoder that Classifies Token Replacements Accurately (ELECTRA). Moreover, to automate the component selection of abstracts, we developed the Abstract Component Classifier Model and created component datasets using this classifier model classification. Using the component datasets classified using the Abstract Component Classifier Model, we created 10 component-composition datasets used by the top 10 screening models with the highest performance when implementing screening models using the component datasets that were classified manually. Ten screening models were implemented using these datasets, and their performances were compared with those of models developed using manually classified component-composition datasets. The primary evaluation metric was the F10-Score weighted by the recall. Results: A total of 256 included articles and 1261 excluded articles were extracted from the selected systematic review. In the screening models implemented using manually classified datasets, the performance of some surpassed that of models trained on all components (BERT: 9 models, BioLinkBERT: 6 models, and BioM-ELECTRA: 21 models). In models implemented using datasets classified by the Abstract Component Classifier Model, the performances of some models (BERT: 7 models and BioM-ELECTRA: 9 models) surpassed that of the models trained on all components. These models achieved an 88.6{\%} reduction in manual screening workload while maintaining high recall (0.93). Conclusions: Component selection from the title and abstract can improve the performance of screening models and substantially reduce the manual screening workload in systematic review updates. Future research should focus on validating this approach across different systematic review domains. |
| 書誌情報 |
en : JMIR Medical Informatics
巻 13,
ページ数 15,
発行日 2025-03-27
|
| 出版者 |
|
|
出版者 |
JMIR Publications |
| ISSN |
|
|
収録物識別子タイプ |
EISSN |
|
収録物識別子 |
2291-9694 |
| 出版者版DOI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
DOI |
|
|
関連識別子 |
https://doi.org/10.2196/65371 |
| 出版者版URI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
URI |
|
|
関連識別子 |
https://medinform.jmir.org/2025/1/e65371/ |
| 権利 |
|
|
権利情報Resource |
https://creativecommons.org/licenses/by/4.0/ |
|
権利情報 |
©Tatsuki Hasegawa, Hayato Kizaki, Keisho Ikegami, Shungo Imai, Yuki Yanagisawa, Shuntaro Yada, Eiji Aramaki, Satoko Hori. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 27.03.2025. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included. |
| 著者版フラグ |
|
|
出版タイプ |
NA |
