| アイテムタイプ |
学術雑誌論文 / Journal Article(1) |
| 公開日 |
2025-10-23 |
| タイトル |
|
|
タイトル |
Toward Cross-Hospital Deployment of Natural Language Processing Systems: Model Development and Validation of Fine-Tuned Large Language Models for Disease Name Recognition in Japanese |
| 言語 |
|
|
言語 |
eng |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
clinical NLP |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
Japanese language |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
named entity recognition |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
large language models |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
out-of-domain robustness |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
clinical corpus |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
clinical natural language processing |
| 資源タイプ |
|
|
資源タイプ |
journal article |
| アクセス権 |
|
|
アクセス権 |
open access |
| 著者 |
Shimizu, Seiji
Nishiyama, Tomohiro
Nagai, Hiroyuki
若宮, 翔子
荒牧, 英治
|
| 抄録 |
|
|
内容記述タイプ |
Abstract |
|
内容記述 |
Background: Disease name recognition is a fundamental task in clinical natural language processing, enabling the extraction of critical patient information from electronic health records. While recent advances in large language models (LLMs) have shown promise, most evaluations have focused on English, and little is known about their robustness in low-resource languages such as Japanese. In particular, whether these models can perform reliably on previously unseen in-hospital data, which differs from training data in writing styles and clinical contexts, has not been thoroughly investigated. Objective: This study evaluated the robustness of fine-tuned LLMs for disease name recognition in Japanese clinical notes, with a particular focus on their performance on in-hospital data that was not included during training. Methods: We used two corpora for this study: (1) a publicly available set of Japanese case reports denoted as CR, and (2) a newly constructed corpus of progress notes, denoted as PN, written by ten physicians to capture stylistic variations of in-hospital clinical notes. To reflect real-world deployment scenarios, we first fine-tuned models on CR. Specifically, we compared a LLM and a baseline-masked language model (MLM). These models were then evaluated under two conditions: (1) on CR, representing the in-domain (ID) setting with the same document type, similar to training, and (2) on PN, representing the out-of-domain (OOD) setting with a different document type. Robustness was assessed by calculating the performance gap (ie, the performance drop from in-domain to out-of-domain settings). Results: The LLM demonstrated greater robustness, with a smaller performance gap in F1-scores (ID–OOD = −8.6) compared to the MLM baseline performance (ID–OOD = −13.9). This indicated more stable performance across ID and OOD settings, highlighting the effectiveness of fine-tuned LLMs for reliable use in diverse clinical settings. Conclusions: Fine-tuned LLMs demonstrate superior robustness for disease name recognition in Japanese clinical notes, with a smaller performance gap. These findings highlight the potential of LLMs as reliable tools for clinical natural language processing in low-resource language settings and support their deployment in real-world health care applications, where diversity in documentation is inevitable. |
| 書誌情報 |
en : JMIR Medical Informatics
巻 13,
ページ数 11,
発行日 2025-07-08
|
| 出版者 |
|
|
出版者 |
JMIR Publications |
| ISSN |
|
|
収録物識別子タイプ |
EISSN |
|
収録物識別子 |
2291-9694 |
| 出版者版DOI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
DOI |
|
|
関連識別子 |
https://doi.org/10.2196/76773 |
| 出版者版URI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
URI |
|
|
関連識別子 |
https://medinform.jmir.org/2025/1/e76773 |
| 権利 |
|
|
権利情報Resource |
https://creativecommons.org/licenses/by/4.0/ |
|
権利情報 |
© Seiji Shimizu, Tomohiro Nishiyama, Hiroyuki Nagai, Shoko Wakamiya, Eiji Aramaki. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 08.07.2025. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included. |
| 著者版フラグ |
|
|
出版タイプ |
NA |
| 助成情報 |
|
|
|
助成機関名 |
National Center for Global Health and Medicine (NCGM) |
|
|
研究課題番号 |
JPJ012425 |
|
|
研究課題名 |
Cross-ministerial Strategic Innovation Promotion Program (SIP) on “Integrated Health Care System” |
| 助成情報 |
|
|
|
助成機関名 |
Japan Science and Technology Agency (JST) |
|
|
研究課題番号 |
JPMJCR22N1 |
|
|
研究課題番号URI |
https://projectdb.jst.go.jp/grant/JST-PROJECT-22717060/ |
|
|
研究課題名 |
リアルワールドテキスト処理の深化によるデータ駆動型探薬 |
