| アイテムタイプ |
会議発表論文 / Conference Paper(1) |
| 公開日 |
2026-02-19 |
| タイトル |
|
|
タイトル |
Investigating Neurons and Heads in Transformer-based LLMs for Typographical Errors |
| 言語 |
|
|
言語 |
eng |
| 資源タイプ |
|
|
資源タイプ |
conference paper |
| アクセス権 |
|
|
アクセス権 |
open access |
| 著者 |
Tsuji, Kohei
Hiraoka, Tatsuya
Cheng, Yuchang
荒牧, 英治
Iwakura, Tomoya
|
| 抄録 |
|
|
内容記述タイプ |
Abstract |
|
内容記述 |
This paper investigates how LLMs encode inputs with typos. We hypothesize that specific neurons and attention heads recognize typos and fix them internally using local and global contexts. We introduce a method to identify typo neurons and typo heads that work actively when inputs contain typos. Our experimental results suggest the following: 1) LLMs can fix typos with local contexts when the typo neurons in either the early or late layers are activated, even if those in the other are not. 2) Typo neurons in the middle layers are the core of typo-fixing with global contexts. 3) Typo heads fix typos by widely considering the context not focusing on specific tokens. 4) Typo neurons and typo heads work not only for typo-fixing but also for understanding general contexts. |
| 書誌情報 |
en : Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
p. 6145-6163,
ページ数 19,
発行日 2025
|
| 会議情報 |
|
|
|
会議名 |
The 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP2025) |
|
|
開始年 |
2025 |
|
|
開始月 |
11 |
|
|
開始日 |
04 |
|
|
終了年 |
2025 |
|
|
終了月 |
11 |
|
|
終了日 |
09 |
|
|
開催期間 |
2025-11-04 - 2025-11-09 |
|
|
開催地 |
Suzhou, China |
|
開催国 |
CHN |
| 出版者 |
|
|
出版者 |
Association for Computational Linguistics |
| 出版者版DOI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
DOI |
|
|
関連識別子 |
https://doi.org/10.18653/v1/2025.emnlp-main.313 |
| 出版者版URI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
URI |
|
|
関連識別子 |
https://aclanthology.org/2025.emnlp-main.313/ |
| 権利 |
|
|
権利情報Resource |
https://creativecommons.org/licenses/by/4.0/ |
|
権利情報 |
©2025 Association for Computational Linguistics. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License. |
| 著者版フラグ |
|
|
出版タイプ |
NA |
| 助成情報 |
|
|
|
助成機関名 |
National Center for Global Health and Medicine (NCGM) |
|
|
研究課題番号 |
JPJ012425 |
|
|
研究課題名 |
Cross-ministerial Strategic Innovation Promotion Program (SIP) |
