| アイテムタイプ |
学術雑誌論文 / Journal Article(1) |
| 公開日 |
2025-12-26 |
| タイトル |
|
|
タイトル |
KeyMPs: One-Shot Vision-Language Guided Motion Generation by Sequencing DMPs for Occlusion-Rich Tasks |
| 言語 |
|
|
言語 |
eng |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
Dynamic movement primitives |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
motion generation |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
vision-language models |
| 資源タイプ |
|
|
資源タイプ |
journal article |
| アクセス権 |
|
|
アクセス権 |
open access |
| 著者 |
Anarossi, Edgar
Kwon, Yuhwan
Tahara, Hirotaka
Tanaka, Shohei
Shirai, Keisuke
Hamaya, Masashi
Beltran-Hernandez, Cristian C.
Hashimoto, Atsushi
松原, 崇充
|
| 抄録 |
|
|
内容記述タイプ |
Abstract |
|
内容記述 |
Dynamic Movement Primitives (DMPs) provide a flexible framework wherein smooth robotic motions are encoded into modular parameters. However, they face challenges in integrating multimodal inputs commonly used in robotics like vision and language into their framework. To fully maximize DMPs’ potential, enabling them to handle multimodal inputs is essential. In addition, we also aim to extend DMPs’ capability to handle object-focused tasks requiring one-shot complex motion generation, as observation occlusion could easily happen mid-execution in such tasks (e.g., knife occlusion in cake icing, hand occlusion in dough kneading, etc.). A promising approach is to leverage Vision-Language Models (VLMs), which process multimodal data and can grasp high-level concepts. However, they typically lack enough knowledge and capabilities to directly infer low-level motion details and instead only serve as a bridge between high-level instructions and low-level control. To address this limitation, we propose Keyword Labeled Primitive Selection and Keypoint Pairs Generation Guided Movement Primitives (KeyMPs), a framework that combines VLMs with sequencing of DMPs. KeyMPs use VLMs’ high-level reasoning capability to select a reference primitive through keyword labeled primitive selection and VLMs’ spatial awareness to generate spatial scaling parameters used for sequencing DMPs by generalizing the overall motion through keypoint pairs generation, which together enable one-shot vision-language guided motion generation that aligns with the intent expressed in the multimodal input. We validate our approach through experiments on two occlusion-rich tasks: object cutting, conducted in both simulated and real-world environments, and cake icing, performed in simulation. These evaluations demonstrate superior performance over other DMP-based methods that integrate VLM support. |
| 書誌情報 |
en : IEEE Access
巻 13,
p. 125420-125441,
ページ数 22,
発行日 2025-07-14
|
| 出版者 |
|
|
出版者 |
IEEE |
| ISSN |
|
|
収録物識別子タイプ |
EISSN |
|
収録物識別子 |
2169-3536 |
| 出版者版DOI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
DOI |
|
|
関連識別子 |
https://doi.org/10.1109/ACCESS.2025.3588975 |
| 出版者版URI |
|
|
関連タイプ |
isReplacedBy |
|
|
識別子タイプ |
URI |
|
|
関連識別子 |
https://ieeexplore.ieee.org/document/11079980 |
| 権利 |
|
|
権利情報Resource |
https://creativecommons.org/licenses/by/4.0/ |
|
権利情報 |
c 2025 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ |
| 著者版フラグ |
|
|
出版タイプ |
NA |
| 助成情報 |
|
|
|
助成機関名 |
Japan Society for the Promotion of Science (JSPS) |
|
|
研究課題番号 |
JP21H04910 |
|
|
研究課題番号URI |
https://kaken.nii.ac.jp/ja/grant/KAKENHI-PROJECT-21H04910/ |
|
|
研究課題名 |
自然言語指示に応じて多様な作業を行うロボット実現のための動作生成技術の開発 |
| 助成情報 |
|
|
|
助成機関名 |
Japan Society for the Promotion of Science (JSPS) |
|
|
研究課題番号 |
JP24K03018 |
|
|
研究課題番号URI |
https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-24K03018/ |
|
|
研究課題名 |
日常作業データによる模倣学習技術基盤の確立 |
