Interdomain Attention: Beyond Token-Level Key-Value Memory
Naoki Kiyohara*, Harrison Bo Hua Zhu*, Zhuo Sun, Wenlong Chen, Samir Bhatt, Yingzhen Li
* equal contribution
PhD Candidate, Imperial College London
Researcher, Canon Inc.
London, UK
I am a PhD candidate at the Department of Computing, Imperial College London, on a study-abroad programme from Canon Inc., where I am a researcher, jointly supervised by Prof. Yingzhen Li and Prof. Edward Johns. I previously earned a BSc in Physics from Tokyo University of Science under Prof. Hideaki Sakata and an MSc in Materials Science at The University of Tokyo under Prof. Satoru Nakatsuji. At Canon, I conduct R&D in machine learning and fluid–structure analysis for medical systems and semiconductor manufacturing equipment.
My research focuses on building theoretically grounded deep sequence models, spanning attention mechanisms, state-space models (HiPPO, S4, Mamba), and their unification through kernel methods. I work on continuous-time modelling with ordinary/stochastic differential equations, and develop Bayesian methods for uncertainty quantification and inference. I am particularly interested in applying these principled approaches to problems in science and engineering.
For a complete list, see Google Scholar.
Naoki Kiyohara*, Harrison Bo Hua Zhu*, Zhuo Sun, Wenlong Chen, Samir Bhatt, Yingzhen Li
* equal contribution
Junichi Ooida, Naoki Kiyohara, Hironaga Noguchi, Yuichiro Oguchi, Kohei Nagane, Takuya Sakaguchi, Gakuto Aoyama, Fumimasa Shige, James V. Chapman, Masahiko Asami, Klaus F. Kofoed, Michael H. C. Pham, Koshiro Suzuki
J. Seki, Y. Oguchi, N. Kiyohara, K. Suzuki, K. Nagane, S. Narioka, T. Nakayama, Y. Shiode, S. Aihara, T. Asano
Feel free to reach out to me at:
n.kiyohara23 [at] imperial [dot] ac [dot] uk