XIE LAB — Human-Centered AI Laboratory

What We Work On

研究内容

Generative AI

Human-Centered Content Generation

Sketch-guided, Controllable image generation, 3D model generation, and video generation using diffusion models with sketch-based control.

Computational Design

Creative AI for Professional Design

AI-driven architectural design, anime design, fashion design, and content design from multimodal inputs such as sketch and texts.

Physical AI

Physical Intelligence & Robotics Learning

Physics-guided content generation such as fluid simulation, smoke illustration, and robotic learning with diffusion policy and VLA models.

UI & HCI

Human-AI Interaction

Supernumerary robotic limbs, soft actuators, compliant mechanisms, and interactive interfaces for creativity support.

Funded by: JST CREST · JST BOOST · JSPS KAKENHI (B) · JSPS KAKENHI Exploratory · OLM Digital Inc. · Live2D Inc. ·　NEDO · Kayamori Foundation · Nakayama Foundation · NSF IRES

Team

Members

構成員

Research Staff / 教員 (5)

Haoran Xie

謝浩然

PI, Associate Professor
JAIST & Waseda

Hongbo Zhang

張宏博

Assistant Professor
特任助教, JAIST

Xusheng Du

杜旭升

Researcher
次席研究員, Waseda

Hengyuan Chang

常恒遠

Researcher
研究員, JAIST

Xianwen Zheng

Researcher
研究員, JAIST

PhD Students / 博士学生 (14 — IS:10, KS:4)

Yongming Zhang

張永銘

IS, Sketch + Image

Tianyu Zhang

張天宇

IS, Sketch + Image

Hao Jin

金昊

IS, Video Generation

Mikiya Kusunoki

楠木幹也

KS, Origami + AH
JAIST-SPRING

Fangyuan Tu

涂方圓

IS, Video Generation

Yuan Ma

馬源

IS, Medical Image
JAIST-SPRING

Xiaoxuan Xie

謝晓玄

IS, Archeology + GenAI
CSC Scholarship

Zhengyang Wang

王征洋

IS, Architecture + GenAI
CSC Scholarship

Bofei Huang

黄柏飛

KS, Human-centered AI

Chuang Chen

陳闘

IS, Sketch + Image

Minhao Li

李旻昊

IS, Robot + GenAI

Kohei Shinada

品田康平

KS, Robot + GenAI
JAIST-SPRING

Ryuichi Miyauchi

宮内竜一

IS, Image Generation
JAIST-SPRING

Hisato Shirata

白田陽彩人

KS, Fabrication + GenAI
JAIST-SPRING

Master's Students / 修士学生 (7 — IS:5, KS:2)

Daishi Iwai

岩井大志

IS, Tokyo Satellite

Takuya Namai

生井琢也

IS, Tokyo Satellite

Ippei Amano

天野一平

IS

Mufeng Guo

郭沐風

IS

Haoran Zhuo

卓浩然

KS

Xu Zhao

趙旭

KS

Fusang Wan

万扶桑

IS, Tokyo Satellite

Publications

Selected Papers

主要発表論文

2026

AI-Driven Urban Evolution Forecasting: A Unified Memory-Aware Multi-Conditional Generation Framework

X. Du, C. Li, Q. Li, Y. Lu, Y. Xu, Y. Zhang, Z. Xu, H. Xie

Sustainable Cities and Society, 2026 (JCR Q1, IF 12.0)

Controllable Generation of Building Representations with Multi-Stage Retrieval-Augmented Diffusion Models

Z. Wang, Y. Ren, H. Jin, J. Feng, X. Du, Y. Zhang, H. Xie

Frontiers of Architectural Research, 2026 (JCR Q1)

Sketch-Guided Stylized Landscape Cinemagraph Synthesis

H. Jin, H. Chang, X. Xie, Z. Wang, X. Du, S. Hu, H. Xie

Computer & Graphics, 2026

2025

LineArt: Knowledge-guided Training-free Appearance Transfer for Design Drawing with Diffusion Model

X. Wang, H. Li, H. Fang, Y. Peng, H. Xie, X. Yang, C. Li

CVPR 2025

CompAct: Designing Interconnected Compliant Mechanisms with Active Material Integration

H. Yang, I. Shen, N. Martelaro, B. Zhu, H. Xie, T. Igarashi, L. Yao

ACM CHI 2025

Sketch-Guided Scene Image Generation with Diffusion Model

T. Zhang, X. Xie, X. Du, H. Xie

Computer & Graphics, 2025

Interactive Multilayer Gaussian Garments for Low-Cost Try-On

R. Zesch, I. Shen, H. Xie, B. Zhu, S. Sueda, T. Igarashi

Graphics Interface 2025 Best Graphics Paper

PromptNavi: Text-to-Image Generation through Interactive Prompt Visual Exploration

B. Huang, H. Xie

Computer & Graphics, 2025

2024

DiffOBI: Diffusion-based Image Generation of Oracle Bone Inscription Style Characters

X. Xie, X. Du, M. Li, X. Yang, H. Xie

ACM SIGGRAPH Asia 2024

TexControl: Sketch-Based Two-Stage Fashion Image Generation using Diffusion Model

Y. Zhang, T. Zhang, H. Xie

NICOGRAPH International 2024 Best Paper

2023

AniFaceDrawing: Anime Portrait Exploration during Your Sketching

Z. Huang, H. Xie, T. Fukusato, K. Miyata

ACM SIGGRAPH 2023

Earlier

Precomputed Panel Solver for Aerodynamics Simulation

H. Xie, T. Igarashi, K. Miyata

ACM SIGGRAPH 2018 (TOG) (JCR Q1, IF 9.5)

→ Full list on Publications and Google Scholar.

Graduates

Alumni

卒業生

PhD Alumni / 卒業生(博士) (2)

2026

XD
Xusheng Du (杜旭升) — PhD(IS). Generative AI-Driven 3D Modeling and Applications using Intuitive Control: From Architectural Design to Urban Planning. 優秀修了者 JAIST-SPRING → Waseda Researcher
HC
Hengyuan Chang (常恒遠) — PhD(IS). Smoke Effect Generation and Design with Sketch Control using Diffusion Models. → JAIST Researcher

Master's Alumni / 卒業生(修士) (16)

2026

RM
Ryuichi Miyauchi (宮内竜一) — MS(IS). 生成モデルベースのスライダー型画像属性制御手法の提案. 情報処理学会北陸支部優秀学生賞 → PhD (JAIST)
KN
Kai Niu (牛凱) — MS(IS). Sketch-based 3D Anime-Style Character Generation with Geometry Guidance. 優秀修了者
ML
Minhao Li (李旻昊) — MS(IS). Interactive Robotic Sketch Painting based on Large Language Model and Diffusion Models. → PhD (JAIST)
YL
Yude Lu (盧玉德) — MS(KS). VR Space Drawing with Adjustable and Directional Brush.
HS
Hisato Shirata (白田陽彩人) — MS(KS). スケッチベース可動オブジェクトの設計支援インターフェースの研究. → PhD (JAIST)
KS
Kohei Shinada (品田康平) — MS(KS). ロボット設計のためのスケッチベースインターフェース. → PhD (JAIST)

2025

CC
Chuang Chen (陳闘) — MS(IS). Real-Time Drawing Assistance with Sketch-Based Control via Diffusion Models. → PhD (JAIST)
SL
Shiwen Liu (劉詩雯) — MS(KS). Augmenting Beginner's Drawing Skills with Decontextualized Methods. → JAIST Research Staff
XX
Xiaoxuan Xie (謝晓玄) — MS(IS). Diffusion-based Image Generation of Oracle Bone Inscription Style Characters. 優秀修了者 → PhD (JAIST)
ZW
Zhengyang Wang (王征洋) — MS(IS). Building Image Generation Using Retrieval Augmented Diffusion Model. 優秀修了者 → PhD (JAIST)
BH
Bofei Huang (黄柏飛) — MS(KS). Text-to-Image Generation through Interactive Prompt Visual Exploration. → PhD (JAIST)

2024

SY
Shihao Yang (楊世浩) — MS(IS). Sketch-Guided Consistent Image Generation using Multi-Aggregation Attention Mechanism. → Sangfor Technology
MK
Mikiya Kusunoki (楠木幹也) — MS(KS). スケーラブルかつ伸縮可能な折り紙構造を用いた余剰肢ロボットの開発. 優秀修了者 → PhD (JAIST)

2023

TZ
Tianyu Zhang (張天宇) — MS(IS). Sketch-Based Scene Image Generation with Two-Stage Latent Diffusion Model. → PhD (JAIST)
XD
Xusheng Du (杜旭升) — MS(IS). Real-Time 3D Shadow Guidance for Sketch-Based 3D Shape Generation. → PhD (JAIST)
JW
Jiahao Weng (翁嘉豪) — MS(IS). Sketching Interface for Document Layout and Content Design. → China State Shipbuilding Corporation

Past Members & Visitors

RG
Ruihan Gui — 2025.10–2026.03, Tianjin Univ., Visitor
TN
Takeru Nakaoka (中岡建) — 2024.04–2026.03, KS, Master Student
SL
Shiwen Liu — 2025.08–2025.12, JAIST, Research Assistant
JC
Jaemin Chung — 2025.08, Korea Aerospace Univ., Visitor
JL
Jereth Liu — 2025.07–08, Univ. of Pennsylvania, Visitor
NR
Nuttapong Rochanavibhata — 2025.05–08, Mahidol Univ., Visitor
AK
Athiwat Kongkaeo — 2025.06–08, Chulalongkorn Univ., Visitor
WB
Warissara Booranamaitree — 2025.06–08, Chulalongkorn Univ., Visitor
ZP
Zhenglin Pan — 2024.10–2025.05, Research Student → Research Assistant
SY
Shogo Yoshida (吉田匠吾) — 2024.05–2025.03, JAIST, Research Assistant
XG
Xinyue Gui — 2024.02–03, Univ. of Tokyo, Visitor
ZL
Ziyu Liao — 2023.08–11, NUAA, Visitor
YH
Yi He (何毅) — 2022.10–2023.03, JAIST, Research Staff
RM
Ryoma Miyauchi (宮内涼将) — 2021.04–06, JAIST, Research Staff
NF
Naihui Fang — 2017.10–2018.04, UTokyo, Research Staff
SL
Sicheng Li (李思成) — 2020–2025, Miyata Lab Co-supervision, PhD → Hithink RoyalFlush
ZH
Zhengyu Huang (黄正宇) — 2020–2023, Miyata Lab Co-supervision, PhD → HKUST(GZ)
YP
Yichen Peng (彭以琛) — 2019–2024, Miyata Lab Co-supervision, PhD → Institute of Science Tokyo