Chi-Chang Lee

I am a CS Ph.D. student at the University of Maryland, College Park, where I am a member of the Perception and Robotics Group, advised by Prof. Yiannis Aloimonos. My current research focuses on large-scale tactile simulation for dexterous manipulation, specifically on calibrating imperfect contact approximations in physics engines with real-world data, aiming for tactile simulation that is both accurate and massively parallel. Previously, I earned both my Bachelor's and Master's degrees from National Taiwan University.

From 2023 to 2025, I was honored to collaborate with the Improbable AI Lab at MIT CSAIL, led by Prof. Pulkit Agrawal. My work there focused on Reinforcement Learning for robots under a finite interaction budget, aiming to disentangle heuristic signals from the true task objective while still optimizing real but sparse success outcomes. I contributed to the real-world deployment of an advanced RL algorithm that achieves consistent gains on 31 robotic tasks (ICRA'24 & NeurIPS'24).

For hobbies outside the lab, I am a big fan of EDM, so I used AI to create an automated EDM mix. I am also fascinated by exploring museums in DC; here are my random reflections in Mandarin.

Experience

	Research Intern May 2026 – Aug. 2026 Origami Robotics (YC W26), San Francisco, CA Investigating embodiment gap across morphological, visual, and action-space dimensions to bridge human demonstrations with dexterous robot execution.
	Research Collaborator Jul. 2023 – Sep. 2025 Improbable AI Lab at Massachusetts Institute of Technology Advisor: Prof. Pulkit Agrawal
	Research Assistant Mar. 2019 – Mar. 2024 Bio-ASP Lab at Academia Sinica CITI, Taiwan Advisor: Prof. Yu Tsao
	Visiting Researcher Nov. 2022 – Feb. 2023 Yamagishi Laboratory at National Institute of Informatics, Japan Advisor: Prof. Junichi Yamagishi

Publications in Robot Learning and Embodied AI
(∗ indicates equal contribution)

Optimal robot learning under finite interaction budgets - In practice, RL for robotics often struggles with complex environments because it relies on brittle, hand-engineered dense rewards or assumes unrealistic sampling budgets (e.g., relying on proximal heuristics while the true target is a sparse success outcome). I developed a novel RL framework that optimizes for target success outcomes without requiring (1) infinite sampling assumptions or (2) restrictive reward shaping.

Going Beyond Heuristics by Imposing Policy Improvement as a Constraint
Chi-Chang Lee*, Zhang-Wei Hong*, Pulkit Agrawal
Conference on Neural Information Processing Systems (NeurIPS), 2024
Paper | OpenReview | Video | Code

Maximizing Velocity by Minimizing Energy
Srinath Mahankali*, Chi-Chang Lee*, Gabriel B. Margolis, Zhang-Wei Hong, Pulkit Agrawal
International Conference on Robotics and Automation (ICRA), 2024
Paper | Website | Code

Publications in Machine Learning for Audio Modalities

Machine Learning for Audio Modalities - At National Taiwan University, I worked closely with Prof. Yu Tsao and Prof. Hsin-Min Wang. My research focused on robust and adaptive machine learning for audio and speech modalities, particularly on “normalizing” speech from diverse acoustic conditions for downstream tasks such as Speech-to-Text and Speaker Identification. This work followed two main directions: (1) leveraging multiple label modalities for auxiliary learning and data reuse by combining synthetic noisy-clean speech pairs with real-world noisy-speech and text-label pairs to improve robustness (ICLR'23 & ASRU'23), and (2) fast adaptation and continual learning for acoustic normalization (Interspeech'20 & Interspeech'22). Because speech and acoustic environments are too diverse for models to generalize reliably on their own, I studied how to automatically acquire informative data and rapidly recalibrate these models to mitigate out-of-domain degradation.

D4AM: A General Denoising Framework for Downstream Acoustic Models
Chi-Chang Lee, Yu Tsao, Hsin-Min Wang, Chu-Song Chen
International Conference on Learning Representations (ICLR), 2023
Paper | Code

LC4SV: A Denoising Framework Learning to Compensate for Unseen Speaker Verification Models
Chi-Chang Lee, Hong-Wei Chen, Chu-Song Chen, Hsin-Min Wang, Tsung-Te Liu, Yu Tsao
IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023
Paper | Code

NASTAR: Noise Adaptive Speech Enhancement with Target-Conditional Resampling
Chi-Chang Lee, Cheng-Hung Hu, Yu-Chen Lin, Chu-Song Chen, Hsin-Min Wang, Yu Tsao
INTERSPEECH, 2022
Paper | Website | Code

SERIL: Noise adaptive speech enhancement using regularization-based incremental learning
Chi-Chang Lee, Yu-Chen Lin, Hsuan-Tien Lin, Hsin-Min Wang, Yu Tsao
INTERSPEECH, 2020
Paper | Code

Updated VLM/LLM side projects in 2025

In-Context Bayesian Optimization incorporating improvement constraints

This is a 14-day side project in 2025 that aims to expand in-context learning for Bayesian optimization by introducing a reference policy and a policy-improvement view. I target two goals: (1) acquiring improvement / convergence support via constrained updates relative to the reference policy, and (2) replacing domain-specific task descriptions (e.g., dataset/model info in hyperparameter tuning) with task-agnostic statistical signals produced by the reference policy (e.g., mean/uncertainty summaries), improving scalability to unseen BO tasks.
Paper | Code

Text-to-Appearance Inverse Rendering with Vision–Language Feedback

We study unpaired text-to-appearance inverse rendering, where a differentiable renderer is optimized using only text prompts, without any reference images. Since direct CLIP-based optimization from text is often vague and under-constrained, we introduce vision–language feedback as an optimization confiner. The VLM (1) augments ambiguous textual descriptions into concrete, parameter-aware instructions and (2) determines which scene parameters should be updated, while simultaneously tightening feasible ranges for all parameters. This guidance stabilizes optimization and improves prompt adherence and visual clarity compared to naive CLIP-only methods.
Paper | Code

Teaching

Teaching Assistant, Machine Learning, National Taiwan University, Taiwan 2021 Fall

Teaching Assistant, Time Frequency Analysis and Wavelet Transforms, National Taiwan University, Taiwan 2018 Fall

Honor

Second Place, IC/CAD Contest 2019

template from jonbarron