Bachelor Dissertation: To Be Determined
Bigeye Lab, Tsinghua University
Bachelor Dissertation (Advisor: Changshui Zhang)

  • Some topics related to deep reinforcement learning.

Visual Attention for Deep Reinforment Learning
Visual, Cognition, Action Virtual Reality Lab, The University of Texas at Austin
Visiting Student (Advisor: Dana Ballard)

  • To understand the visual features of deep reinforcement learning algorithms, designed a toy problem Catch.
  • Reimplemented all the details of Deep Q-Network, including: target network, replay buffer, CNN, preprocessing, gradient clipping (huber loss) and visualized the feature maps using Tensorboard.
  • Combined human visual attention with A2C and tested it on Atari games.

Reinforcement Learning: An introduction (2nd Edition, Chineses Version)
Speech Lab, Shanghai Jiao Tong University
Student Reviewer (Advisor: Kai Yu)

  • Proofread the chinese translation of chapter 5 (Monte Carlo methods), chapter 6 (TD methods), chapter 7 (n-step methods), chapter 8 (model-based RL), chapter 12 (eligibility traces) and chapter 13 (policy gradient methods).

Object Detection and Manipulation Demo on UR5 Robot Arm
Machine Intelligence Group, Tsinghua University
Research Assistant (Advisor: Chongjie Zhang)

  • Helped to implement a manipulation demo on our dual-arm collaborative robot.
  • Built a human computer interface based on rviz, using ROS to communicate with MoveIt! movement control node and RGB-D object detection node.
  • Particapated a weakly reinforcement learning reading group and gave several paper sharing talks, including DDPG, HER and SAC-X.

RoboCup 2017 Nagoya Humanoid League (Adult Size)
Robotics Lab, Tsinghua University
Team Member (Advisor: Mingguo Zhao)

  • Led a group of three for kicking trajectory generation task of an adult size (1.5m, 30kg+) humanoid, using cubic spline intersection and inverse dynamics.
  • Finished 90% of the C++ codes related to ball kicking and our robot was the only adult size humanoid which was able to perform high kick in Techical Chanllenge.
  • My work helped our team win the 2nd place in the Technical Challenge.