Workshop

Workshop on Visual Computing for Social and Cognitive Robots

Chair: Daniel Thalmann

 

---

Workshop for EU IRSES project “AniNex”

Chair: Ruofeng Tong

 

---

Prof. Dinesh Manocha 

Title:

Motion Planning: From Robots to Humans

Abstract:

Algorithmic motion planning has been actively studied in robotics and related areas for more than three decades. There is a rich collection of techniques that have been successfully used for CAD/CAM, bioinformatics, computer gaming and other applications. More recently, there is a lot of attention paid to extending these techniques to physical robots that take into account sensing errors, uncertainty and dynamics constraints. We will give an overview of our work based on optimization-based planning and highlight its performance on industrial and mobile manipulators. Moreover, we address the problem of human-like motion planning. This problem arises in the context of human trajectory and behavior computation in the physical work that takes into environmental, sociological and psychological factors into account. We give an overview of our work on velocity-based approaches for multi-agent human motion planning and demonstrate its application to urban simulation, loading and unloading of aircrafts, stadium evacuation and planning for large religious and political events.

Speaker’s Biography:

Prof. Dinesh Manocha is currently the Phi Delta Theta/Mason Distinguished Professor of Computer Science at the University of North Carolina at Chapel Hill. He has co-authored than 400 papers in the leading conferences and journals on computer graphics, robotics, and scientific computing. He has also served program chair for many conferences and editorial boards of many leading journals. Some of the software systems related to collision detection, GPU-based algorithms and geometric computing developed by his group have been downloaded by more than 150,000 users and are widely used in the industry. Manocha has received awards including Alfred P. Sloan Fellowship, NSF Career Award, Office of Naval Research Young Investigator Award, and 12 best paper awards at the leading conferences. He is a Fellow of ACM, AAAS, and IEEE, and received Distinguished Alumni Award from Indian Institute of Technology, Delhi.

---

Prof. Magnenat-Thalmann Nadia

Title:

Modelling Awareness and Social Behavior for Virtual Humans and Social Robots

Abstract:

This presentation will discuss the concept of embodied Machine represented by Virtual Humans (VH) or Social Robots that interact with real people. We will describe the various most recent methods of capturing gestures and understanding them. It will also address the kind of interactions between real users and virtual humans based on gaze. Multimodal fusion and decision making models will be presented. The presentation will discuss situations when several real users, robots and VH are interacting and who is taking the lead and turn in the conversation. Case studies will be shown with the humanoid robot Nadine, one of the most realistic female robot in the world, which is able to recognize gestures and emotions and speak in a natural way.

Speaker’s Biography:

Prof. Nadia Magnenat Thalmann has pioneered research into virtual humans over the last 30 years. Her global research area is the Virtual Human and Social Robot. Besides directing her research group MIRALab in Switzerland, she is presently Visiting Professor and Director of the Institute for Media Innovation (IMI) at Nanyang Technological University, Singapore. She has acquired a great experience of collaborative research through her strong participation to more than 50 European Research Projects. During her Career, she has received more than 30 Awards such as “Woman of the Year”, an early recognition in Montreal in 1987. Among the recent ones, she was awarded a Doctor Honoris Causa in Natural Sciences from the Leibniz University of Hanover in Germany in 2009, the Distinguished Career Award from the European Association for Computer Graphics in Norrkoping, Sweden in 2010, an Honorary Doctorate of the University in Ottawa in 2010 and a Career Achievement Award from the Canadian Human Computer Communications Society in Toronto in 2012. Recently, she received the prestigious Humboldt Research Award in Germany given to “academics whose fundamental discoveries, new theories, or insights have had a significant impact on their own discipline and who are expected to continue producing cutting-edge achievements in the future”.

---

Prof. Hongbin Zha

Title:

Visual Tracking and Behavior Analysis for Human-Machine Interaction

Abstract:

For robots working in a daily-life environment, interaction between human and robot or among robots themselves is essential to successful accomplishment of their tasks. For the purpose, the robots have to possess ability to track the objects which they want to interact with and further to recognize their behaviors by using computer vision techniques. In the talk, the speaker will introduce some researches carried out in the 3DVCR (3D Visual Computing and Robotics) group in Peking University for the object tracking and behavior analysis. Main topics includes: (1) Salient object detection for localizing objects that attract machine’s or human’s attention. It is a learning-based technique to exploit global features from multiple saliency indicators to directly predict the existence and the position of a salient object. (2) Detection of visual focus of attention (VFoA) of human. It is achieved by both estimating the human’s head pose and localizing salient areas in the environment which may attract the human’s attention. (3) Analysis of human drivers’ social behaviors for an intelligent driving system. The behaviors are learnt through on-road vehicle trajectories collection in real-world traffic, consisting of on-road vehicle trajectory collection, and lane change and car following action analysis where the social behaviors between an ego and surround vehicles are taken into account.

Speaker’s Biography:

Prof. Hongbin Zha received PhD degree in electrical engineering from Kyushu University, Japan, in 1990. After working as a research associate at Kyushu Institute of Technology, he joined Kyushu University in 1991 as an associate professor. He was also a visiting professor in the Centre for Vision, Speech, and Signal Processing, Surrey University, Unite Kingdom, in 1999. Since 2000, he has been a professor at the Key Lab on Machine Perception (MOE), Peking University, China. His research interests include computer vision, 3D data processing and modeling, and robotics. He has published more than 300 technical publications in journals, books, and international conference proceedings. He received the Franklin V. Taylor Award from the IEEE Systems, Man, and Cybernetics Society in 1999, and Best Paper Awards in Euromed 2010.

---

Dr. Jianfei Cai

Title:

RGB-D image segmentation and understanding

Abstract:

Commodity RGB-D sensors such as Microsoft Kinect have received significant attention in the recent years due to their low cost and the ability to capture synchronized color images and depth maps in real time. Although the available depth information has been proven to be extremely useful for many visual computing problems, there are still challenges remaining on finding the best way to harness the low-resolution, noisy and unstable depth data and to complement the existing methods using RGB data alone. In this talk, I will show you a series of works from my group on using shadow for RGB-D image segmentation, using unsupervised feature learning for scene labeling and using deep learning technology for RGB-D object recognition. We believe the presented works would be beneficial for robotics applications, particularly on using RGB-D cameras for scene understanding.

Speaker’s Biography:

Dr. Cai received his PhD degree from the University of Missouri-Columbia. He is currently an Associate Professor and has served as the Head of Visual & Interactive Computing Division and the Head of Computer Communication Division at the School of Computer Engineering, Nanyang Technological University, Singapore. His major research interests include visual computing and multimedia networking. He has published more than 150 technical papers in international conferences and journals. He is a co-recipient of paper awards in ACCV, IEEE ICIP and MMSP. He has been actively participating in program committees of various conferences. He has served as the leading Technical Program Chair for IEEE International Conference on Multimedia & Expo (ICME) 2012 and the leading General Chair for Pacific-rim Conference on Multimedia (PCM) 2012. He is currently an Associate Editor for IEEE Trans on Image Processing (T-IP) and has served as an Associate Editor for IEEE Trans on Circuits and Systems for Video Technology (T-CSVT), and Guest Editor for IEEE Trans on Multimedia (TMM), etc. He is also a senior member of IEEE.

---

Dr. Charlie C. L. Wang

Title:

Foot-Rooted Body Motion Capture by IMU Sensors

Abstract:

This talk presents a motion capture system using IMU sensors, which can capture full-body motions with height change in real-time. Unlike prior IMU-based approaches using hip-joint as the root of forward kinematic model, a foot-rooted kinematic model is developed in this work. An enhanced state-machine has also been investigated to allow dynamically switching the root of kinematic trees between the left and the right foot. Benefitted from this, full-body motions can be well captured in our system as long as there is at least one static foot in the motion. The ‘floating’ artifact of hip-joint rooted methods has been eliminated in our approach. Comparing to those vision-based systems, more flexibility on capturing outdoor motions can be provided by our system. The performance of our system has been verified by experimental tests taken on our inexpensive hardware setup with multiple IMU sensors. Application of this work in the imitation of humanoid will also be presented.

Speaker’s Biography:

Dr. Charlie C. L. Wang is a Fellow of American Society of Mechanical Engineers (ASME) with expertise in geometric computing and solid modeling. His work is well recognized in the community of computer-aided design and manufacturing. Dr. Wang joined Department of Mechanical and Automation Engineering at Chinese University of Hong Kong as a faculty member in 2003. He has received a few awards from professional societies including the ASME CIE Young Engineer Award (2009), the CUHK Young Researcher Award (2009), the Best Paper Awards of ASME CIE Conferences (in 2008 and 2001), the Prakash Krishnaswami CAPPD Best Paper Award of ASME CIE Conference in 2011, and NAMRI/SME Outstanding Paper Award in 2013. Dr. Wang serves on the editorial board of a few journals including Computer-Aided Design, IEEE Transactions on Automation Science and Engineering, ASME Journal of Computing and Information Science in Engineering, etc. Besides research, he is also a consistent and superb teacher and received the CUHK Vice-Chancellor’s Exemplary Teaching Award (2008). His philosophy of teaching is that a good teach not only imparts knowledge but also inspires students to seek truth beyond tradition, definition and image.

---

Dr. Jia Pan

Title:

Robotic Manipulation Leveraging Plane Contacts

Abstract:

General robotic manipulation is challenging. To successfully manipulate an object, the robot must first accurately estimate the object’s pose from the sensor data, and then find a good grasp of the object via grasp planning, and finally use motion planning to compute a trajectory to approach the object. In some cases, the manipulation cannot be achieved in a single grasp due to cluttered environments or kinematics singularities, and regrasping is necessary for these cases. All these components of manipulation, including pose estimation, grasp planning, motion planning, and regrasp planning, can be difficult to solve. In this talk, we will discuss about how to use plane contacts to tackle some of challenges in robotic manipulation. The first part is about pose estimation. We present a new approach which effectively estimates the target object’s pose given its partial observation, by leveraging two facts: 1) in most manipulation scenarios, the object is posed in a stable pose sitting on flat support surfaces, and 2) many household objects can only keep stable on a planar surface under a small set of poses. In this way, we can reduce the 6-dimensional search of pose estimation into a set of 3-dimensional search problem, and thus greatly improve the accuracy and efficiency of the estimated pose. The second part is about regrasping. Similar to previous work, we use one intermediate location for the temporary placement between two pickups. However, unlike previous work which assumed this intermediate location to be horizontal, we show that a tilted surface can improve the object’s stability and manipulability while being regrasped, and thus increase the flexibility of regrasping.

Speaker’s Biography:

Dr. Jia Pan is a Research Assistant Professor in the Department of Computer Science, the University of Hong Kong. His research area is intelligent grasping and manipulation, including motion planning, deformable object manipulation, learning from demonstration, reinforcement learning and planning with uncertainty.

---

Prof. Lizhuang Ma

Title:

Intelligent Generation and Processing System for Big Data of Visual Media

Abstract:

With the rapid development of information technology and its applications such as mobile Internet, cloud computing and social network, visual media data which is mainly consist of images, videos and 3D models, is explosively growing. Visual media has accounted for more than 60% of Internet resources. The big data of visual media has four characteristics: large-scale, variety aspects, diverse sources and updated dynamically. The big data of visual media is being formed in Internet industry, also becoming an important information asset. Visual media intelligent generation and processing technology, is to generate, process and analyze visual media data for obtaining valuable information. The key technical problems include massive data representation, dynamic storage management, constructions of complex models and rendering of large-scale scenes. Personalized service should be provided to customers by analyzing big data with intelligent techniques. This talk presents our recent progresses in intelligent generation and processing system for big data of visual media, finished by Shanghai Jiao Tong University & Tencent Technology Company Limited in Shanghai.

The talk will cover massive internet images re-compression, image structure extraction and enhancement, face detection, face alignment and face verification for Internet bank.

Speaker’s Biography:

Prof. Lizhuang Ma received his B. Sc. and Ph.D. degree at Zhejiang University, China in 1985 and 1991 respectively. Dr. Ma was promoted as an Associative Professor and Professor in 1993 and 1995 respectively. Dr. Ma stayed at Zhejiang University from 1991 to 2002. He was a visiting Professor at Frounhofer IGD, Darmstadt, Germany from July to Dec. 1998, and visiting Professor at Center for Advanced Media Technology, Nanyang Technological University, Singapore from Sept. 1999 to Oct. 2000. He is now a Professor, PhD tutor, and the head of Digital Media Technology and Data Reconstruction Lab. at the Department of Computer Science and Engineering, Shanghai Jiao Tong University, China since 2002. He is also the Chairman of the Center of Information Science and Technology for Traditional Chinese Medicine at Shanghai Traditional Chinese Medicine University. Dr. Ma is the recipient of China National Excellent Young Scientist Foundation, first class member of China National Hundred-Thousand-Ten -thousand Talent Plan. Dr. Ma has published more than 100 academic research papers. His research interests include CAD, computer graphics, computer animation and digital media technology. Please refer http://www.cs.sjtu.edu.cn/en/PeopleDetail.aspx?id=154