Scientist

Institute for Infocomm Research (I2R)
A*STAR
Level 13 Connexis (South Tower)
Singapore 138632
E-mail: xux AT i2r.a-star.edu.sg Google Scholar GitHub LinkedIn

I am currently a senior scientist with the Institute for Infocomm Research (I2R), A-STAR. Prior to that, I was a Research Fellow working with Assist Prof. Lee Gim Hee in the Computer Vision and Robotic Perception (CVRP) Lab of School of Computing, Natinoal University of Singapore (NUS). I also worked with Prof. Cheong Loong Fah in Visual Interactive Media Lab of NUS from Sep 2016 to Mar 2019. I received a PhD degree from Computer Vision Group in the School of Electronic Engineering and Computer Science at Queen Mary, University of London (QMUL), jointly supervised by Prof. Shaogang Gong and Dr. Tao Xiang. I was also working closely with Dr. Timothy Hospedales .

Before joining the Computer Vision Group in QMUL I received my B.Eng in Automation from Sichuan University and then had spent 2 years in 3D Vision Lab at Sichuan University supervised by Prof. Yusheng Liu and Prof. Kai Liu.

My research interests include label-efficient learning and robust AI with applications to 3D point cloud data.

Openings

TOP: We are always looking for highly self-motivated students. Please feel free to contact me if you wish to consider the following opportunities
- Short-term internship with paid allowance & eligible to non-Singaporean undergrad and master students.
- Visiting PhD/Master/Ungrad funded by CSC, ARAP and SIPGA scholarships are available.
- Pursuing PhD with Singapore universities and A*STAR funded by SINGA or ARAP scholarship.
- Find more details for eligible scholarships
- Find details for the above scholarships in a single PDF

Research

Projects

- May 2023: A*STAR MTC Programmatic Fund "Towards Realistic Deep Learning for 3D Vision" (SGD$ 1.1M allocated) will kickstart in Aug 2023

• We shall develop 3D deep learning techniques robust to imperfect visibility, adversarial attacks and incremental data to enable deployment in real-world applications.
  

- May 2021: A*STAR CDA Project "Exploiting Unlabeled Data, Cheaper Labels and Efficient Annotation for 3D Point Cloud Deep Learning" (EUDEA) ($SGD 238k allocated)

• My project on exploring label-efficient learning on 3D point cloud data started from Apr. 2021.
  
• We will be looking into improving the efficiency of 3D point cloud learning from several perspectives.
  

Publications & Services

- Apr 2024: Our on improving generalization of object detection on remote sensing images and open-set semi-supervised object detection on remote sensing images are accepted by IEEE IGARSS and International Journal of Applied Earth Observation and Geoinformation. Congratulations to Nanqing Liu !
- Feb 2024: We are happy to share our most recent work on improving the generalization of Segment Anything model - WeSAM, accepted by CVF/IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Congratulations to Haojie Zhang !
- Feb 2024: Our work on Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Training was accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Congratulations to Yongyi Su!
- Jan 2024: Our work on COFT-AD: COntrastive Fine-Tuning for Few-Shot Anomaly Detection was accepted by IEEE Transactions on Image Processing (TIP). Congratulations to Jingyi Liao!
- Jan 2024: I will be serving as the Area Chair for ACM Multimedia (ACM MM) 2024!
- Dec 2023: Our work on Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalization was accepted by The Annual AAAI Conference on Artificial Intelligence (AAAI). Congratulations to Yongyi Su!
- Nov 2023: Our work on Transformation-Invariant Network for Few-Shot Object Detection in Remote Sensing Images was accepted by IEEE Transactions on Geoscience and Remote Sensing (TGRS). Congratulations to Nanqing Liu!
- Oct 2023: Our work on Revisiting Pretraining for Semi-Supervised Learning in the Low-Label Regime was accepted by Neurocomputing. Congratulations!
- Jul 2023: Our work on On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion was accepted by ICCV 2023 as Oral presentation (1.8% acceptance rate). Congratulations to Yushu Li!
- May 2023: Our work on Weakly Supervised 3D Point Cloud Segmentation via Multi-Prototype Learning was accepted by IEEE Transactions on Circuits and System for Video Technology (TCSVT). Congratulations to Yongyi Su!
- Jan 2023: Our work on Diverse and consistent multi-view networks for semi-supervised regression was accepted by ECML PKDD Journal Track 2023.
- Sep 2022: Our work on Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering was accepted by NeurIPS 2022. Congratulations to Su Yongyi!
- Jun 2022: Our work on Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding was accepted by ICPR 2022 as Oral presentation.
- May 2022: Our work on SemiCurv: Semi-Supervised Curvilinear Structure Segmentation was accepted by IEEE Transactions on Image Processing (TIP).
- May 2022: Our work on MA-GANet: A Multi-Attention Generative Adversarial Network for Defocus Blur Detection was accepted by IEEE Transactions on Image Processing (TIP).
- Nov 2021: Our work on Automatic Data Augmentation for 3D Point Cloud has appeared in BMVC 2021. Code is available here.
- Jun 2021: The first 3D affordance prediction dataset 3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding has appeared in CVPR 2021. Code is available here.
- Mar 2021: Our work on Learning Clustering for Motion Segmentation has appeared in IEEE Transactions on Circuits and Systems for Video Technology (TCSVT).

Education Background

• PhD in Computer Science, Queen Mary University of London, 2016
  Thesis: Semantic Spaces for Video Analysis of Behaviour
  Supervisors: Prof. Shaogang Gong and Dr. Tao Xiang
• M.Sc in Control Theory and Engineering, Sichuan University, 2012
  Supervisor: Prof. Yusheng Liu
• B.Eng in Automation, Sichuan University, 2010
  

Professional Experience

• Scientist with I2R, A-STAR, Singpaore
  2019.12-Now
  
• Research Fellow in SoC, National University of Singapore
  2019.4-2019.12
  Supervisors: Prof. Gim Hee Lee
• Research Fellow in ECE, National University of Singapore
  2016.9-2019.3
  Supervisors: Prof. Loong-Fah Cheong

Academic Services

Member

• IEEE Senior Member

Area Chair

• ACM Multimedia (ACM MM) 2024

Journal Reviewer

• TPAMI, IJCV, TIP, TNNLS, TMLR, TCSVT, etc.

Conference Reviewer

• NeurIPS, ICLR, ICML, CVPR, ECCV, ICCV, etc.

Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Training PDF
Yongyi Su#, Xun Xu#*, Tianrui Li and Kui Jia
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024

Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptation PDF ProjectPage
Haojie Zhang, Yongyi Su, Xun Xu*, and Kui Jia
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

COFT-AD: COntrastive Fine-Tuning for Few-Shot Anomaly Detection PDF Bibtex
Jingyi Liao, Xun Xu*, Manh Cuong Nguyen, Adam Goodge and Chuan Sheng Foo
IEEE Transactions on Image Processing (TIP), 2024

Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalization PDF ProjectPage
Yongyi Su, Xun Xu* and Kui Jia
Annual AAAI Conference on Artificial Intelligence (AAAI), 2024

Semi-Supervised Object Detection with Uncurated Unlabeled Data for Remote Sensing Images PDF
Nanqing Liu, Xun Xu*, Yingjie Gao and Heng-Chao Li
International Journal of Applied Earth Observation and Geoinformation, 2024

CLIP-guided Source-free Object Detection in Aerial Images PDF
Nanqing Liu, Xun Xu, Yongyi Su, Chengxin Liu, Peiliang Gong, Heng-Chao Li
The International Geoscience and Remote Sensing Symposium (IGARSS), 2024

On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion PDF ProjectPage Bibtex
Yushu Li, Xun Xu*, Yongyi Su and Kui Jia
IEEE/CVF International Conference on Computer Vision (ICCV), 2023 (Oral Presentation 1.8% acceptance rate)

Revisiting Pretraining for Semi-Supervised Learning in the Low-Label Regime PDF
Xun Xu, Jingyi Liao, Lile Cai, Manh Cuong Nguyen, Kangkang Lu, Wanyue Zhang, Yasin Yazici and Chuan Sheng Foo
Neurocomputing, 2023

Weakly Supervised 3D Point Cloud Segmentation via Multi-Prototype Learning PDF Bibtex
Yongyi Su, Xun Xu* and Kui Jia
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2023

Transformation-Invariant Network for Few-Shot Object Detection in Remote Sensing Images PDF
Nanqing Liu, Xun Xu*, Turgay Celik, Zongxin Gan and Heng-Chao Li
IEEE Transactions on Geoscience and Remote Sensing (TGRS), 2023

Diverse and consistent multi-view networks for semi-supervised regression PDF
Cuong Nguyen, Arun Raja, Le Zhang, Xun Xu, Balagopal Unnikrishnan, Mohamed Ragab, Kangkang Lu and Chuan-Sheng Foo
Machine Learning (ML), 2023

On Adversarial Robustness of Audio Classifiers PDF
Kangkang Lu, Manh Cuong Nguyen, Xun Xu and Chuan Sheng Foo International Conference on Acoustics, Speech, & Signal Processing (ICASSP 2023)

STFAR: Improving Object Detection Robustness at Test-Time by Self-Training with Feature Alignment Regularization PDF
Yijin Chen, Xun Xu*, Yongyi Su, Kui Jia
Preprint, 2023

Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering PDF ProjectPage
Yongyi Su#, Xun Xu#* and Kui Jia
Advances in Neural Information Processing Systems (NeurIPS 2022)

SemiCurv: Semi-Supervised Curvilinear Structure Segmentation PDF ProjectPage Bibtex
Xun Xu, Manh Cuong Nguyen, Yasin Yazici, Kangkang Lu, Hlaing Min, Chuan Sheng Foo
IEEE Transactions on Image Processing (TIP), 2022

MA-GANet: A Multi-Attention Generative Adversarial Network for Defocus Blur Detection PDF Bibtex
Zeyu Jiang, Xun Xu*, Le Zhang, Chao Zhang, Chuan Sheng Foo and Ce Zhu
IEEE Transactions on Image Processing (TIP), 2022

Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding PDF Bibtex
Xian Shi, Xun Xu*, Wanyue Zhang, Xiatian Zhu, Chuan Sheng Foo and Kui Jia
International Conference on Pattern Recognition (ICPR 2022) (Oral Presentation)

On Representation Knowledge Distillation for Graph Neural Networks PDF ProjectPage Bibtex
Chaitanya K. Joshi, Fayao Liu, Xun Xu, Jie Lin and Chuan Sheng Foo
IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022

Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving PDF
Zhihao Liang, Xun Xu*, Shengheng Deng, Lile Cai, Tao Jiang and Kui Jia
Preprint, 2022

Learning Clustering for Motion Segmentation PDF ProjectPage Bibtex
Xun Xu, Loong-Fah Cheong, Zhuwen Li, Le Zhang and Ce Zhu
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2021

3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding PDF ProjectPage Bibtex
Shengheng Deng# Xun Xu#, Chaozheng Wu, Ke Chen and Kui Jia
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021)

Revisiting Superpixels for Active Learning in Semantic Segmentation with Realistic Annotation Costs PDF Supplementary ProjectPage Bibtex
Lile Cai, Xun Xu, Junhao Lieuw and Chuan Sheng Foo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021)

ARMOURED: Adversarially Robust MOdels using Unlabeled data by REgularizing Diversity PDF Bibtex
Kangkang Lu, Cuong M Nguyen, Xun Xu, Kiran Chari, Yu Jing Goh and Chuan Sheng Foo
International Conference on Learning Representations (ICLR 2021)

Exploring Spatial Diversity for Region-based Active Learning PDF Bibtex
Lile Cai, Xun Xu, Lining Zhang and Chuan Sheng Foo
IEEE Transactions on Image Processing (TIP), 2021

On Automatic Data Augmentation for 3DPoint Cloud Classification PDF ProjectPage Bibtex
Wanyue Zhang, Xun Xu*, Fayao Liu, Le Zhang and Chuan Sheng Foo
British Machine Vision Conference (BMVC 2021)

Label-Efficient Point Cloud Semantic Segmentation: An Active Learning Approach PDF
Xian Shi#, Xun Xu#*, Ke Chen, Lile Cai, Chuan Sheng Foo and Kui Jia
Preprint, 2021

Learning Category-level Shape Saliency via Deep Implicit Surface Networks PDF
Chaozheng Wu, Sun Lin, Xun Xu and Kui Jia
Preprint, 2021

MultiANet: a Multi-Attention Network for DefocusBlur Detection PDF ProjectPage Bibtex
Zeyu Jiang, Xun Xu, Chao Zhang, Xiaoning Liu and Ce Zhu
IEEE International Workshop on Multimedia Signal Processing (MMSP 2020)

Weakly Supervised Semantic Point Cloud Segmentation: Towards 10× Fewer Labels PDF ProjectPage Bibtex
Xun Xu and Gim Hee Lee
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2020)

3D Rigid Motion Segmentation with Mixed and Unknown Number of Models PDF ProjectPage Bibtex
Xun Xu， Loong-Fah Cheong and Zhuwen Li
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019

C3AE: Exploring the Limits of Compact Model for Age Estimation PDF
Chao Zhang, Shuaicheng Liu, Xun Xu and Ce Zhu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019)

Motion Segmentation by Exploiting Complementary Geometric Models PDF ProjectPage Bibtex
Xun Xu, Loong-Fah Cheong and Zhuwen Li
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018)

Robust Video Background Identification by Dominant Rigid Motion Estimation PDF Bibtex
Kaimo Lin, Nianjuan Jiang, Loong Fah Cheong, Jiangbo Lu and Xun Xu
Asian Conference on Computer Vision (ACCV 2018)

Image Ordinal Classification and Understanding: Grid Dropout with Mask Label PDF Bibtex [Oral Presentation]
Chao Zhang, Ce Zhu, Jimin Xiao, Xun Xu and Yipeng Liu
IEEE Conference on Multimedia and Expo (ICME 2018)

Visual aesthetic understanding: Sample-specific aesthetic classification and deep activation map visualization PDF Bibtex
Chao Zhang, Ce Zhu, Xun Xu, Yipeng Liu, Jimin Xiao and Tammam Tillo
Signal Processing: Image Communication (SPIG), 2018

Image ordinal classification with deep multi-view learning PDF Bibtex
Chao Zhang, Xun Xu and Ce Zhu
Electronic Letters, 2018

Transductive Zero-Shot Action Recognition by Word-Vector Embedding PDF Data Bibtex
Xun Xu, Timothy Hospedales and Shaogang Gong
International Journal of Computer Vision (IJCV), 2017

Discovery of Shared Semantic Spaces for Multi-Scene Video Query and Summarization PDF Supplementary Slides Project Page Bibtex
Xun Xu, Timothy Hospedales and Shaogang Gong
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2017

Zero-Shot Crowd Behaviour Recognition PDF
Xun Xu, Shaogang Gong and Timothy Hospedales
In Murino, Shah, Cristani, Savarese (Eds.), Group and Crowd Behaviour Understanding in Computer Vision , Elsevier, April 2017.

Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation PDF Data Bibtex
Xun Xu, Timothy Hospedales and Shaogang Gong
European Conference on Computer Vision (ECCV 2016)

Semantic Embedding Space for Zero ­Shot Action Recognition PDF Slides Demo Bibtex [★ recognized as top 10% papers]
Xun Xu, Timothy Hospedales and Shaogang Gong
International Conference on Image Processing (ICIP 2015)

Cross-Domain Traffic Scene Understanding by Motion Model Transfer PDF Code Bibtex
Xun Xu, Shaogang Gong and Timothy Hospedales
ACM International Conference on Multimedia, Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Streams, 2013