Jianbo Ye

Applied Researcher & Developer, Ph.D.

As a Senior Applied Scientist, I am working on cutting-edge initiatives in 3D computer vision and spatial AI at AWS AI Labs. Prior to this role, I was a member of the team that launched Amazon Astro from 2018 to 2021. I hold a PhD from Pennsylvania State University, where I was supervised by Professors James Z. Wang and Jia Li. My undergraduate studies were in mathematics at the University of Science and Technology of China (USTC). My current research interests include machine learning, optimal transport, 3D computer vision, and affective computing.

What's New

(2022-04-19) Our work on Amazon Astro's SLAM were featured in Amazon Science blog [link1, link2]
(2020-06-30) See our BoLD dataset and ECCV bodily expressed emotion challenge [link].
(2019-10-11) Our research team at Penn State was interviewed by China Meteorological News (Chinese).
(2019-09-05) Our research work was reported in TechXplore: Can computers be trained to understand body language?
(2019-07-02) Our research work was reported in phys.org, ScienceDaily: Using artificial intelligence to better predict severe weather.
(2018-01-31) Penn State News: Jianbo will join Amazon Lab126 as an applied scientist.


Keywords: selected optimization learning pattern recognition optimal transport

Selected Journal Publications

(sorted chronologically)

SCOTT: Shape-Location Combined Tracking with Optimal Transport
Xinye Zheng, Jianbo Ye, James Z. Wang, Jia Li
SIAM Journal on Mathematics of Data Science, 2020
DOI code pattern recognition optimal transport optimization

ARBEE: Towards Automated Recognition of Bodily Expression of Emotion In the Wild
Yu Luo, Jianbo Ye, Reginald B. Adams, Jr., Jia Li, Michelle G. Newman, James Z. Wang
International Journal of Computer Vision, 2019, Springer (arXiv:1808.09568 [cs.CV], August 2018)
DOI learning pattern recognition (TechXplore)

Aggregated Wasserstein Metric and State Registration for Hidden Markov Models
Yukun Chen, Jianbo Ye, Jia Li
IEEE Transactions on Pattern Analysis and Machine Intelligence, April 2019 (arXiv:1711.05792 [cs.LG], November 2017)
DOI pattern recognition optimal transport

Detecting Comma-shaped Clouds for Severe Weather Forecasting using Shape and Motion
Xinye Zheng, Jianbo Ye, Yukun Chen, Stephen Wistar, Jia Li, Jose A. Piedra-Fernández, Michael A. Steinberg, James Z. Wang
IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 6, pp. 3788-3801, 2019 (arXiv:1802.08937 [cs.CV], Feb 2018)
DOI pattern recognition (phys.org, ScienceDaily, PSC electronic magazine, XSEDE newsletter)

Probabilistic Multigraph Modeling for Improving the Quality of Crowdsourced Affective Data
Jianbo Ye, Jia Li, Michelle G. Newman, Reginald B. Adams, Jr., James Z. Wang
IEEE Transactions on Affective Computing, Jan.-March 2019. (arXiv:1701.01096 [stat.ML], Jan 2017)
DOI g-scholar code dataset learning

This project also develops a scalable data analytic tool, called accelerated D2-clustering, to process large scale distribution data. It could potentially leverage hundreds of CPUs with a very decent scaling efficiency (say, 70-80%).

If you are a government agency, an education institution, or a non-profit organization, we may offer you a FREE academic license of the C/MPI package to run on clusters. Please contact authors by email to discuss details. If you are commercial and would like to use our software, let us know and we will try to arrange to let you use.

Fast Discrete Distribution Clustering Using Wasserstein Barycenter with Sparse Support
Jianbo Ye, Panruo Wu, James Z. Wang and Jia Li
IEEE Transactions on Signal Processing, January 2017 (arXiv:1510.00012 [stat.CO], September 2015)
DOI g-scholar code optimization learning optimal transport

Selected Peer-reviewed Conference Proceedings

Overlapping Displacement Error: Are Your SLAM poses Map-consistent?
Christian Mostegel, Jianbo Ye, Yu Luo, and Yang Liu
IROS 2021

Active Learning of Strict Partial Orders: A Case Study on Concept Prerequisite Relations
Chen Liang, Jianbo Ye, Han Zhao, Bart Pursel, C. Lee Giles
EDM 2019 (arXiv:1801:06481 [cs.LG], January 2018)
pdf learning

Investigating Capsule Networks with Dynamic Routing for Text Classification
Wei Zhao, Jianbo Ye, Min Yang, Zeyang Lei, Suofei Zhang, Zhou Zhao
EMNLP 2018 (arXiv:1804.00538 [cs.CL], March 2018)
pdf code learning

Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers
Jianbo Ye, Xin Lu, Zhe Lin, and James Z. Wang
ICLR 2018
pdf open review TensorFlow PyTorch summary 知乎 learning optimization

A Simulated Annealing based Inexact Oracle for Wasserstein Loss Minimization
Jianbo Ye, James Z. Wang and Jia Li
ICML 2017 (arXiv:1608.03859 [stat.CO], August 2016)
pdf & supp g-scholar video optimization learning optimal transport

Determining Gains Acquired from Word Embedding Quantitatively Using Discrete Distribution Clustering
Jianbo Ye, Yanran Li, Zhaohui Wu, James Z. Wang, Wenjie Li, Jia Li
ACL 2017 (Long paper)
pdf g-scholar code dataset learning optimal transport

Doctoral Thesis

Jianbo Ye, Computational Modeling of Compositional and Relational Data Using Optimal Transport and Probabilistic Models, Ph.D. thesis, The Pennsylvania State University, May 2018
pdf g-scholar


A Faster Drop-in Implementation for Leaf-wise Exact Greedy Decision Tree Induction Using Pre-sorted Deque
Jianbo Ye
(arXiv:1712.06989 [cs.DS], December 2017)
code learning

Yet another model reduction technique for deformable meshes based on approximation quality controllable subspace.
On the Approximation Theory of Linear Variational Subspace Design
Jianbo Ye and Zhixin Yan
(arXiv:1506.08459 [cs.GR], June 2015)
pdf g-scholar gitxiv software executable demo: linux-x86_64 video (40M) optimization


Mar 2018, A Simulated Annealing based Inexact Oracle for Wasserstein Loss Minimization, 42nd SIAM-SEAS Conference, UNC Chapel Hill. slides
Sep 2017, Optimal transport for machine learning: the state-of-the-art numerical tools, Artificial Intelligence Seminar Series, sponsored by Apple, CMU. website
Aug 2017, Oral presentation at ICML, Sydney. video
May 2017, New numerical tools for optimal transport and their machine learning applications BIRS-CMO Workshop (Optimal Transport meets Probability, Statistics and Machine Learning), Oaxaca. video video-2
Oct 2015, Accelerated Discrete Distribution Clustering under Wasserstein Distance
Apr 2014, Probabilistic Graphical Models and Their Applications in Vision and Graphics
Oct 2013, Emerging Technologies in Computer Graphics


As a PhD student (2011-2018), I wrote a couple of open source softwares. I no longer actively maintain those codes. Hope they are still useful!

[2018] Decision_Tree_PDeque: A very fast (probably the fastest) and generic decision tree implementation in C++ templates, supporting classification, regression and more. code

[2017] batchnorm_prune: Pruning ConvNet channels by enforcing sparsity on BatchNorm layers using ISTA. code
Python Tensorflow

[2015-2016] d2_kmeans: a parallel clustering algorithm for discrete distributions, including normalized histogram as a special case, under the Wasserstein metric. project page

[2014-2015] neuron: a full-fledged Scala library for composing and training neural network of complex topologies with parameter sharing, supporting different activations, metrics, regularization, and optimization methods. It also includes different variants of multilayer perceptron and auto-encoders. project page
Scala breeze

[2014-2015] dmfCramer: Discrete martrix factorizatin with Cramer risk minimization, which is yet another probabilistic matrix factorization method with loss function based on large deviations theory rather than conventional MLE/MAP framework. project page
Scala breeze

[2013] Pocket Avatar (intern project at Intel): I developed an efficient data-driven framework for real-time facial expression retargeting. demo

[2012-2013] iMeshDeform: A C++ mesh deformation framework based on linear variational subspace. project page video

[2011-2012] R-BiHDM: state-of-the-art, simple and fast signature for nonrigid shapes, which have been tested upon multiple benchmarks. project page