Sebastian Tschiatschek About Publications Teaching Research Projects People Contact

Probabilistic Models and Inference

HM-VAEs: a Deep Generative Model for Real-valued Data with Heterogeneous Marginals

By Chao Ma, Sebastian Tschiatschek, Yingzhen Li, Richard Turner, José Miguel Hernández-Lobato, Cheng Zhang
In Proceedings of The 2nd Symposium on Advances in Approximate Bayesian Inference, 2020
In this paper, we propose a very simple but effective VAE model (HM-VAE) that can handle real-valued data with heterogeneous marginals, meaning that they have drastically distinct marginal distributions, statistical properties as well as semantics. Preliminary resultsshow that the HM-VAE can learn distributions with heterogeneous marginal distributions,whereas the vanilla VAEs fails.

VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data

By Chao Ma, Sebastian Tschiatschek, José Miguel Hernández-Lobato, Richard Turner, Cheng Zhang
In arXiv, 2020
Our VAEM accurately generates marginals and pairwise-marginals of real-world datasets
Deep generative models often perform poorly in real-world applications due to the heterogeneity of natural data sets. Heterogeneity arises from data containing different types of features (categorical, ordinal, continuous, etc.) and features of the same type having different marginal distributions. We propose an extension of variational autoencoders (VAEs) called VAEM to handle such heterogeneous data. VAEM is a deep generative model that is trained in a two stage manner such that the first stage provides a more uniform representation of the data to the second stage, thereby sidestepping the problems caused by heterogeneous data. We provide extensions of VAEM to handle partially observed data, and demonstrate its performance in data generation, missing data prediction and sequential feature selection tasks. Our results show that VAEM broadens the range of real-world applications where deep generative models can be successfully deployed.

Bayesian EDDI: Sequential Variable Selection with Bayesian Partial VAE

By Chao Ma, Wenbo Gong, Sebastian Tschiatschek, Sebastian Nowozin, José Miguel Hernández-Lobato, Cheng Zhang
In Workshop on Real-World Sequential Decision Making: Reinforcement Learning and Beyond at NeurIPS, 2019

Icebreaker: Element-wise Active Information Acquisition with Bayesian Deep Latent Gaussian Model

By Wenbo Gong, Sebastian Tschiatschek, Richard Turner, Sebastian Nowozin, José Miguel Hernández-Lobato
In Neural Information Processing Systems (NeurIPS), 2019
Accumulated feature statistics as active selection progresses.
In this paper we introduce the ice-start problem, i.e., the challenge of deploying machine learning models when only little or no training data is initially available, and acquiring each feature element of data is associated with costs. This setting is representative for the real-world machine learning applications. For instance, in the health-care domain, when training an AI system for predicting patient metrics from lab tests, obtaining every single measurement comes with a high cost. Active learning, where only the label is associated with a cost does not apply to such problem, because performing all possible lab tests to acquire a new training datum would be costly, as well as unnecessary due to redundancy. We propose Icebreaker, a principled framework to approach the ice-start problem. Icebreaker uses a full Bayesian Deep Latent Gaussian Model (BELGAM) with a novel inference method. Our proposed method combines recent advances in amortized inference and stochastic gradient MCMC to enable fast and accurate posterior inference. By utilizing BELGAM's ability to fully quantify model uncertainty, we also propose two information acquisition functions for imputation and active prediction problems. We demonstrate that BELGAM performs significantly better than the previous VAE (Variational autoencoder) based models, when the data set size is small, using both machine learning benchmarks and real-world recommender systems and health-care applications. Moreover, based on BELGAM, Icebreaker further improves the performance and demonstrate the ability to use minimum amount of the training data to obtain the highest test time performance.

Differentiable Submodular Maximization

By Sebastian Tschiatschek, Aytunc Sahin, Andreas Krause
In Joint Conference on Artificial Intelligence (IJCAI), 2018
We consider learning of submodular functions from data. These functions are important in machine learning and have a wide range of applications, e.g. data summarization, feature selection and active learning. Despite their combinatorial nature, submodular functions can be maximized approximately with strong theoretical guarantees in polynomial time. Typically, learning the submodular function and optimization of that function are treated separately, i.e. the function is first learned using a proxy objective and subsequently maximized. In contrast, we show how to perform learning and optimization jointly. By interpreting the output of greedy maximization algorithms as distributions over sequences of items and smoothening these distributions, we obtain a differentiable objective. In this way, we can differentiate through the maximization algorithms and optimize the model to work well with the optimization algorithm. We theoretically characterize the error made by our approach, yielding insights into the tradeoff of smoothness and accuracy. We demonstrate the effectiveness of our approach for jointly learning and optimizing on synthetic maximum cut data, and on real world applications such as product recommendation and image collection summarization.
[pdf] [long]

Variational Inference for Data-Efficient Model Learning in POMDPs

By Sebastian Tschiatschek, Kai Arulkumaran, Jan Stühmer, Katja Hofmann
In arXiv preprint arXiv:1805.09281, 2018

Hybrid Generative-Discriminative Training of Gaussian Mixture Models

By Wolfgang Roth, Robert Peharz, Sebastian Tschiatschek, Franz Pernkopf
In Pattern Recognition Letters, 2018

Sum-Product Networks for Sequence Labeling

By Martin Ratajczak, Sebastian Tschiatschek, Franz Pernkopf
In arXiv preprint arXiv:1807.02324, 2018

Optimally Searching for Cancer Genes Using Submodular Models

By Gideon Dresdner, Sebastian Tschiatschek, Viktor Gál, Gunnar Rätsch
In , 2018