Publications/Reports

Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition

Neural reflectance decomposition from unconstrained image collections with a novel illumination representation and pre-integration network.

M. Boss, V. Jampani, R. Braun, C. Liu, J. T. Barron, H. Lensch

Neural Information Processing Systems, NeurIPS’21

pdf / project page / video / code (github)

ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction

Category-agnostic and template-free learning of articulated 3D reconstruction and dense 3D trajectories from a monocular video.

G. Yang, D. Sun, V. Jampani, D. Vlasic, F. Cole, C. Liu, D. Ramanan

Neural Information Processing Systems, NeurIPS’21

pdf / project page / code (github)

Robust Visual Reasoning via Language Guided Neural Module Networks

A novel language-adaptive convolution layer for tackling vision-language tasks such as VQA and REF.

A. Akula, V. Jampani, S. Changpinyo, S. C. Zhu

Neural Information Processing Systems, NeurIPS’21

pdf

Aligning Silhouette Topology for Self-Adaptive 3D Human Pose Recovery

3D human pose learning with silhouette supervision using a differentiable topological-skeleton extraction.

M. Rakesh*, J. N. Kundu*, V. Jampani, R. V. Babu (*equal contribution)

Neural Information Processing Systems, NeurIPS’21

pdf / project page / video

Non-local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation

Domain adaption like technique to learn 3D pose estimation given only unpaired videos and 3D pose datasets.

J. N. Kundu, S. Seth, A. Jamkhandi, Y. Pradyumna, V. Jampani, R. V. Babu, A. Chakraborty

Neural Information Processing Systems, NeurIPS’21

pdf / project page / code (github)

SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting

A fast, modular and unified network for single image 3D photography that can preserve intricate hair-like structures.

V. Jampani*, H. Chang*, K. Sargent, A. Kar, R. Tucker, M. Krainin, D. Kaeser, W. T. Freeman, D. Salesin, B. Curless, C. Liu (*equal contribution)

International Conference on Computer Vision, ICCV’21 (oral)

pdf / project page / video / results

NeRD: Neural Reflectance Decomposition from Image Collections

Estimating re-lightable 3D asset (shape and material properties) from the image collection of an object captured under unconstrained illumination conditions.

M. Boss, R. Braun, V. Jampani, J. T. Barron, C. Liu, H. Lensch

International Conference on Computer Vision, ICCV’21

pdf / project page / video / code (soon)

Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image

Perpetual novel view generation corresponding to an arbitrarily long camera trajectory given a single image of a natural scene.

A. Liu*, R. Tucker*, V. Jampani, A. Makadia, N. Snavely, A. Kanazawa (*equal contribution)

International Conference on Computer Vision, ICCV’21 (oral)

pdf / project page / video / code (github)

Discoverying 3D Parts from Image Collections

Learning 3D shape reconstuction while discovering constituent 3D parts from category-specific and posed 2D image collections.

C-H. Yao, W-C. Hung, V. Jampani, M-H. Yang

International Conference on Computer Vision, ICCV’21

pdf / project page / video / code (soon)

Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation

Multi-head network with multiple augmentations for domain generalization and adaptation for semantic segmentation.

J. N. Kundu*, A. Kulkarni*, A. Singh, V. Jampani, R. V. Babu (*equal contribution)

International Conference on Computer Vision, ICCV’21

pdf / project page / code (soon)

Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold

A method to predict probability distributions over the rotation manifold that is particularly useful for pose estimation of symmetric objects.

K. Murphy*, C. Esteves*, V. Jampani, S. Ramalingam, A. Makadia (*equal contribution)

International Conference on Machine Learning, ICML’21 (spotlight)

pdf / project page / video / code (github)

Decoupled Dynamic Filter Networks

DDF is a lighter-weight, high-performing content-adaptive convolution layer that can readily replace standard convolution layers in CNNs.

J. Zhou, V. Jampani, Z. Pi, Q. Liu, M-H. Yang.

Computer Vision and Pattern Recognition, CVPR’21

pdf / project page / video / code (github)

LASR: Learning Articulated Shape Reconstruction from a Monocular Video

Template-free and category-agnostic articulated 3D shape reconstruction from a given monocular video.

G. Yang, D. Sun, V. Jampani, D. Vlasic, F. Cole, H. Chang, D. Ramanan, W. T. Freeman, C. Liu

Computer Vision and Pattern Recognition, CVPR’21

pdf / project page / code (github)

AutoFlow: Learning a Better Training Set for Optical Flow

Jointly rendering training data while learning flow model can result in state-of-the-art pre-training and fine-tuning optical flow results.

D. Sun, D. Vlasic, C. Hermann, V. Jampani, M. Krainin, H. Chang, R. Zabih, W. T. Freeman, C. Liu

Computer Vision and Pattern Recognition, CVPR’21 (oral)

pdf / project page / code (coming soon)

Adaptive Prototype Learning and Allocation for Few-Shot Segmentation

Few-shot segmentation with superpixel guided adaptive prototype learning and allocation.

G. Li, V. Jampani, L. Sevilla-Lara, D.Sun, J. Kim, J. Kim

Computer Vision and Pattern Recognition, CVPR’21

pdf / project page / video / code (github)

Generative View Synthesis: From Single-view Semantics to Novel-view Images

Learning to generate semantically consistent novel-view images from a given single 2D semantics.

T. A. Habtegebrial, V. Jampani, O. Gallo, D. Stricker

Neural Information Processing Systems, NeurIPS’20

pdf / video / project page / code (github)

Appearance Consensus Driven Self-Supervised Human Mesh Recovery

Learning human mesh recovery from unannotated video frames and unpaired pose data.

J. N. Kundu*, M. Rakesh*, V. Jampani, R. M. Venkatesh, R. V. Babu (*equal contribution)

European Conference on Computer Vision, ECCV’20 (oral)

pdf / video / project page / code (github)

Self-supervised Single-view 3D Reconstruction via Semantic Consistency

Learning 3D object reconstruction from unannotated image collections via self-supervised semantic consistency.

X. Li, S. Liu, K. Kim, S. D. Mello, V. Jampani, M. H. Yang and J. Kautz

European Conference on Computer Vision, ECCV’20

pdf / video / project page / code (github)

DeepGMR: Learning Latent Gaussian Mixture Models for Registration

First learning-based point cloud registration method that explicitly leverages a probabilistic registration paradigm.

W. Yuan, B. Eckart, K. Kim, V. Jampani, D. Fox and J. Kautz

European Conference on Computer Vision, ECCV’20

pdf / video / project page / code (github)

Self-supervised Viewpoint Learning from Image Collections

Learning 3D object viewpoint from unannotated image collections via self-supervision.

S. K. Mustikovela, V. Jampani, S. D. Mello, S. Liu, U. Iqbal, C. Rother and J. Kautz

Computer Vision and Pattern Recognition, CVPR’20

pdf / video / project page / code (github)

Two-shot Spatially-varying BRDF and Shape Estimation

Practical SVBRDF and shape estimation with two-shot mobile capture.

M. Boss, V. Jampani, K. Kim, H. Lensch and J. Kautz

Computer Vision and Pattern Recognition, CVPR’20

pdf / video / project page / code (github)

Self-Supervised 3D Human Pose Estimation via Part-Guided Novel Image Synthesis

3D human pose estimation technique that is self-supervised using image pairs from in-the-wild videos.

J. N. Kundu*, S. Seth*, V. Jampani, M. Rakesh, R. V. Babu and A. Chakraborty (*equal contribution)

Computer Vision and Pattern Recognition, CVPR’20 (oral)

pdf / video / code (google drive)

From Image Collections to Point Clouds with Self-supervised Shape and Pose Networks

Point cloud reconstruction network trained with self-supervision using image collections.

K. L. Navaneet, A. Mathew, S. Kashyap, W-C. Hung, V. Jampani and R. V. Babu

Computer Vision and Pattern Recognition, CVPR’20

pdf / code (github)

Learning Propagation for Arbitrarily-structured Data

A generalized spatial propagation network module that learns to propagate information across arbitrarily-structured data such as superpixels and point clouds.

S. Liu, X. Li, V. Jampani, S. D. Mello and J. Kautz

Internationl Conference on Computer Vision, ICCV’19

pdf

SENSE: A Shared Encoder Network for Scene Flow Estimation

A compact shared-encoder network along with semi-supervised loss functions for depth and optical flow estimation.

H. Jiang, D. Sun, V. Jampani, Z. Lv, E. Learned-Miller and J. Kautz

Internationl Conference on Computer Vision, ICCV’19 (oral)

pdf / code (github)

Gated-SCNN: Gated Shape CNNs for Semantic Segmentation

State-of-the-art semantic segmentation with two-stream CNN architecture that explicitly wires shape information as separate processing branch.

T. Takikawa*, D. Akuna*, V. Jampani and S. Fidler (*equal contribution)

International Conference on Computer Vision, ICCV’19

pdf / code (github) / project page / video

Pixel-Adaptive Convolutional Neural Networks

Generalization of spatially-invariant convolutions to spatially-varying convolutions with applications in joint-image upsampling, conditional random fields and layer hot-swapping.

H. Su, V. Jampani, D. Sun, O. Gallo, E. Learned-Miller and J. Kautz

Computer Vision and Pattern Recognition, CVPR’19

pdf / poster / video / project page / code (github)

SCOPS: Self-Supervised Co-Part Segmentation

Learning to discover semantically consistent object part segments from image collections.

W-C. Hung, V. Jampani, S. Liu, P. Molchanov, M-H. Yang and J. Kautz

Computer Vision and Pattern Recognition, CVPR’19

pdf / poster / video / project page / code (github)

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation

A general unsupervised deep learning framework for learning depth, optical flow, camera motion and motion segmentation from videos.

A. Ranjan, V. Jampani, L. Balles, K. Kim, D. Sun, J. Wulff and M. J. Black

Computer Vision and Pattern Recognition, CVPR’19

pdf / code (github)

DIFFER: Moving Beyond 3D Reconstruction with Differntiable Feature Rendering

Single Image 3D object reconstruction along with texture, segmentation and normal prediction with differentiable feature rendering.

K. L. Navaneet, P. Mandikal, V. Jampani and R. V. Babu

3D-WiDGET, CVPR-Workshops’19 (Oral)

pdf / code (github)

Superpixel Sampling Networks

An end-to-end trainable deep superpixel algorithm that allows learning with flexible loss functions resulting in the learning of task-specific superpixels.

V. Jampani, D. Sun, M-Y. Liu, M-H. Yang and J. Kautz

European Conference on Computer Vision, ECCV’18

pdf / poster / video / project page / code (github)

Switchable Temporal Propagation Network

A generic fast deep learning framework for propagating a variety of visual properties (such as color, HDR, segmentation) across video frames.

S. Liu, G. Zhong, S. D. Mello, J. Gu, V. Jampani, M-H. Yang and J. Kautz

European Conference on Computer Vision, ECCV’18

pdf / code (coming soon)

On the Integration of Optical Flow and Action Recognition

An investigate study on why optical flow is helpful, what makes a flow method good for action recognition, and how we can make it better.

L. Sevilla, Y. Liao, F. Güney, V. Jampani, A. Geiger and M. J. Black

German Conference on Pattern Recognition, GCPR’18 (oral)

pdf

SPLATNet: Sparse Lattice Networks for Point Cloud Processing

A fast and end-to-end trainable neural network that directly works on point clouds and can also do joint 2D-3D processing.

H. Su, V. Jampani, D. Sun, S. Maji, E. Kalogerakis, M-H. Yang and J. Kautz

Computer Vision and Pattern Recognition, CVPR’18 (oral, best paper honorable mention, NVAIL Pioneering Research Award)

pdf / poster / video / CVPR talk / code (github)

Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation

Variable-length multi-frame video interpolation via self-supervised optical flow estimation and occlusion reasoning.

H. Jiang, D. Sun, V. Jampani, M-H. Yang, E. Learned-Miller and J. Kautz

Computer Vision and Pattern Recognition, CVPR’18 (spotlight)

pdf / supplementary / news / CVPR talk / video results

Learning Superpixels with Segmentation-Aware Affinity Loss

A learning technique that leverages deep networks to predict pixel affinities useful for graph-based superpixel segmentation.

W-C. Tu, M-Y. Liu, V. Jampani, D. Sun, S-Y. Chien, M-H. Yang and J. Kautz

Computer Vision and Pattern Recognition, CVPR’18

pdf / project page / code (github)

Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization

A study exploring the use of domain randomization for real-world computer vision tasks such as object detection.

J. Tremblay*, A. Prakash*, D. Acuna*, M. Brophy*, V. Jampani, C. Anil, T. To, E. Cameracci, S. Boochoon and S. Birchfield (*equal contribution)

Workshop on Autonomous Driving, CVPR-Workshops’18

pdf

Semantic Video CNNs through Representation Warping

A fast and end-to-end trainable approach for converting image CNNs to video CNNs for semantic segmentation.

R. Gadde, V. Jampani and P. V. Gehler

International Conference on Computer Vision, ICCV’17 (oral)

pdf / video / code (github) / ICCV talk / poster

Video Propagation Networks

A fast, generic and end-to-end trainable approach for propagating information across video frames.

V. Jampani, R. Gadde and P. V. Gehler

Computer Vision and Pattern Recognition, CVPR’17

pdf / supplementary / project page / code (github) / poster

Efficient 2D and 3D Facade Segmentation using Auto-Context

A fast auto-context based facade segmentation approach for segmenting both 2D images and 3D point clouds.

R. Gadde*, V. Jampani*, R. Marlet and P. V. Gehler (*equal contribution)

Pattern Analysis and Machine Intelligence, PAMI’17

arXiv pre-print / code (bitbucket)

Superpixel Convolutional Networks using Bilateral Inceptions

An inception style fast bilateral filtering module that can be used in existing image segmentation CNNs.

R. Gadde*, V. Jampani*, M. Kiefel, D. Kappler and P. V. Gehler (*equal contribution)

European Conference on Computer Vision, ECCV’16

pdf / supplementary / project page / code (github) / poster

Learning Sparse High-Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks

Parameterization and learning of sparse high-dimensional filters with wide range of applications in image filtering, CRFs and neural networks.

V. Jampani, M. Kiefel and P. V. Gehler

Computer Vision and Pattern Recognition, CVPR’16

pdf / supplementary / project page / code (github) / poster

Optical Flow with Semantic Segmentation and Localized Layers

An approach for incorporating semantics of the scene for better optical flow estimation.

L. Sevilla, D. Sun, V. Jampani and M. J. Black

Computer Vision and Pattern Recognition, CVPR’16

pdf / video / project page / code (zip)

Permutohedral Lattice CNNs

CNNs with high-dimensional permutohedral kernels instead of standard spatial kernels.

M. Kiefel, V. Jampani and P. V. Gehler

International Conference on Learning Representations, ICLR Workshops’15

pdf

Consensus Message Passing for Layered Graphical Models

Improved message passing inference in layered vision models that can be used in probabilistic programs like Infer.NET.

V. Jampani*, S. M. A. Eslami*, D. Tarlow, P. Kohli and J. Winn (*equal contribution)

Artificial Intelligence and Statistics, AISTATS’15

pdf / supplementary / arXiv

Efficient Facade Segmentation using Auto-Contex

A state-of-the-art approach for facade segmentation that is faster than most existing approaches.

V. Jampani*, R. Gadde* and P. V. Gehler (*equal contribution)

Winter Conference on Applications of Computer Vision, WACV’15

pdf / supplementary / code (bitbucket) / project page

The Informed Sampler: A Discriminative Approach to Bayesian Inference in Generative Computer Vision Models

A generic MCMC sampling technique for inverting graphic engines.

V. Jampani, S. Nowozin, M. Loper and P. V. Gehler

Computer Vision and Image Understanding Journal, CVIU’15

pdf / arXiv

Assessment of Computational Visual Attention Models on Medical Images

Analysis and extensions of visual attention models for medical image perception.

V. Jampani, Ujjwal, J. Sivaswamy and V. Vaidya

Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP’12

pdf / online

Role of Expertise and Contralateral Symmetry in the Diagnosis of Pneumoconiosis: An Experimental Study

An experimental eye-tracking study on X-ray image perception, with observers ranging from novices to experienced radiologists.

V. Jampani, V. Vaidya, J. Sivaswamy and K. L. Tourani

SPIE, Medical Imaging’11

online

ImageFlow: Streaming Image Search

A novel image search user interface that explores a different alternative to the traditional grid interfaces.

V. Jampani, G. Ramos and S. Drucker

Microsoft Research Technial Report, Redmond, MSR-TR-2010-148

pdf / news article

 

Theses

Learning Inference Models for Computer Vision

Learning based techniques for better inference in several computer vision models ranging from inverse graphics to freely parameterized neural networks.

V. Jampani

PhD Thesis, MPI for Intelligent Systems and University of Tübingen, December, 2016

pdf / slides / library

A Study of X-Ray Image Perception for Pneumoconiosis Detection

Eye tracking experimental studies and models for X-Ray Image Perception and Diagnosis.

V. Jampani

Master Thesis, IIIT-Hyderabad, January, 2013

pdf