A Sparsity-Promoting Image Decomposition Model for Depth Recovery

Xinchen Ye*¹, Mingliang Zhang¹, Jingyu Yang², Xin Fan¹, Fangfang Guo¹

¹ Dalian University of Technology  ²Tianjin University

* Corresponding author

Abstract

This paper proposes a novel image decomposition model for scene depth recovery from low-quality depth measurements and its corresponding high resolution color image. Through our observation, the depth map mainly contains smooth regions separated by additive step discontinuities, and can be simulta- neously decomposed into a local smooth surface and an approximately piecewise constant component. Therefore, the proposed unified model combines the least square polynomial approximation (for smooth surface) and a sparsity-promoting prior (for piecewise constant) to better portray the 2D depth signal intrinsically. As we know, the representation of the piecewise constant signal in gradient domain is ex- tremely sparse. Previous researches using total variation filter based on L 1 -norm or L p -norm (0 < p < 1) are both sub-optimal when addressing the tradeoffbetween enhancing the sparsity and keeping the model convex. We propose a novel non-convex penalty based on Moreau envelope, which promotes the prior sparsity and simultaneously maintains the convexity of the whole model for each variable. We prove the convexity of the proposed model and give the convergence analysis of the algorithm. We also introduce an iterative reweighted strategy applied on the sparsity prior to deal with the depth-color inconsistent problem and to locate the depth boundaries. Moreover, we provide an accelerated algorithm to deal with the problem of non-uniform down-sampling when transforming the depth observation matrix into the Fourier domain for fast processing. Experimental results demonstrate that the proposed method can han- dle various types of depth degradation and achieve promising performance in terms of recovery accuracy and running time.

Index Terms—Image decomposition Depth recovery Depth discontinuities Depth cameras

Method

Publications

[1]  Xinchen Ye*, Mingliang Zhang, Jingyu Yang, Xin Fan, Fangfang Guo, A Sparsity-Promoting Image Decomposition Model for Depth Recovery, Pattern Recognition, 107: 107506, 2020. （CCF-B, 中科院1区TOP）

[2]  Xinchen Ye, Xiaolin Song and Jingyu Yang*. Depth Recovery via Decomposition of Polynomial and Piece-wise Constant Signals. Visual Communications and Image Processing, 2016, Chengdu, China.