Hits:

Indexed by:Journal Papers

Date of Publication:2020-11-01

Journal:PATTERN RECOGNITION

Included Journals:SCIE

Volume:107

ISSN No.:0031-3203

Key Words:Image decomposition; Depth recovery; Depth discontinuities; Depth cameras

Abstract:This paper proposes a novel image decomposition model for scene depth recovery from low-quality depth measurements and its corresponding high resolution color image. Through our observation, the depth map mainly contains smooth regions separated by additive step discontinuities, and can be simultaneously decomposed into a local smooth surface and an approximately piecewise constant component. Therefore, the proposed unified model combines the least square polynomial approximation (for smooth surface) and a sparsity-promoting prior (for piecewise constant) to better portray the 2D depth signal intrinsically. As we know, the representation of the piecewise constant signal in gradient domain is extremely sparse. Previous researches using total variation filter based on L-1-norm or L-p-norm (0 < p < 1) are both sub-optimal when addressing the tradeoff between enhancing the sparsity and keeping the model convex. We propose a novel non-convex penalty based on Moreau envelope, which promotes the prior sparsity and simultaneously maintains the convexity of the whole model for each variable. We prove the convexity of the proposed model and give the convergence analysis of the algorithm. We also introduce an iterative reweighted strategy applied on the sparsity prior to deal with the depth-color inconsistent problem and to locate the depth boundaries. Moreover, we provide an accelerated algorithm to deal with the problem of non-uniform down-sampling when transforming the depth observation matrix into the Fourier domain for fast processing. Experimental results demonstrate that the proposed method can handle various types of depth degradation and achieve promising performance in terms of recovery accuracy and running time. (C) 2020 Elsevier Ltd. All rights reserved.