Sampling and (Sparse) Stochastic Processes: A Tale of Splines and Innovation

The commonality between splines and Gaussian or sparse stochastic processes is that they are ruled by the same type of differential equations. Our purpose here is to demonstrate that this has profound implications for the three primary forms of sampling: uniform, nonuniform, and compressed sensing. The connection with classical sampling is that there is a one-to-one correspondence between spline interpolation and the minimum-mean-square-error reconstruction of a Gaussian process from its uniform or nonuniform samples. The caveat, of course, is that the spline type has to be matched to the operator that whitens the process. The connection with compressed sensing is that the non-Gaussian processes that are ruled by linear differential equations generally admit a parsimonious representation in a wavelet-like basis. There is also a construction based on splines that yields a wavelet-like basis that is matched to the underlying differential operator. It has been observed that expansions in such bases provide excellent M-term approximations of sparse processes. This property is backed by recent estimates of the local Besov regularity of sparse processes.