Sparse Representation in Redundant Systems

CSIC Building (#406), Seminar Room 4122.
Directions: home.cscamm.umd.edu/directions

Sublinear Algorithms to Track Functions with Sparse Fourier Representation

Dr. Jing Zou

Mathematics at Princeton University

Abstract:   We analyze a family of sublinear algorithms that find a near-optimal B-term Sparse Representation R for a given discrete signal S of length N, in time and space poly(B, log(N), log(delta), epsilon^-1). These representations are expansions with respect to a particular basis or a family of bases; examples are wavelet bases, wavelet packets or Fourier bases. We shall use the acronym RAlSTA (Randomized Algorithm for Sparse Transform Analysis) for this family of algorithms. We here restrict ourselves to the Fourier case and thus RAlSFA (Randomized Algorithm for Sparse Fourier Analysis), where the poly(log(N)) time of RAlSFA should be compared with the superlinear O(N log N) time requirement of the Fast Fourier Transform (FFT). However, a straightforward implementation of the original RAlSFA, as presented in the theoretical paper cite{GGIMS}, turns out to be very slow in practice. Our main result is a greatly improved and practical RAlSFA implementation. We introduce several new ideas and techniques to speed up the algorithm; tests on numerical examples show that our implementation is about four thousand times faster than the original RAlSFA. Both rigorous and heuristic arguments for parameter choices are presented. Our empirically improved RAlSFA constructs, with probability at least 1-delta, a near-optimal B-term representation R in time poly(B)log(N)O(log(delta)/epsilon²) such that ||S-R||² < (1+epsilon)||S-R_opto||². Furthermore, the improved RAlSFA already beats FFT for not unreasonably large N. We extend the algorithm to higher dimensional cases both theoretically and numerically. The crossover point lies at N~25,000 in one dimension, and at N~460 for data on a N*N grid in two dimensions for small B signals where there is no noise. This result can also be extended to process sparse signals in unevenly spaced data.