Progress Report
Summary of what we've done so far:
-
We've experimented using the matlab program from the MIT paper.
-
Learning how complex-steerable-pyramids work - this has been quite difficult as the math is complex.
-
Started writing our program in python - so far we have only implemented the non-phase based approach.
Complex-Steerable-Pyramids
​
The paper we are basing our project on "Phase-Based Video Motion Processing" uses complex steerable pyramids to decompose the image and extract the motion information. Now what are these?
​
Image Pyramids
​
Image pyramids are a way of representing an image at multiple resolutions. We do this to extract information from the image more effectively. For example, analyzing high frequency information with small oscillations is more effective at a higher resolution. But lower frequency information can be adequately recovered from a lower resolution. This process effectively splits the image into different frequency bands.
​
Steerable Filters
​
Steerable filters are a class of filters with the property that a filter with arbitrary orientation can be represented as a linear combination of a set of basis filters, though the basis can be over-complete. The basis filters for a steerable basis are all rotated copies of each other.
​
The Steerable Pyramid
​
​
Project Plan
1 / Refine filtering function for Eulerian code
One significant issue that we have identified in our current code is the filtering values we are using for some of the data. We based these values off of the code associated with the academic papers used to build this project. In experimentation with the code and other values, we found that we can reduce noise and increase amplification by adjusting the filter. This process needs to be applied to our full dataset before moving into more advanced processing.
2 / Implement phase-based processing
One of the main goals of our project, and the main challenges, is the application of phase-based processing to the motion amplification. This will involve rewriting our main functions and adding additional helper functions to perform the Fourier decomposition. See our home page for more information on phase-based processing.
3 / Capture & analyze own dataset
After implementing and testing our code on existing datasets, our goal is to capture and analyze our own datasets that are more familiar to students at U of M. Our first target is the swing set on the Grove. This will provide interesting analysis of both stationary objects with small vibrations, and the response of the algorithm to large movements. The second dataset we are targeting is the vibration of a string from the grand piano in Pierpont. This will show its ability to identify and magnify harmonic motions.
Expanding Our Knowledge: The SciPy Toolbox
One interesting tool that we have learned and applied in our project is the Signal Processing toolbox for Python. This was recommended to us by Professor Balzano, as a way of implementing similar commands to the toolbox in Matlab. What we found most useful and interesting about this was the filtering functions. There are a variety of pre-implemented filter functions in this toolbox, allowing us to quickly and easily implement bandpass filters for our project.