Feature Extraction with Binarized Descriptors: ASIC Implementation and FPGA Environment

Final layout of the ASIC. It comprises 254 kGE, including 29 kbit SRAM. Its operating frequency is 100 MHz and its simulated power consumption is 252 mW.

Short Description

Image feature extraction is an important analysis tool in many computer vision applications. In this work, we look at sparse depth estimation in stereo video, using image features based on binarized semantic kernels (SKB, Zilly et al. 2011). When the stereo camera parameters are known, the depth of a real world point can be easily calculated from its position in the two stereo images. If a sparse depth map of an imaged scene is sufficient for the given application, the point correspondences can be established using image features. This process generally comprises four steps. First, informative regions (interest points) are localized in the images. Second, small image patches around those interest points are extracted and normalized in size. Third, feature descriptors are calculated from those normalized patches; and finally, the interest points of two images are matched by calculating the “distance” between their descriptors and by solving an assignment problem.

The SKB feature extraction algorithm is well suited for a hardware implementation because it only requires the evaluation of a small number of filter kernels. Further, those filter responses are thresholded in order to yield a binary descriptor, which is small in size and can be matched very efficiently using the Hamming distance.

In this semester project we developed a hybrid ASIC/FPGA system which implements the first three steps of this process. Our system is able to extract SKB features from one 720p video stream in real time (30 frames/s). Computationally intensive parts have been put on the ASIC, and the video interface and memory oriented parts on the FPGA. The target technology of the ASIC is UMC180L and the FPGA is an Altera Cyclone IV E (EP4CE115).

Status: Completed

Fall Semester 2012 (sem12h3, sem12h10)

Lukas Cavigelli

Pascal Hager

Oliver With

Supervisors: Michael Schaffner, Pierre Greisen

Professor

Hubert Kaeslin

Partners

Disney Research Zurich

Publications

• M. Schaffner, P. Hager, L. Cavigelli, P. Greisen, F. K. Gürkaynak, H. Kaeslin, "A Real-Time 720p Feature Extraction Core Based on Semantic Kernels Binarized", Proceedings of the International Conference on Very Large Scale Integration (VLSI-SoC), Istanbul, Turkey, 2013

Links

• Chip gallery page of Sandstorm.

↑ top