This PDF file contains the front matter associated with SPIE Proceedings Volume 9393, including the Title Page, Copyright information, Table of Contents, Authors, and Conference Committee listing.

In this paper, we propose a new approach for object matching in videos. By applying our novel descriptor on points of interest we obtain point descriptors or signatures. Points of interest are extracted from the object using a simple color Harris detector. This novel descriptor is issued from a rotating filtering stage. The rotating filtering stage is made of oriented anisotropic half-Gaussian smoothing convolution kernels. Further, the dimension of our descriptor can be controlled by varying the angle of the rotating filter by small steps. Our descriptor with a dimension as small as 36 can give a matching performance similar to that of the well-known SIFT descriptor. The small dimension of our descriptor is the main motivation for extending the matching process to videos. By construction, our descriptor is not Euclidean invariant, hence we achieve Euclidean invariance by FFT correlation between the two signatures. Moderate deformation invariance is achieved using Dynamic Time Warping (DTW). Then, using a cascade verification scheme we improve the robustness of our matching method. Eventually, our method is illumination invariant, rotation invariant, moderately deformation invariant and partially scale invariant.

In this paper, we present a method for temporal propagation of depth data that is available for so called key-frames through video sequence. Our method requires that full frame depth information is assigned. Our method utilizes nearest preceding and nearest following key-frames with known depth information. The propagation of depth information from two sides is essential as it allows to solve most occlusion problems correctly. Image matching is based on the coherency sensitive hashing (CSH) method and is done using image pyramids. Disclosed results are compared with temporal interpolation based on motion vectors from optical flow algorithm. The proposed algorithm keeps sharp depth edges of objects even in situations with fast motion or occlusions. It also handles well many situations, when the depth edges don’t perfectly correspond with true edges of objects.

When a picture is shot all the information about the distance between the object and the camera gets lost. Depth estimation from a single image is a notable issue in computer vision. In this work we present a hardware and software framework to accomplish the task of 3D measurement through structured light. This technique allows to estimate the depth of the objects, by projecting specific light patterns on the measuring scene. The potentialities of the structured light are well-known in both scientific and industrial contexts. Our framework uses a picoprojector module provided by STMicroelectronics, driven by the designed software projecting time- multiplexing Gray code light patterns. The Gray code is an alternative method to represent binary numbers, ensuring that the hamming distance between two consecutive numbers is always one. Because of this property, this binary coding gives better results for depth estimation task. Many patterns are projected at different time instants, obtaining a dense coding for each pixel. This information is then used to compute the depth for each point in the image. In order to achieve better results, we integrate the depth estimation with the inverted Gray code patterns as well, to compensate projector-camera synchronization problems as well as noise in the scene. Even though our framework is designed for laser picoprojectors, it can be used with conventional image projectors and we present the results for this case too.

Three-dimensional (3-D) meshes are already profusely used in lot of domains. In this paper, we propose a new high capacity data hiding scheme for vertex cloud. Our approach is based on very small displacements of vertices, that produce very low distortion of the mesh. Moreover this method can embed three bits per vertex relying only on the geometry of the mesh. As an application, we show how we embed a large binary logo for copyright purpose.

Locksmith forensics is an important and very challenging part of classic crime scene forensics. In prior work, we propose a partial transfer to the digital domain, to effectively support forensic experts and present approaches for a full process chain consisting of five steps: Trace positioning, 2D/3D acquisition with a confocal 3D laser scanning microscope, detection by segmentation, trace type determination, and determination of the opening method. In particular the step of trace segmentation on high-resolution 3D surfaces thereby turned out to be the part most difficult to implement. The reason for that is the highly structured and complex surfaces to be analyzed. These surfaces are cluttered with a high number of toolmarks, which overlap and distort each other. In Clausing et al., we present an improved approach for a reliable segmentation of relevant trace regions but without the possibility of separating single traces out of segmented trace regions. However, in our past research, especially features based on shape and dimension turned out to be highly relevant for a fully automated analysis and interpretation. In this paper, we consequently propose an approach for this separation. To achieve this goal, we use our segmentation approach and expand it with a combination of the watershed algorithm with a graph-based analysis. Found sub-regions are compared based on their surface character and are connected or divided depending on their similarity. We evaluate our approach with a test set of about 1,300 single traces on the exemplary locking cylinder component ’key pin’ and thereby are able of showing the high suitability of our approach.

The reconstruction of broken artifacts is a common task in archeology domain; it can be supported now by 3D data acquisition device and computer processing. Many works have been dedicated in the past to reconstructing 2D puzzles but very few propose a true 3D approach. We present here a complete solution including a dedicated transportable 3D acquisition set-up and a virtual tool with a graphic interface allowing the archeologists to manipulate the fragments and to, interactively, reconstruct the puzzle. The whole lateral part is acquired by rotating the fragment around an axis chosen within a light sheet thanks to a step-motor synchronized with the camera frame clock. Another camera provides a top view of the fragment under scanning. A scanning accuracy of 100μm is attained. The iterative automatic processing algorithm is based on segmentation into facets of the lateral part of the fragments followed by a 3D matching providing the user with a ranked short list of possible assemblies. The device has been applied to the reconstruction of a set of 1200 fragments from broken tablets supporting a Latin inscription dating from the first century AD.

Stereo matching is a fundamental topic in computer vision. Usually, stereo matching is mainly composed of four stages: cost computation, cost aggregation, disparity optimization and disparity refinement. In this paper, we propose a novel stereo matching method with space-constrained cost aggregation and segmentation-based disparity refinement. Stateof- the-art methods are used for cost aggregation and disparity optimization stages. Three technical contributions are given in this paper. First, applying space-constrained cross-region in cost aggregation stage; second, utilizing both color and disparity information in image segmentation; third, using image segmentation and occlusion region detection to aid disparity refinement. The performance of our platform ranks second in the Middlebury evaluation.

Structured light is a robust and accurate method for 3D range imaging in which one or more light patterns are projected onto the scene and observed with an off-axis camera. Commercial sensors typically utilize DMD- or LCD-based LED projectors, which produce good results but have a number of drawbacks, e.g. limited speed, limited depth of focus, large sensitivity to ambient light and somewhat low light efficiency.

We present a 3D imaging system based on a laser light source and a novel tip-tilt-piston micro-mirror. Optical interference is utilized to create sinusoidal fringe patterns. The setup allows fast and easy control of both the frequency and the phase of the fringe patterns by altering the axes of the micro-mirror. For 3D reconstruction we have adapted a Dual Frequency Phase Shifting method which gives robust range measurements with sub-millimeter accuracy.

The use of interference for generating sine patterns provides high light efficiency and good focusing properties. The use of a laser and a bandpass filter allows easy removal of ambient light. The fast response of the micro-mirror in combination with a high-speed camera and real-time processing on the GPU allows highly accurate 3D range image acquisition at video rates.

Although a lot of progress has been made in the development of 2D objective video quality metrics, the area of 3D video quality metrics is still in its infancy. Many of the proposed metrics are simply adaptations of 2D quality metrics that consider the depth channel as an extra color channel. In this paper, we propose a 3D no-reference objective quality metric that estimates 3D quality taking into account spatial distortions, excessive disparity, depth representation and temporal information of the video. The metric is resolution and frame-rate independent. To estimate the amount of spatial distortion in the video, the proposed metric uses a blockiness metric. The contribution of motion and excessive disparity to 3D quality is calculated using a non-linear relative disparity measure and a frame-rate proportional motion measure. The metric's performance is verified against the COSPAD1 database. The MOS predicted using the proposed metric obtained good correlation values with the subjective scores. The performance was on average better than the performance of two simple 2D full reference metrics: SIMM and PSNR.

The 3D acquisition-compression-processing chain is, most of the time, sequenced into independent stages. As resulting, a large amount of 3D points are acquired whatever the geometry of the object and the processing to be done in further steps. It appears, particularly in mechanical part 3D modeling and in CAD, that the acquisition of such an amount of data is not always mandatory. We propose a method aiming at minimizing the number of 3D points to be acquired with respect to the local geometry of the part and therefore to compress the cloud of points during the acquisition stage. The method we propose is based on a new coarse to …ne approach in which from a coarse set of 2D points associated to the local normals the 3D object model is segmented into a combination of primitives. The obtained model is enriched where it is needed with new points and a new primitive extraction stage is performed in the re…ned regions. This is done until a given precision of the reconstructed object is attained. It is noticeable that contrary to other studies we do not work on a meshed model but directly on the data provided by the scanning device.

3D object shapes (represented by meshes) include both areas that attract the visual attention of human observers and others less or not attractive at all. This visual attention depends on the degree of saliency exposed by these areas. In this paper, we propose a technique for detecting salient regions in meshes. To do so, we define a local surface descriptor based on local patches of adaptive size and filled with a local height field. The saliency of mesh vertices is then defined as its degree measure with edges weights computed from adaptive patch similarities. Our approach is compared to the state-of-the-art and presents competitive results. A study evaluating the influence of the parameters establishing this approach is also carried out. The strength and the stability of our approach with respect to noise and simplification are also studied.

This paper presents a new pre-processing algorithm for Time-of-Flight (TOF) depth map denoising. Typically, denoising algorithms use the raw depth map as it comes from the sensor. Systematic artifacts due to the measurement principle are not taken into account which degrades the denoising results. For phase measurement TOF sensing, a major artifact is observed as salt-and-pepper noise caused by the measurement’s ambiguity. Our pre-processing algorithm is able to isolate and unwrap affected pixels deploying the physical behavior of the capturing system yielding Gaussian noise. Using this pre-processing method before applying the denoising step clearly improves the parameter estimation for the denoising filter together with its final results.

In this paper, we describe a consideration of a method of a camera calibration for TOF depth camera in the case of using with color video camera to combine their images into colored 3D models of a scene. Mainly, there are two problems with the calibration to combine them. One is stability of the TOF measurements, and another is deviation between the measured depth values and actual distances that are based on a geometrical camera model. To solve them, we propose a calibration method for it. At first, we estimate an optimum offset distance and intrinsic parameters for the depth camera to match both the measured depth value and its ideal one. Using estimated offset to consecutive frames and compensating the measured values to the actual distance values each frame, we try to remove the difference of camera models and suppress the noise as temporal variation For the estimation, we used the Zhang’s calibration method for the intensity image from the depth camera and the color video image of a chessboard pattern. Using this method, we can get the 3D models which are matched between depth and color information correctly and stably. We also explain effectiveness of our approach by showing several experimental results.

In this paper, we present a free viewpoint video generation system with billboard representation for soccer games. Free viewpoint video generation is a technology that enables users to watch 3-D objects from their desired viewpoints. Practical implementation of free viewpoint video for sports events is highly demanded. However, a commercially acceptable system has not yet been developed. The main obstacles are insufficient user-end quality of the synthesized images and highly complex procedures that sometimes require manual operations. In this work, we aim to develop a commercially acceptable free viewpoint video system with a billboard representation. A supposed scenario is that soccer games during the day can be broadcasted in 3-D, even in the evening of the same day. Our work is still ongoing. However, we have already developed several techniques to support our goal. First, we captured an actual soccer game at an official stadium where we used 20 full-HD professional cameras. Second, we have implemented several tools for free viewpoint video generation as follow. In order to facilitate free viewpoint video generation, all cameras should be calibrated. We calibrated all cameras using checker board images and feature points on the field (cross points of the soccer field lines). We extract each player region from captured images manually. The background region is estimated by observing chrominance changes of each pixel in temporal domain (automatically). Additionally, we have developed a user interface for visualizing free viewpoint video generation using a graphic library (OpenGL), which is suitable for not only commercialized TV sets but also devices such as smartphones. However, practical system has not yet been completed and our study is still ongoing.

Observing cloud lifecycles and obtaining measurements on cloud features are significant problems in atmospheric cloud research. Scanning radars have been the most capable instruments to provide such measurements, but they have shortcomings when it comes to spatial and temporal resolution. High spatial and temporal resolution is particularly important to capture the variations in developing convections. Stereo photogrammetry can complement scanning radars with the potential to observe clouds as distant as tens of kilometers and to provide high temporal and spatial resolution, although it comes with the calibration challenges peculiar to various outdoor settings required to collect measurements on atmospheric clouds. This work explores the use of stereo photogrammetry in atmospheric cloud research, focusing on tracking vertical motion in developing convections. Calibration challenges and strategies to overcome these challenges are addressed within two different stereo settings in Miami, Florida and in the plains of Oklahoma. A feature extraction and matching algorithm is developed and implemented to identify cloud features of interest. A two-level resolution hierarchy is exploited in feature extraction and matching. 3D positions of cloud features are reconstructed from matched pixel pairs, and cloud tops of developing turrets in shallow to deep convection are tracked in time to estimate vertical accelerations. Results show that stereophotogrammetry provides a useful tool to observe cloud lifecycles and track the vertical acceleration of turrets exceeding 10 km height.

Conventional stereo matching methods provide the unsatisfactory results for stereo pairs under uncontrolled environments such as illumination distortions and camera device changes. A majority of efforts to address this problem has devoted to develop robust cost function. However, the stereo matching results by cost function cannot be liberated from a false correspondence when radiometric distortions exist. This paper presents a robust stereo matching approach based on probabilistic Laplacian propagation. In the proposed method, reliable ground control points are selected using weighted mutual information and reliability check. The ground control points are then propagated with probabilistic Laplacian. Since only reliable matching is propagated with the reliability of GCP, the proposed approach is robust to a false initial matching. Experimental results demonstrate the effectiveness of the proposed method in stereo matching for image pairs taken under illumination and exposure distortions.

This paper presents a probabilistic optimization approach to enhance the resolution of a depth map. Conventionally, a high-resolution color image is considered as a cue for depth super-resolution under the assumption that the pixels with similar color likely belong to similar depth. This assumption might induce a texture transferring from the color image into the depth map and an edge blurring artifact to the depth boundaries. In order to alleviate these problems, we propose an efficient depth prior exploiting a Gaussian mixture model in which an estimated depth map is considered to a feature for computing affinity between two pixels. Furthermore, a fixed-point iteration scheme is adopted to address the non-linearity of a constraint derived from the proposed prior. The experimental results show that the proposed method outperforms state-of-the-art methods both quantitatively and qualitatively.

In the last few years, due to the growing use of stereoscopic images, much effort has been spent by the scientific community to develop algorithms for stereoscopic image compression. Stereo images represent the same scene from two different views, and therefore they typically contain a high degree of redundancy. It is then possible to implement some compression strategies devoted to exploit the intrinsic characteristics of the two involved images that are typically embedded in a MPO (Multi Picture Object) data format. MPO files represents a stereoscopic image by building a list of JPEG images. Our previous work introduced a simple block-matching approach to compute local residual useful to reconstruct during the decoding phase, stereoscopic images that maintain high perceptual quality; this allows to the encoder to force high level of compression at least for one of the two involved images. On the other hand the matching approach, based only on the similarity of the blocks, results rather inefficient. Starting from this point, the main contribution of this paper focuses on the improvement of both matching step effectiveness and its computational cost. Such alternative approach aims to greatly enhance matching step by exploiting the geometric properties of a pair of stereoscopic images. In this way we significantly reduce the complexity of the method without affecting results in terms of quality.

Time-of-Flight cameras have become one of the most widely-spread low-cost 3D-sensing devices. Most of them do not actually measure the time the light needs to hit an object and come back to the camera, but the difference of phase with respect to a reference signal. This requires special pixels with complex spatial structure, such as PMD pixels, able to sample the cross-correlation function between the incoming signal, reflected by the scene, and the reference signal. The complex structure, together with the presence of in-pixel electronics and the need for a compact readout circuitry for both pixel channels, suggests that systematic crosstalk effects will come up in this kind of devices. For the first time, we take profit of recent results on subpixel spatial responses of PMD pixels to detect and characterize crosstalk occurrences. Well-defined crosstalk patterns have been identified and quantitatively characterized through integration of the inter-pixel spatial response over each sensitive area. We cast the crosstalk problem into an image convolution and provide deconvolution kernels for cleaning PMD raw images from crosstalk. Experiments on real PMD raw images show that our results can be used to undo the lowpass filtering caused by crosstalk in high contrast image areas. The application of our kernels to undo crosstalk effects leads to reductions of the depth RMSE up to 50% in critical areas.

In this paper a procedure for crosstalk (CT) measurements on spatial-multiplexed multi-user autostereoscopic 3D displays with so-called viewing distance control (VDC) is presented. VDC makes use of a rendering method which allows shifting of the viewing distance for multiview displays by using a novel distribution of the content at sub-pixel level. Methods for CT measurements to date cannot be used as the measurements have to be executed at distances that are not defined in the standard procedures for stereoscopic displays. The measuring procedures used so far are not applicable, as neither a measurement process nor any test images are defined for the use at different viewing distances. As separate CT-measurement specifications for two-view and multiview autostereoscopic displays already exist, the authors propose a unified measurement process. This process is supposed to utilize both, the equipment, as well as the physical arrangement of measuring subject and instrument that are used so far. It has to be considered that, due to the basic functional principles, several quality measurement and evaluation criteria for 3D displays have emerged. Different autostereoscopic display technologies lead to different measurement procedures. A unified method for analyzing image quality features in 3D displays, requiring no enhanced effort but offering comparable results, is desirable.

Time of Flight (ToF) imaging is one of the most used range image acquisition method. It is generally based on synchronous detection of a modulated light coming back from the target; this light being emitted by a controlled emitting device. Many commercial devices exist using ad hoc designed camera. One drawback is their low spatial resolution due to low resolution of the dedicated camera (typically 320x240). We propose to overcome this limitation by using a standard of the shelf camera. This can be achieved if the same light modulator is used both for direct and re‡ected light so that the light sensor has only to measure a mean intensity on a number of periods. Our proposition is based on Pockels cell modulator. A ToF imaging set-up based on it is designed and its performance are studied from a dedicated simulation software we have developed. The system we propose allows to extract a range image directly from the camera by computing the ratio of two successive images each one being shot with a known modulation frequency. The spatial resolution will be the same as the one of the camera and the imaging rate the half.

Understanding and evaluation of potential evidence, as well as evaluation of automated systems for forensic examinations currently play an important role within the domain of digital crime scene analysis. The application of 3D sensing and pattern recognition systems for automatic extraction and comparison of firearm related tool marks is an evolving field of research within this domain. In this context, the design and evaluation of rotation-invariant features for use on topography data play a particular important role. In this work, we propose and evaluate a 3D imaging system along with two novel features based on topography data and multiple profile-measurement-lines for automatic matching of firing pin shapes. Our test set contains 72 cartridges of three manufactures shot by six different 9mm guns. The entire pattern recognition workflow is addressed. This includes the application of confocal microscopy for data acquisition, preprocessing covers outlier handling, data normalization, as well as necessary segmentation and registration. Feature extraction involves the two introduced features for automatic comparison and matching of 3D firing pin shapes. The introduced features are called ‘Multiple-Circle-Path’ (MCP) and ‘Multiple-Angle-Path’ (MAP). Basically both features are compositions of freely configurable amounts of circular or straight path-lines combined with statistical evaluations. During the first part of evaluation (E₁), we examine how well it is possible to differentiate between two 9mm weapons of the same mark and model. During second part (E₂), we evaluate the discrimination accuracy regarding the set of six different 9mm guns. During the third part (E₃), we evaluate the performance of the features in consideration of different rotation angles. In terms of E₁, the best correct classification rate is 100% and in terms of E₂ the best result is 86%. The preliminary results for E₃ indicate robustness of both features regarding rotation. However, in future work these results have to be validated using an enlarged test set.

In order to ensure safety, long term stability and quality control in modern tunneling operations, the acquisition of geotechnical information about encountered rock conditions and detailed installed support information is required. The limited space and time in an operational tunnel environment make the acquiring data challenging. The laser scanning in a tunneling environment, however, shows a great potential.

The surveying and mapping of tunnels are crucial for the optimal use after construction and in routine inspections. Most of these applications focus on the geometric information of the tunnels extracted from the laser scanning data. There are two kinds of applications widely discussed: deformation measurement and feature extraction.

The traditional deformation measurement in an underground environment is performed with a series of permanent control points installed around the profile of an excavation, which is unsuitable for a global consideration of the investigated area. Using laser scanning for deformation analysis provides many benefits as compared to traditional monitoring techniques. The change in profile is able to be fully characterized and the areas of the anomalous movement can easily be separated from overall trends due to the high density of the point cloud data. Furthermore, monitoring with a laser scanner does not require the permanent installation of control points, therefore the monitoring can be completed more quickly after excavation, and the scanning is non-contact, hence, no damage is done during the installation of temporary control points.

The main drawback of using the laser scanning for deformation monitoring is that the point accuracy of the original data is generally the same magnitude as the smallest level of deformations that are to be measured. To overcome this, statistical techniques and three dimensional image processing techniques for the point clouds must be developed.

For safely, effectively and easily control the problem of Over Underbreak detection of road and solve the problemof the roadway data collection difficulties, this paper presents a new method of continuous section extraction and Over Underbreak detection of road based on 3D laser scanning technology and image processing, the method is divided into the following three steps: based on Canny edge detection, local axis fitting, continuous extraction section and Over Underbreak detection of section. First, after Canny edge detection, take the least-squares curve fitting method to achieve partial fitting in axis. Then adjust the attitude of local roadway that makes the axis of the roadway be consistent with the direction of the extraction reference, and extract section along the reference direction. Finally, we compare the actual cross-sectional view and the cross-sectional design to complete Overbreak detected. Experimental results show that the proposed method have a great advantage in computing costs and ensure cross-section orthogonal intercept terms compared with traditional detection methods.

In this paper, we propose an efficient framework for edge-preserving stereo matching. Local methods for stereo matching are more suitable than global methods for real-time applications. Moreover, we can obtain accurate depth maps by using edge-preserving filter for the cost aggregation process in local stereo matching. The computational cost is high, since we must perform the filter for every number of disparity ranges if the order of the edge-preserving filter is constant time. Therefore, we propose an efficient iterative framework which propagates edge-awareness by using single time edge preserving filtering. In our framework, box filtering is used for the cost aggregation, and then the edge-preserving filtering is once used for refinement of the obtained depth map from the box aggregation. After that, we iteratively estimate a new depth map by local stereo matching which utilizes the previous result of the depth map for feedback of the matching cost. Note that the kernel size of the box filter is varied as coarse-to-fine manner at each iteration. Experimental results show that small and large areas of incorrect regions are gradually corrected. Finally, the accuracy of the depth map estimated by our framework is comparable to the state-of-the-art of stereo matching methods with global optimization methods. Moreover, the computational time of our method is faster than the optimization based method.

Three-dimensional content (3D) creation has received a lot of attention due to numerous successes of 3D entertainment. Accurate stereo correspondence is necessary for efficient 3D content creation. In this paper, we propose a disparity map estimation method based on stereo correspondence. The proposed system utilizes depth and stereo camera sets. While the stereo set carries out disparity estimation, depth camera information is projected to left and right camera positions using 3D transformation and upsampling is processed in accordance with the image size. The upsampled depth is used for obtaining disparity data of left and right positions. Finally, disparity data from each depth sensor are combined. In order to evaluate the proposed method, we applied view synthesis from the acquired disparity map. The experimental results demonstrate that our method produces more accurate disparity maps compared to the conventional approaches which use the single depth sensors.