Computer methods for 3D motion tracking in real-time

Boguslaw Rymut; Computer methods for 3D motion tracking in real-time; PhD Thesis; 2016
Supervisor: Bogdan Kwolek, DSc, PhD, Eng., Associate Prof
Keywords: model-based 3D motion tracking, parallel computing, computer vision

Abstract

This thesis is devoted to marker-less 3D human motion tracking in calibrated and synchronized multicamera systems. Pose estimation is based on a 3D model, which is transformed into the image plane and then rendered. Owing to elaborated techniques the tracking of the full body has been achieved in real-time via dynamic optimization or dynamic Bayesian filtering. The objective function of a particle swarm optimization algorithm and the observation model of a particle filter are based on matching between the rendered 3D models in the required poses and image features representing the extracted person. In such an approach the main part of the computational overload is associated with the rendering of 3D models in hypothetical poses as well as determination of value of objective function. Effective methods for rendering of 3D models in real-time with support of OpenGL as well as parallel methods for determining the objective function on the GPU were developed. Several variants of objective function using both software and hardware rendering were proposed and evaluated on real data. Methods for effective rendering of 3D models in OpenGL, as well as data mapping between OpenGL and CUDA were developed and evaluated. Programmable streams in OpenGL were designed and configured to achieve real-time rendering of considerable number of 3D models in desired poses. Methods for parallel execution of particle swarm optimization as well as objective function calculation were developed to achieve effective utilization of hardware resources and the best possible tracking accuracies and frequencies. The elaborated solutions permit 3D tracking of full body motion in real-time. Certain solutions to enable selection of the number of particles and the number of iterations in the PSO algorithm, which determine the number of processed frames per second, and which in turn determines the change in the pose between consecutive frames were investigated and proposed. They make it possible to achieve the lowest errors in real-time 3D motion tracking.

Media

PhD Thesis in PDF format
Citation in BibTeX format

Citation


@PhdThesis{RymutPhdThesis2016, author = {Bogusław Rymut}, title = {Komputerowe algorytmy ekstrakcji i śledzenia obiektów w czasie rzeczywistym}, year = {2016}, }

Full Body Motion Tracking in Monocular Images Using Particle Swarm Optimization

Bogusław Rymut and Tomasz Krzeszowski and Bogdan Kwolek: Full Body Motion Tracking in Monocular Images Using Particle Swarm Optimization; Int. Conf. on Artificial Intelligence and Soft Computing, 2012, Artificial Intelligence and Soft Computing, Lecture Notes in Computer Science, Vol. 7267, Springer Berlin Heidelberg, 2012, pp. 600-607.

Abstract

The estimation of full body pose in monocular images is a very difficult problem. In 3D-model based motion tracking the challenges arise as at least one-third of degrees of freedom of the human pose that needs to be recovered is nearly unobservable in any given monocular image. In this paper, we deal with high dimensionality of the search space through estimating the pose in a hierarchical manner using Particle Swarm Optimization. Our method fits the projected body parts of an articulated model to detected body parts at color images with support of edge distance transform. The algorithm was evaluated quantitatively through the use of the motion capture data as ground truth.

Citation

@incollection{
    title={Full Body Motion Tracking in Monocular Images Using Particle Swarm Optimization},
    year={2012},
    isbn={978-3-642-29346-7},
    booktitle={Artificial Intelligence and Soft Computing},
    volume={7267},
    series={Lecture Notes in Computer Science},
    editor={Rutkowski, Leszek and Korytkowski, Marcin and Scherer, Rafał and Tadeusiewicz, Ryszard and Zadeh, LotfiA. and Zurada, JacekM.},
    doi={10.1007/978-3-642-29347-4_70},
    url={http://dx.doi.org/10.1007/978-3-642-29347-4_70},
    publisher={Springer Berlin Heidelberg},
    author={Rymut, Bogusław and Krzeszowski, Tomasz and Kwolek, Bogdan},
    pages={600-607}
}

Parallel Appearance-Adaptive Models for Real-Time Object Tracking Using Particle Swarm Optimization

Boguslaw Rymut and Bogdan Kwolek: Parallel Appearance-Adaptive Models for Real-Time Object Tracking Using Particle Swarm Optimization; Int. Conf. on Computational Collective Intelligence – Technologies and Applications, 2011, Computational Collective Intelligence. Technologies and Applications, Lecture Notes in Computer Science, Vol. 6923, Springer Berlin Heidelberg, 2011, pp. 455-464.

Abstract

This paper demonstrates how appearance adaptive models can be employed for real-time object tracking using particle swarm optimization. The parallelization of the code is done using OpenMP directives and SSE instructions. We show the performance of the algorithm that was evaluated on multi-core CPUs. Experimental results demonstrate the performance of the algorithm in comparison to our GPU based implementation of the object tracker using appearance-adaptive models. The algorithm has been tested on real image sequences.

Citation


@incollection{ title={Parallel Appearance-Adaptive Models for Real-Time Object Tracking Using Particle Swarm Optimization}, author={Rymut, Boguslaw and Kwolek, Bogdan}, year={2011}, isbn={978-3-642-23937-3}, booktitle={Computational Collective Intelligence. Technologies and Applications}, volume={6923}, series={Lecture Notes in Computer Science}, editor={Jędrzejowicz, Piotr and Nguyen, NgocThanh and Hoang, Kiem}, pages={455-464} doi={10.1007/978-3-642-23938-0_46}, url={http://dx.doi.org/10.1007/978-3-642-23938-0_46}, publisher={Springer Berlin Heidelberg}, }

Face Tracking with Guaranteed Framerates on Mobile Phones

Boguslaw Rymut and Bogdan Kwolek: Face Tracking with Guaranteed Framerates on Mobile Phones; Image Processing and Communications Challenges 3, Advances in Intelligent and Soft Computing, Vol. 102, Springer Berlin Heidelberg, 2011, pp. 165-172.

Abstract

This paper addresses the problem of face tracking with guaranteed framerates at mobile devices. The frame rate of the computationally inexpensive algorithm is not affected by the image content. An ellipse with fixed orientation is used to model the head. The position and the size of the ellipse are determined with respect to intensity gradient near the edge of the ellipse and skin color probability in the ellipse’s interior. The tracking is achieved using particle swarm optimization. The experiments were done using Lenovo S10 netbook and Nokia N900 smart phone.

Media

Presentation in PDF format
Article preprint in PDF format

Citation


@incollection{ title={Face Tracking with Guaranteed Framerates on Mobile Phones}, author={Rymut, Boguslaw and Kwolek, Bogdan}, year={2011}, publisher={Springer Berlin Heidelberg} series={Advances in Intelligent and Soft Computing}, volume={102}, booktitle={Image Processing and Communications Challenges 3}, editor={Choraś, RyszardS.}, isbn={978-3-642-23153-7}, pages={165-172}, doi={10.1007/978-3-642-23154-4_19}, url={http://dx.doi.org/10.1007/978-3-642-23154-4_19} }

Face Tracking Using Adaptive Appearance Models and Convolutional Neural Network

Boguslaw Rymut and Bogdan Kwolek: Face Tracking Using Adaptive Appearance Models and Convolutional Neural Network; Int. Conf. on Hybrid Artificial Intelligence Systems, 2011, Hybrid Artificial Intelligent Systems, Lecture Notes in Computer Science, Vol. 6678, Springer Berlin Heidelberg, 2011, pp. 271-279.

Abstract

One inherent problem of online learning based trackers is drift consisting in a gradual accommodation of the tracker to non-targets. This paper proposes an algorithm that does not suffer from the template drift inherent in a naive implementation of the online appearance models. The tracking is done via particle swarm optimization algorithm built on adaptive appearance models. The convolutional neural network based face detections are employed to support the re-diversification of the swarm in the course of the tracking. Such candidate solutions vote simultaneously towards true location of the face through correcting the fitness function. In particular, the hybrid algorithm has better recovery capability in case of tracking failure.

GPU-Accelerated Object Tracking Using Particle Filtering and Appearance-Adaptive Models

Bogusław Rymut and Bogdan Kwolek: GPU-Accelerated Object Tracking Using Particle Filtering and Appearance-Adaptive Models; Image Processing and Communications Challenges 2, Advances in Intelligent and Soft Computing, Vol. 84, Springer Berlin Heidelberg, 2010, pp. 337-344.

Abstract

In this work we present an object tracking algorithm running on GPU. The tracking is achieved by a particle filter using appearance-adaptive models. The main focus of our work is parallel computation of the particle weights. The tracker yields promising GPU/CPU speed-up. We demonstrate that the GPU implementation of the algorithm that runs with 256 particles is about 30 times faster than the CPU implementation. Practical implementation issues in the CUDA framework are discussed. The algorithm has been tested on freely available test sequences.

GPU-Supported Object Tracking Using Adaptive Appearance Models and Particle Swarm Optimization

Boguslaw Rymut and Bogdan Kwolek: GPU-Supported Object Tracking Using Adaptive Appearance Models and Particle Swarm Optimization; Int. Conf. on Computer Vision and Graphics, 2010, Computer Vision and Graphics, Lecture Notes in Computer Science, Vol. 6375, Springer Berlin Heidelberg, 2010, pp. 227-234.

Abstract

In this paper we present a particle swarm optimization (PSO) based approach for marker-less full body motion tracking. The objective function is smoothed in an annealing scheme and then quantized. This allows us to extract a pool of candidate best particles. The algorithm selects a global best from such a pool to force the PSO jump out of stagnation. Experiments on 4-camera datasets demonstrate the robustness and accuracy of our method. The tracking is conducted on 2 PC nodes with multi-core CPUs, connected by 1 GigE. This makes our system capable of accurately recovering full body movements with 14 fps.

Citation

GPU-Accelerated Human Motion Tracking Using Particle Filter Combined with PSO

GPU-Accelerated Object Tracking Using Particle Filtering and Appearance-Adaptive Models; Image Processing and Communications Challenges 2, Advances in Intelligent and Soft Computing, Vol. 84, Springer Berlin Heidelberg, 2010, pp. 337-344.

Abstract

In this work we present an object tracking algorithm running on GPU. The tracking is achieved by a particle filter using appearance-adaptive models. The main focus of our work is parallel computation of the particle weights. The tracker yields promising GPU/CPU speed-up. We demonstrate that the GPU implementation of the algorithm that runs with 256 particles is about 30 times faster than the CPU implementation. Practical implementation issues in the CUDA framework are discussed. The algorithm has been tested on freely available test sequences.

GPU-based object tracking

Boguslaw Rymut; GPU-based object tracking; Master’s Thesis; Rzeszow University of Technology; 2010
Supervisor: Bogdan Kwolek, DSc, PhD, Eng., Associate Prof
Keywords: image processing, object tracking, gpu programming, nvidia cuda, parallel programming

Abstract

The purpose of this master thesis was to propose the implementation of particle swarm optimization and particles filter methods for GPU class devices based on NVIDIA CUDA technology. The thesis presents issues related to programming applications in NVIDIA CUDA technology and methods of object tracking based on adaptive model using particle swarm optimization and particles filtering. his thesis also describes the construction of two demonstration applications: an implementation of particle swarm optimization and implementation of particles filter. In particular, the thesis demonstrated that it is possible to obtain significant GPU/CPU speed-up. The thesis presents the advantages and disadvantages of implementation of these algorithms on the GPU devices.

Media

Master’s Thesis in PDF format
Citation in BibTeX format

Citation


@MastersThesis{ author = {Bogusław Rymut}, title = {{GPU-based object tracking}}, school = {Rzeszow University of Technology}, address = {the Poland}, year = {2010}, }