Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Spider monkey optimisation assisted particle filter for robust object tracking

Spider monkey optimisation assisted particle filter for robust object tracking IntroductionObject tracking is one of the major research areas in computer vision. The idea is to first detect one or more targets, and then obtain a record of trajectory of those targets in the video sequence. Over the years, object tracking has created a niche for itself in computer vision applications ranging from vehicle tracking [1] and monitoring and surveillance systems [2], to drive assistance [3] and human tracking [4].However, the task of object tracking is suffused with challenges. Often little information is made available about the target. Even if the target information is given, the target appearance may change over time and space. Occlusion, background clutter, camouflaged foreground, and changing illumination in frame mar the efforts to achieve robust tracking. Furthermore, varying scale, rotating object, moving camera, and shadowing make the already difficult task more arduous.To efficiently tackle the difficulties arising in vision tracking, researchers have proposed algorithms that can be roughly categorised under deterministic approach or stochastic approach. Deterministic methods localise tracked object in each frame by iteratively searching for region which maximises the similarity between the tracked region and target. Though these methods are computationally efficient, they may converge to a local optima. Additionally, they are prone to background clutter and occlusion. Lately, [5, 6] have been proposed for tracking objects. These techniques use online learning mechanisms to track target. In [6], algorithm proposed divides the task of object tracking into three components: tracking, learning, and detection. A novel learning method has been proposed called P–N learning that estimates the errors in detection by a pair of experts named P‐expert (for estimating missed target detections) and N‐expert (for estimating false alarms).However, this research focusses mainly on the drawbacks of particle filters (PFs) (a stochastic approach) that limit their efficiency in tracking objects and how nature‐inspired optimisation techniques can overcome those drawbacks. PF was first reported in [7] as bootstrap filter. Over the years improved versions of PFs have been introduced as condensation filter [8], interacting particle approximation [9], and sequential importance sampling [10].PF represents the posterior PDF of system's state by random particles with associated weights and estimates the final state based on these particles. Keeping this in mind, PF views the tracking algorithm as a state estimation problem under Bayesian model and is considered to be one of the most useful tools to estimate Bayesian models with non‐linear and non‐Gaussian noise [10, 11].Nevertheless, increased efficiency of PF entails increased computation cost that can create a delay in applications involving real‐time systems. The increased dimensionality of state space adds to the computational burden associated with PFs.Apart from these challenges, particle degeneration is a common problem that limits the efficiency of PFs. Degeneracy is a phenomena in which after few iterations most of the particles have negligible weight and their contribution towards the state approximation is almost zero. Degeneracy causes wastage of computational efforts invested in updating particles having negligible weights. Two common methods to deal with degeneracy are:Resampling.Good choice of importance density.Resampling is a method by which particles with small weights are eliminated from the state vector estimate and are replaced with particles having large weights. The authors in [12–15] present systematic resampling, adaptive resampling, residual resampling, multinomial resampling, and stratified resampling methodologies, respectively. Nonetheless, these resampling methods partially solve the particle degeneracy problem. Instead, resampling creates a new problem of sample impoverishment. Resampling strategies lead to decreased diversity in the particle set. This results in poor construction of posterior PDF. Lately, more efficient approaches for resampling particles have been advanced by the researchers. Zuo [16] discusses a dynamic resampling (DR) method in which resampling operation is only performed on part of the particles in a step‐by‐step manner and the number of particles to be resampled is decided dynamically by a termination criterion based on the effective sample size. New born particles produced by resampling operation are helpful in alleviating degeneracy, whereas particles that are not resampled maintain the diversity. This technique has been used in this work to resample particles in spider monkey optimisation based PF (SMO‐PF). In [17], authors have introduced a novel resampling technique named as spline resampling that consists of two parts: spline transformation of weights and the spread transformation of states. The former is based on a spline transformation and achieves accurate particle filtering and latter is based on point spread transformation on states of particles to prevent sample impoverishment. Zhai and Yeary [18] proposes a PF that uses Markov Chain Monte Carlo method based Metropolis–Hastings algorithm to resample particles in the PF.To address the issue of sample impoverishment, researchers have also developed more accurate posterior distribution. In [19], authors have developed unscented PF that is a parametric/non‐parametric hybrid of unscented Kalman filter (UKF) and PF so that the PF provides the general probabilistic framework to handle non‐linearity and non‐Gaussianity and UKF generates better proposal distribution.Lately, nature‐inspired optimisation algorithms have captured great deal of attention of the researchers working on PF‐based object tracking. These algorithms provide strategies that guide the search process in the fitness space. The given paper proposes an object tracking algorithm based on PF and SMO algorithm that models the foraging behaviour of spider monkeys (SMs). The proposed work parallels the line of thought used in [20] and optimises the distribution of sample particles before the DR step. Extensive experiments were carried out to test the efficiency of the new algorithm. The algorithm was first tested on generic one‐dimensional (1D) state estimation problem, and then on bearings only tracking problem so as to provide a theoretical backing to novel state estimation method for object tracking. SMO‐PF using DR [SMO (DR)‐PF] was then tested on standard videos obtained from CAVIAR dataset, and indigenous video consisting of various object tracking challenges. The results were also obtained for simple PF, particle swarm optimisation assisted PF (PSO‐PF), SMO‐PF with normal resampling, and cuckoo search optimisation assisted PF (CS‐PF) to provide comparative analysis.The rest of the paper is organised as follows. Section 2 details some advancements made in the field of nature‐inspired optimisation assisted PFs. Section 3 delineates the PF algorithm and SMO algorithm. In Section 4, the SMO (DR)‐PF algorithm is discussed for 1D and 2D state estimation, and object tracking problem. Section 5 discusses the performance parameters used for evaluating the algorithms. In Section 6, results obtained from the new PF for the three problems are discussed in detail. Finally, the paper is concluded in Section 7.Related workNature‐inspired meta‐heuristics approach to optimise sample distribution in the state hypothesis obtained from PF has been an area of active research in recent years. All nature‐inspired optimisation methods share two common aspects: exploration and exploitation. Exploration is the ability of expanding search space, whereas exploitation is the ability of finding optima around a good solution [21]. First few iterations of optimisation algorithm are used for exploring the search space so as to avoid getting trapped in local optima. After some iterations, exploration dampens and exploitation surfaces and the system reaches a semi‐optimal solution. In case of PF, the combination results in concentration of particles in high probability area aiding in creation of effective posterior hypothesis.In [22], authors implemented ant colony optimisation (ACO) algorithm in PF to solve the sample impoverishment problem. ACO provides guaranteed convergence and can be used to optimise dynamic problems; time to convergence, however, is uncertain. A non‐linear economic model was employed to test the performance of the algorithm. The results were also obtained for other improved PFs for comparison. The research, however, did not test the algorithm for bearings only tracking problem and object tracking. In [23], authors used artificial fish swarm optimisation (AFSO) for optimising particle distribution before resampling step so as to centralise particles in high‐likelihood area. The algorithm was tested for manoeuvring target tracking. AFSO technique provides high convergence speed, high accuracy, and fault tolerance. However, it has high time complexity associated with it [24]. In [25], authors proposed a genetic algorithm (GA) filter wherein they replaced the prediction step with mutation and crossover operators in GA. Though GA is fast in exploring the search space and has low memory requirements, the solution obtained by GA can be sub‐optimal or, local maxima/minima.PSO provides various advantages over many swarm‐based optimisation techniques. The fitness function can be non‐differentiable and PSO rapidly produces quality solution for problems involving many dimensions. However, since there is no general optimisation technique for practical and multidimensional problems, tuning and experimenting with input parameters of PSO method is necessary. Additionally, for noisy non‐linear problems, stochastic variability of results obtained from the PSO method is very high. In [26], authors exploited capabilities of PSO and proposed a PF based on PSO to overcome particle deprivation problem, but its implementation did not cover object tracking problem. In [27], authors proposed a PSO assisted PF for object tracking in video sequences. They also exploited the ability of PF to fuse different cues and used the mixture of histogram of oriented gradients (HOG) and colour histogram to construct feature vector for target association. Later in [3], authors proposed an adaptive PSO assisted PF called APSO‐PF to track manoeuvring object. The standard parameters in PSO‐PF were made adaptive and were changed according to motion state of the object.CS optimisation can effectively deal with multimodal problems and requires tuning of only single parameter. Walia and Kapoor [28] proposed a novel PF that embeds cuckoo search via levy flight to estimate the state of generic 1D state estimation problem and classic bearings only 2D state estimation problem. The new algorithm surpassed standard PF and PSO‐PF in terms of root mean squared error (RMSE) and number of effective particles (NOEP). Later in [29], authors detailed the implementation of improved CS‐based PF for object tracking. Comprehensive qualitative and quantitative analysis was done and the new algorithm was proved to be more reliable. In optimisation problems with high level of noise and non‐linear fitness function, firefly optimisation algorithm (FA) outperforms PSO in finding the optimal solution and time taken to reach that optimum solution. In [30], authors proposed a novel PF based on FA to increase the number of meaningful particles for better approximation of state vector. However, FA sometimes get trapped in the local optima and is unable to completely get rid of it. Results were obtained for different video sequences consisting of various object tracking challenges. The new algorithm outperformed standard PF for tracking motile object. In [31], a general optimisation‐based tracking architecture based on FA is proposed. The speed and accuracy of new FA‐based tracking algorithm is compared with standard PF, meanshift, and PSO.In [20], it was observed that SMO successfully overcame the problem of premature convergence and stagnation in the search space experienced by many state‐of‐the‐art nature‐inspired meta‐heuristic optimisation techniques. SMO performed better than artificial bee colony optimisation, PSO, differential evolution, and covariance matrix adaption‐evolution strategy for different benchmark problems.Particle filtering and SMOThis section discusses the PF and SMO algorithm in detail. Additionally, the methodology of incorporating optimisation algorithm within a PF has been also expounded.Particle filterThe key idea of particle filtering is to represent required PDF of state pxt|Zt, where Zt=z1,z2,…,zt is the set of all available observations and xt is the state estimate at time t, by set of random particles with associated weights. It is assumed that initial PDF pxo|zo is known a priori.Particle filtering consists of two steps:Prediction of state using motion model.Update state using measurement model.These steps are applied recursively to obtain PDF pxt|Zt.The prediction step uses the motion model xt=ft(xt−1,wt−1) to obtain the prior PDF at time t, provided pxt−1|Zt−1 is known, where Zt−1=z1,z2,…,zt−1 is set of previously observed states up till time t−1. Here, ft is a non‐linear function of target state and wt−1 is zero‐mean white noise sequence that is used to model unknown disturbances in the state prediction process. This model defines the translation, rotation, scale, and other state parameters of the target.Online measurements are made according to model zt=htxt,ut, here ht is a non‐linear function and ut is a zero‐mean white noise sequence. The measurement zt forms the basis of the update step and is used to update the predicted state using the Bayesian rule as follows:1pxt|z1:zt=pzt|xtpxt|z1:zt−1pzt|z1:zt−12pzt|z1:zt−1=∫pzt|xtpxt|z1:zt−1dxtAs stated, particle filtering approximates the state by a weighted particle set S={xtn,wtn,n=1,2,…,Ns}. Here, Ns is equal to total number of particles. Each particle depicts one hypothetical state xt of the target and has a weight wt that is obtained in terms of degree of its association with the observation. These weights are normalised such that ∑i=1Nswti=1.The phenomena of importance sampling comes into play to obtain this weighted approximation of posterior p(xt|Zt). It states that, if px∝π(x) is a probability density through which it is difficult to obtain particles, but for which πx can be evaluated, then a weighted approximation of the posterior PDF is given by3p(x)≃∑i=1Ns⁡wiδ(x−xi)Here, wi∝πxi/vxi is the normalised weight of the ith particle. x≃vx,i=1,2,…,Ns are the samples that are generated from a proposal v⋅ known as the importance density. These updated weighted approximation acts as prior for the next state and the process is repeated until last state is achieved. However, after several iterations most of the particle's weight is almost negligible and NOEP, as expressed in (4), rapidly decreases. Resampling was introduced to tackle this problem4Neff=1/∑i=1N⁡(wki)2Spider monkey optimisationSMO is a nature‐inspired heuristic approach to solve optimisation problem. It is based on the foraging behaviour of SMs who fall under the category of fission‐fusion social structure (FFSS) based animals [20]. The FFSS‐based animals are social and live in groups of 40–50 individuals. The FFSS of swarm helps in reducing the competition amongst the group members by dividing them into smaller groups. A female global leader searches the food sources and divides the group into smaller sub‐groups in case she is unable to procure sufficient food for the group. These smaller sub‐groups then forage independently. These sub‐groups also have a female local leader who decides an efficient foraging route every day. Members of these sub‐groups communicate within and outside the sub‐group about availability of food resources.The SMO process used consists of three steps, and they are:Step 1: Initialisation of populationIn this step, Ns SMs, uniformly distributed, are generated. Each SM SMn{n=1,2,…,Ns} is a D‐dimensional vector, where D the number of state variables, and each SMn{n=1,2,…,Ns} is a potential state.Step 2: Local leader phase (LLP)Here each SM SMn updates its position based on the information of the local leader and corresponding group member experience. After obtaining the new position, fitness value of that position is obtained. If new fitness value is higher than the previous fitness value, then SMn, member of the lth local group, updates its position with the new one according to the following equation:5SMnewnd=SMnd+U0,1×LLld−SMnd+U−1,1×(SMrd−SMnd)SMnd is the dth dimension of nth SM, LLld represents the dth dimension of the lth local group leader position, and SMrd is the dth dimension of rth SM that is chosen randomly from lth local group such that r≠n, and U(0,1) is a uniformly distributed random number between 0 and 1. Fig. 1a shows the position update process used in LLP. L is the number of groups in total, N is the group size, and pr is the perturbation rate that controls the magnitude of perturbation in current state.Step 3: Global leader phase (GLP)1Fig.SMO algorithm for object tracking(a) Optimisation algorithm used to improve particle distribution, (b) Improved particle distribution using SMOAll SMs update the position based on global leader experience and local group member experience. The position is updated according to the following equation:6SMnewnd=SMnd+U0,1×GLd−SMnd+U−1,1×(SMrd−SMnd)GLd represents the dth dimension of the global leader position and d={1,2,…,D} is a randomly chosen index.In this step, position of SMs (SMn) is updated according to probability pn that is calculated according to the following equation:7pn=0.9×fitnessnmax_fitness+0.1Here, fitnessn is fitness of nth SM and max_fitness is the maximum fitness in the group. By assigning probability to a SM, it is made sure that good candidates have higher chances to make themselves better. Furthermore, new fitness values are compared with old fitness values and if new value is greater than old values, SMn updates its position. Fig. 1a also discusses the algorithm for GLP.SMO‐based PFSMO (DR) algorithm is used in conjunction with PF to improve the efficiency of PF. First, the initial state PDF is obtained from the resampled particles of the previous time step. Next, motion model and measurement model (described in Section 4) are used to obtain the predicted PDF p(xt|zt). With the help of a likelihood function, described in [4] ((25)), a weighted approximation of the state PDF, p1(x)≃∑i=1Nswiδ(x−xi), is obtained wherein weight is assigned to a particle corresponding to the degree of its association (as measured with the measurement model) with the true state. After obtaining a weighted approximation of the state PDF, SMO algorithm is used to improve the weights of the particle. Every particle in the PF is treated as a SM. The set of particles with NsSMs is divided into L local groups with each group having N SMs. The particle with highest fitness value/weight in the set is treated as the global leader and the particle with highest weight in each local group is treated as the local leader. Next, LLP phase is applied that helps the particles to explore the fitness space with high perturbation rate pr. This helps to overcome the problem of stagnation in the fitness space experienced by many optimisation techniques. After the LLP phase, GLP phase is applied that allows the particles with high weights to update their position and further improve their fitness values. As can be noticed from Fig. 1b, particles having low weights before SMO step have increased fitness values and are more close to the global leader. Also, the global leader has further improved its weight value. After obtaining weighted approximation via SMO, resampling step is applied to obtain the final PDF of the state. All in all, the optimisation step, in combination with DR, increases the diversity and quality of particles, and helps in effectively modelling the posterior PDF of the target.One‐dimensional state estimation, bearings only tracking, and object tracking by SMO (DR)‐PFIn this section, SMO (DR)‐PF algorithm used for generic 1D state estimation problem, bearing only tracking (2D state estimation) problem, and object tracking problem is discussed in detail. This section also discusses the motion models, measurement models and other implementation details.One‐dimensional state estimation problem and 2D state estimation problemOne‐dimensional non‐linear problemThe following non‐linear model was considered to evaluate the performance of SMO (DR)‐PF:8xk=0.5xk−1+25xk−11+xk−12+8cos1.2k−1+wk9zk=xk220+ukThe above model was reported in [32]. It is a highly non‐linear model, both in system (8) and the measurement (9) equation. Here wk and uk are zero‐mean Gaussian white noise with variance 10.0 and 1.0, respectively.Bearings only tracking problemIn this problem, the target's motion is governed by second‐order model represented by10xk=αxk−1+ρwkxk=(x,x.,y,y.)kTandwk=(wx,wy)kTHere, x and y denote Cartesian coordinates of the target and x. and y. represent velocity of the target. wk denotes system noise and is zero‐mean Gaussian white noise process with covariance C:EwkwjT=Cδjk, where C=cI2 and I2 is 2×2 identity matrix. Measurement used to track target is bearing or angle zk and is governed by measurement11zk=tan−1ykxk+ukHere, uk is measurement noise and is zero‐mean Gaussian white noise with variance r:Eukuj=rδjk. Before taking measurements at k = 1, the initial state vector is assumed to have a Gaussian distribution with known mean x1¯ and covariance C1=diag(σ12,σ22,σ32,σ42).Fig. 2a presents the algorithm that has been implemented for 1D non‐linear state estimation problem and bearings only tracking 2D state estimation problem.2Fig.SMO assisted PF algorithm(a) For 1D and 2D state estimation problem, (b) For object tracking problemSMO (DR)‐PF for object trackingThis subsection discusses the algorithm implemented (Fig. 2b) for object tracking in standard videos, obtained from CAVIAR dataset [33], and synthetic videos.First, the acquired frame is searched for the object to be tracked. This is done with the help of Gaussian mixture model (GMM) based foreground detection technique. In this method, intensity value of every pixel from the first few frames is modelled using mixture of Gaussian distributions. Aberrant pixel values that do not fit the background distribution are labelled as foreground. Number of Gaussian modes in mixture model used were 5. Algorithm presented in [34] was used to detect foreground object. Although GMM‐based detection allows for some variability in the states of the pixel, the model assumes that this variability is derived from noise and not structured motion patterns such as swaying trees, moving clouds, and so on. Another drawback of the used technique is the lack of consistency between the states of adjacent pixels that often leads to abysmal foreground–background segmentation [35]. As will be noticed in Section 6.3, these shortcomings limit the efficiency achieved through the proposed object tracking framework. However, PF's ability to maintain multiple hypothesis of target's state helps in overcoming these drawbacks to some extent.Thereafter, N particles are initialised to represent that state of object. After updating particles through (13), every particle is assigned a weight using the Bhattacharyya coefficient and weighted approximation of the state PDF is obtained. Next, these particles are divided into sub‐groups for implementing the LLP and GLP of SMO algorithm. This step reduces degeneracy in the particle set and increases NOEP. Then the newly generated particles are resampled using DR. Finally, the state of the target is estimated by taking weighted mean of resampled particles. The loop is continued until all states are predicted.Motion modelMotion of an object to be tracked can greatly vary from simple translation to complex rotation. Constant velocity model for translation motion and random walk model for the scaling and rotation have been considered. The used multicomponent model was proposed in [4].The complete state vector is described as12x=(x,x.,y,y.,θ,s)TThe state is updated according to the following equation:13xk+1=Gxk+wk,wk∼N(0,C)14G=1V0102×202×102×102x21V0102×102×101×201×21001×201×201Here wk is zero‐mean Gaussian white noise, V is the sampling interval, and C is the covariance matrix with covariance values for translation motion, rotation, and scaling as σx2,σy2,σθ2 and σs2, respectively.Measurement modelAfter updating the state, every particle is assigned a weight. The weights are proportional to the degree of association of reference feature with the target feature.This work uses colour histogram for computing weights of the particles, since colour histograms in general can accommodate small changes in the appearance of the object.Histogram, ht=(h1,t,h2,t,…,hn,t) for a region St corresponding to state t is given by15hi,t=∑uϵSt⁡δi(bu),i=1,2,…,nHere, δi is the Kronecker‐delta function at the bin index i. bu∈1,…,n is the histogram bin index associated the intensity at pixel location u=(x,y) and n is the number of bins in every channel. Then histogram is normalised such that ∑1nhi,t=1. 8×8×8 Bin RGB histogram was constructed.After obtaining the target and reference histogram, similarity between the two histograms is measured. This manuscript uses Bhattacharyya distance for measuring similarity. Bhattacharyya distance (B.D.) has been previously used in [29, 30]16B.D.=1−B.C.Here, B.C. is Bhattacharyya coefficient and is given by17B.C.href,htgt=∑i=1n⁡href,i∗htgt,iHere, href and htgt are the colour histogram of reference and target region, respectively. If the colour histograms are similar then B.C. is 1 indicating a perfect match.Based on (16), distance D is defined which takes into consideration all three colour channels18Dhref,htgt=∑c∈(R,G,B}B.D.2(hrefc,htgtc)3Based on value of Dhref,htgt, weight is assigned to every particle.Performance evaluation parametersThis section discusses the performance parameters used for evaluating the performance of the novel algorithm.For 1D non‐linear problem and bearings only tracking problemA. RMSE: It is an excellent general purpose error measuring parameter and is defined as19RMSEx=∑i=1N(xi−x^i)2NForone−dimensionHere, xi is the true value of state and x^i is the predicted value of the state. N is total number of estimated states20RMSExy=∑i=1N(xi−x^i)2+(yi−y^i)2NFortwo−dimensionHere, (xi,yi) are the true location coordinates of the target in –xy plane, whereas (x^i,y^i) are the estimated location coordinates. N is total number of estimated states.B. Variance: Results were obtained for 25 iterations where 1 iteration comprises 50 states for 1D non‐linear problem, and 24 states for bearings only tracking problem. Variance depicts the stability of PF algorithm. Less the variance more stable is the performance of that algorithm.C. Time cost: The total time taken by the algorithm to obtain the estimates of all states.D. NOEP: ‘NOEP’ versus ‘state index’ graph was obtained to monitor the benefits brought by optimisation.E. Number of distinct particles (NODP): ‘NODP’ versus. ‘state index’ graph was obtained to monitor the benefits brought by DR.For object trackingA. RMSE (in pixels): The combined RMSE error of pixel coordinate (xi,yi) to the estimated coordinate (x^i,y^i) was used and is defined as (20). N = 10 is equal to the total number of iterations for which the algorithm was run. This process is repeated for every frame. ‘RMSE’ versus ‘frame number’ graph is obtained.B. F‐measure: To calculate F‐measure, it is paramount to understand recall and precision. Recall is the proportion of positive pixels that are correctly classified as target to be trackedRecall=numberofcorrecltyclassifiedtargetpixelsnumberoftargetpixelsinGTPrecision is the proportion of predicted positive pixels that were correctly identified as targetPrecision=numberofcorrecltyclassifiedtargetpixelsnumberoftargetpixelsestimatedValues of recall and precision are directly proportional to the performance of the algorithm; however, each performance evaluator can be misleading alone. Hence, their harmonic mean, F‐measure was introduced to accommodate both.F−measure=2×Recall×PrecisionRecall+PrecisionC. Success rate: It is the percentage of total frames for which target object is accurately localised.D. Time cost: It is the average amount of time (in seconds) taken to process one video frame of the video sequence to localise the target.E. NOEP: Same as mentioned for 1D and 2D state estimation problem.F. NODP: Same as mentioned for 1D and 2D state estimation problem.Results and discussionThis section will cover the performance analysis of SMO (DR)‐PF for 1D state estimation, 2D state estimation, and object tracking. Algorithms were implemented on MATLAB running on 1.4 GHz Dual Core Processor.Generic 1D non‐linear modelThe initial state was assumed to be xo = 0.1. The PFs were initialised with prior PDF pxo=N(0,2).Fig. 3a shows the 50 states estimated through different PFs. Green line represents the true state obtained via (8), whereas black line represents the estimated state via PFs. Number of particles was equal to 500. Optimisation process brought an increase in efficiency by decreasing the difference between the estimated state and the true state. Figs. 3b and c display the NOEP and NODP at each state. Table 1 compares the mean RMSE for 25 iterations, variance of these RMSE values, and time cost for the five algorithms. Variance (RMSE) suggests increased stability of the proposed algorithms. It can be observed from Fig. 3c that the DR step brought a significant increase in the NODP than the systematic resampling technique used for other algorithms. Also, it can be observed from Fig. 3b that there is not much difference between NOEP for SMO‐PF and SMO (DR)‐PF. Hence, it can be concluded that SMO overcomes the particle degradation problem, while DR overcomes the particle impoverishment problem. Additionally, it can be observed from Table 1 that SMO (DR)‐PF, though has higher RMSE than SMO‐PF (perhaps because of the large measurement and process noise in the system), performs better than other improved PFs.1TableMean RMSE, variance, and time cost for all PFs (1D non‐linear model)Mean RMSEVariance (RMSE)Time cost, sSimple‐PF5.421.760.23PSO‐PF4.921.390.84SMO‐PF3.620.681.02SMO(DR)‐PF4.650.520.96CS‐PF4.841.120.783Fig.Comparative performance evaluation(a) State estimation using simple PF, PSO‐PF, SMO‐PF, SMO (DR)‐PF, and CS‐PF (left to right), (b) Number of effective particles, (c) Number of distinct particlesBearings only trackingThe initial state of target was assumed to be x1=(−0.05,0.001,0.7,−0.055)T. About 500 particles were taken to estimate the state. The prior distribution parameters were set to x1¯=(0.0,0.0,0.4,−0.08)T, and σ1=0.5,σ2=0.005,σ3=0.3, and σ4=0.01. The target trajectory is presented by red line and true trajectory by black line.Fig. 4a presents the results 24 states. It can be observed that SMO‐PF closely follows the true trajectory, while others fail. Fig. 4b presents NOEP at 24 states. An interesting observation can be made from Fig. 4c. It can be seen that SMO (DR)‐PF maintains excellent diversity in its particle set. On further exploring, it was found that SMO step alone increased the effective weight of particles above the threshold (0.35×Total number of particles) and made the DR step redundant. Hence, no resampling step was required to increase effective weight of the particles. However, it can also be observed that RMSE of SMO (DR)‐PF is more than SMO‐PF. Table 2 compares the mean RMSE for 25 iterations, variance of these RMSE values, and time cost for the five algorithms. Low variance demonstrates stable performance of the proposed algorithm.2TableMean RMSE, variance, and time cost for all PFs (bearings only tracking)Mean RMSEVariance (RMSE)Time, sSimple PF0.200.00310.62PSO‐PF0.160.000540.75SMO‐PF0.060.000371.01SMO(DR)‐PF0.140.000300.98CS‐PF0.150.000450.694Fig.Comparative performance evaluation(a) State estimation using simple PF, PSO‐PF, SMO‐PF, SMO (DR)‐PF, and CS‐PF (left to right), (b) Number of effective particles, (c) Number of distinct particlesObject tracking problemThis sub‐section discusses the performance of the proposed method. Algorithms were tested on three videos. Two videos were taken from the standard CAVIAR dataset and one video was synthesised indigenously. Number of particles N used to estimate the state of the target was set to 150. PFing infuses randomness in the object tracking process. To accommodate this randomness, every PF‐based algorithm was run for ten times and results discussed are average of those observations. To maintain the even‐handedness, same target model and motion model have been used for all algorithms, but normal resampling was used for other algorithms to exhibit benefits of DR. Table 3 reports the parameters used for tracking process.3TableFeatures and parameters used for object tracking (bearings only tracking)Number of particles150Number of bins for each histogram channel8V,σx2,σy2,σs2,σθ2 for (13)0.9, 0.04, 0.04, 0.0045, 0.0004PSO parametersc1 = c2 = 2, number of iterations = 3SMO parameterspr = 0.8, L = 10, N = 15, number of iterations = 2DR parametersNth = 0.35×number of particlesCS parametersP (discovery of alien eggs) = 0.8, number of iterations = 2Processor1.4 GHz dual coreCamera speed23 fpsOneLeaveShopReenter1cor.mpgIn this video, the person with dark red shirt is tracked. The person walks towards the camera thereby inducing scale transformation. During the course of the video, illumination in the scene varies substantially. The target object is occluded from frame 18 to frame 36. The video sequence incorporates shadowing challenge.It can be noticed from Fig. 5d that SMO‐PF maintained better F‐measure than others. It can be further noticed that optimised PF (esp. SMO‐PF) maintained higher NOEP (Fig. 5b) during occlusion. Fig. 5c present the total positional RMSE. As it can be noticed, positional RMSE for SMO‐PF was lower than that for others (save few frames). Table 4 also compares the success rate and timing cost of the five algorithms. Fig. 5a presents some of the frames from the video sequence. Similar to the bearings only tracking problem, for SMO (DR)‐PF, SMO alone overcame the particle degradation problem and retained the diversity (Fig. 5e).4TableTime cost and success rate for OneLeaveShopReenter1cor.mpgTime cost, sSuccess rate, %Simple PF0.1564.17PSO‐PF0.3192.50SMO‐PF0.3699.17SMO(DR)‐PF0.3598.33CS‐PF0.3090.005Fig.Comparison of performance for five PFs on the basis of(a) Sample frames from OneLeaveShopReenter1cor.mpg, (b) Number of effective particles, (c) Positional RMSE (in pixels), (d) F‐measure, (e) Number of distinct particlesShopAssistant1cor.mpgIn this video, the person with white top is being tracked. The person walks away from the camera, thereby inducing scale changes. Change in velocity occurs in both directions. To make the sequence more challenging camouflaged background was used and the bottom of the target had significant similarity with clothes of the other person. The other person partially occluded the target in the frames 90–125. Additionally, as the target moves away, illumination changes and shadowing further complicate the task of tracking.Fig. 6a presents some of the frames from the video sequence. Fig. 6b presents NOEP for all PFs. Fig. 6c presents the distance (in pixels) between the estimated centre and true centre. As it can be noticed, this distance is very less for SMO‐PF and SMO (DR)‐PF when compared with others. Fig. 6d shows the F‐measure for all algorithms. As evident from these figures, overall performance of SMO‐PF transcends over other PFs; however, performance of SMO (DR)‐PF is very close to that of SMO‐PF and in few frames even better. Additionally, in SMO (DR)‐PF, SMO alone overcomes the particle degradation problem and retains the diversity in the particle set (Fig. 6e). Hence, resampling step became redundant. This helped to reduce the timing cost associated with SMO‐PF as can be noticed from Table 5.5TableTime cost and success rate for ShopAssistant1cor.mpgTime cost, sSuccess rate, %Simple PF0.1820.00PSO‐PF0.4679.20SMO‐PF0.4884.00SMO(DR)‐PF0.4780.00CS‐PF0.4568.446Fig.Comparison of performance for five PFs on the basis of(a) Sample frames from ShopAssistant1cor.mpg, (b) Number of effective particles, (c) Positional RMSE (in pixels), (d) F‐measure, (e) Number of distinct particles2StudentsConsiderableRotation.mpgIn this video, the person with the black shirt is being tracked. As the person moves from left to right, there are slight scale changes. The resolution was kept low and the colour of target's clothes was not significantly different from the background. Waving trees in the background added to the complexity in tracking the target. From frames 48 to 56, the target was occluded by another person. From Frame 93 to 105, the target rotated significantly. Fig. 7a presents some of the frames. It can be seen that simple PF failed to capture the rotation of target in these frames. On the other hand, SMO (DR)‐PF, SMO‐PF, CS‐PF, and PSO‐PF initially struggled to capture the rotation but eventually they did (Figs. 7c and d). It is interesting to note that NOEP maintained by PSO‐PF is, on average, better than CS‐PF for every video. Higher number of iterations used for PSO‐PF can be the cause of these results. Additionally, SMO alone handled the particle degeneracy and impoverishment issue (Figs. 7b and e).7Fig.Comparison of performance for five PFs on the basis of(a) Sample frames from 2StudentsConsiderableRotation.mpg, (b)Number of effective particles, (c) Positional RMSE (in pixels), (d) F‐measure, (e) Number of distinct particlesTable 6 discusses the timing cost and success rate of the five algorithms. Though timing cost for SMO‐PF and SMO (DR)‐PF is higher than the others, the higher success rate of the proposed algorithms ensures improved performance in applications where efficiency is primary and timing is secondary.6TableTime cost and success rate for 2StudentsConsiderableRotation.mpgTime cost, sSuccess rate, %Simple PF0.2665.71PSO‐PF0.4980.95SMO‐PF0.5895.24SMO (DR)‐PF0.5596.19CS‐PF0.4475.24ConclusionThis paper presents a PF based on SMO and DR for objet tracking. The SMO (DR)‐PF implements SMO to overcome particle degradation problem and DR strategy to overcome the particle impoverishment problem caused by the sub‐optimal resampling strategies. SMO strategy is a nature‐inspired optimisation algorithm that emulates the foraging behaviour of a group of SMs. A group of SMs consists of one global female leader (having maximum fitness), and is divided into smaller groups. Each group has a local leader having maximum fitness value in the local group. Every SM updates its position with the help of the experience of local and global leader, thereby improving its fitness value. Using this line of thought in PF process results in improved quality of the particles in the particle set. DR strategy was employed to overcome particle impoverishment problem in PFs. For object tracking and bearings only tracking, it is also observed that SMO alone increases the effective weight of particles and obviates the need of resampling step. The proposed PF is tested on standard videos obtained from the CAVIAR dataset and indigenously synthesised video containing various object tracking challenges. Proposed PF is also used to estimate state of non‐linear 1D model and bearings only tracking problem. SMO (DR)‐PF is compared with simple PF, SMO‐PF based on normal resampling, CS‐PF, and PSO‐PF for the above mentioned state estimation problems. Though the SMO‐PF and SMO (DR)‐PF have slightly higher timing cost than simple‐PF, PSO‐PF, and CS‐PF, they use fewer number of iterations than PSO‐PF to converge to an optimum solution. It is interesting to note that SMO‐based PFs perform better than CS‐based PF for same number of iterations. Since, no direct comparison between SMO and CS exist in literature, it is a subject of discussion whether SMO is better than CS optimisation on benchmark functions. On the basis of results obtained through various performance evaluation parameters, it can be concluded that the SMO‐based PFs perform better than other improved PFs for object tracking. This work will be extended to track multiple targets and more cues will be added to track those targets.AcknowledgmentsThe authors thank anonymous reviewers for their valuable comments to improve the quality of this research.9 References[1]Pan, X., Chen, X., Men, A.: ‘Occlusion handling based on particle filter in surveillance system’. 2nd Int. Conf. on Computer Modelling and Simulation, Hainan, 2010, pp. 179–183[2]Bradski, G.R.: ‘Real time face and object tracking as a component of a perceptual user interface’. IEEE Workshop on Applications of Computer Vision, Princeton, 1998, pp. 214–219[3]Qie, Z., Li, J.X.: ‘Adaptive particle swarm optimization‐based particle filter for tracking maneuvering object’. Chinese Control Conf., Nanjing, 2014, pp. 4685–4690[4]Brasnett, P., Mihaylova, L., Bull, D., et al: ‘Sequential Monte Carlo tracking by fusing multiple cues in video sequences’, Image Vis. Comput., 2007, 25, (8), pp. 1217–1227[5]Babenko, B., Yang, M., Belongie, S.: ‘Visual tracking with online multiple instance learning’. IEEE Int. Conf. on Computer Vision and Pattern Recognition, 2009, pp. 983–990[6]Kalal, Z., Mikolajczyk, K., Matas, J.: ‘Tracking‐learning‐detection’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (7), pp. 1409–1422[7]Gordon, N., Salmond, D., Smith, A.: ‘A novel approach for nonlinear/non‐Bayesian state estimation’, IEE Proc. F, Radar Signal Process., 1993, 140, (2), pp. 107–113[8]Isard, M., Blake, A.: ‘Condensation – conditional density propagation for visual tracking’, Int. J. Comput. Vis., 1998, 29, (1), pp. 5–28[9]Crisan, D., Moral, P., Lyons, T.: ‘Discrete filtering using branching and interacting particle systems’, Markov Process. Relat. Fields, 1999, 5, (3), pp. 293–319[10]Arulampalam, M., Maskell, S., Gordon, N., et al: ‘A tutorial on particle filters for online nonlinear/non‐Gaussian Bayesian tracking’, IEEE Trans. Signal Process., 2002, 50, (2), pp. 174–188[11]Thrun, S., Burgard, W., Fox, D.: ‘Probabilistic robotics’ (The MIT Press, 2005)[12]Carpenter, J., Clifford, P., Fearnhead, P.: ‘An improved particle filter for non‐linear problems’, IEE Proc. Radar Sonar Navig., 1999, 146, (1), pp. 2–7[13]Wang, Z., Liu, Z., Liu, W., et al: ‘Particle Filter algorithm based on adaptive resampling strategy’. IEEE Conf. on Electronic and Mechanical Engineering and Information Technology, China, 2011, pp. 3138–3141[14]Liu, J.S., Chen, R.: ‘Sequential Monte Carlo methods for dynamical systems’, J. Am. Stat. Assoc., 1998, 93, pp. 1032–1044[15]Efron, B., Tibshirani, R.: ‘An introduction to the bootstrap’ (Chapman & Hall, 1993)[16]Zuo, J.: ‘Dynamic resampling for alleviating sample impoverishment of particle filter’, IET Radar Sonar Navig., 2013, 7, (9), pp. 968–977[17]Choe, J., Wang, T., Liu, F., et al: ‘Particle filter with spline resampling and global transition model’, IET Comput. Vis., 2014, 9, (2), pp. 184–197[18]Zhai, J., Yeary, M.: ‘Implementing particle filters with Metropolis–Hastings algorithms’. Region 5 Conf.: Annual Technical and Leadership Workshop, 2004, pp. 149–152[19]Van Der, R., Doucet, A., De Freitas, N., et al: ‘The unscented particle filter’. NIPS, 2000, pp. 584–590[20]Bansal, J., Sharma, H., Jadon, S., et al: ‘Spider monkey optimization algorithm for numerical optimization’, Memet. Comput., 2014, 6, (1), pp. 31–47[21]Rashedi, E., Nezamabadi, H., Saryazdi, S.: ‘GSA: a gravitational search algorithm’, Inf. Sci., 2009, 179, (13), pp. 2232–2248[22]Zhong, J., Fung, Y.: ‘A biological inspired improvement strategy for particle filter’. IEEE Conf. on Industrial Technology, Gippsland, February 2009, pp. 1–6[23]Liang, X., Feng, J., Li, Q., et al: ‘A swarm intelligence optimization for particle filter’. Proc. of World Congress Intelligent Control and Automation, Chongqing, 2008, pp. 1986–1991[24]Neshat, M., Sepidnam, G., Sargolzaei, M.: ‘Artificial fish swarm algorithm: a survey of the state‐of‐the‐art, hybridization, combinatorial and indicative applications’, Artif. Intell. Rev., 2014, 42, (4), pp. 965–997[25]Higuchi, T.: ‘Monte Carlo filter using the genetic algorithm operators’, J. Stat. Comput. Simul., 1997, 59, (1), pp. 1–23[26]Tong, G., Fang, Z., Xu, X.: ‘A particle swarm optimized particle filter for nonlinear system state estimation’. IEEE Congress Evolutionary Computation, Vancouver, 2006, pp. 438–442[27]Zhao, J., Li, Z.: ‘Particle filter based on particle swarm optimization resampling for vision tracking’, Expert Syst. Appl., 2010, 37, (12), pp. 8910–8914[28]Walia, G.S., Kapoor, R.: ‘Particle filter based on Cuckoo search for nonlinear state estimation’. Int. Advance Computation Conf., India, 2013, pp. 918–924[29]Walia, G.S., Kapoor, R.: ‘Intelligent video target tracking using an evolutionary particle filter based upon improved cuckoo search’, Expert Syst. Appl., 2014, 41, (14), pp. 6315–6326[30]Gao, M., Li, L.‐L., Sun, X., et al: ‘Firefly algorithm (FA) based particle filter method for visual tracking’, Optik, 2015, 126, (18), pp. 1705–1711[31]Gao, M.L., He, X.H., Luo, D.S., et al: ‘Object tracking using firefly algorithm’, IET Comput. Vis., 2013, 7, (4), pp. 227–237[32]Kitagawa, G.: ‘Non‐Gaussian state‐space modelling of nonstationary time series’, J. Am. Stat. Assoc., 1987, 82, (400), pp. 1032–1063[33]‘CAVIAR Test Case Scenarios’. Available at http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/, accessed 15 May 2016[34]Stauffer, C., Grimson, W.: ‘Adaptive background mixture models for real‐time tracking’. IEEE Computer Society Conf. on Computer Vision Pattern Recognition, Fort Collins, June 1999, pp. 246–252[35]Chan, A.B., Mahadevan, V., Vasconcelos, N.: ‘Generalized Stauffer–Grimson background subtraction for dynamic scenes’, Mach. Vis. Appl., 22, (5), 2011, pp. 751–766 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png IET Computer Vision Wiley

Spider monkey optimisation assisted particle filter for robust object tracking

Download PDF
13 pages

Loading next page...
 
/lp/wiley/spider-monkey-optimisation-assisted-particle-filter-for-robust-object-rcjyNX3MKl

References (43)

Publisher
Wiley
Copyright
© The Institution of Engineering and Technology
eISSN
1751-9640
DOI
10.1049/iet-cvi.2016.0201
Publisher site
See Article on Publisher Site

Abstract

IntroductionObject tracking is one of the major research areas in computer vision. The idea is to first detect one or more targets, and then obtain a record of trajectory of those targets in the video sequence. Over the years, object tracking has created a niche for itself in computer vision applications ranging from vehicle tracking [1] and monitoring and surveillance systems [2], to drive assistance [3] and human tracking [4].However, the task of object tracking is suffused with challenges. Often little information is made available about the target. Even if the target information is given, the target appearance may change over time and space. Occlusion, background clutter, camouflaged foreground, and changing illumination in frame mar the efforts to achieve robust tracking. Furthermore, varying scale, rotating object, moving camera, and shadowing make the already difficult task more arduous.To efficiently tackle the difficulties arising in vision tracking, researchers have proposed algorithms that can be roughly categorised under deterministic approach or stochastic approach. Deterministic methods localise tracked object in each frame by iteratively searching for region which maximises the similarity between the tracked region and target. Though these methods are computationally efficient, they may converge to a local optima. Additionally, they are prone to background clutter and occlusion. Lately, [5, 6] have been proposed for tracking objects. These techniques use online learning mechanisms to track target. In [6], algorithm proposed divides the task of object tracking into three components: tracking, learning, and detection. A novel learning method has been proposed called P–N learning that estimates the errors in detection by a pair of experts named P‐expert (for estimating missed target detections) and N‐expert (for estimating false alarms).However, this research focusses mainly on the drawbacks of particle filters (PFs) (a stochastic approach) that limit their efficiency in tracking objects and how nature‐inspired optimisation techniques can overcome those drawbacks. PF was first reported in [7] as bootstrap filter. Over the years improved versions of PFs have been introduced as condensation filter [8], interacting particle approximation [9], and sequential importance sampling [10].PF represents the posterior PDF of system's state by random particles with associated weights and estimates the final state based on these particles. Keeping this in mind, PF views the tracking algorithm as a state estimation problem under Bayesian model and is considered to be one of the most useful tools to estimate Bayesian models with non‐linear and non‐Gaussian noise [10, 11].Nevertheless, increased efficiency of PF entails increased computation cost that can create a delay in applications involving real‐time systems. The increased dimensionality of state space adds to the computational burden associated with PFs.Apart from these challenges, particle degeneration is a common problem that limits the efficiency of PFs. Degeneracy is a phenomena in which after few iterations most of the particles have negligible weight and their contribution towards the state approximation is almost zero. Degeneracy causes wastage of computational efforts invested in updating particles having negligible weights. Two common methods to deal with degeneracy are:Resampling.Good choice of importance density.Resampling is a method by which particles with small weights are eliminated from the state vector estimate and are replaced with particles having large weights. The authors in [12–15] present systematic resampling, adaptive resampling, residual resampling, multinomial resampling, and stratified resampling methodologies, respectively. Nonetheless, these resampling methods partially solve the particle degeneracy problem. Instead, resampling creates a new problem of sample impoverishment. Resampling strategies lead to decreased diversity in the particle set. This results in poor construction of posterior PDF. Lately, more efficient approaches for resampling particles have been advanced by the researchers. Zuo [16] discusses a dynamic resampling (DR) method in which resampling operation is only performed on part of the particles in a step‐by‐step manner and the number of particles to be resampled is decided dynamically by a termination criterion based on the effective sample size. New born particles produced by resampling operation are helpful in alleviating degeneracy, whereas particles that are not resampled maintain the diversity. This technique has been used in this work to resample particles in spider monkey optimisation based PF (SMO‐PF). In [17], authors have introduced a novel resampling technique named as spline resampling that consists of two parts: spline transformation of weights and the spread transformation of states. The former is based on a spline transformation and achieves accurate particle filtering and latter is based on point spread transformation on states of particles to prevent sample impoverishment. Zhai and Yeary [18] proposes a PF that uses Markov Chain Monte Carlo method based Metropolis–Hastings algorithm to resample particles in the PF.To address the issue of sample impoverishment, researchers have also developed more accurate posterior distribution. In [19], authors have developed unscented PF that is a parametric/non‐parametric hybrid of unscented Kalman filter (UKF) and PF so that the PF provides the general probabilistic framework to handle non‐linearity and non‐Gaussianity and UKF generates better proposal distribution.Lately, nature‐inspired optimisation algorithms have captured great deal of attention of the researchers working on PF‐based object tracking. These algorithms provide strategies that guide the search process in the fitness space. The given paper proposes an object tracking algorithm based on PF and SMO algorithm that models the foraging behaviour of spider monkeys (SMs). The proposed work parallels the line of thought used in [20] and optimises the distribution of sample particles before the DR step. Extensive experiments were carried out to test the efficiency of the new algorithm. The algorithm was first tested on generic one‐dimensional (1D) state estimation problem, and then on bearings only tracking problem so as to provide a theoretical backing to novel state estimation method for object tracking. SMO‐PF using DR [SMO (DR)‐PF] was then tested on standard videos obtained from CAVIAR dataset, and indigenous video consisting of various object tracking challenges. The results were also obtained for simple PF, particle swarm optimisation assisted PF (PSO‐PF), SMO‐PF with normal resampling, and cuckoo search optimisation assisted PF (CS‐PF) to provide comparative analysis.The rest of the paper is organised as follows. Section 2 details some advancements made in the field of nature‐inspired optimisation assisted PFs. Section 3 delineates the PF algorithm and SMO algorithm. In Section 4, the SMO (DR)‐PF algorithm is discussed for 1D and 2D state estimation, and object tracking problem. Section 5 discusses the performance parameters used for evaluating the algorithms. In Section 6, results obtained from the new PF for the three problems are discussed in detail. Finally, the paper is concluded in Section 7.Related workNature‐inspired meta‐heuristics approach to optimise sample distribution in the state hypothesis obtained from PF has been an area of active research in recent years. All nature‐inspired optimisation methods share two common aspects: exploration and exploitation. Exploration is the ability of expanding search space, whereas exploitation is the ability of finding optima around a good solution [21]. First few iterations of optimisation algorithm are used for exploring the search space so as to avoid getting trapped in local optima. After some iterations, exploration dampens and exploitation surfaces and the system reaches a semi‐optimal solution. In case of PF, the combination results in concentration of particles in high probability area aiding in creation of effective posterior hypothesis.In [22], authors implemented ant colony optimisation (ACO) algorithm in PF to solve the sample impoverishment problem. ACO provides guaranteed convergence and can be used to optimise dynamic problems; time to convergence, however, is uncertain. A non‐linear economic model was employed to test the performance of the algorithm. The results were also obtained for other improved PFs for comparison. The research, however, did not test the algorithm for bearings only tracking problem and object tracking. In [23], authors used artificial fish swarm optimisation (AFSO) for optimising particle distribution before resampling step so as to centralise particles in high‐likelihood area. The algorithm was tested for manoeuvring target tracking. AFSO technique provides high convergence speed, high accuracy, and fault tolerance. However, it has high time complexity associated with it [24]. In [25], authors proposed a genetic algorithm (GA) filter wherein they replaced the prediction step with mutation and crossover operators in GA. Though GA is fast in exploring the search space and has low memory requirements, the solution obtained by GA can be sub‐optimal or, local maxima/minima.PSO provides various advantages over many swarm‐based optimisation techniques. The fitness function can be non‐differentiable and PSO rapidly produces quality solution for problems involving many dimensions. However, since there is no general optimisation technique for practical and multidimensional problems, tuning and experimenting with input parameters of PSO method is necessary. Additionally, for noisy non‐linear problems, stochastic variability of results obtained from the PSO method is very high. In [26], authors exploited capabilities of PSO and proposed a PF based on PSO to overcome particle deprivation problem, but its implementation did not cover object tracking problem. In [27], authors proposed a PSO assisted PF for object tracking in video sequences. They also exploited the ability of PF to fuse different cues and used the mixture of histogram of oriented gradients (HOG) and colour histogram to construct feature vector for target association. Later in [3], authors proposed an adaptive PSO assisted PF called APSO‐PF to track manoeuvring object. The standard parameters in PSO‐PF were made adaptive and were changed according to motion state of the object.CS optimisation can effectively deal with multimodal problems and requires tuning of only single parameter. Walia and Kapoor [28] proposed a novel PF that embeds cuckoo search via levy flight to estimate the state of generic 1D state estimation problem and classic bearings only 2D state estimation problem. The new algorithm surpassed standard PF and PSO‐PF in terms of root mean squared error (RMSE) and number of effective particles (NOEP). Later in [29], authors detailed the implementation of improved CS‐based PF for object tracking. Comprehensive qualitative and quantitative analysis was done and the new algorithm was proved to be more reliable. In optimisation problems with high level of noise and non‐linear fitness function, firefly optimisation algorithm (FA) outperforms PSO in finding the optimal solution and time taken to reach that optimum solution. In [30], authors proposed a novel PF based on FA to increase the number of meaningful particles for better approximation of state vector. However, FA sometimes get trapped in the local optima and is unable to completely get rid of it. Results were obtained for different video sequences consisting of various object tracking challenges. The new algorithm outperformed standard PF for tracking motile object. In [31], a general optimisation‐based tracking architecture based on FA is proposed. The speed and accuracy of new FA‐based tracking algorithm is compared with standard PF, meanshift, and PSO.In [20], it was observed that SMO successfully overcame the problem of premature convergence and stagnation in the search space experienced by many state‐of‐the‐art nature‐inspired meta‐heuristic optimisation techniques. SMO performed better than artificial bee colony optimisation, PSO, differential evolution, and covariance matrix adaption‐evolution strategy for different benchmark problems.Particle filtering and SMOThis section discusses the PF and SMO algorithm in detail. Additionally, the methodology of incorporating optimisation algorithm within a PF has been also expounded.Particle filterThe key idea of particle filtering is to represent required PDF of state pxt|Zt, where Zt=z1,z2,…,zt is the set of all available observations and xt is the state estimate at time t, by set of random particles with associated weights. It is assumed that initial PDF pxo|zo is known a priori.Particle filtering consists of two steps:Prediction of state using motion model.Update state using measurement model.These steps are applied recursively to obtain PDF pxt|Zt.The prediction step uses the motion model xt=ft(xt−1,wt−1) to obtain the prior PDF at time t, provided pxt−1|Zt−1 is known, where Zt−1=z1,z2,…,zt−1 is set of previously observed states up till time t−1. Here, ft is a non‐linear function of target state and wt−1 is zero‐mean white noise sequence that is used to model unknown disturbances in the state prediction process. This model defines the translation, rotation, scale, and other state parameters of the target.Online measurements are made according to model zt=htxt,ut, here ht is a non‐linear function and ut is a zero‐mean white noise sequence. The measurement zt forms the basis of the update step and is used to update the predicted state using the Bayesian rule as follows:1pxt|z1:zt=pzt|xtpxt|z1:zt−1pzt|z1:zt−12pzt|z1:zt−1=∫pzt|xtpxt|z1:zt−1dxtAs stated, particle filtering approximates the state by a weighted particle set S={xtn,wtn,n=1,2,…,Ns}. Here, Ns is equal to total number of particles. Each particle depicts one hypothetical state xt of the target and has a weight wt that is obtained in terms of degree of its association with the observation. These weights are normalised such that ∑i=1Nswti=1.The phenomena of importance sampling comes into play to obtain this weighted approximation of posterior p(xt|Zt). It states that, if px∝π(x) is a probability density through which it is difficult to obtain particles, but for which πx can be evaluated, then a weighted approximation of the posterior PDF is given by3p(x)≃∑i=1Ns⁡wiδ(x−xi)Here, wi∝πxi/vxi is the normalised weight of the ith particle. x≃vx,i=1,2,…,Ns are the samples that are generated from a proposal v⋅ known as the importance density. These updated weighted approximation acts as prior for the next state and the process is repeated until last state is achieved. However, after several iterations most of the particle's weight is almost negligible and NOEP, as expressed in (4), rapidly decreases. Resampling was introduced to tackle this problem4Neff=1/∑i=1N⁡(wki)2Spider monkey optimisationSMO is a nature‐inspired heuristic approach to solve optimisation problem. It is based on the foraging behaviour of SMs who fall under the category of fission‐fusion social structure (FFSS) based animals [20]. The FFSS‐based animals are social and live in groups of 40–50 individuals. The FFSS of swarm helps in reducing the competition amongst the group members by dividing them into smaller groups. A female global leader searches the food sources and divides the group into smaller sub‐groups in case she is unable to procure sufficient food for the group. These smaller sub‐groups then forage independently. These sub‐groups also have a female local leader who decides an efficient foraging route every day. Members of these sub‐groups communicate within and outside the sub‐group about availability of food resources.The SMO process used consists of three steps, and they are:Step 1: Initialisation of populationIn this step, Ns SMs, uniformly distributed, are generated. Each SM SMn{n=1,2,…,Ns} is a D‐dimensional vector, where D the number of state variables, and each SMn{n=1,2,…,Ns} is a potential state.Step 2: Local leader phase (LLP)Here each SM SMn updates its position based on the information of the local leader and corresponding group member experience. After obtaining the new position, fitness value of that position is obtained. If new fitness value is higher than the previous fitness value, then SMn, member of the lth local group, updates its position with the new one according to the following equation:5SMnewnd=SMnd+U0,1×LLld−SMnd+U−1,1×(SMrd−SMnd)SMnd is the dth dimension of nth SM, LLld represents the dth dimension of the lth local group leader position, and SMrd is the dth dimension of rth SM that is chosen randomly from lth local group such that r≠n, and U(0,1) is a uniformly distributed random number between 0 and 1. Fig. 1a shows the position update process used in LLP. L is the number of groups in total, N is the group size, and pr is the perturbation rate that controls the magnitude of perturbation in current state.Step 3: Global leader phase (GLP)1Fig.SMO algorithm for object tracking(a) Optimisation algorithm used to improve particle distribution, (b) Improved particle distribution using SMOAll SMs update the position based on global leader experience and local group member experience. The position is updated according to the following equation:6SMnewnd=SMnd+U0,1×GLd−SMnd+U−1,1×(SMrd−SMnd)GLd represents the dth dimension of the global leader position and d={1,2,…,D} is a randomly chosen index.In this step, position of SMs (SMn) is updated according to probability pn that is calculated according to the following equation:7pn=0.9×fitnessnmax_fitness+0.1Here, fitnessn is fitness of nth SM and max_fitness is the maximum fitness in the group. By assigning probability to a SM, it is made sure that good candidates have higher chances to make themselves better. Furthermore, new fitness values are compared with old fitness values and if new value is greater than old values, SMn updates its position. Fig. 1a also discusses the algorithm for GLP.SMO‐based PFSMO (DR) algorithm is used in conjunction with PF to improve the efficiency of PF. First, the initial state PDF is obtained from the resampled particles of the previous time step. Next, motion model and measurement model (described in Section 4) are used to obtain the predicted PDF p(xt|zt). With the help of a likelihood function, described in [4] ((25)), a weighted approximation of the state PDF, p1(x)≃∑i=1Nswiδ(x−xi), is obtained wherein weight is assigned to a particle corresponding to the degree of its association (as measured with the measurement model) with the true state. After obtaining a weighted approximation of the state PDF, SMO algorithm is used to improve the weights of the particle. Every particle in the PF is treated as a SM. The set of particles with NsSMs is divided into L local groups with each group having N SMs. The particle with highest fitness value/weight in the set is treated as the global leader and the particle with highest weight in each local group is treated as the local leader. Next, LLP phase is applied that helps the particles to explore the fitness space with high perturbation rate pr. This helps to overcome the problem of stagnation in the fitness space experienced by many optimisation techniques. After the LLP phase, GLP phase is applied that allows the particles with high weights to update their position and further improve their fitness values. As can be noticed from Fig. 1b, particles having low weights before SMO step have increased fitness values and are more close to the global leader. Also, the global leader has further improved its weight value. After obtaining weighted approximation via SMO, resampling step is applied to obtain the final PDF of the state. All in all, the optimisation step, in combination with DR, increases the diversity and quality of particles, and helps in effectively modelling the posterior PDF of the target.One‐dimensional state estimation, bearings only tracking, and object tracking by SMO (DR)‐PFIn this section, SMO (DR)‐PF algorithm used for generic 1D state estimation problem, bearing only tracking (2D state estimation) problem, and object tracking problem is discussed in detail. This section also discusses the motion models, measurement models and other implementation details.One‐dimensional state estimation problem and 2D state estimation problemOne‐dimensional non‐linear problemThe following non‐linear model was considered to evaluate the performance of SMO (DR)‐PF:8xk=0.5xk−1+25xk−11+xk−12+8cos1.2k−1+wk9zk=xk220+ukThe above model was reported in [32]. It is a highly non‐linear model, both in system (8) and the measurement (9) equation. Here wk and uk are zero‐mean Gaussian white noise with variance 10.0 and 1.0, respectively.Bearings only tracking problemIn this problem, the target's motion is governed by second‐order model represented by10xk=αxk−1+ρwkxk=(x,x.,y,y.)kTandwk=(wx,wy)kTHere, x and y denote Cartesian coordinates of the target and x. and y. represent velocity of the target. wk denotes system noise and is zero‐mean Gaussian white noise process with covariance C:EwkwjT=Cδjk, where C=cI2 and I2 is 2×2 identity matrix. Measurement used to track target is bearing or angle zk and is governed by measurement11zk=tan−1ykxk+ukHere, uk is measurement noise and is zero‐mean Gaussian white noise with variance r:Eukuj=rδjk. Before taking measurements at k = 1, the initial state vector is assumed to have a Gaussian distribution with known mean x1¯ and covariance C1=diag(σ12,σ22,σ32,σ42).Fig. 2a presents the algorithm that has been implemented for 1D non‐linear state estimation problem and bearings only tracking 2D state estimation problem.2Fig.SMO assisted PF algorithm(a) For 1D and 2D state estimation problem, (b) For object tracking problemSMO (DR)‐PF for object trackingThis subsection discusses the algorithm implemented (Fig. 2b) for object tracking in standard videos, obtained from CAVIAR dataset [33], and synthetic videos.First, the acquired frame is searched for the object to be tracked. This is done with the help of Gaussian mixture model (GMM) based foreground detection technique. In this method, intensity value of every pixel from the first few frames is modelled using mixture of Gaussian distributions. Aberrant pixel values that do not fit the background distribution are labelled as foreground. Number of Gaussian modes in mixture model used were 5. Algorithm presented in [34] was used to detect foreground object. Although GMM‐based detection allows for some variability in the states of the pixel, the model assumes that this variability is derived from noise and not structured motion patterns such as swaying trees, moving clouds, and so on. Another drawback of the used technique is the lack of consistency between the states of adjacent pixels that often leads to abysmal foreground–background segmentation [35]. As will be noticed in Section 6.3, these shortcomings limit the efficiency achieved through the proposed object tracking framework. However, PF's ability to maintain multiple hypothesis of target's state helps in overcoming these drawbacks to some extent.Thereafter, N particles are initialised to represent that state of object. After updating particles through (13), every particle is assigned a weight using the Bhattacharyya coefficient and weighted approximation of the state PDF is obtained. Next, these particles are divided into sub‐groups for implementing the LLP and GLP of SMO algorithm. This step reduces degeneracy in the particle set and increases NOEP. Then the newly generated particles are resampled using DR. Finally, the state of the target is estimated by taking weighted mean of resampled particles. The loop is continued until all states are predicted.Motion modelMotion of an object to be tracked can greatly vary from simple translation to complex rotation. Constant velocity model for translation motion and random walk model for the scaling and rotation have been considered. The used multicomponent model was proposed in [4].The complete state vector is described as12x=(x,x.,y,y.,θ,s)TThe state is updated according to the following equation:13xk+1=Gxk+wk,wk∼N(0,C)14G=1V0102×202×102×102x21V0102×102×101×201×21001×201×201Here wk is zero‐mean Gaussian white noise, V is the sampling interval, and C is the covariance matrix with covariance values for translation motion, rotation, and scaling as σx2,σy2,σθ2 and σs2, respectively.Measurement modelAfter updating the state, every particle is assigned a weight. The weights are proportional to the degree of association of reference feature with the target feature.This work uses colour histogram for computing weights of the particles, since colour histograms in general can accommodate small changes in the appearance of the object.Histogram, ht=(h1,t,h2,t,…,hn,t) for a region St corresponding to state t is given by15hi,t=∑uϵSt⁡δi(bu),i=1,2,…,nHere, δi is the Kronecker‐delta function at the bin index i. bu∈1,…,n is the histogram bin index associated the intensity at pixel location u=(x,y) and n is the number of bins in every channel. Then histogram is normalised such that ∑1nhi,t=1. 8×8×8 Bin RGB histogram was constructed.After obtaining the target and reference histogram, similarity between the two histograms is measured. This manuscript uses Bhattacharyya distance for measuring similarity. Bhattacharyya distance (B.D.) has been previously used in [29, 30]16B.D.=1−B.C.Here, B.C. is Bhattacharyya coefficient and is given by17B.C.href,htgt=∑i=1n⁡href,i∗htgt,iHere, href and htgt are the colour histogram of reference and target region, respectively. If the colour histograms are similar then B.C. is 1 indicating a perfect match.Based on (16), distance D is defined which takes into consideration all three colour channels18Dhref,htgt=∑c∈(R,G,B}B.D.2(hrefc,htgtc)3Based on value of Dhref,htgt, weight is assigned to every particle.Performance evaluation parametersThis section discusses the performance parameters used for evaluating the performance of the novel algorithm.For 1D non‐linear problem and bearings only tracking problemA. RMSE: It is an excellent general purpose error measuring parameter and is defined as19RMSEx=∑i=1N(xi−x^i)2NForone−dimensionHere, xi is the true value of state and x^i is the predicted value of the state. N is total number of estimated states20RMSExy=∑i=1N(xi−x^i)2+(yi−y^i)2NFortwo−dimensionHere, (xi,yi) are the true location coordinates of the target in –xy plane, whereas (x^i,y^i) are the estimated location coordinates. N is total number of estimated states.B. Variance: Results were obtained for 25 iterations where 1 iteration comprises 50 states for 1D non‐linear problem, and 24 states for bearings only tracking problem. Variance depicts the stability of PF algorithm. Less the variance more stable is the performance of that algorithm.C. Time cost: The total time taken by the algorithm to obtain the estimates of all states.D. NOEP: ‘NOEP’ versus ‘state index’ graph was obtained to monitor the benefits brought by optimisation.E. Number of distinct particles (NODP): ‘NODP’ versus. ‘state index’ graph was obtained to monitor the benefits brought by DR.For object trackingA. RMSE (in pixels): The combined RMSE error of pixel coordinate (xi,yi) to the estimated coordinate (x^i,y^i) was used and is defined as (20). N = 10 is equal to the total number of iterations for which the algorithm was run. This process is repeated for every frame. ‘RMSE’ versus ‘frame number’ graph is obtained.B. F‐measure: To calculate F‐measure, it is paramount to understand recall and precision. Recall is the proportion of positive pixels that are correctly classified as target to be trackedRecall=numberofcorrecltyclassifiedtargetpixelsnumberoftargetpixelsinGTPrecision is the proportion of predicted positive pixels that were correctly identified as targetPrecision=numberofcorrecltyclassifiedtargetpixelsnumberoftargetpixelsestimatedValues of recall and precision are directly proportional to the performance of the algorithm; however, each performance evaluator can be misleading alone. Hence, their harmonic mean, F‐measure was introduced to accommodate both.F−measure=2×Recall×PrecisionRecall+PrecisionC. Success rate: It is the percentage of total frames for which target object is accurately localised.D. Time cost: It is the average amount of time (in seconds) taken to process one video frame of the video sequence to localise the target.E. NOEP: Same as mentioned for 1D and 2D state estimation problem.F. NODP: Same as mentioned for 1D and 2D state estimation problem.Results and discussionThis section will cover the performance analysis of SMO (DR)‐PF for 1D state estimation, 2D state estimation, and object tracking. Algorithms were implemented on MATLAB running on 1.4 GHz Dual Core Processor.Generic 1D non‐linear modelThe initial state was assumed to be xo = 0.1. The PFs were initialised with prior PDF pxo=N(0,2).Fig. 3a shows the 50 states estimated through different PFs. Green line represents the true state obtained via (8), whereas black line represents the estimated state via PFs. Number of particles was equal to 500. Optimisation process brought an increase in efficiency by decreasing the difference between the estimated state and the true state. Figs. 3b and c display the NOEP and NODP at each state. Table 1 compares the mean RMSE for 25 iterations, variance of these RMSE values, and time cost for the five algorithms. Variance (RMSE) suggests increased stability of the proposed algorithms. It can be observed from Fig. 3c that the DR step brought a significant increase in the NODP than the systematic resampling technique used for other algorithms. Also, it can be observed from Fig. 3b that there is not much difference between NOEP for SMO‐PF and SMO (DR)‐PF. Hence, it can be concluded that SMO overcomes the particle degradation problem, while DR overcomes the particle impoverishment problem. Additionally, it can be observed from Table 1 that SMO (DR)‐PF, though has higher RMSE than SMO‐PF (perhaps because of the large measurement and process noise in the system), performs better than other improved PFs.1TableMean RMSE, variance, and time cost for all PFs (1D non‐linear model)Mean RMSEVariance (RMSE)Time cost, sSimple‐PF5.421.760.23PSO‐PF4.921.390.84SMO‐PF3.620.681.02SMO(DR)‐PF4.650.520.96CS‐PF4.841.120.783Fig.Comparative performance evaluation(a) State estimation using simple PF, PSO‐PF, SMO‐PF, SMO (DR)‐PF, and CS‐PF (left to right), (b) Number of effective particles, (c) Number of distinct particlesBearings only trackingThe initial state of target was assumed to be x1=(−0.05,0.001,0.7,−0.055)T. About 500 particles were taken to estimate the state. The prior distribution parameters were set to x1¯=(0.0,0.0,0.4,−0.08)T, and σ1=0.5,σ2=0.005,σ3=0.3, and σ4=0.01. The target trajectory is presented by red line and true trajectory by black line.Fig. 4a presents the results 24 states. It can be observed that SMO‐PF closely follows the true trajectory, while others fail. Fig. 4b presents NOEP at 24 states. An interesting observation can be made from Fig. 4c. It can be seen that SMO (DR)‐PF maintains excellent diversity in its particle set. On further exploring, it was found that SMO step alone increased the effective weight of particles above the threshold (0.35×Total number of particles) and made the DR step redundant. Hence, no resampling step was required to increase effective weight of the particles. However, it can also be observed that RMSE of SMO (DR)‐PF is more than SMO‐PF. Table 2 compares the mean RMSE for 25 iterations, variance of these RMSE values, and time cost for the five algorithms. Low variance demonstrates stable performance of the proposed algorithm.2TableMean RMSE, variance, and time cost for all PFs (bearings only tracking)Mean RMSEVariance (RMSE)Time, sSimple PF0.200.00310.62PSO‐PF0.160.000540.75SMO‐PF0.060.000371.01SMO(DR)‐PF0.140.000300.98CS‐PF0.150.000450.694Fig.Comparative performance evaluation(a) State estimation using simple PF, PSO‐PF, SMO‐PF, SMO (DR)‐PF, and CS‐PF (left to right), (b) Number of effective particles, (c) Number of distinct particlesObject tracking problemThis sub‐section discusses the performance of the proposed method. Algorithms were tested on three videos. Two videos were taken from the standard CAVIAR dataset and one video was synthesised indigenously. Number of particles N used to estimate the state of the target was set to 150. PFing infuses randomness in the object tracking process. To accommodate this randomness, every PF‐based algorithm was run for ten times and results discussed are average of those observations. To maintain the even‐handedness, same target model and motion model have been used for all algorithms, but normal resampling was used for other algorithms to exhibit benefits of DR. Table 3 reports the parameters used for tracking process.3TableFeatures and parameters used for object tracking (bearings only tracking)Number of particles150Number of bins for each histogram channel8V,σx2,σy2,σs2,σθ2 for (13)0.9, 0.04, 0.04, 0.0045, 0.0004PSO parametersc1 = c2 = 2, number of iterations = 3SMO parameterspr = 0.8, L = 10, N = 15, number of iterations = 2DR parametersNth = 0.35×number of particlesCS parametersP (discovery of alien eggs) = 0.8, number of iterations = 2Processor1.4 GHz dual coreCamera speed23 fpsOneLeaveShopReenter1cor.mpgIn this video, the person with dark red shirt is tracked. The person walks towards the camera thereby inducing scale transformation. During the course of the video, illumination in the scene varies substantially. The target object is occluded from frame 18 to frame 36. The video sequence incorporates shadowing challenge.It can be noticed from Fig. 5d that SMO‐PF maintained better F‐measure than others. It can be further noticed that optimised PF (esp. SMO‐PF) maintained higher NOEP (Fig. 5b) during occlusion. Fig. 5c present the total positional RMSE. As it can be noticed, positional RMSE for SMO‐PF was lower than that for others (save few frames). Table 4 also compares the success rate and timing cost of the five algorithms. Fig. 5a presents some of the frames from the video sequence. Similar to the bearings only tracking problem, for SMO (DR)‐PF, SMO alone overcame the particle degradation problem and retained the diversity (Fig. 5e).4TableTime cost and success rate for OneLeaveShopReenter1cor.mpgTime cost, sSuccess rate, %Simple PF0.1564.17PSO‐PF0.3192.50SMO‐PF0.3699.17SMO(DR)‐PF0.3598.33CS‐PF0.3090.005Fig.Comparison of performance for five PFs on the basis of(a) Sample frames from OneLeaveShopReenter1cor.mpg, (b) Number of effective particles, (c) Positional RMSE (in pixels), (d) F‐measure, (e) Number of distinct particlesShopAssistant1cor.mpgIn this video, the person with white top is being tracked. The person walks away from the camera, thereby inducing scale changes. Change in velocity occurs in both directions. To make the sequence more challenging camouflaged background was used and the bottom of the target had significant similarity with clothes of the other person. The other person partially occluded the target in the frames 90–125. Additionally, as the target moves away, illumination changes and shadowing further complicate the task of tracking.Fig. 6a presents some of the frames from the video sequence. Fig. 6b presents NOEP for all PFs. Fig. 6c presents the distance (in pixels) between the estimated centre and true centre. As it can be noticed, this distance is very less for SMO‐PF and SMO (DR)‐PF when compared with others. Fig. 6d shows the F‐measure for all algorithms. As evident from these figures, overall performance of SMO‐PF transcends over other PFs; however, performance of SMO (DR)‐PF is very close to that of SMO‐PF and in few frames even better. Additionally, in SMO (DR)‐PF, SMO alone overcomes the particle degradation problem and retains the diversity in the particle set (Fig. 6e). Hence, resampling step became redundant. This helped to reduce the timing cost associated with SMO‐PF as can be noticed from Table 5.5TableTime cost and success rate for ShopAssistant1cor.mpgTime cost, sSuccess rate, %Simple PF0.1820.00PSO‐PF0.4679.20SMO‐PF0.4884.00SMO(DR)‐PF0.4780.00CS‐PF0.4568.446Fig.Comparison of performance for five PFs on the basis of(a) Sample frames from ShopAssistant1cor.mpg, (b) Number of effective particles, (c) Positional RMSE (in pixels), (d) F‐measure, (e) Number of distinct particles2StudentsConsiderableRotation.mpgIn this video, the person with the black shirt is being tracked. As the person moves from left to right, there are slight scale changes. The resolution was kept low and the colour of target's clothes was not significantly different from the background. Waving trees in the background added to the complexity in tracking the target. From frames 48 to 56, the target was occluded by another person. From Frame 93 to 105, the target rotated significantly. Fig. 7a presents some of the frames. It can be seen that simple PF failed to capture the rotation of target in these frames. On the other hand, SMO (DR)‐PF, SMO‐PF, CS‐PF, and PSO‐PF initially struggled to capture the rotation but eventually they did (Figs. 7c and d). It is interesting to note that NOEP maintained by PSO‐PF is, on average, better than CS‐PF for every video. Higher number of iterations used for PSO‐PF can be the cause of these results. Additionally, SMO alone handled the particle degeneracy and impoverishment issue (Figs. 7b and e).7Fig.Comparison of performance for five PFs on the basis of(a) Sample frames from 2StudentsConsiderableRotation.mpg, (b)Number of effective particles, (c) Positional RMSE (in pixels), (d) F‐measure, (e) Number of distinct particlesTable 6 discusses the timing cost and success rate of the five algorithms. Though timing cost for SMO‐PF and SMO (DR)‐PF is higher than the others, the higher success rate of the proposed algorithms ensures improved performance in applications where efficiency is primary and timing is secondary.6TableTime cost and success rate for 2StudentsConsiderableRotation.mpgTime cost, sSuccess rate, %Simple PF0.2665.71PSO‐PF0.4980.95SMO‐PF0.5895.24SMO (DR)‐PF0.5596.19CS‐PF0.4475.24ConclusionThis paper presents a PF based on SMO and DR for objet tracking. The SMO (DR)‐PF implements SMO to overcome particle degradation problem and DR strategy to overcome the particle impoverishment problem caused by the sub‐optimal resampling strategies. SMO strategy is a nature‐inspired optimisation algorithm that emulates the foraging behaviour of a group of SMs. A group of SMs consists of one global female leader (having maximum fitness), and is divided into smaller groups. Each group has a local leader having maximum fitness value in the local group. Every SM updates its position with the help of the experience of local and global leader, thereby improving its fitness value. Using this line of thought in PF process results in improved quality of the particles in the particle set. DR strategy was employed to overcome particle impoverishment problem in PFs. For object tracking and bearings only tracking, it is also observed that SMO alone increases the effective weight of particles and obviates the need of resampling step. The proposed PF is tested on standard videos obtained from the CAVIAR dataset and indigenously synthesised video containing various object tracking challenges. Proposed PF is also used to estimate state of non‐linear 1D model and bearings only tracking problem. SMO (DR)‐PF is compared with simple PF, SMO‐PF based on normal resampling, CS‐PF, and PSO‐PF for the above mentioned state estimation problems. Though the SMO‐PF and SMO (DR)‐PF have slightly higher timing cost than simple‐PF, PSO‐PF, and CS‐PF, they use fewer number of iterations than PSO‐PF to converge to an optimum solution. It is interesting to note that SMO‐based PFs perform better than CS‐based PF for same number of iterations. Since, no direct comparison between SMO and CS exist in literature, it is a subject of discussion whether SMO is better than CS optimisation on benchmark functions. On the basis of results obtained through various performance evaluation parameters, it can be concluded that the SMO‐based PFs perform better than other improved PFs for object tracking. This work will be extended to track multiple targets and more cues will be added to track those targets.AcknowledgmentsThe authors thank anonymous reviewers for their valuable comments to improve the quality of this research.9 References[1]Pan, X., Chen, X., Men, A.: ‘Occlusion handling based on particle filter in surveillance system’. 2nd Int. Conf. on Computer Modelling and Simulation, Hainan, 2010, pp. 179–183[2]Bradski, G.R.: ‘Real time face and object tracking as a component of a perceptual user interface’. IEEE Workshop on Applications of Computer Vision, Princeton, 1998, pp. 214–219[3]Qie, Z., Li, J.X.: ‘Adaptive particle swarm optimization‐based particle filter for tracking maneuvering object’. Chinese Control Conf., Nanjing, 2014, pp. 4685–4690[4]Brasnett, P., Mihaylova, L., Bull, D., et al: ‘Sequential Monte Carlo tracking by fusing multiple cues in video sequences’, Image Vis. Comput., 2007, 25, (8), pp. 1217–1227[5]Babenko, B., Yang, M., Belongie, S.: ‘Visual tracking with online multiple instance learning’. IEEE Int. Conf. on Computer Vision and Pattern Recognition, 2009, pp. 983–990[6]Kalal, Z., Mikolajczyk, K., Matas, J.: ‘Tracking‐learning‐detection’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (7), pp. 1409–1422[7]Gordon, N., Salmond, D., Smith, A.: ‘A novel approach for nonlinear/non‐Bayesian state estimation’, IEE Proc. F, Radar Signal Process., 1993, 140, (2), pp. 107–113[8]Isard, M., Blake, A.: ‘Condensation – conditional density propagation for visual tracking’, Int. J. Comput. Vis., 1998, 29, (1), pp. 5–28[9]Crisan, D., Moral, P., Lyons, T.: ‘Discrete filtering using branching and interacting particle systems’, Markov Process. Relat. Fields, 1999, 5, (3), pp. 293–319[10]Arulampalam, M., Maskell, S., Gordon, N., et al: ‘A tutorial on particle filters for online nonlinear/non‐Gaussian Bayesian tracking’, IEEE Trans. Signal Process., 2002, 50, (2), pp. 174–188[11]Thrun, S., Burgard, W., Fox, D.: ‘Probabilistic robotics’ (The MIT Press, 2005)[12]Carpenter, J., Clifford, P., Fearnhead, P.: ‘An improved particle filter for non‐linear problems’, IEE Proc. Radar Sonar Navig., 1999, 146, (1), pp. 2–7[13]Wang, Z., Liu, Z., Liu, W., et al: ‘Particle Filter algorithm based on adaptive resampling strategy’. IEEE Conf. on Electronic and Mechanical Engineering and Information Technology, China, 2011, pp. 3138–3141[14]Liu, J.S., Chen, R.: ‘Sequential Monte Carlo methods for dynamical systems’, J. Am. Stat. Assoc., 1998, 93, pp. 1032–1044[15]Efron, B., Tibshirani, R.: ‘An introduction to the bootstrap’ (Chapman & Hall, 1993)[16]Zuo, J.: ‘Dynamic resampling for alleviating sample impoverishment of particle filter’, IET Radar Sonar Navig., 2013, 7, (9), pp. 968–977[17]Choe, J., Wang, T., Liu, F., et al: ‘Particle filter with spline resampling and global transition model’, IET Comput. Vis., 2014, 9, (2), pp. 184–197[18]Zhai, J., Yeary, M.: ‘Implementing particle filters with Metropolis–Hastings algorithms’. Region 5 Conf.: Annual Technical and Leadership Workshop, 2004, pp. 149–152[19]Van Der, R., Doucet, A., De Freitas, N., et al: ‘The unscented particle filter’. NIPS, 2000, pp. 584–590[20]Bansal, J., Sharma, H., Jadon, S., et al: ‘Spider monkey optimization algorithm for numerical optimization’, Memet. Comput., 2014, 6, (1), pp. 31–47[21]Rashedi, E., Nezamabadi, H., Saryazdi, S.: ‘GSA: a gravitational search algorithm’, Inf. Sci., 2009, 179, (13), pp. 2232–2248[22]Zhong, J., Fung, Y.: ‘A biological inspired improvement strategy for particle filter’. IEEE Conf. on Industrial Technology, Gippsland, February 2009, pp. 1–6[23]Liang, X., Feng, J., Li, Q., et al: ‘A swarm intelligence optimization for particle filter’. Proc. of World Congress Intelligent Control and Automation, Chongqing, 2008, pp. 1986–1991[24]Neshat, M., Sepidnam, G., Sargolzaei, M.: ‘Artificial fish swarm algorithm: a survey of the state‐of‐the‐art, hybridization, combinatorial and indicative applications’, Artif. Intell. Rev., 2014, 42, (4), pp. 965–997[25]Higuchi, T.: ‘Monte Carlo filter using the genetic algorithm operators’, J. Stat. Comput. Simul., 1997, 59, (1), pp. 1–23[26]Tong, G., Fang, Z., Xu, X.: ‘A particle swarm optimized particle filter for nonlinear system state estimation’. IEEE Congress Evolutionary Computation, Vancouver, 2006, pp. 438–442[27]Zhao, J., Li, Z.: ‘Particle filter based on particle swarm optimization resampling for vision tracking’, Expert Syst. Appl., 2010, 37, (12), pp. 8910–8914[28]Walia, G.S., Kapoor, R.: ‘Particle filter based on Cuckoo search for nonlinear state estimation’. Int. Advance Computation Conf., India, 2013, pp. 918–924[29]Walia, G.S., Kapoor, R.: ‘Intelligent video target tracking using an evolutionary particle filter based upon improved cuckoo search’, Expert Syst. Appl., 2014, 41, (14), pp. 6315–6326[30]Gao, M., Li, L.‐L., Sun, X., et al: ‘Firefly algorithm (FA) based particle filter method for visual tracking’, Optik, 2015, 126, (18), pp. 1705–1711[31]Gao, M.L., He, X.H., Luo, D.S., et al: ‘Object tracking using firefly algorithm’, IET Comput. Vis., 2013, 7, (4), pp. 227–237[32]Kitagawa, G.: ‘Non‐Gaussian state‐space modelling of nonstationary time series’, J. Am. Stat. Assoc., 1987, 82, (400), pp. 1032–1063[33]‘CAVIAR Test Case Scenarios’. Available at http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/, accessed 15 May 2016[34]Stauffer, C., Grimson, W.: ‘Adaptive background mixture models for real‐time tracking’. IEEE Computer Society Conf. on Computer Vision Pattern Recognition, Fort Collins, June 1999, pp. 246–252[35]Chan, A.B., Mahadevan, V., Vasconcelos, N.: ‘Generalized Stauffer–Grimson background subtraction for dynamic scenes’, Mach. Vis. Appl., 22, (5), 2011, pp. 751–766

Journal

IET Computer Vision – Wiley

Published: Apr 1, 2017

Keywords: particle filtering (numerical methods); Monte Carlo methods; object tracking; optimisation; state estimation; state-space methods; spider monkey optimisation assisted particle filter; sequential Monte Carlo methods; particle representation; state-space model; recursive Bayesian filter; PF; robust object tracking framework; population-based heuristic optimisation algorithms; exploration; exploitation; dynamic resampling method; one dimensional state estimation problem; particle degeneracy; particle impoverishment

There are no references for this article.