Agriculture is becoming increasingly reliant upon accurate data from sensor arrays, with localization an emerging application in the livestock industry. Ground‑based time difference of arrival ( TDoA) radio location methods have the advantage of being lightweight and exhibit higher energy efficiency than methods reliant upon Global Navigation Satellite Systems (GNSS). Such methods can employ small primary battery cells, rather than rechargeable cells, and still deliver a multi‑ year deployment. In this paper, we present a novel deep learning algorithm adapted from a one‑ dimensional U-Net implementing a convolutional neural network (CNN) model, originally developed for the task of semantic segmentation. The presented model (ResUnet-1d) both converts TDoA sequences directly to posi‑ tions and reduces positional errors introduced by sources such as multipathing. We have evaluated the model using simulated animal movements in the form of TDoA position sequences in combination with real‑ world distributions of TDoA error. These animal tracks were simulated at various step intervals to mimic potential TDoA transmission inter‑ vals. We compare ResUnet-1d to a Kalman filter to evaluate the performance of our algorithm to a more traditional noise reduction approach. On average, for simulated tracks having added noise with a standard deviation of 50 m, the described approach was able to reduce localization error by between 66.3% and 73.6%. The Kalman filter only achieved a reduction of between 8.0% and 22.5%. For a scenario with larger added noise having a standard deviation of 100 m, the described approach was able to reduce average localization error by between 76.2% and 81.9%. The Kalman filter only achieved a reduction of between 31.0% and 39.1%. Results indicate that this novel 1D CNN U-Net like encoder/decoder for TDoA location error correction outperforms the Kalman filter. It is able to reduce average localization errors to between 16 and 34 m across all simulated experimental treatments while the uncorrected aver‑ age TDoA error ranged from 55 to 188 m. Keywords: Radiolocation, Machine learning, Encoder/decoder, Convolutional neural network Introduction exist between the types of geolocation employed includ- The development and implementation of precision farm - ing price, precision, accuracy, power consumption, and ing practices are enabled by location-aware platforms. the frequency of position updates. Geolocation systems Such platforms can track assets across holdings enabling include angle of arrival (AoA), Doppler approaches, more efficient management strategies. power on arrival (PoA), time of arrival (ToA) and time Animal tracking has increasingly become an active difference of arrival (TDoA). area of both research and applied innovation. Trade-offs The oldest forms of radio-tracking animals used AoA to triangulate the position of individuals fitted with a radio transmitter with the first publications appearing *Correspondence: firstname.lastname@example.org in the 1960s, such as work conducted on the early sum- Agriculture and Food, CSIRO, 4067 Brisbane, Australia mer activities of porcupines . Modern versions of this Full list of author information is available at the end of the article © The Author(s) 2021. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Wang et al. Anim Biotelemetry (2021) 9:26 Page 2 of 12 technique estimate the AoA by comparing the amplitude protocols such as LoRaWAN, for example, Kolmostar’s variation of an antenna array at a signal receiver and JEDI-200 module. typically six or more antennas are used for this purpose. Time difference of arrival (TDoA) localization meth - These techniques can yield very low power use depend - ods require no transmitter based location processing ing upon the duty cycle and strength of the transmission. and only short transmission bursts. Locations using this Satellite tracking of wildlife has a long history starting method are estimated by examining the time difference with Craighead Jr et al.  tracking elk via the Nimbus of a transmitted signal arriving at multiple fixed time- meteorological satellites using a bulky 11.3 kg collar in synchronized receivers. In this paper, we have chosen April of 1970. However, it was the creation of the ARGOS to examine Taggle’s proprietary TDoA localization sys- system in 1978 , utilizing the Doppler shift of a carrier tem, each location requires only 0.12 J of energy making frequency over successive transmissions, that initiated it suitable for tracking solutions using non-rechargeable the first generation of relatively accessible animal satel - batteries, allowing for a greater number of location trans- lite tracking devices. Early deployments using ARGOS to missions per energy consumed. Taggle’s radio transmit- track wildlife included basking sharks in June of 1982  ter operates in a frequency range of 912–927 MHz using and wandering albatrosses in 1989 . direct sequence spread spectrum modulation at a power The ARGOS system employs a repetition period output of 14 dBm. Its high capacity receivers can accom- between two consecutive payloads, of between 45 and modate at least 14,000 device transmission per hour. 200 s for as little as 360 ms (PPT-A3 Argos Specifica - Along with a message header, single transmissions can tion), to estimate the location of a platform transmitter hold 12–15.5 bytes of user data with a transmission dura- terminal. The accuracy of this system is based upon seven tion of around 300 ms. The theoretical TDoA localization location qualities ranging from 150 m to tens of kilome- accuracy of the Taggle’s system is approximate: ters, with a low power consumption using as little as 0.8 J speed of light per location estimate (assuming two ARTIC R2 transmis- accuracy = ≈ 5 m. (1) 4 × bandwidth sions plus amplification to 0.5 W). Power of arrival (PoA) localization methods rely upon However, the clock synchronization of the receivers is the received signal strength at a minimum of two receiv- ±20 nanoseconds resulting in a potential error of 12 m. ers. Implementation of PoA in LPWANs (low-power Menzies et al.  conducted a small-scale trial with wide area networks) over long distances is not practi- Taggel’s localization system using twelve Taggle tags on a cal due to the inverse square law and signal attenuation, plot, the size of which was approximately 5 ha. They found where the received signal strength quickly becomes too that the positions had a mean precision of ±22 m with an weak for the receiver to meaningfully differentiate small SD of 49 m. Ground-based TDoA location systems can changes in received signal strength. These techniques are experience substantial noise due to multipathing where most suitable for factory-scale localization. the time differences are exaggerated due to signal paths Time of arrival (ToA) localization methods use the that are not line-of-sight, this is the kind of error we time of reception of signals received by a roaming device seek to address in this study. The examined localization from multiple transmitters of a known location; the most method uses TDoA among four fixed receivers to esti - ubiquitous implementation of this technology is Global mate the origin of the transmission; the two most basic Navigation Satellite Systems (GNSS). Efficient imple - analytical methods are the Taylor series method  and mentations of GNSS networks, such as those using the Chan method  that can localize objects with minimal Ublox Zoe-M8B, require around 1.8 J to acquire a loca- error in the absence of multipathing. To address noise in tion from a cold start; this signal then needs to be re- TDoA location estimates, machine learning-based meth- transmitted adding additional energy overhead. For many ods are introduced for localization in an arbitrarily com- animal tracking systems using GNSS, it is the single larg- plex system. Current approaches to this problem fall into est power drain on the system. The advantage of GNSS two main methodologies. based systems is the lack of ground-based infrastructure; The first uses fingerprint references and machine learn - however, they require expensive space-based infrastruc- ing models to derive insights about the geometrical struc- ture. GNSS yield excellent spatial fidelity of around 2.5 ture of the environment which can provide information m, with some systems achieving cm accuracy. Modern about TDoA error and improve localization accuracy. approaches to ToA use LPWAN to unload the on-device processing to remote services over communication Kolmostar 48531 Warm Springs Blvd, Suite 407, Fremont, CA 94539 (www. kolmostar.com). Taggle Systems Pty Ltd. Sydney, N.S.W. 2000, Australia (taggle.com). W ang et al. Anim Biotelemetry (2021) 9:26 Page 3 of 12 de Sousa and Thomã Electronic  applied the Random denoising encoder/decoder algorithm based on this Forest algorithm embedded in a machine learning frame- framework with 1D CNN layers. The algorithm (ResU - work to extract a reference dataset of TDoA fingerprints net-1d) is a deep learning approach for TDoA localiza- in outdoor scenarios. In the experiment, four TDoA tion error correction, using noisy TDoA tracks to correct sensors were deployed in an area of 2 km in the City of for multipathing. The performance of this algorithm was Ilmenau in Germany, representative of a typical subur- tested on simulated animal track TDoA sequences with ban scenario with small buildings and spaced streets. The added noise derived from real-word data. The results empirical cumulative density function (CDF) used in the show that this algorithm can recover animal tracks from experiment showed 210 m of error for 65% of location noisy TDoAs. We then compared our approach with a estimates, compared to 300 m for the raw TDoA calcula- Kalman filter to see how our approach compared to this tions. Similarly, Alonso-González et al.  implemented widely employed strategy for reducing statistical noise in a neural network model to estimate the positions for time series data. TDoAs using an indoor fingerprint approach to predict The remainder of the article is organized as follows. In a transmitters positions in a 3D environment. They tested Sect. 2, we discuss the problem of TDoA localization in their model in a 4 × 4 × 3 m room; their experimental terrestrial systems. Section 3 describes the model archi- results indicated a substantial improvement in accuracy, tecture and the methodology of the experiment. Sec- with a best average error of 390 μm. tion 4 describes the animal movement simulation and the The second approach applies denoising neural net - method for generating the TDoA data. The final Sect. 5 works to reduce the TDoA error to improve localization presents the performance of the developed algorithm accuracy. Wu et al.  proposed a radial basis function in comparison to the non-corrected TDoA location (RBF) neural network to improve localization accuracy. estimates. They tested the model on simulated TDoAs within a 2D 500 × 500 m space with seven receivers, the root mean Problem overview and formulation square error (RMSE) of the localization was 17 m, while Classical TDoA localization methods assume that radio the Chan algorithm leads to an ≈ 30 m RMSE. Zhang signals travel without obstruction in line-of-sight with et al.  presented a novel localization algorithm based localization solutions based on solving the following upon a neural network ensemble to estimate the posi- hyperbolic equations: tions of objects in indoor multipathing environments. The ensemble method was tested on the simulated TDoA TDoA = ToA − ToA , ij i j (2) in a 2D 60 × 60 cm space. The best RMSE of localization was < 1 cm and the ensemble method also showed better where ToA is the time of arrival to the ith receiver, generalization and stability than a single neural network. defined as These approaches demonstrate the utility of machine learning models for localization in reducing the initial 2 2 ToA = (x − X ) + (y − Y ) . (3) i i i TDoA error or for correcting location estimates from noisy TDoAs. In this work, we present a novel denois- Here, X , and Y are the coordinates of the ith receiver sta- i i ing 1D convolutional neural network. The denoising tion, while x and y are the coordinates of the transmission encoder/decoder we propose has many similarities with origin track, and c is the speed of light in air. a denoising autoencoder; however, the autoencoder lacks The height of receivers and transmitters in TDoA net - the skip connections we employ. Denoising autoencoders works can vary adding systematic error; however, in are an extension of simple autoencoders and were origi- most practical cases, the introduced error is negligible. nally invented to reduce the risk of overfitting [2, 27]. For instance, the error introduced by a 100 m elevation Denoising autoencoders can be applied to remove the across 1 km is < 5 m. Of most concern is the occlusion effect of stochastic noise to inputs, for example, to clean of line-of-sight between transmitters and receivers due the noise from corrupted images. Convolutional layers to vegetation, topography, or man-made objects. Such emphasize local features of structured data, such as those obstructions can lead to increased path length, and hence evident in images or sequence data. The information the time of arrival, between a transmitter and receiver. from these local features help the model to reduce noise This multipathing phenomenon can add significant error from TDoA sequences and their associated movement to TDoA localization estimates. sequences. Diakogiannis  proposed a novel deep learn- In multipathing scenarios, signals always travel along ing framework for semantic segmentation of remotely a longer path than line-of-sight, therefore the error of sensed data; this framework consisted of stacked CNN ToAs is always positive, but the error for TDoAs, accord- layers in a U-Net-like backbone . We propose a ing to the Eq. (2) can be either positive or negative. Since Wang et al. Anim Biotelemetry (2021) 9:26 Page 4 of 12 Fig. 1 Histogram (bin 8 m) of sampled error distribution of TDoAs measured on a static reference tag (tag ID: 130114) deployed in Warina Park, Townsville, Australia. A total of 2215 transmissions were used to estimate the error distribution using two Taggle receivers (tower ID: taggle‑067 and taggle ‑058). The fitted distribution demonstrates the Gaussian nature of the error with an X ∼ 0 and σ ∼ 100 the real path of each signal traveled corresponds to a Fig. 2 ResUnet-1d architecture.The left (downward) branch is unique multipathing scenario, it is not possible to predict the encoder. The right (upward) branch is the decoder. Conv1D is the or model the error of TDoA from a single transmission. standard 1D CNN layer, and Conv1DN is the standard 1D CNN layer To get an estimate of a real-world TDoA error dis- with batch normalization.The B in data size represents the batch size, tribution, we placed a single static transmitter, tag ID and the N represents the sequence length of the input data time is 130114, in Warina Park, Townsville Australia (Lat. −19.279, Long. 146.771) and made 2215 localization transmissions at two-minute intervals (approximately 3 implemented. The network implements a conditioned days). The TDoA was examined by looking at two receiv - multitasking approach, estimating the semantic classes, ers in Townsville’s Taggle network, towers taggle-058 and their boundaries, and their distance transforms. taggle-067. The resulting error distribution in meters This model was chosen as it has a few advantages for is shown in Fig. 1, using a bin size of 8 m. The noise is the problem of TDoA positioning and localization error normally distributed and the problem can be framed as reduction. First, the U-Net backbone architecture is reducing Gaussian noise from TDoA measurements. recognized in the field of computer vision for achieving state-of-art image denoising [13, 14]. Second, the residual Model framework connections  allow for the efficient gradient propaga - This section gives an overview of the architecture of the tion in deep architectures, thus guaranteeing fast con- model ResUnet-1d (Sect. 3.1) which reduces the locali- vergence and improved performance. Heinrich et al.  zation errors of animal tracks. The ResUnet-1d model integrated ResNet into the fully convolutional neural combined two tasks, converting the TDoAs to posi- network (FCN) with U-Net architectures for Low-Dose tions and localization denoising. Section 3.2 introduces Computerized Tomography (CT) image denoising, show- the process of model training and the use of the trained ing that U-Net combined with ResNet yields the most model. promising result with an enhanced peak signal–noise ratio. Therefore, we have migrated this successful frame - work from the domain of image denoising and applied it Architecture to our TDoA animal tracking problem. The architecture In this study, we implement a modified 1D version of the of the framework is shown in Fig. 2. ResUnet-a model  that is designed for semantic seg- In the proposed model, ResUnet-1d, we introduce mentation of mono-temporal very high-resolution aerial several changes to the original ResUnet-a that make images. ResUnet-a uses a UNet encoder/decoder back- it suitable for our application. First, the TDoAs and the bone and residual building blocks with atrous convolu- positions are multi-channel one-dimension time-series tions for feature extraction. In the middle and at the end data, the 2D convolution layers in the model are replaced of the network, a pyramid scene parsing pooling layer is W ang et al. Anim Biotelemetry (2021) 9:26 Page 5 of 12 series tracks should match the length of the tracks on which the model will be applied. The input data are a sequence of noised TDoAs, with the shape 256 × N , where N is 3, which is the number of t t the TDoAs at each transmission. The simulated ground truth animal tracks, that were used to generate the noisy TDoAs are the ground truth values that the algorithm is trying to recover. The shape of the output track is 256 × 2 , as each position only has two coordinates, x, and y. The output of the model can predict denoised tracks using noised TDoAs as an input, hence reducing TDoA multipathing error. In principle, the proposed model could be trained by real-world animal tracks and TDoAs collected from the paddock. However, in practise, training of the deep learn- ing model requires thousands of data samples, which may not be feasible for most real-world applications of this problem. Furthermore, it is unlikely that an animal will be fitted with both a GNSS and TDoA tracking sys - tem outside of a research setting. We, therefore, propose the application of this model be proceeded by two initial steps, deployment of static tags to estimate the TDoA error distribution followed by the simulation of TDoAs tracks. We will introduce the simulation methods in the next section. The simulated data can be used to train the model and the trained model will be able to reduce the Fig. 3 The architecture of the HEAD block. The two inputs of this localization error. block are the outputs of the first Conv1DN in the encoder branch named First and the output of the final ResUnit in the decoder branch named Final.The B and N have same meaning as in Fig. 2 time Kalman filter We compared our results against those obtained with a Kalman filter to observe the efficacy of our approach to by 1D convolutions. Second, compared with seman- this widely employed method for noise reduction in time tic segmentation tasks, the time series denoising tasks series data. A Kalman filter is a recursive algorithm to should be simpler in both the input data format and the estimate the state of a dynamic system having certain difficulty of the tasks. Therefore, in ResUnet-1d, the types of random behavior and demonstrates the capabil- encoder part only consists of three ResBlock-a build- ity of noise reduction . We applied this approach to ing blocks followed by a 1D PSPPooling layer. Each reduce localization noise. feature extraction unit is a standard residual unit (we did The Kalman filter describes the system by the state vec - not use atrous convolutions). This shallower model can tor x = (x, y, v , v ) and updates the state vector and x y potentially prevent overfitting issues while reducing the error covariance matrix P in each iteration. To update computational burden. Lastly, as PSPPooling does not the state vector and error covariance matrix, we need the perform well on regression problems , the last PSP- state transmission matrix F, measurement noise covari- Pooling layer is replaced by an attention block which is ance matrix Q, system noise covariance matrix R, and the embedded in the HEAD block for increased performance, measurement vector z . In this case, the state transmis- details of this block are illustrated in Fig. 3. sion matrix is: 10 10 Methodology 01 01 F = . As the received TDoAs of each transmitter denote time- (4) 001 0 series data, we needed to segment the continuous time- 0 00 1 series data into fixed-length sequences. The proposed The measurement noise covariance matrix Q and system model expects 256 records in one piece of track data. But noise covariance matrix R are estimated from the dataset. it is a free parameter determined by the specific task, the The measurement vector z is the localization position only requirement is that the length of the training time t Wang et al. Anim Biotelemetry (2021) 9:26 Page 6 of 12 estimated from the raw TDoAs at each time t. The update movement simulation required specifying the prob- of the state vector and the error covariance matrix is abilities of transition between the random walk state and given by: correlated random walk state . We used a transition matrix to define the probabilities of transition between x¯ = Fx , (5) t t−1 states. The transition matrix is a square matrix where all values are probabilities, and the element at row i column P = FP F + Q, (6) j defines the probability of the individual changing from t t−1 state i to state j. The transition matrix in this study is as T T −1 same as the example in Quaglietta and Porto , which ¯ ¯ K = P H (HP H + R) , (7) t t t is x = x¯ + K (z − Hx¯ ), t t t t t (8) 0.995 0.005 (10) 0.01 0.99 P = (I − K H )P , (9) t t t The step length, in meters, for both states are set to a ran - dom number from the uniform distribution U(0, 1). where I is the identity matrix, x is the priori estimate of the state vector, and K is the Kalman filter gain. In each TDoA simulation time step, the Kalman filter estimates the state vector In the simulation, the coordinates of the receivers are: by combining the prior estimate of the state vector and (−2000, 2000) m, (−2000, 2000) m, (2000,−2000) m, and the measurement vector with the Kalman filter gain. In (2000, 2000) m, an area of 16,000 ha. To mimic real-world this work, we implement the Kalman filter using an open animal movements at different time scales the simula - source Kalman filter module, pykalman, on Github. tions recorded the positions of the track at different step intervals, N . We converted the recorded positions into Data simulation and preprocessing ToA using Eq. (3). The TDoAs were obtained by the sub - Within the field of animal movement behavior, the mod - stitution of two ToAs using Eq. (2). The input TDoAs in eling of movement data is implemented in many ways. our model are TDoA , TDoA ,and TDoA . For com- 12 13 14 Quaglietta and Porto  introduced an algorithm, Sim- putational simplicity, we multiplied the derived times by Riv, to simulate individual-based, spatially explicit move- the speed of light, c for all the input TDoAs to get dis- ments in river networks and heterogeneous landscapes. tances. We added Gaussian noise N (0, σ) , based upon In this study, we simulate a cows’ movement using this the static tag observation, into the simulated TDoAs to approach on a totally homogeneous landscape. generate noised TDoAs, where σ is the standard devia- tion of the TDoA error distribution. In this work, we Animal track modeling evaluate our model’s ability to reduce noise for TDoAs Animal movements are considered to be Brownian exhibiting error standard deviations of either 50 m or 100 motion and multistate. The main states of the movement m. include random walking, correlated random walking, and rest. The random walk state is a random movement state, Training data simulation and preprocessing in which the direction of the steps is completely inde- For the training simulation dataset, we first generated pendent. The correlated random walk means the direc - multiple animal movement tracks within a virtual pad- tion taken in one step by an individual animal should be dock. Each track started from a random position within correlated with the direction of the previous step [23, 26]. the paddock and picked a direction for the first step with The correlation, which is the turning angle concentra - equal probabilities. The subsequent steps were gener - tion parameter of the wrapped normal distribution, has ated with the method discussed in Sect. 4. As discussed a value between [0, 1], where 0 means there is no corre- in Sect. 4.1, we assumed the device recorded the posi- lation between two steps (yielding a random walk state), tion of the animal every N steps. The number of steps and 1 means the direction does not change. In this study, in each simulated track sequence was N × 256 . We then we use 0.98 (as chosen by Quaglietta and Porto ) as down-sample the track by extracting the first position the value of correlation. The resting state corresponds in every N steps. The down-sampling mimics receivers to a state where the individual animal does not change only recording the animal’s positions at set time intervals, position. the length of all tracks after down-sampling is 256. We Following Quaglietta and Porto , we assume the saved these down-sampled tracks as the target outputs cows are Lévy-like walkers who alternate between ran- of the model. When the tracks were generated, we used dom walks and correlated random walks. This multi-state the method detailed in Sect. 4.1 to calculate the TDoA W ang et al. Anim Biotelemetry (2021) 9:26 Page 7 of 12 sequence of each track and then add random Gaussian Table 1 Hyperparameters of the ResUnet-1d noise. Hyperparameters Value The resultant noised TDoAs (model input) and the Depth 3 corresponding ground truth positions (model output) Number of filters 32 pairs were split into a training dataset and a testing data- Batch size 256 set in the fraction of 8 : 2. Meeting the requirements of −4 Learning rate the ResUnet-1d, the values of the model inputs and Optimizer Adam the model outputs should be in the range of [0, 1], we re-scaled the input and output data in both training and Loss Function L1Loss testing datasets through min–max normalization. As all data used in this work are simulated, we do not need to tackle the issue of missing. However, the occurrence of which is equivalent to a time interval ranging from 0.6 − 30 missing data in time-series data is very common, we will min when considering the average speed of a cow [20, 25]. discuss this issue in Sect. 5.3. The values of this time interval would be much higher for grazing individuals. Evaluation The optimized hyperparameters of ResUnet-1d are In this section, the predictive accuracy of the ResUnet- summarized in Table 1. For all models, we used the Adam 1d is evaluated by using simulated animal movement −4  optimizer, with a learning rate of 10 . We chose the data described in the previous section. The localization L1Loss loss function to obtain the best training perfor- results from our model are compared with the results of mance for ResUnet-1d. Our model was built and trained the analytical method implemented in Menzies et al. . using the MXNet deep learning library , under the GLUON API. Each of the models was trained on 8000 sim- Design of experiment ulated tracks with a batch size of 256 on a single NVIDIA In this work, we implemented the ResUnet-1d model Tesla P100 GPU using the CSIRO’s HPC facilities. We to reduce localization error, the main source of this error used 2000 simulated tracks to test the performance of our in real-world TDoA deployments is multipathing. We model. evaluated our model on the simulated Lévy-like tracks with different recording steps N and two different TDoA Performance of ResUnet‑1d error standard deviations σ . We aimed to investigate if Figures 4 and 5 illustrate simulated tracks, the orange lines ResUnet-1d could reduce the localization error signifi - are the ground truth movement track generated from cantly and how the ResUnet-1d model performance the animal movement simulations and the faded blue varied with step interval N and the original TDoA error’s points are the measured positions calculated with noised standard deviation σ. TDoAs. The green lines represent the recovered tracks by As observed from the static tag in Townsville, the the ResUnet-1d model, they reproduce the shape of the error distribution of TDoAs is Gaussian, and the stand- ground truth tracks and recovered most of their features. ard deviation of the error in the range of ∼ 100 m. We The gray lines are the tracks recovered by the Kalman fil - use this value to simulate a urban-like environment. ter and also exhibit noise reduction. However, the tracks However, we suspect that the urban environment has recovered by the Kalman filter have larger errors than those increased multipathing issues due to large metallic mov- recovered by our ResUnet-1d model. ing objects, such as vehicles, and other effects of the built To evaluate the performance of the model quantitatively, environment. In more remote agricultural areas, these we measure the root mean square error (RMSE) of noised aforementioned obstructions are greatly reduced, thus tracks and recovered tracks to the ground truth tracks in we can hypothesize that the standard deviation of the the following way TDoA error in these locations is likely to be lower. To mimic this expectation, we chose to halve the standard 2 2 O O RMSE = (x − x ) + (y − y ) , (11) i i i i deviation of the error to 50 m to emulate a more rural i=1 setting. The choice of this value for the SD is corrobo - rated by Menzies et al. . where the (x, y) represents the position of noised tracks In our simulation, the step size was selected from a uni- or recovered tracks, and (x ˆ , y ˆ) represents the position of form distribution U(0, 1) m to mimic continuous move- ground truth tracks. The probability density function ment. As discussed in Sect. 4.2, we down sampled the track (PDF) of the errors is illustrated in Figs. 6 and 7. In both positions by recording only one position in every N posi- figures, the distribution of the error of the ResUnet-1d tions, where N ∈ [10, 20, 40, 60, 80, 100, 200, 300, 500] s Wang et al. Anim Biotelemetry (2021) 9:26 Page 8 of 12 Fig. 4 Examples of simulated ground truth tracks (orange), ResUnet-1d corrected tracks (green) and Kalman filter corrected tracks (gray). Blue points are the positions from noised TDoAs. The black vertical and horizontal lines indicate 100 m distance. The standard deviation of the raw TDoA noise was σ = 50 m. Inset black numbers indicate step down‑sampling increment Fig. 5 Examples of simulated ground truth tracks (orange), ResUnet-1d corrected tracks (green) and Kalman filter corrected tracks (gray). Blue points are the positions from noised TDoAs. The black vertical and horizontal lines indicate 100 m distance. The standard deviation of the raw TDoA noise was σ = 100 m. Inset black numbers indicate step down‑sampling increment W ang et al. Anim Biotelemetry (2021) 9:26 Page 9 of 12 Fig. 6 Probability density function for localization errors after noise with a standard deviation of 50 m had been applied to simulated tracks. Results are displayed for uncorrected TDOAs, ResUnet-1d recovered tracks and Kalman filter recovered tracks. Inset black numbers indicate step down‑sampling increment recovered tracks is significantly narrowed and of a lower deviation of 100 m the ResUnet-1d approach was able value compared with the distribution of the original error to reduce average localization error by between 76.2% of the noised tracks. Comparison of the distribution and 81.9%. The Kalman filter only achieved a reduction of of the error of the ResUnet-1d recovered tracks and between 31.0% and 39.1%. Kalman filter recovered tracks confirm what Figs. 4 and Results indicate that ResUnet-1d is able to reduce 5 illustrate, the distribution of the error from the tracks average localization errors to between 16 and 34 m recovered by ResUnet-1d are clearly narrower and across all simulated experimental treatments while the lower than those from Kalman filter. Therefore, we dem - corresponding uncorrected average TDoA location onstrate that our ResUnet-1d model can effectively error ranged from 55 to 188 m, and in the case of the reduce localization error introduced by random Gaussian Kalman filter between 48 and 115 m. Our ResUnet-1d errors in TDoA position estimates, and can outperform a approach was robust to the down sampling (step) inter- Kalman filter. val; however, the Kalman filter error correction tended to Figure 8 compares the RMSE of uncorrected and cor- improve at greater step intervals but never approached rected tracks from the two simulations with different the levels observed by ResUnet-1d. On average across noise levels. The RMSE of the two recovered tracks by the two noise treatments ResUnet-1d exhibited a 54% ResUnet-1d are significantly lower than the RMSE (noise σ = 50 m) and 44% (noise σ = 100 m) difference in of the two noised tracks and the tracks recovered by localization error correction. Kalman filter. When the original localization error was lower, orange lines, the efficacy of the recovered tracks Discussion dropped slightly. On average, for simulated tracks hav- Conventional analytic methods for calculating locations ing added noise with a standard deviation of 50 m, the based upon TDoAs work well in the absence of multip- ResUnet-1d approach was able to reduce localization athing; in real-world settings, multipathing degrades the error by between 66.3% and 73.6%. The Kalman filter performance of these systems. Previous machine learn- only achieved a reduction of between 8.0% and 22.5%. ing methods have shown a reduction in localization noise For a scenario with larger added noise having a standard but have only considered a single static position. Wang et al. Anim Biotelemetry (2021) 9:26 Page 10 of 12 Fig. 7 Probability density function for localization errors after noise with a standard deviation of 100 m had been applied to simulated tracks. Results are displayed for uncorrected TDOAs, ResUnet-1d recovered tracks and Kalman filter recovered tracks. Inset black numbers indicate step down‑sampling increment Due to the nature of track localization, the positions heterogeneous environment, such as creek lines, would of neighboring points have information which can help elicit less random animal movements resulting in a contribute to a reduction in localization error. There - higher correlation between neighboring positions. This fore, a model which can combine information about kind of correlation should translate to increased per- neighboring points to make an enhanced prediction formance when compared to animal tracks simulated should provide a better solution to this problem. A under the assumption of a homogeneous environment. Kalman filter is a conventional method to reduce noise The use of this method requires a sequence of the in just such a scenario. The method proposed in this TDoAs without missing data. Missing data are likely work implements a CNN layer to consider the connec- to be common in TDoA localization networks. In real tion between neighboring positions for animal move- deployments, the signal transmission can not only be ment tracks. The combination of the CNN layer and reflected by objects but also blocked by them. Moreover, U-Net like encoder/decoder architecture enhances network communication outages along with transmitter the ability of the model to reduce noise in the data. maintenance or failure will also lead to data gaps. The complexity of the model we proposed here simul - It would be possible to implement a simple interpola- taneously considers and extracts hidden patterns from tion prepossessing step to the proposed model to recover the random noise over a long portion of the track. A missing data. Two promising methods for enhance the Kalman filter only looks at the position before the given interpolation are the implementation of a momentum position and assumes a linear function to make the term or a Taylor approximation. Zhang et al.  pro- prediction. By testing this algorithm on a correlated posed a new sequence-to-sequence imputation model for random walk simulation, we have made the assump- recovering missing data in wireless sensor networks. This tion of a homogeneous paddock environment using method could also be employed in the data prepossessing SimRiv. It is recognized that SimRiv does not take into pipeline to address missing data. account the impacts of a heterogeneous landscape, We acknowledge that the simulated data in this paper where the willingness of an animal to cross a specific represent an oversimplification of this problem; we environment is not taken into account. We anticipate a expect real-world deployments to pose challenges such as W ang et al. Anim Biotelemetry (2021) 9:26 Page 11 of 12 Fig. 8 Comparison of localization errors for each step interval (n=2000 tracks) for uncorrected TDoA location estimates, ResUnet-1d recovered location estimates and Kalman filter location estimates. Results are given for both a standard deviation of 50 m and 100 m noise applied to simulated tracks shifting error means and standard deviations for TDoA and is more robust to changes in the down-sampling data spatially. This paper does, however, demonstrate the interval. These findings need to be validated in a work - potential utility for this approach. We intend to take this ing cattle station in conjunction with an assessment of simulated model and apply it to a working cattle station missing data prepossessing. to see if the demonstrated gains hold true for a real-world Acknowledgements scenario. This will include the deployment of multiple The authors acknowledge the support of the Scientific Computing team static nodes that will inform the TDoA stochasticity both of CSIRO. We would also like to acknowledge Gordon Foyster and Richard Keaney for their useful discussions and comments on this manuscript. spatially and temporally. Open Access This article is distributed under the terms of the Creative Commons Attribu‑ Conclusion tion 4.0 International License (http:// creat iveco mmons. org/ licen ses/ by/4. 0/), In this paper, we developed and investigated a 1D which permits unrestricted use, distribution, and reproduction in any medium, CNN-based U-Net like encoder/decoder model for provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were denoising TDoA position estimates for animal track- made. ing using simulated animal movement data (ResU- net-1d). We have demonstrated that our model can Author Contributions LW and FD conceived of the presented idea. LW, FD, SM, and NB planned the successfully recover simulated animal movement tracks experiments. LW carried out data simulation, deep learning model develop‑ from noised TDoA sequences, and reduce localization ing, training and data interpretation. FD supported the development of deep error by between 66.3% and 81.9%. As the results for learning model architecture. LW took the lead in writing the manuscript in consultation with FD, SM and NB. All authors discussed the results and ResUnet-1d tracks with different step intervals do contributed to the final manuscript. All authors read and approved the final not show a clear trend, it would be possible the algo- manuscript. rithm to be implemented for animal tracks constructed Funding from lower frequency transmissions, at down-sampling This document is the result of a research project funded by Advance Queens‑ intervals greater than 500 m. Our model outperforms land Innovation Partnerships (AQIP)—Smart Ear Tag for Livestock, 2016. a Kalman filter for this TDoA noise reduction problem Wang et al. Anim Biotelemetry (2021) 9:26 Page 12 of 12 Availability of data and materials 13. Komatsu R, Gonsalves T. Eec ff tiveness of u‑net in denoising rgb images. The datasets during and/or analyzed during the current study are available Comput Sci Inf Techn. 2019; pp 1–10, https:// doi. org/ 10. 5121/ csit. 2019. from the corresponding author on reasonable request. 90201. 14. Liu D, Wen B, Liu X, Wang Z, Huang TS. When image denoising meets high‑level vision tasks: a deep learning approach. 2017. arXiv preprint Declarations arXiv: 17060 4284https:// doi. org/ 10. 24963/ ijcai. 2018/ 117. 15. Marselli C, Daudet D, Amann HP, Pellandini F. Application of kalman Ethics approval and consent to participate filtering to noisereduction on microsensor signals. In: Proceedings du Not applicable. Colloque interdisciplinaire en instrumentation, C2I, 18‑19 novembre 98, pp. 443‑450, Ecole Normale Supérieure de Cachan, France, pp 443–450. Consent for publication Not applicable. 16. Marshall WH, Gullion GW, Schwab RG. Early summer activities of porcu‑ pines as determined by radio‑positioning techniques. J Wildlife Manage. Competing interests 1962;26(1):75–9. https:// doi. org/ 10. 2307/ 37981 69. The authors declare that they have no competing interests. 17. Menzies D, Patison KP, Fox DR, Swain DL. A scoping study to assess the precision of an automated radiolocation animal tracking system. Comput Author details Electron Agricult. 2016;124:175–83. https:// doi. org/ 10. 1016/j. compag. 1 2 Agriculture and Food, CSIRO, 4814 Townsville, Australia. Agriculture 2016. 04. 001. and Food, CSIRO, 4067 Brisbane, Australia. ICRAR , The University of West‑ 18. Mo J, Deng Z, Jia B, Bian X. A pseudorange measurement scheme ern Australia, 6009 Perth, Australia. Data61, CSIRO, 6014 Perth, Australia. based on snapshot for base station positioning receivers. Sensors. eResearch Centre, James Cook University, 4814 Townsville, Australia. 2017;17(12):2783. https:// doi. org/ 10. 3390/ s1712 2783. 19. Morales JM, Haydon DT, Frair J, Holsinger KE, Fryxell JM. Extracting Received: 10 February 2021 Accepted: 18 June 2021 more out of relocation data: building movement models as mixtures of random walks. Ecology. 2004;85(9):2436–45. https:// doi. org/ 10. 1890/ 03‑ 0269. 20. O’Driscoll K, Schutz MM, Lossie A, Eicher S. The effect of floor surface on dairy cow immune function and locomotion score. J Dairy Sci. References 2009;92(9):4249–61. https:// doi. org/ 10. 3168/ jds. 2008‑ 1906. 1. Alonso‑ González I, Sánchez‑Rodríguez D, Ley‑Bosch C, Quintana‑Suárez 21. Priede I. A basking shark (cetorhinus maximus) tracked by satellite MA. Discrete indoor three‑ dimensional localization system based on neu‑ together with simultaneous remote sensing. Fish Res. 1984;2:201–16. ral networks using visible light communication. Sensors. 2018;18(4):1040. https:// doi. org/ 10. 1016/ 0165‑ 7836(84) 90003‑1. https:// doi. org/ 10. 3390/ s1804 1040. 22. Quaglietta L, Porto M. Simriv: an r package for mechanistic simulation of 2. Bengio Y, Delalleau O. Justifying and generalizing contrastive divergence. individual, spatially‑ explicit multistate movements in rivers, heterogene‑ Neural Computat. 2009;21(6):1601–21. https:// doi. org/ 10. 1162/ neco. ous and homogeneous spaces incorporating landscape bias. Movement 2008. 11‑ 07‑ 647. Ecology. 2019;7. https:// doi. org/ 10. 1186/ s40462‑ 019‑ 0154‑8. 3. Chan Y T, Ho K. Joint time‑scale and tdoa estimation: analysis and fast 23. Reynolds AM. Mussels realize weierstrassian lévy walks as composite cor‑ approximation. IEEE Trans Signal Process. 2005;53(8):2625–34. https:// doi. related random walks. Sci Rep. 2014;4:4409. https:// doi. org/ 10. 1038/ srep0 org/ 10. 1109/ TSP. 2005. 850336. 4. Chen T, Li M, Li Y, Lin M, Wang N, Wang M, Xiao T, Xu B, Zhang C, Zhang Z. 24. Ronneberger O, Fischer P, Brox T. U‑net: Convolutional networks for Mxnet: a flexible and efficient machine learning library for heterogeneous biomedical image segmentation. In: International Conference on Medical distributed systems. 2015. arXiv preprint arXiv: 15120 1274https:// arxiv. image computing and computer‑assisted intervention, Springer, 2015; p org/ abs/ 1512. 01274 v1 234–241, https:// doi. org/ 10. 1007/ 978‑3‑ 319‑ 24574‑4_ 28. 5. Clark DD. Overview of the argos system. Proc Oceans. 1989;3:934–9. 25. Swain D, Wark T, Bishop‑Hurley G. Using high fix rate gps data to deter ‑ https:// doi. org/ 10. 1109/ OCEANS. 1989. 586711. mine the relationships between fix rate, prediction errors and patch 6. Craighead Jr FC, Craighead JJ, Cote CE, Buechner HK. Satellite and ground selection. Ecol Model. 2008;212(3):273–9. https:// doi. org/ 10. 1016/j. ecolm radiotracking of elk. NASA, Washington Animal Orientation and Naviga‑ odel. 2007. 10. 027. tion. 1972. https:// ntrs. nasa. gov/ citat ions/ 19720 017418. 26. Turchin P. Quantitative analysis of movement: measuring and modeling 7. de Sousa MN, Thomä RS. Enhanced localization systems with multipath population redistribution in animals and plants. Sinauer Associates. 1998. fingerprints and machine learning. In: 2019 IEEE 30th Annual Interna‑ https:// doi. org/ 10. 1086/ 393125. tional Symposium on Personal, Indoor and Mobile Radio Communica‑ 27. Vincent P, Larochelle H, Bengio Y, Manzagol PA. Extracting and composing tions (PIMRC), IEEE, 2019; pp 1–6, https:// doi. org/ 10. 1109/ PIMRC. 2019. robust features with denoising autoencoders. In: Proceedings of the 89041 20. 25th international conference on Machine learning, 2008;p 1096–1103, 8. Diakogiannis F, Waldner F, Caccetta P, Wu C. Resunet‑a: a deep learning https:// doi. org/ 10. 1145/ 13901 56. 13902 94. framework for semantic segmentation of remotely sensed data. ISPRS J 28. Wu L, Li F, Chen S. A improved wireless location algorithm in nlos envi‑ Photogram Remote Sens. 2020;162:94–114. https:// doi. org/ 10. 1016/j. isprs ronment. Inform Technol J. 2013;12(24):8563. https:// doi. org/ 10. 3923/ itj. jprs. 2020. 01. 013. 2013. 8563. 8569. 9. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 29. Zhang YF, Thorburn PJ, Xiang W, Fitch P. Ssim—a deep learning approach In: Proceedings of the IEEE conference on computer vision and pattern for recovering missing time series sensor data. IEEE Internet Things J. recognition, 2016; pp 770–778, https:// doi. org/ 10. 1109/ CVPR. 2016. 90. 2019;6(4):6618–28. https:// doi. org/ 10. 1109/ JIOT. 2019. 29090 38. 10. Heinrich MP, Stille M, Buzug TM. Residual u‑net convolutional neural 30. Zhang Z, Jiang F, Li B, Zhang B. A novel time difference of arrival localiza‑ network architecture for low‑ dose ct denoising. Curr Direct Biomed Eng. tion algorithm using a neural network ensemble model. Int J Distrib 2018;4(1):297–300. https:// doi. org/ 10. 1515/ cdbme‑ 2018‑ 0072. Sensor Netw. 2018;14(11):1550147718815798. https:// doi. org/ 10. 1177/ 11. Jouventin P, Weimerskirch H. Satellite tracking of wandering albatrosses. 15501 47718 815798. Nature. 1990;343(6260):746–8. https:// doi. org/ 10. 1038/ 34374 6a0. 12. Kingma DP, Ba J. Adam: a method for stochastic optimization. 2014. arXiv preprint arXiv: 14126 980https:// arxiv. org/ abs/ 1412. 6980v9.
Animal Biotelemetry – Springer Journals
Published: Aug 1, 2021
Keywords: Radiolocation; Machine learning; Encoder/decoder; Convolutional neural network