Towards Bandwidth Guaranteed Virtual Cluster Reallocation in the Cloud

Towards Bandwidth Guaranteed Virtual Cluster Reallocation in the Cloud Abstract Cloud data center traffic is experiencing a rapid growth as more and more data-intensive applications are required to process big data in a cloud data center. Although fat-tree networks own rich path multiplicity, and have been widely adopted as network topologies in cloud data center networks to transmit vast bisection bandwidth, they lead to bandwidth-related bottlenecks. To address this issue, in this paper, we propose a traffic-aware virtual cluster reallocation approach via biogeography-based optimization to allocate some reallocated virtual machines (VMs) as compact as possible with those VMs in the same virtual clusters. In order to validate our approach, we build a system model to perform a thorough evaluation of its performance. Experimental results show that our proposed approach outperforms six existed approaches in term of total transmission cost, total processing time, and total network resource consumption. 1. INTRODUCTION Recent improvement of information technology already results in a wide exploitation of cloud computing, in which several prior concepts and technologies are given a new perspective [1], such as virtualization technology [2] and data center. In this computing paradigm, its background support technology is data-intensive computing technology [3] and its computing resources are provided for cloud users on a pay-per-use basis [4]. Data-intensive computing is a kind of parallel computing applications, it takes most of their processing time to I/O (i.e. the movement and manipulation of data). The parallel processing of data-intensive applications typically partitions or subdivides the processed data into multiple segments, which can be processed independently using the same executable application program in parallel on an appropriate computing platform, and then the processing results are reassembled to produce the completed output data. Thus, the greater the aggregate distribution of the data, the more the benefit is in the parallel processing of the data [5]. Generally, a set of data-intensive applications (i.e. cloud services) are processed in a cloud. Each data-intensive application can be divided into multiple tasks to jointly process the partitioned or subdivided multiple segments, in which each task is processed by a virtual machine (VM). Massive scalable resources are dynamically organized into many VMs to address cloud services in a cloud data center. Their multiple isolated execution environments constitute the resources of a single physical machine (PM). These VMs are usually instantiated to perform a user’s specific computation. Once a VM is instantiated, it can be used to process the target task such that the VMs are designed to handle cloud services on an on-demand cloud basis for the final users. After processing the task, the VM is released and the PM can accommodate another VM. When a cloud provider wants to create a number of VMs, an important issue is the VM (re)allocation problem [6], which is essential in a cloud data center. This is because most PM hardware capacities are under-utilized. The virtualization technology is exploited to greatly improve the resource utilization of PMs, such as higher per-PM utilization, increased flexibility and availability, and reduction in hardware costs and operational expenses related to power, cooling, physical space, etc. Recently cloud data centers have emerged as common platforms for serving data-intensive applications: a variety of system architectures have been designed to process these data-intensive applications, such as MapReduce, Hadoop and HPCC. In such a framework, cloud users usually request a set of VMs (i.e. virtual cluster) rather than an individual VM; the virtual cluster can jointly process these data-intensive applications and its intermediate results are transferred iteratively through multiple stages. Further, the flows not only between VMs but also into the Internet are created by the traffic generated by the application [7]. Therefore, it often accounts for most of the processing time of a data-intensive application. For example, more than 50% of the processing time is consumed while transferring the data of 26% of the jobs in a Facebook data center [8]. As the data-intensive applications increase, the traffic will experience rapid growth; the global cloud data center traffic will be nearly a tripling of data from 2015 to 2020 with a combined annual growth rate of 27%, more specifically, from 4.7 ZB/year in 2015 to 15.3 ZB/year in 2020 [9]. Thus, the bandwidth resources, especially the bandwidth usage in a network core, may experience a bottleneck [10]. Moreover, interference of the bandwidth-related bottleneck could lead to unpredictable processing times and hurt cloud service quality, causing financial losses for users. To achieve expected network performance and to avoid the network bottlenecks, one strategy used is the exploitation of the network topology (e.g. fat-tree networks [11]). The advantages of this topology include (1) the exploitation of commodity Ethernet switches; (2) the relief of bandwidth-related bottlenecks through rich path multiplicity. However, only such strategy is not enough. Another strategy is the exploitation of a traffic-aware virtual cluster (re)allocation approach, it allocates the VMs in the same virtual cluster to make them as compact as possible to have less transmission cost taken by the above flows. That is, these flows visit the lower number of switches and links. Further, the network bottlenecks can be relieved. In this paper, we propose a traffic-aware virtual cluster reallocation (TVCR) approach via biogeography-based optimization (BBO) [12], abbreviated as TVCR, to address network bottlenecks for reallocated VMs in some virtual clusters. We first map the foregoing optimization problem to an ecosystem model of the BBO algorithm and propose TVCR to address the single-objective optimization problem. And then we perform a thorough performance evaluation of TVCR in terms of the total transmission cost, total processing time and total network resource consumption. 1.1. Organization Section 2 presents the background and related work. Section 3 introduces the technical details of our proposed approach; Section 4 provides the performance evaluation of our approach; Section 5 concludes with research recommendations. 2. BACKGROUND AND RELATED WORK As we all know, virtual cluster (re)allocation is a useful approach for many cloud services that require expected network performance, especially, when considering the more and more data-intensive computing services in today’s cloud data centers [13]. Next, we will introduce some relevant research regarding the traffic-aware virtual cluster (re)allocation problem. 2.1. Traffic Pattern or Matrix Most of the existing traffic-aware virtual cluster (re)allocation approaches focus on resources and communication. Meng, Papps and Zhang [14] examined the traffic in real-data center-like systems and proposed a Traffic-aware VM placement scheme (TVMPP) to improve the network scalability. They formulated the problem of assigning VMs to PMs to minimize the total communication costs. This goal can be achieved by placing VM pairs with a large mutual traffic rate in PMs with close proximity. However, TVMPP allocates only one VM on each PM. In light of their study, many researchers [15–18] have observed that optimizing the VM placement is crucial and studied optimization models for VM placement. Dias and Costa [15] proposed a traffic-aware VM placement algorithm based on the current traffic matrix, CPU and memory usage to allocate VMs to the data center PMs. By analyzing the formation of community patterns in terms of traffic using graph theory, these VMs which exchange high amounts of data are aggregated and allocated to PMs as close as possible to reduce traffic congestion. Breitgand, Epstein and Glikson [16] studied the problem of placing VM instances on physical containers in such a way to reduce communication cost and latency. You, Tang and Ding [17] proposed a network-aware VM allocation algorithm with the aim of reducing the bandwidth usage in modern virtualization-based data centers. Biran et al. [18] proposed a new network-aware VM placement problem, called Min Cut Ratio-aware VM Placement (MCRVMP); they strived to allocate a placement that not only satisfies the predicted communication demand but is also resilient to demand time-variations. Furthermore, most of the related works formulate the problem as bin-packing related optimization problems and consider the physical network topology only through the traffic pattern. As far as we know, the exception is the study of Cohen et al. [19], which concentrated merely on the networking aspects and considered the VM placement problem with intense bandwidth requirements. They focused on maximizing the benefit from the overall traffic sent by VMs to a single point in the data center (i.e. the root), proposed an algorithm and simulated different widely used data center network topologies. However, the defined problem is very limited from a scenario perspective. Then, Ilkhechi, Korpeoglu and Ulusoy [20] came up with the problem by generalizing the mentioned problem to a scenario including more than one root or sink. 2.2. Data Access Latency Currently, more and more cloud services are data-intensive applications that require the processing of a large data set. To minimize the impact of data access latencies between processing nodes and data nodes and completion times, Kuo, Yang and Tsai [21] proposed an approximation algorithm for the VM placement problem to minimize the maximum access latency. Alicherry and Lakshman [22] considered lower bandwidth usage costs of cloud applications for optimal placement of computational nodes and presented algorithms for assigning VMs to data nodes that minimize various latency metrics under different constraints. 2.3. Network Cost To minimize the network cost caused by network traffic in cloud data centers, Jiang et al. [23] solved a joint tenant placement and route selection problem to minimize traffic costs by exploiting the multipath routing capability and dynamic VM migration. Li et al. [24] and Fukunaga, Hirahara and Yoshikawa [25] proposed optimized VM placement. However, compared with TVCR, they did not take cloud data center topology into account and considered traffic demands only to perform PM-level consolidation. As a result, network capacity violations in a data center are possible. 2.4. Network Bandwidth While both TVMPP and MCRVMP assume static and well-defined traffic demands, measurements from production data centers show that the network bandwidth demands of VMs are dynamic. Therefore, Zhu et al. [26] proposed an abstraction model of heterogeneous VM network bandwidth allocations and mathematically proved the problem of VM allocations with heterogeneous bandwidth demands to be NP-hard. Meanwhile, they proposed an online heuristic VM assignment algorithm to place the VMs with heterogeneous bandwidth demands into an appropriate subtree of the data center network. Jin et al. [27] studied efficient VM placement in data centers with multiple deterministic and stochastic resources. They mentioned that the bandwidth demands of VMs in data centers can be approximated based on the normal distribution [28]. To better represent the uncertainty of the future bandwidth demand, Wang, Meng and Zhang [29] used random variables to characterize the future bandwidth usage. They formulated the VM consolidation problem into a NP-hard Stochastic Bin-Packing problem and introduced a new heuristic approach to solve it. 2.5. Energy Efficiency To reduce the energy consumption of PMs and switches in a cloud data center, da Silva and da Fonseca [30] introduced the topology-aware VM placement algorithm to allocate groups of communicating VMs to the data centers, and exploited small regions of the data center network to consolidate the network flows produced by these VMs. The proposed algorithm avoids network bottlenecks, and thus accepts more requests of VMs without compromising the energy efficiency. Compared with previous related approaches, we first exploit the PM fault prediction model to anticipate the deteriorating PM, which is selected on the basis of the CPU temperature model in [31]. That is, when the temperature exceeds the upper threshold of the normal CPU temperature range (e.g. 68°C) for the duration, the PM is considered to be deteriorating. Then, we propose an approach via the BBO algorithm to allocate the reallocated VMs to healthy PMs with the objective of minimizing total transmission cost. Finally, we build a system model to evaluate our proposed approach. 3. PROPOSED TVCR APPROACH In this section, we describe the execution flow of TVCR, as shown in Fig. 1. The TVCR includes an initial virtual cluster allocation (IVCA) module, PM fault prediction module, traffic-aware virtual cluster reallocation module, data-intensive application scheduler module and task scheduler module. Figure 1. View largeDownload slide The execution flow chart of TVCR. Figure 1. View largeDownload slide The execution flow chart of TVCR. As shown in Fig. 1, the TVCR first exploits IVCA approach [31] to initially allocate a set of VMs to a set of PMs. In the IVCA algorithm, when a VM is allocated to a PM, it first identify all other VMs which are in the same virtual cluster as the VM. If such VMs exist, the VM will be allocated to the same subnet or pod as these VMs. Otherwise, it will be allocated such that overall energy consumption is minimized. Note that a virtual cluster is constructed by choosing randomly three VMs when creating the VMs, e.g. a general application model in Section 4.1. Next, the PM fault prediction module is exploited to anticipate a deteriorating PM. For example, PM2 hosts multiple VMs (e.g. vm1, vm3, vm6, vm7, vm8, vm14 and vm15) and is detected to deteriorate. And then the traffic-aware virtual cluster reallocation module is exploited to identify some near-optimal and healthy target PMs (e.g. PM1, PM3 and PM5) for hosting these VMs on the PM2. Note that these reallocated VMs are not isolated; they are all in some virtual clusters. Thus, in the process of reallocation, the total transmission cost must be minimum. Finally, to analyze the performance of TVCR, the data-intensive application scheduler module is exploited to assign some cloud services (i.e. a set of data-intensive applications) to these virtual clusters including the reallocated VMs, and then the task scheduler module is invoked to assign each task of these data-intensive applications to a VM of these virtual clusters. 3.1. Traffic-Aware Virtual Cluster Reallocation When a virtual cluster jointly processes a data-intensive application, its network resource consumption and processing time have a direct relationship with the transmission cost. This relationship is observed because the transmission cost is lower and the communication cost (e.g. network resource consumption and communication time) is also less. 3.1.1. Total Network Resource Consumption Total network resource consumption is evaluated using three sub-metrics: Packetroot, Packetagg and Packetedge, and it can be formulated as follows: Packetroot+Packetagg+Packetedge, (1) s.t. Packetedge=∑i=1nSi×size(packeti), (2) Packetagg=∑i=1nAi×size(packeti), (3) Packetroot=∑i=1nRi×size(packeti), (4) where n denotes the size of a set of data-intensive applications. Packetedge, Packetagg and Packetroot denote the total size of packets transferred by the edge, aggregation, and core switches, respectively. Ai, Ri and Si denote the frequencies that have been transferred by the edge, aggregation and core switches, respectively. 3.1.2. Total Processing Time The total processing time is the time which all virtual clusters including reallocated VMs jointly process a set of data-intensive applications, and it can be formulated as follows: ∑i=1n(Tend(ti)−Tstart(ti)), (5) where n denotes the size of a set of data-intensive applications; Tstart(ti) and Tend(ti) are the start time and end time of the ith data-intensive application, respectively. 3.1.3. Total Transmission Cost Total transmission cost is the total traffic load induced by the reallocation of VMs hosted on deteriorating PM, and it can be formulated as follows: TCtotal=∑i=1m∑k=1VLik(dikbwik+dkibwki) (6) where V and m are the virtual cluster size and number of VMs on a deteriorating PM, respectively; bwik (or bwki) is the bandwidth usage between the ith (or kth) VM and the kth (or ith) VM; Lik is the transmission cost between the ith reallocated VM and other VMs in the same virtual cluster. When the deteriorating PM is detected, if the kth (or ith) VM is the VM sender, the value of the binary variable dki (dik) is 1, otherwise 0. 3.1.4. Problem Formulation The total transmission cost of reallocated VMs can be formulated as an optimal objective function to minimize it under some constraints, and it can be formulated as follows: Objective:minTCtotal (7) s.t. ∑i=1MDibwxij<Cjbw, (8) ∑i=1MDimemxij<Cjmem, (9) ∑i=1MDicpuxij<Cjcpu, (10) ∑j=1Nxij=1,xij=0or1, (11) where N and M represent the number of PMs and VMs of the cloud data center, respectively; Dibw,Dimem and Dicpu denote the maximum resource requirements of the ith VM including bandwidth capacity, memory capacity and processing capacity, respectively; and Cjbw, Cjmem and Cjcpu denote the bandwidth idle capacity, memory idle capacity and processing idle capacity of the jth PM, respectively. Equations (8–10) denote that the PM’s idle resource capacity must be more than the sum of the resource requirements for each VM; Equation (11) denotes that a VM can only be placed on a PM. That is, if the ith VM is placed on the jth PM, then xij = 1, otherwise xij = 0. 3.2. PM Selection Optimization As we all know, BBO is a new stochastic global evolutionary algorithm, it has certain similar features with other biology-based algorithms (e.g. particle swarm optimization (PSO) [32]) and has good performance compared with other algorithms. For example, compared with BBO algorithm, PSO algorithm is more likely to clump together in similar groups [12]. In the standard BBO algorithm [12], the population of candidate solutions represents an ecosystem (i.e. archipelago of islands), in which each candidate solution is an island. A Habitat Suitability Index (HSI) is exploited to measure the goodness (i.e. fitness) of a candidate solution, that is, an island with a high (or low) HSI denotes a good (or poor) solution, and decided by Suitability Index Variables (SIVs) (e.g. temperature, rainfall, etc.). Therefore, a vector of SIVs represents a solution. There are two key operators (i.e. migration and mutation) in BBO. Among them, the migration is designed to probabilistically share SIVs between solutions, and thus increases the quality of low HSI solutions. In particular, it is a distinguishing feature of BBO from other population-based optimization algorithms. The mutation follows the migration and probabilistically exploits randomly generating new SIVs to replace SIVs in a solution. Further, the initial population can evolve repeatedly from generation to generation until a termination criterion is met. Such heuristics can find the correct solution in a small time and is therefore suitable for managing large data centers. Next, we will apply the standard BBO algorithm to solve the discrete single-objective virtual cluster reallocation optimization problem. We first propose ecosystem model and definition of parameters and operators of the BBO algorithm. Finally, we present implementation scheme of TVCR. 3.2.1. The Ecosystem Model of the BBO algorithm It is well known that the population of the virtual cluster reallocation optimization problem consists of multiple individuals, and each individual is a candidate virtual cluster reallocation solution. In TVCR, the population is treated as an ecosystem, each island represents an individual. Therefore, each island is a candidate virtual cluster reallocation solution. These islands have the same optimization objective (as shown in Equation (7)) and constraints (as shown in Equations (8–11)), such that each island exchange information with other islands with respect to the optimization objective and constraints to not only optimize itself but also optimize an entire ecosystem. For the ease of understanding, we define a matrix EPL (as shown in Equation (12)) to represent the ecosystem. The row number P represents the total number of islands in the ecosystem (i.e. the size of the population). The column number L represents the total number of reallocated VMs. The matrix element ei, j represents the PM number assigned to the jth reallocated VM of the ith island. Therefore, the candidate solution {ei,1, ei,2,…, ei, j, …, ei,L} denotes the ith island: EPL=|e1,1e1,2..e1,Le2,1e2,2..e2,L........eP,1eP,2..eP,L|. (12) Let E = {E1, E2,…, EP} denotes an ecosystem of P islands and Ek = {ek,1, ek,2,…, ek,L, O, C1, C2, C3, C4} represents the kth island, which contains a vector of L SIVs (i.e. the list of reallocated VMs), one objective O and four constraints C1, C2, C3 and C4. Each SIV is an integer index of the PM hosting a VM. O represents an objective defined in Equation (7). The four constraints C1, C2, C3 and C4 correspond to Equations (8–11). The HSI of the island Ek is computed using the objective function O. Combined with the above ecosystem and the specific characteristics of the virtual cluster reallocation problem, the parameters and operators of the BBO are defined as follows. According to the related literature [12], the immigration rate λ and emigration rate μ of a habitat (i.e. island) are a function of the number of species s (i.e. the number of used PMs), as shown in Equations (13) and (14). Smax is the largest possible number of species in the habitat (i.e. the number of reallocated VMs). As the number of species s gradually increases, the immigration rate λs gradually decreases, and emigration rate μs gradually increases. When λs is equal to μs, the number of species S0 in the habitat reaches equilibrium; the equilibrium state S0 will migrate as the environment changes. There are multiple function types of λs and μs; one of them is a linear function type, which is used in this paper. Especially, the maximum emigration rate is usually considered to equal the maximum immigration rate I = 1: μs=I⋅sSmax, (13) λs=I(1−sSmax). (14) We define Ps as the probability of the habitat including s species. When the probability changes from time t to time (t+Δt), it can be formulated as follows: Ps(t+Δt)=Ps(t)(1−λsΔt−μsΔt)+Ps−1λs−1Δt+Ps+1μs+1Δt. (15) Suppose that Δt is small enough, the probability of more than one immigration or emigration can be ignored. Taking the limit of (15) as Δt→0, the steady-state value of probability Ps with respect to the number of each species can be formulated as follows [33]: Ps={11+∑s=1Smaxλ0λ1...λs−1μ1μ2...μss=0λ0λ1...λs−1μ1μ2...μs(1+∑s=1Smaxλ0λ1...λs−1μ1μ2...μs)1≤s≤Smax. (16) To ensure the existence of the foregoing probabilities, the emigration rate μs cannot equal to 0 for all species. Definition 1 (Migration Operator): E → Ek is a probabilistic operator that adjusts habitat Ek based on the ecosystem E. The probability that Ek is modified is proportional to its immigration rate λk, and the probability that the source of the modification comes from Ei is proportional to the emigration rate μi. Definition 2 (Mutation Operator): Ek→ Ek is a probabilistic operator that randomly modifies habitat SIVs based on the habitat’s a priori probability of existence. The relationship between the number of biological species s in a habitat and its corresponding probability can be obtained by solving Equation (16). The mutation probability of the habitat is inversely proportional to the number of species s in the habitat; the mutation rate ms is calculated as ms=mmax(1−PsPmax), (17) where mmax represents the mutation probability given in advance which is assigned 0.1 [33] and Pmax=max{Ps}. Definition 3 (Removal Operator): Ei → Ei is an operator that identifies overloaded PMs of the habitat and replace them with normal PMs. In standard BBO algorithm [12], the mutation operator simply replaces the original SIV with a randomly generated SIV, and the migration operator replaces the immigrated SIV with the emigrated SIV. The two operators are easy to produce similar solution and lead to poor diversity of population. Therefore, the standard BBO algorithm designs the removal operator to eliminate these similar solutions and improve the diversity of population. When they are applied to TVCR, the two operators will generate overloaded PMs, that is, the resource requirement of the VMs placed in one PM is far more than the maximum capacity of the PM in a virtual cluster reallocation solution. Therefore, to remove these overloaded PMs and to improve the diversity of population, the removal operator is designed to find these overloaded PMs, and to replace these overloaded PMs with some normal PMs meeting the resource requirement of the VMs placed. Finally, to enhance the performance of TVCR, we exploit elitism to make sure that the best habitat is not lost from one generation to the next. It is common to save the best habitats at the beginning of each generation in a set and then replace the worst habitats with the set at the end of the generation after migration, mutation and removal have been completed. The size of the set (i.e. Nelite) is a tuning parameter, but it typically includes the best two habitats [34]. Meanwhile, when we use the migration rates to decide how much information to share between habitats, we replace selected SIVs (i.e. PMs) in such a way that the modified habitat is always feasible and better than the original habitat. Since TVCR exploits the mutation operator to avoid local extrema, the two strategies can improve its performance. 3.2.2. The Pseudo-code of TVCR In this section, we propose TVCR (as shown in Algorithm 1) to solve the above optimization problem. The TVCR performs the following steps: (a) the algorithm initially allocates a set of VMs (i.e. vmList) to a set of PMs (i.e. hostList) via the IVCA approach. (b) The algorithm anticipates the deteriorating PM of the cloud data center via the PM fault prediction module. (c) The algorithm maps the traffic-aware virtual cluster reallocation optimization problem to an ecosystem including SIVs and habitats, and initializes the size of the population P, the number of generations G, the maximum species Smax, the maximum immigration rates I, the maximum mutation rate mmax and the number of elites Nelite. (d) The algorithm initializes and sorts a random set of habitats via HSI. (e) The algorithm saves the elites, probabilistically exploits immigration, emigration and mutation to modify and mutate each non-elite habitat via Definitions 1 and 2, respectively, and removes the overloaded SIVs in each habitat via Definition 3. (f) The algorithm recomputes each HSI to sort all habitats in the ecosystem and select the elites. (h) Finally, the algorithm returns to Step (e) for the next iteration until a predefined number of generations. Algorithm 1 Our proposed TVCR approach 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution Algorithm 1 Our proposed TVCR approach 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution 4. PERFORMANCE EVALUATION In this section, we perform the thorough performance evaluation of TVCR approach via the simulation experiment. 4.1. Experiment Setup We build a system model in WebCloudSim system [31,36], which is based on CloudSim [35]. In this system, all of the experiments are executed in a 16-port fat-tree data center network [11], which comprises 64 core switches, 128 aggregation switches and 128 edge switches. To more intuitively observe experimental results, the transfer delays of the core, aggregation and edge switches are 1 s, 1 s and 2 s, respectively [36]. The system comprises 1024 heterogeneous PMs with resource capacity equivalent to 3720 or 5320 MIPS, 10 GB of RAM, 10 GB/s network bandwidth and 1 TB of storage and 4000 heterogeneous VMs [37]. The processing capacity of each VM is chosen randomly from seven values: 360, 490, 540, 620, 720, 860 or 1000 MIPS. The bandwidth requirement of each VM is set at 900 Mbps. The disk and the memory capacity of each VM are 1 and 1 GB [31], respectively. Appropriate parameter values of Algorithm 1 are determined on the basis of the related literatures and preliminary experiments, the size of the population was set at 20 and the number of generations was set at 30. To measure the performance of different reallocation approaches, a data-intensive application model including multiple different VMs is exploited. In fact, there have been some studies about the number of VMs processing a data-intensive application, which shows that more than 80% of applications use less than 10 VMs [38]. There are mainly three types of data-intensive applications: multi-tiered data-intensive application [39], workflow data-intensive application [40] and batch data-intensive application (i.e. MapReduce) [41]. According to the characteristics of these data-intensive applications, we exploit a set of the general data-intensive applications as our case in our experiments, in which each application consists of three tasks (such as Task 1, Task 2 and Task 3); Task 2 can perform its task after the processing results of Tasks 1 and 3 are transferred to Task 2. Note that each VM only processes a task which comprises some computation and communication stages [31]. We compare our approach with six other heuristic approaches: First-Fit (FF), Best-fit (BF), Random First-fit (RFF), IVCA, Modified Best-Fit Decreasing (MBFD) [42] and PSO [43]. As we all know, BF, FF and RFF are three classical approaches. When a deteriorating PM is monitored, there are multiple suitable PMs. BF identifies the PMs with the minimum CPU utilization to place these reallocated VMs. FF always allocates these reallocated VMs to the PMs that first satisfy the constraints. MBFD always places these reallocated VMs at the near-optimal PMs, which satisfy simultaneously the minimum transmission cost and energy consumption. However, different from former algorithms, RFF allocates these reallocated VMs by randomly selecting some PMs. 4.2. Experimental Results and Evaluation In this section, we compare the performance of our approach with six related approaches in terms of total transmission cost, algorithm execution time (i.e. the time which the approach finds the minimum transmission cost), total network resource consumption, and total processing time. In our experiments, the numbers of reallocated VMs, VMs and data-intensive applications were set at 9, 4000 and 1000, respectively. For each approach and each scenario, we conducted the experiments 10 times to compute the average results, which are discussed below. 4.2.1. Comparison of Total Transmission Cost The first set of experiments is conducted to analyze the algorithm execution time and the total transmission cost incurred due to the reallocated VMs migrating to healthy PMs. According to Equation (7), the transmission cost determines the total network resource consumption and the total processing time. The algorithm execution time determines and affects the efficiency and the execution results. As shown in Figs. 2 and 3, the experimental results indicate that TVCR has the least amount of total transmission cost and has a moderate algorithm execution time. This is because TVCR exploits the BBO global optimization algorithm to select healthy PMs for the reallocated VMs. Although PSO is also a heuristic algorithm, its solutions are more likely to clump together in similar group; the other five related approaches do not adopt the heuristic algorithm. Thus, the total transmission cost using the PSO algorithm is more than that of the BBO algorithm and far less than that of the five other related approaches. We can also see that RFF, BF and FF have almost similar higher total transmission cost and lower algorithm execution time; this is because that these approaches do not consider the total transmission cost of the VC allocation scheme. On the contrary, MBFD and IVCA both take into account the total transmission cost. Therefore, the total transmission cost is lower and algorithm execution time of them is higher. However, they still exceed the TVCR and PSO approaches. MBFD has more the total transmission cost than IVCA; this is due to the fact that MBFD needs to consider both the total transmission cost and the total energy consumption. However, IVCA does not consider simultaneously the above two metrics. Figure 2. View largeDownload slide Total transmission cost of all approaches. Figure 2. View largeDownload slide Total transmission cost of all approaches. Figure 3. View largeDownload slide Algorithm execution time of all approaches. Figure 3. View largeDownload slide Algorithm execution time of all approaches. 4.2.2. Comparison of Total Processing Time We have shown that the total transmission cost of TVCR is the least of the seven approaches. Meanwhile, there exists a direct relationship between the total transmission cost and the total processing time. Next, to validate the relationship, we will compare and analyze the total processing time of the seven approaches while jointly processing a set of data-intensive applications separately. As shown in Fig. 4, the experimental results show that the total processing time of TVCR approach is the least of all approaches. This is due to the fact that the VMs forming a virtual cluster are allocated more concentratedly by TVCR. To be precise, the more aggregation and edge switches and less core switches are required by TVCR. Thus, TVCR takes less total processing time to transfer the packets between the VMs in virtual clusters including the reallocated VMs. That is, less total processing time is needed to process a set of data-intensive applications than for the other approaches. Figure 4. View largeDownload slide Total processing time of all approaches. Figure 4. View largeDownload slide Total processing time of all approaches. Although the differences of these total processing times are small, the total processing time is used by the virtual clusters including the reallocated VMs; moreover, the numbers of virtual clusters and data-intensive applications are both too small. If there are then many data-intensive applications processed by massive virtual clusters in a cloud data center, the difference of these approaches will be large. Because the service response time and the quality of service have a direct relationship, the longer the response time, the worse the quality of service. Therefore, TVCR would be better from the quality of service perspective as well compared with the related approaches. 4.2.3. Comparison of Total Network Resource Consumption As shown in Figs. 5–8, TVCR consumes the least amount of network resources. This finding is observed because TVCR exploits the BBO-based approach to allocate the reallocated VMs to some healthy PMs. Therefore, the VMs of the virtual cluster which includes the reallocated VMs are allocated as compact as possible, e.g. in the same subnet. On the contrary, other approaches allocate these reallocated VMs more dispersedly. Meanwhile, according to the characteristics of the fat-tree, when one packet is routed by the core switches, it will also be transferred by the related aggregation and edge switches. Therefore, there are more core switches to be exploited. Further, the core link easily becomes a bottleneck. Thus, we should exploit our approach to reduce the number of used core switches. Figure 5. View largeDownload slide The number of edge switch packets processed by the all approaches. Figure 5. View largeDownload slide The number of edge switch packets processed by the all approaches. Figure 6. View largeDownload slide The number of aggregation switch packets processed by all approaches. Figure 6. View largeDownload slide The number of aggregation switch packets processed by all approaches. Figure 7. View largeDownload slide The number of root switch packets processed by all approaches. Figure 7. View largeDownload slide The number of root switch packets processed by all approaches. Figure 8. View largeDownload slide The total number of switch packets processed by all approaches. Figure 8. View largeDownload slide The total number of switch packets processed by all approaches. Based on the above analysis of the experimental results, TVCR outperforms six other approaches. Next, we further analyze the value of these metrics by varying the number of data-intensive applications (as shown in Figs. 9–11) and VMs (as shown in Figs. 12–14), respectively. Finally, we also analyze the impact of the number of reallocated VMs on these metrics (as shown in Figs. 15–17). Figure 9. View largeDownload slide The impact of different numbers of data-intensive applications on total transmission cost for all approaches. Figure 9. View largeDownload slide The impact of different numbers of data-intensive applications on total transmission cost for all approaches. Figure 10. View largeDownload slide The impact of different numbers of data-intensive applications on total processing time for TVCR. Figure 10. View largeDownload slide The impact of different numbers of data-intensive applications on total processing time for TVCR. Figure 11. View largeDownload slide The impact of different numbers of data-intensive applications on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 11. View largeDownload slide The impact of different numbers of data-intensive applications on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 12 View largeDownload slide The impact of different numbers of VMs on total transmission cost for all approaches. Figure 12 View largeDownload slide The impact of different numbers of VMs on total transmission cost for all approaches. Figure 13. View largeDownload slide The impact of different numbers of VMs on total processing time for TVCR. Figure 13. View largeDownload slide The impact of different numbers of VMs on total processing time for TVCR. Figure 14. View largeDownload slide The impact of different numbers of VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 14. View largeDownload slide The impact of different numbers of VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 15 View largeDownload slide The impact of different numbers of reallocated VMs on the total transmission cost for all approaches. Figure 15 View largeDownload slide The impact of different numbers of reallocated VMs on the total transmission cost for all approaches. Figure 16. View largeDownload slide The impact of different numbers of reallocated VMs on total processing time for TVCR. Figure 16. View largeDownload slide The impact of different numbers of reallocated VMs on total processing time for TVCR. Figure 17. View largeDownload slide The impact of different numbers of reallocated VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 17. View largeDownload slide The impact of different numbers of reallocated VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). 4.2.4. Impact of the Number of Data-Intensive Applications Figures 9–11 show the impact of the number of data-intensive applications on three metrics (i.e. total transmission cost, total processing time and total network resource consumption) for all approaches. In these experiments, the virtual cluster size, number of VMs and number of reallocated VMs were set at 3, 4000 and 9, respectively. Meanwhile, the number of data-intensive applications was varied from 100 to 1000 with a step value of 100. These figures indicate that the total transmission cost of all approaches was not substantially affected by the number of data-intensive applications. However, the total processing time of TVCR and the total network resource consumption of all approaches steadily increased with the increase in the number of data-intensive applications. Then, for each number of data-intensive applications, the total transmission cost and the total network resource consumption of TVCR were both the smallest of all approaches. 4.2.5. Impact of the Number of VMs Figures 12–14 show the impact of the number of VMs on all approaches. In this experiment, the virtual cluster size, number of VMs and number of reallocated data-intensive applications were set at 3, 9 and 1000, respectively. Meanwhile, we varied the number of VMs from 1000 to 5500, with a step value of 500. These figures show that the total transmission cost and the total network resource consumption for all approaches were not substantially affected by the number of VMs. However, the total processing time of TVCR fluctuates as the number of VMs varies. This is due to the fact that although the total transmission cost was not substantially affected by the change of the number of VMs, the routing path of each packet was diverse. Therefore, it is very likely that some packets need to be forwarded by the same switches; as a result, some forwarded packets need to line up to wait for the same switches. Therefore, there exists a time fluctuation under different numbers of VMs. 4.2.6. Impact of the Number of Reallocated VMs Figures 15–17 show the impact of the number of reallocated VMs on three metrics (i.e. total processing time, total transmission cost and total network resource consumption) of the all approaches. In this experiment, the virtual cluster size, number of data-intensive applications and number of VMs were set at 3, 1000 and 4000, respectively. Meanwhile, the number of reallocated VMs was varied from 2 to 9, with a step value of 1. These figures indicate that the total transmission cost of all approaches and the total processing time of TVCR increased as the number of reallocated VMs increased. However, the total network consumption of all approaches was not substantially affected by the number of reallocated VMs. 5. CONCLUSIONS AND FUTURE WORK In this work, we proposed TVCR to reallocate VMs hosted on a deteriorating PM to healthy PMs. To implement TVCR, we exploited a three-step process, in which we first exploited an IVCA approach to initially allocate a set of VMs, then exploited the PM fault prediction module to anticipate a deteriorating PM, and finally identified some healthy PMs to host these VMs on the deteriorating PM. Next, we performed the thorough performance evaluation of TVCR in term of total transmission cost, total processing time and total network resource consumption. The experimental results showed that TVCR outperforms six existed approaches. In our experimentations, we will exploit TVCR to initially allocate virtual clusters to a set of PMs to minimize the bandwidth usage in network core and the risk of violating the reliability requirement of a data-intensive application. ACKNOWLEDGEMENTS The work presented in this paper is supported by the NSFC (61472047 and 61602054), and Beijing Natural Science Foundation (4174100). REFERENCES 1 Foster , I. , Zhao , Y. , Raicu , I. and Lu , S. , ( 2008 ) Cloud computing and grid computing 360-degree compared. Proc. GCE 08, Austin, TX, USA, 12–16 November, pp.1–10, IEEE, New York. 2 Barham , P. , Dragovic , B. , Fraser , K. , Hand , S. , Harris , T.L. , Ho , A. , Neugebauer , R. , Pratt , I. and Warfield , A. , ( 2003 ) Xen and the Art of Virtualization. Proc. SOSP 03, Bolton Landing, New York, USA, 19–22 October, pp. 164–177, ACM, New York. 3 Middleton , A. ( 2010 ) Data-Intensive Technologies for Cloud Computing. Borko Furht and Armando Escalante. In Handbook of Cloud Computing . Springer , US, New York . 4 Buyya , R. , Yeo , C.S. , Venugopal , S. , Broberg , J. and Brandic , I. ( 2009 ) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility . Future Gener. Comput. Syst. , 25 , 599 – 616 . Google Scholar CrossRef Search ADS 5 Skillicorn , D. and Talia , D. ( 1998 ) Models and languages for parallel computation . ACM Comput. Sruveys , 30 , 123 – 169 . Google Scholar CrossRef Search ADS 6 Liang , Q. , Zhang , J. , Zhang , Y. and Liang , J. ( 2014 ) The placement method of resources and applications based on request prediction in cloud data center . Inf. Sci. , 279 , 735 – 745 . Google Scholar CrossRef Search ADS 7 Ho , J. , Hsiu , P. and Chen , M. , ( 2015 ) Improving Serviceability for Virtual Clusters in Bandwidth-Constrained Datacenters. Proc. CLOUD 15, New York, USA, 27 June–2 July, pp. 710–717, IEEE, New York. 8 Chowdhury , M. , Zaharia , M. , Ma , J. , Jordan , M.I. and Stoica , I. , ( 2011 ) Managing Data Transfers in Computer Clusters with Orchestra. Proc. SIGCOMM 11, Toronto, Ontario, Canada, 15–19 August, pp. 98–109, ACM, New York. 9 Networking , C.V. ( 2017 ) Cisco Global Cloud Index: Forecast and Methodology, 2015–20, Cisco Systems, California. 10 Breitgand , D. and Epstein , A. , ( 2012 ) Improving Consolidation of Virtual Machines with Risk-Aware Bandwidth Oversubscription in Compute Clouds. Proc. INFOCOM 12, Orlando, FL, USA, 25–30 March, pp. 2861–65, IEEE, New York. 11 Al-Fares , M. , Loukissas , A. and Vahdat , A. , ( 2008 ) A Scalable, Commodity Data Center Network Architecture. Proc. SIGCOMM 08, Seattle, WA, USA, 17–22 August, pp. 63–74, ACM, New York. 12 Simon , D. ( 2008 ) Biogeography-based optimization . IEEE Trans. Evol. Comput , 12 , 702 – 713 . Google Scholar CrossRef Search ADS 13 Duan , J. , Guo , Z. and Yang , Y. , ( 2015 ) Cost Efficient and Performance Guaranteed Virtual Network Embedding in Multicast Fat-Tree DCNs. Proc. INFOCOM 15, Kowloon, Hong Kong, 26 April–1 May, pp.136–144, IEEE, New York. 14 Meng , X. , Papps , V. and Zhang , L. , ( 2010 ) Improving the Scalability of Data Center Networks with Traffic-Aware Virtual Machine Placement. Proc. INFOCOM 10, San Diego, CA, USA, 14–19 March, pp.1–9, IEEE, New York. 15 Dias , D. and Costa , L. , ( 2012 ) Online Traffic-Aware Virtual Machine Placement in Data Center Networks. Proc. GIIS 12, Choroni, Venezuela, 17–19 December, pp. 1–8, IEEE, New York. 16 Breitgand , D. , Epstein , A. and Glikson , A. , ( 2013 ) Network Aware Virtual Machine and image Placement in a Cloud. Proc. CNSM 13, Zurich, Switzerland, 14–18 October, pp. 9–17, IEEE, New York. 17 You , K. , Tang , B. and Ding , F. , ( 2013 ) Near-Optimal Virtual Machine Placement with Product Traffic Pattern in Data Centers. Proc. ICC 13, Budapest, Hungary, 9–13 June, pp. 3705–3709, IEEE, New York. 18 Biran , O. , Corradi , A. , Fanelli , M. , Foschini , L. , Nus , A. , Raz , D. and Silvera , E. , ( 2012 ) A Stable Network-Aware VM Placement for Cloud Systems. Proc. CCGRID 12, Ottawa, Canada, 13–16 May, pp. 498–506, IEEE, New York. 19 Cohen , R. , Eytan , L. , Naor , J. and Raz , D. , ( 2013 ) Almost Optimal Virtual Machine Placement for Traffic Intense Data Centers. Proc. INFOCOM 13, Turin, Italy, 14–19 April, pp. 355–359, IEEE, New York. 20 Ilkhechi , A. , Korpeoglu , I. and Ulusoy , O. ( 2015 ) Network-aware virtual machine placement in cloud data centers with multiple traffic-intensive components . Comput. Netw. , 91 , 508 – 527 . Google Scholar CrossRef Search ADS 21 Kuo , J. , Yang , H. and Tsai , M. , ( 2014 ) Optimal Approximation Algorithm of Virtual Machine Placement for Data Latency Minimization in Cloud Systems. Proc. INFOCOM 14, Toronto, ON, Canada, 27 April–2 May, pp. 1303–1311, IEEE, New York. 22 Alicherry , M. and Lakshman , T. , ( 2013 ) Optimizing Data Access Latencies in Cloud Systems by Intelligent Virtual Machine Placement. Proc. INFOCOM 13, Turin, Italy, 14–19 April, pp. 647–655, IEEE, New York. 23 Jiang , J. , Lan , T. , Ha , S. and Chen , M. , ( 2012 ) Joint VM Placement and Routing for Data Center Traffic Engineering. Proc. INFOCOM 12, Orlando, FL, USA, 25–30 March, pp. 2876–2880, IEEE, New York. 24 Li , X. , Wu , J. , Tang , S. and Lu , S. , ( 2014 ) Let’s Stay Together: Towards Traffic Aware Virtual Machine Placement in Data Centers. Proc. INFOCOM 14, Toronto, ON, Canada, 27 April−2 May, pp. 1842–1850, IEEE, New York. 25 Fukunaga , T. , Hirahara , S. and Yoshikawa , H. , ( 2015 ) Virtual Machine Placement for Minimizing Connection Cost in Data Center Networks. Proc. INFOCOM WKSHPS 15, Hong Kong, China, 26 April− May, pp. 486–491,IEEE, New York. 26 Zhu , J. , Li , D. , Wu , J. and Liu , H. , ( 2012 ) Towards Bandwidth Guarantee in Multi-tenancy Cloud Computing Networks. Proc. ICNP 12, Austin, TX, USA, 30 October−2 November, pp. 1–10, IEEE, New York. 27 Jin , H. , Pan , D. , Xu , J. and Pissinou , N. , ( 2012 ) Efficient VM Placement with Multiple Deterministic and Stochastic Resources in Data Centers. Proc. GLOBECOM 12, Anaheim, CA, USA, 3–7 December, pp. 2505–2510, IEEE, New York. 28 Kandula , S. , Sengupta , S. , Greenberg , A. , Patel , P. and Chaiken , R. , ( 2009 ) The Nature of Data Center Traffic: Measurements & Analysis. Proc. IMC 09, Chicago, IL, USA, 04–06 November, pp. 202–208, ACM, New York. 29 Wang , M. , Meng , X. and Zhang , L. , ( 2011 ) Consolidating Virtual Machines with Dynamic Bandwidth Demand in Data Centers. Proc. INFOCOM 11, Shanghai, China, 10–15 April, pp. 71–75, IEEE, New York. 30 da Silva , R.A.C. and da Fonseca , N.L.S. , ( 2015 ) Algorithm for the Placement of Groups of Virtual Machines in Data Centers. Proc. ICC 15, London, UK, 8–12 June, pp. 6080–6085, IEEE, New York. 31 Liu , J. , Wang , S. , Zhou , A. , Kumar , S. , Yang , F. and Buyya , R. ( 2016 ) Using proactive fault-tolerance approach to enhance cloud service reliability . IEEE Trans. Cloud Comput. , doi:10.1109/TCC.2016.2567392. 32 Kenndy , J. and Eberhart , R. , ( 1995 ) Particle Swarm Optimization. Proc. ICNN 95, Perth, Western Australia, 27 November−1 December, pp. 1942–1948, IEEE, New York. 33 Ma , H. , Ni , S. and Sun , M. , ( 2009 ) Equilibrium Species Counts and Migration Model Tradeoffs for Biogeography-Based Optimization. Proc. CDC/CCC 09, Shanghai, China, 15–18 December, pp. 3306–3310, IEEE, New York. 34 Simon , D. , Ergezer , M. and Du , D. , ( 2009 ) Population Distributions in Biogeography-Based Optimization Algorithms with Elitism. Proc. SMC 09, San Antonio,TX, USA, 11–14 October, pp. 991–996, IEEE, New York. 35 Calheiros , R.N. , Ranjan , R. , Beloglazov , A. , De Rose , C.A. and Buyya , R. ( 2011 ) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms . Softw. Pract. Exper. , 41 , 23 – 50 . Google Scholar CrossRef Search ADS 36 Zhou , A. , Wang , S. , Zheng , Z. , Hsu , C.-H. , Lyu , M.R. and Yang , F. ( 2016 ) On cloud service reliability enhancement with optimal resource usage . IEEE Trans. Cloud Comput. , 4 , 452 – 466 . Google Scholar CrossRef Search ADS 37 Beloglazov , A. and Buyya , R. ( 2012 ) Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers . Concur. Comput.: Pract. Exper. , 24 , 1397 – 1420 . Google Scholar CrossRef Search ADS 38 Bodík , P. , Menache , I. , Chowdhury , M. , Mani , P. , Maltz , D.A. and Stoica , I. , ( 2012 ) Surviving Failures in Bandwidth-Constrained Datacenters. Proc. SIGCOMM 12, Helsinki, Finland, 13–17 August, pp. 431–442, ACM, New York. 39 Lee , J. , Turner , Y. , Lee , M. , Popa , L. , Banerjee , S. , Kang , J.-M. and Sharma , P. , ( 2014 ) Application-Driven Bandwidth Guarantees in Datacenters. Proc. SIGCOMM 14, Chicago, IL, USA, 17–22 August, pp. 467–478, ACM, New York. 40 Garg , S.K. and Buyya , R. ( 2013 ) An environment for modeling and simulation of message-passing parallel applications for cloud computing . Softw. Pract. Exper. , 43 , 1359 – 1375 . Google Scholar CrossRef Search ADS 41 Yang , H.-c. , Dasdan , A. , Hsiao , R.-L. and Parker , D.S. , ( 2007 ) Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters. Proc. SIGMOD 07, Beijing, China, 11–14 June, pp. 1029–1040, ACM, New York. 42 Beloglazov , A. , Abawajy , J. and Buyya , R. ( 2012 ) Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing . Future Gener. Comput. Syst. , 28 , 755 – 768 . Google Scholar CrossRef Search ADS 43 Wang , S. , Liu , Z. , Zheng , Z. , Sun , Q. and Yang , F. , ( 2013 ) Particle Swarm Optimization for Energy-Aware Virtual Machine Placement Optimization in Virtualized Data Centers. Proc. ICPADS 13, Seoul, South Korea, 15–18 December, pp. 102–109, IEEE, New York. © The British Computer Society 2017. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Computer Journal Oxford University Press

Towards Bandwidth Guaranteed Virtual Cluster Reallocation in the Cloud

Loading next page...
 
/lp/ou_press/towards-bandwidth-guaranteed-virtual-cluster-reallocation-in-the-cloud-RrXKAI57jF
Publisher
Oxford University Press
Copyright
© The British Computer Society 2017. All rights reserved. For permissions, please email: journals.permissions@oup.com
ISSN
0010-4620
eISSN
1460-2067
D.O.I.
10.1093/comjnl/bxx113
Publisher site
See Article on Publisher Site

Abstract

Abstract Cloud data center traffic is experiencing a rapid growth as more and more data-intensive applications are required to process big data in a cloud data center. Although fat-tree networks own rich path multiplicity, and have been widely adopted as network topologies in cloud data center networks to transmit vast bisection bandwidth, they lead to bandwidth-related bottlenecks. To address this issue, in this paper, we propose a traffic-aware virtual cluster reallocation approach via biogeography-based optimization to allocate some reallocated virtual machines (VMs) as compact as possible with those VMs in the same virtual clusters. In order to validate our approach, we build a system model to perform a thorough evaluation of its performance. Experimental results show that our proposed approach outperforms six existed approaches in term of total transmission cost, total processing time, and total network resource consumption. 1. INTRODUCTION Recent improvement of information technology already results in a wide exploitation of cloud computing, in which several prior concepts and technologies are given a new perspective [1], such as virtualization technology [2] and data center. In this computing paradigm, its background support technology is data-intensive computing technology [3] and its computing resources are provided for cloud users on a pay-per-use basis [4]. Data-intensive computing is a kind of parallel computing applications, it takes most of their processing time to I/O (i.e. the movement and manipulation of data). The parallel processing of data-intensive applications typically partitions or subdivides the processed data into multiple segments, which can be processed independently using the same executable application program in parallel on an appropriate computing platform, and then the processing results are reassembled to produce the completed output data. Thus, the greater the aggregate distribution of the data, the more the benefit is in the parallel processing of the data [5]. Generally, a set of data-intensive applications (i.e. cloud services) are processed in a cloud. Each data-intensive application can be divided into multiple tasks to jointly process the partitioned or subdivided multiple segments, in which each task is processed by a virtual machine (VM). Massive scalable resources are dynamically organized into many VMs to address cloud services in a cloud data center. Their multiple isolated execution environments constitute the resources of a single physical machine (PM). These VMs are usually instantiated to perform a user’s specific computation. Once a VM is instantiated, it can be used to process the target task such that the VMs are designed to handle cloud services on an on-demand cloud basis for the final users. After processing the task, the VM is released and the PM can accommodate another VM. When a cloud provider wants to create a number of VMs, an important issue is the VM (re)allocation problem [6], which is essential in a cloud data center. This is because most PM hardware capacities are under-utilized. The virtualization technology is exploited to greatly improve the resource utilization of PMs, such as higher per-PM utilization, increased flexibility and availability, and reduction in hardware costs and operational expenses related to power, cooling, physical space, etc. Recently cloud data centers have emerged as common platforms for serving data-intensive applications: a variety of system architectures have been designed to process these data-intensive applications, such as MapReduce, Hadoop and HPCC. In such a framework, cloud users usually request a set of VMs (i.e. virtual cluster) rather than an individual VM; the virtual cluster can jointly process these data-intensive applications and its intermediate results are transferred iteratively through multiple stages. Further, the flows not only between VMs but also into the Internet are created by the traffic generated by the application [7]. Therefore, it often accounts for most of the processing time of a data-intensive application. For example, more than 50% of the processing time is consumed while transferring the data of 26% of the jobs in a Facebook data center [8]. As the data-intensive applications increase, the traffic will experience rapid growth; the global cloud data center traffic will be nearly a tripling of data from 2015 to 2020 with a combined annual growth rate of 27%, more specifically, from 4.7 ZB/year in 2015 to 15.3 ZB/year in 2020 [9]. Thus, the bandwidth resources, especially the bandwidth usage in a network core, may experience a bottleneck [10]. Moreover, interference of the bandwidth-related bottleneck could lead to unpredictable processing times and hurt cloud service quality, causing financial losses for users. To achieve expected network performance and to avoid the network bottlenecks, one strategy used is the exploitation of the network topology (e.g. fat-tree networks [11]). The advantages of this topology include (1) the exploitation of commodity Ethernet switches; (2) the relief of bandwidth-related bottlenecks through rich path multiplicity. However, only such strategy is not enough. Another strategy is the exploitation of a traffic-aware virtual cluster (re)allocation approach, it allocates the VMs in the same virtual cluster to make them as compact as possible to have less transmission cost taken by the above flows. That is, these flows visit the lower number of switches and links. Further, the network bottlenecks can be relieved. In this paper, we propose a traffic-aware virtual cluster reallocation (TVCR) approach via biogeography-based optimization (BBO) [12], abbreviated as TVCR, to address network bottlenecks for reallocated VMs in some virtual clusters. We first map the foregoing optimization problem to an ecosystem model of the BBO algorithm and propose TVCR to address the single-objective optimization problem. And then we perform a thorough performance evaluation of TVCR in terms of the total transmission cost, total processing time and total network resource consumption. 1.1. Organization Section 2 presents the background and related work. Section 3 introduces the technical details of our proposed approach; Section 4 provides the performance evaluation of our approach; Section 5 concludes with research recommendations. 2. BACKGROUND AND RELATED WORK As we all know, virtual cluster (re)allocation is a useful approach for many cloud services that require expected network performance, especially, when considering the more and more data-intensive computing services in today’s cloud data centers [13]. Next, we will introduce some relevant research regarding the traffic-aware virtual cluster (re)allocation problem. 2.1. Traffic Pattern or Matrix Most of the existing traffic-aware virtual cluster (re)allocation approaches focus on resources and communication. Meng, Papps and Zhang [14] examined the traffic in real-data center-like systems and proposed a Traffic-aware VM placement scheme (TVMPP) to improve the network scalability. They formulated the problem of assigning VMs to PMs to minimize the total communication costs. This goal can be achieved by placing VM pairs with a large mutual traffic rate in PMs with close proximity. However, TVMPP allocates only one VM on each PM. In light of their study, many researchers [15–18] have observed that optimizing the VM placement is crucial and studied optimization models for VM placement. Dias and Costa [15] proposed a traffic-aware VM placement algorithm based on the current traffic matrix, CPU and memory usage to allocate VMs to the data center PMs. By analyzing the formation of community patterns in terms of traffic using graph theory, these VMs which exchange high amounts of data are aggregated and allocated to PMs as close as possible to reduce traffic congestion. Breitgand, Epstein and Glikson [16] studied the problem of placing VM instances on physical containers in such a way to reduce communication cost and latency. You, Tang and Ding [17] proposed a network-aware VM allocation algorithm with the aim of reducing the bandwidth usage in modern virtualization-based data centers. Biran et al. [18] proposed a new network-aware VM placement problem, called Min Cut Ratio-aware VM Placement (MCRVMP); they strived to allocate a placement that not only satisfies the predicted communication demand but is also resilient to demand time-variations. Furthermore, most of the related works formulate the problem as bin-packing related optimization problems and consider the physical network topology only through the traffic pattern. As far as we know, the exception is the study of Cohen et al. [19], which concentrated merely on the networking aspects and considered the VM placement problem with intense bandwidth requirements. They focused on maximizing the benefit from the overall traffic sent by VMs to a single point in the data center (i.e. the root), proposed an algorithm and simulated different widely used data center network topologies. However, the defined problem is very limited from a scenario perspective. Then, Ilkhechi, Korpeoglu and Ulusoy [20] came up with the problem by generalizing the mentioned problem to a scenario including more than one root or sink. 2.2. Data Access Latency Currently, more and more cloud services are data-intensive applications that require the processing of a large data set. To minimize the impact of data access latencies between processing nodes and data nodes and completion times, Kuo, Yang and Tsai [21] proposed an approximation algorithm for the VM placement problem to minimize the maximum access latency. Alicherry and Lakshman [22] considered lower bandwidth usage costs of cloud applications for optimal placement of computational nodes and presented algorithms for assigning VMs to data nodes that minimize various latency metrics under different constraints. 2.3. Network Cost To minimize the network cost caused by network traffic in cloud data centers, Jiang et al. [23] solved a joint tenant placement and route selection problem to minimize traffic costs by exploiting the multipath routing capability and dynamic VM migration. Li et al. [24] and Fukunaga, Hirahara and Yoshikawa [25] proposed optimized VM placement. However, compared with TVCR, they did not take cloud data center topology into account and considered traffic demands only to perform PM-level consolidation. As a result, network capacity violations in a data center are possible. 2.4. Network Bandwidth While both TVMPP and MCRVMP assume static and well-defined traffic demands, measurements from production data centers show that the network bandwidth demands of VMs are dynamic. Therefore, Zhu et al. [26] proposed an abstraction model of heterogeneous VM network bandwidth allocations and mathematically proved the problem of VM allocations with heterogeneous bandwidth demands to be NP-hard. Meanwhile, they proposed an online heuristic VM assignment algorithm to place the VMs with heterogeneous bandwidth demands into an appropriate subtree of the data center network. Jin et al. [27] studied efficient VM placement in data centers with multiple deterministic and stochastic resources. They mentioned that the bandwidth demands of VMs in data centers can be approximated based on the normal distribution [28]. To better represent the uncertainty of the future bandwidth demand, Wang, Meng and Zhang [29] used random variables to characterize the future bandwidth usage. They formulated the VM consolidation problem into a NP-hard Stochastic Bin-Packing problem and introduced a new heuristic approach to solve it. 2.5. Energy Efficiency To reduce the energy consumption of PMs and switches in a cloud data center, da Silva and da Fonseca [30] introduced the topology-aware VM placement algorithm to allocate groups of communicating VMs to the data centers, and exploited small regions of the data center network to consolidate the network flows produced by these VMs. The proposed algorithm avoids network bottlenecks, and thus accepts more requests of VMs without compromising the energy efficiency. Compared with previous related approaches, we first exploit the PM fault prediction model to anticipate the deteriorating PM, which is selected on the basis of the CPU temperature model in [31]. That is, when the temperature exceeds the upper threshold of the normal CPU temperature range (e.g. 68°C) for the duration, the PM is considered to be deteriorating. Then, we propose an approach via the BBO algorithm to allocate the reallocated VMs to healthy PMs with the objective of minimizing total transmission cost. Finally, we build a system model to evaluate our proposed approach. 3. PROPOSED TVCR APPROACH In this section, we describe the execution flow of TVCR, as shown in Fig. 1. The TVCR includes an initial virtual cluster allocation (IVCA) module, PM fault prediction module, traffic-aware virtual cluster reallocation module, data-intensive application scheduler module and task scheduler module. Figure 1. View largeDownload slide The execution flow chart of TVCR. Figure 1. View largeDownload slide The execution flow chart of TVCR. As shown in Fig. 1, the TVCR first exploits IVCA approach [31] to initially allocate a set of VMs to a set of PMs. In the IVCA algorithm, when a VM is allocated to a PM, it first identify all other VMs which are in the same virtual cluster as the VM. If such VMs exist, the VM will be allocated to the same subnet or pod as these VMs. Otherwise, it will be allocated such that overall energy consumption is minimized. Note that a virtual cluster is constructed by choosing randomly three VMs when creating the VMs, e.g. a general application model in Section 4.1. Next, the PM fault prediction module is exploited to anticipate a deteriorating PM. For example, PM2 hosts multiple VMs (e.g. vm1, vm3, vm6, vm7, vm8, vm14 and vm15) and is detected to deteriorate. And then the traffic-aware virtual cluster reallocation module is exploited to identify some near-optimal and healthy target PMs (e.g. PM1, PM3 and PM5) for hosting these VMs on the PM2. Note that these reallocated VMs are not isolated; they are all in some virtual clusters. Thus, in the process of reallocation, the total transmission cost must be minimum. Finally, to analyze the performance of TVCR, the data-intensive application scheduler module is exploited to assign some cloud services (i.e. a set of data-intensive applications) to these virtual clusters including the reallocated VMs, and then the task scheduler module is invoked to assign each task of these data-intensive applications to a VM of these virtual clusters. 3.1. Traffic-Aware Virtual Cluster Reallocation When a virtual cluster jointly processes a data-intensive application, its network resource consumption and processing time have a direct relationship with the transmission cost. This relationship is observed because the transmission cost is lower and the communication cost (e.g. network resource consumption and communication time) is also less. 3.1.1. Total Network Resource Consumption Total network resource consumption is evaluated using three sub-metrics: Packetroot, Packetagg and Packetedge, and it can be formulated as follows: Packetroot+Packetagg+Packetedge, (1) s.t. Packetedge=∑i=1nSi×size(packeti), (2) Packetagg=∑i=1nAi×size(packeti), (3) Packetroot=∑i=1nRi×size(packeti), (4) where n denotes the size of a set of data-intensive applications. Packetedge, Packetagg and Packetroot denote the total size of packets transferred by the edge, aggregation, and core switches, respectively. Ai, Ri and Si denote the frequencies that have been transferred by the edge, aggregation and core switches, respectively. 3.1.2. Total Processing Time The total processing time is the time which all virtual clusters including reallocated VMs jointly process a set of data-intensive applications, and it can be formulated as follows: ∑i=1n(Tend(ti)−Tstart(ti)), (5) where n denotes the size of a set of data-intensive applications; Tstart(ti) and Tend(ti) are the start time and end time of the ith data-intensive application, respectively. 3.1.3. Total Transmission Cost Total transmission cost is the total traffic load induced by the reallocation of VMs hosted on deteriorating PM, and it can be formulated as follows: TCtotal=∑i=1m∑k=1VLik(dikbwik+dkibwki) (6) where V and m are the virtual cluster size and number of VMs on a deteriorating PM, respectively; bwik (or bwki) is the bandwidth usage between the ith (or kth) VM and the kth (or ith) VM; Lik is the transmission cost between the ith reallocated VM and other VMs in the same virtual cluster. When the deteriorating PM is detected, if the kth (or ith) VM is the VM sender, the value of the binary variable dki (dik) is 1, otherwise 0. 3.1.4. Problem Formulation The total transmission cost of reallocated VMs can be formulated as an optimal objective function to minimize it under some constraints, and it can be formulated as follows: Objective:minTCtotal (7) s.t. ∑i=1MDibwxij<Cjbw, (8) ∑i=1MDimemxij<Cjmem, (9) ∑i=1MDicpuxij<Cjcpu, (10) ∑j=1Nxij=1,xij=0or1, (11) where N and M represent the number of PMs and VMs of the cloud data center, respectively; Dibw,Dimem and Dicpu denote the maximum resource requirements of the ith VM including bandwidth capacity, memory capacity and processing capacity, respectively; and Cjbw, Cjmem and Cjcpu denote the bandwidth idle capacity, memory idle capacity and processing idle capacity of the jth PM, respectively. Equations (8–10) denote that the PM’s idle resource capacity must be more than the sum of the resource requirements for each VM; Equation (11) denotes that a VM can only be placed on a PM. That is, if the ith VM is placed on the jth PM, then xij = 1, otherwise xij = 0. 3.2. PM Selection Optimization As we all know, BBO is a new stochastic global evolutionary algorithm, it has certain similar features with other biology-based algorithms (e.g. particle swarm optimization (PSO) [32]) and has good performance compared with other algorithms. For example, compared with BBO algorithm, PSO algorithm is more likely to clump together in similar groups [12]. In the standard BBO algorithm [12], the population of candidate solutions represents an ecosystem (i.e. archipelago of islands), in which each candidate solution is an island. A Habitat Suitability Index (HSI) is exploited to measure the goodness (i.e. fitness) of a candidate solution, that is, an island with a high (or low) HSI denotes a good (or poor) solution, and decided by Suitability Index Variables (SIVs) (e.g. temperature, rainfall, etc.). Therefore, a vector of SIVs represents a solution. There are two key operators (i.e. migration and mutation) in BBO. Among them, the migration is designed to probabilistically share SIVs between solutions, and thus increases the quality of low HSI solutions. In particular, it is a distinguishing feature of BBO from other population-based optimization algorithms. The mutation follows the migration and probabilistically exploits randomly generating new SIVs to replace SIVs in a solution. Further, the initial population can evolve repeatedly from generation to generation until a termination criterion is met. Such heuristics can find the correct solution in a small time and is therefore suitable for managing large data centers. Next, we will apply the standard BBO algorithm to solve the discrete single-objective virtual cluster reallocation optimization problem. We first propose ecosystem model and definition of parameters and operators of the BBO algorithm. Finally, we present implementation scheme of TVCR. 3.2.1. The Ecosystem Model of the BBO algorithm It is well known that the population of the virtual cluster reallocation optimization problem consists of multiple individuals, and each individual is a candidate virtual cluster reallocation solution. In TVCR, the population is treated as an ecosystem, each island represents an individual. Therefore, each island is a candidate virtual cluster reallocation solution. These islands have the same optimization objective (as shown in Equation (7)) and constraints (as shown in Equations (8–11)), such that each island exchange information with other islands with respect to the optimization objective and constraints to not only optimize itself but also optimize an entire ecosystem. For the ease of understanding, we define a matrix EPL (as shown in Equation (12)) to represent the ecosystem. The row number P represents the total number of islands in the ecosystem (i.e. the size of the population). The column number L represents the total number of reallocated VMs. The matrix element ei, j represents the PM number assigned to the jth reallocated VM of the ith island. Therefore, the candidate solution {ei,1, ei,2,…, ei, j, …, ei,L} denotes the ith island: EPL=|e1,1e1,2..e1,Le2,1e2,2..e2,L........eP,1eP,2..eP,L|. (12) Let E = {E1, E2,…, EP} denotes an ecosystem of P islands and Ek = {ek,1, ek,2,…, ek,L, O, C1, C2, C3, C4} represents the kth island, which contains a vector of L SIVs (i.e. the list of reallocated VMs), one objective O and four constraints C1, C2, C3 and C4. Each SIV is an integer index of the PM hosting a VM. O represents an objective defined in Equation (7). The four constraints C1, C2, C3 and C4 correspond to Equations (8–11). The HSI of the island Ek is computed using the objective function O. Combined with the above ecosystem and the specific characteristics of the virtual cluster reallocation problem, the parameters and operators of the BBO are defined as follows. According to the related literature [12], the immigration rate λ and emigration rate μ of a habitat (i.e. island) are a function of the number of species s (i.e. the number of used PMs), as shown in Equations (13) and (14). Smax is the largest possible number of species in the habitat (i.e. the number of reallocated VMs). As the number of species s gradually increases, the immigration rate λs gradually decreases, and emigration rate μs gradually increases. When λs is equal to μs, the number of species S0 in the habitat reaches equilibrium; the equilibrium state S0 will migrate as the environment changes. There are multiple function types of λs and μs; one of them is a linear function type, which is used in this paper. Especially, the maximum emigration rate is usually considered to equal the maximum immigration rate I = 1: μs=I⋅sSmax, (13) λs=I(1−sSmax). (14) We define Ps as the probability of the habitat including s species. When the probability changes from time t to time (t+Δt), it can be formulated as follows: Ps(t+Δt)=Ps(t)(1−λsΔt−μsΔt)+Ps−1λs−1Δt+Ps+1μs+1Δt. (15) Suppose that Δt is small enough, the probability of more than one immigration or emigration can be ignored. Taking the limit of (15) as Δt→0, the steady-state value of probability Ps with respect to the number of each species can be formulated as follows [33]: Ps={11+∑s=1Smaxλ0λ1...λs−1μ1μ2...μss=0λ0λ1...λs−1μ1μ2...μs(1+∑s=1Smaxλ0λ1...λs−1μ1μ2...μs)1≤s≤Smax. (16) To ensure the existence of the foregoing probabilities, the emigration rate μs cannot equal to 0 for all species. Definition 1 (Migration Operator): E → Ek is a probabilistic operator that adjusts habitat Ek based on the ecosystem E. The probability that Ek is modified is proportional to its immigration rate λk, and the probability that the source of the modification comes from Ei is proportional to the emigration rate μi. Definition 2 (Mutation Operator): Ek→ Ek is a probabilistic operator that randomly modifies habitat SIVs based on the habitat’s a priori probability of existence. The relationship between the number of biological species s in a habitat and its corresponding probability can be obtained by solving Equation (16). The mutation probability of the habitat is inversely proportional to the number of species s in the habitat; the mutation rate ms is calculated as ms=mmax(1−PsPmax), (17) where mmax represents the mutation probability given in advance which is assigned 0.1 [33] and Pmax=max{Ps}. Definition 3 (Removal Operator): Ei → Ei is an operator that identifies overloaded PMs of the habitat and replace them with normal PMs. In standard BBO algorithm [12], the mutation operator simply replaces the original SIV with a randomly generated SIV, and the migration operator replaces the immigrated SIV with the emigrated SIV. The two operators are easy to produce similar solution and lead to poor diversity of population. Therefore, the standard BBO algorithm designs the removal operator to eliminate these similar solutions and improve the diversity of population. When they are applied to TVCR, the two operators will generate overloaded PMs, that is, the resource requirement of the VMs placed in one PM is far more than the maximum capacity of the PM in a virtual cluster reallocation solution. Therefore, to remove these overloaded PMs and to improve the diversity of population, the removal operator is designed to find these overloaded PMs, and to replace these overloaded PMs with some normal PMs meeting the resource requirement of the VMs placed. Finally, to enhance the performance of TVCR, we exploit elitism to make sure that the best habitat is not lost from one generation to the next. It is common to save the best habitats at the beginning of each generation in a set and then replace the worst habitats with the set at the end of the generation after migration, mutation and removal have been completed. The size of the set (i.e. Nelite) is a tuning parameter, but it typically includes the best two habitats [34]. Meanwhile, when we use the migration rates to decide how much information to share between habitats, we replace selected SIVs (i.e. PMs) in such a way that the modified habitat is always feasible and better than the original habitat. Since TVCR exploits the mutation operator to avoid local extrema, the two strategies can improve its performance. 3.2.2. The Pseudo-code of TVCR In this section, we propose TVCR (as shown in Algorithm 1) to solve the above optimization problem. The TVCR performs the following steps: (a) the algorithm initially allocates a set of VMs (i.e. vmList) to a set of PMs (i.e. hostList) via the IVCA approach. (b) The algorithm anticipates the deteriorating PM of the cloud data center via the PM fault prediction module. (c) The algorithm maps the traffic-aware virtual cluster reallocation optimization problem to an ecosystem including SIVs and habitats, and initializes the size of the population P, the number of generations G, the maximum species Smax, the maximum immigration rates I, the maximum mutation rate mmax and the number of elites Nelite. (d) The algorithm initializes and sorts a random set of habitats via HSI. (e) The algorithm saves the elites, probabilistically exploits immigration, emigration and mutation to modify and mutate each non-elite habitat via Definitions 1 and 2, respectively, and removes the overloaded SIVs in each habitat via Definition 3. (f) The algorithm recomputes each HSI to sort all habitats in the ecosystem and select the elites. (h) Finally, the algorithm returns to Step (e) for the next iteration until a predefined number of generations. Algorithm 1 Our proposed TVCR approach 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution Algorithm 1 Our proposed TVCR approach 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution 1 Input: vmList, hostList, dvmList Output: VC reallocation solution 2 initially allocate vmList to hostList via IVCA approach 3 if anticipate the deteriorating PM then 4 Initialize the BBO parameters, Smax, I, mmax, Nelite, G, P 5 Initialize and sort a random set of habitats by HSI 6 for count of generation is not equal to Gdo 7 Save the Nelite elites 9 Use λ and μ to modify each non-elite habitat via definition 1 10 Mutate each non-elite habitat via definition 2 11 Remove the overloaded PM via definition 3 12 Sort all habitats based on HSI recomputed 13 Replace the Nelite habitats at the end with the elites 14 Sort all habitats based on HSI 15 end for 16 end if 17 return VC reallocation solution 4. PERFORMANCE EVALUATION In this section, we perform the thorough performance evaluation of TVCR approach via the simulation experiment. 4.1. Experiment Setup We build a system model in WebCloudSim system [31,36], which is based on CloudSim [35]. In this system, all of the experiments are executed in a 16-port fat-tree data center network [11], which comprises 64 core switches, 128 aggregation switches and 128 edge switches. To more intuitively observe experimental results, the transfer delays of the core, aggregation and edge switches are 1 s, 1 s and 2 s, respectively [36]. The system comprises 1024 heterogeneous PMs with resource capacity equivalent to 3720 or 5320 MIPS, 10 GB of RAM, 10 GB/s network bandwidth and 1 TB of storage and 4000 heterogeneous VMs [37]. The processing capacity of each VM is chosen randomly from seven values: 360, 490, 540, 620, 720, 860 or 1000 MIPS. The bandwidth requirement of each VM is set at 900 Mbps. The disk and the memory capacity of each VM are 1 and 1 GB [31], respectively. Appropriate parameter values of Algorithm 1 are determined on the basis of the related literatures and preliminary experiments, the size of the population was set at 20 and the number of generations was set at 30. To measure the performance of different reallocation approaches, a data-intensive application model including multiple different VMs is exploited. In fact, there have been some studies about the number of VMs processing a data-intensive application, which shows that more than 80% of applications use less than 10 VMs [38]. There are mainly three types of data-intensive applications: multi-tiered data-intensive application [39], workflow data-intensive application [40] and batch data-intensive application (i.e. MapReduce) [41]. According to the characteristics of these data-intensive applications, we exploit a set of the general data-intensive applications as our case in our experiments, in which each application consists of three tasks (such as Task 1, Task 2 and Task 3); Task 2 can perform its task after the processing results of Tasks 1 and 3 are transferred to Task 2. Note that each VM only processes a task which comprises some computation and communication stages [31]. We compare our approach with six other heuristic approaches: First-Fit (FF), Best-fit (BF), Random First-fit (RFF), IVCA, Modified Best-Fit Decreasing (MBFD) [42] and PSO [43]. As we all know, BF, FF and RFF are three classical approaches. When a deteriorating PM is monitored, there are multiple suitable PMs. BF identifies the PMs with the minimum CPU utilization to place these reallocated VMs. FF always allocates these reallocated VMs to the PMs that first satisfy the constraints. MBFD always places these reallocated VMs at the near-optimal PMs, which satisfy simultaneously the minimum transmission cost and energy consumption. However, different from former algorithms, RFF allocates these reallocated VMs by randomly selecting some PMs. 4.2. Experimental Results and Evaluation In this section, we compare the performance of our approach with six related approaches in terms of total transmission cost, algorithm execution time (i.e. the time which the approach finds the minimum transmission cost), total network resource consumption, and total processing time. In our experiments, the numbers of reallocated VMs, VMs and data-intensive applications were set at 9, 4000 and 1000, respectively. For each approach and each scenario, we conducted the experiments 10 times to compute the average results, which are discussed below. 4.2.1. Comparison of Total Transmission Cost The first set of experiments is conducted to analyze the algorithm execution time and the total transmission cost incurred due to the reallocated VMs migrating to healthy PMs. According to Equation (7), the transmission cost determines the total network resource consumption and the total processing time. The algorithm execution time determines and affects the efficiency and the execution results. As shown in Figs. 2 and 3, the experimental results indicate that TVCR has the least amount of total transmission cost and has a moderate algorithm execution time. This is because TVCR exploits the BBO global optimization algorithm to select healthy PMs for the reallocated VMs. Although PSO is also a heuristic algorithm, its solutions are more likely to clump together in similar group; the other five related approaches do not adopt the heuristic algorithm. Thus, the total transmission cost using the PSO algorithm is more than that of the BBO algorithm and far less than that of the five other related approaches. We can also see that RFF, BF and FF have almost similar higher total transmission cost and lower algorithm execution time; this is because that these approaches do not consider the total transmission cost of the VC allocation scheme. On the contrary, MBFD and IVCA both take into account the total transmission cost. Therefore, the total transmission cost is lower and algorithm execution time of them is higher. However, they still exceed the TVCR and PSO approaches. MBFD has more the total transmission cost than IVCA; this is due to the fact that MBFD needs to consider both the total transmission cost and the total energy consumption. However, IVCA does not consider simultaneously the above two metrics. Figure 2. View largeDownload slide Total transmission cost of all approaches. Figure 2. View largeDownload slide Total transmission cost of all approaches. Figure 3. View largeDownload slide Algorithm execution time of all approaches. Figure 3. View largeDownload slide Algorithm execution time of all approaches. 4.2.2. Comparison of Total Processing Time We have shown that the total transmission cost of TVCR is the least of the seven approaches. Meanwhile, there exists a direct relationship between the total transmission cost and the total processing time. Next, to validate the relationship, we will compare and analyze the total processing time of the seven approaches while jointly processing a set of data-intensive applications separately. As shown in Fig. 4, the experimental results show that the total processing time of TVCR approach is the least of all approaches. This is due to the fact that the VMs forming a virtual cluster are allocated more concentratedly by TVCR. To be precise, the more aggregation and edge switches and less core switches are required by TVCR. Thus, TVCR takes less total processing time to transfer the packets between the VMs in virtual clusters including the reallocated VMs. That is, less total processing time is needed to process a set of data-intensive applications than for the other approaches. Figure 4. View largeDownload slide Total processing time of all approaches. Figure 4. View largeDownload slide Total processing time of all approaches. Although the differences of these total processing times are small, the total processing time is used by the virtual clusters including the reallocated VMs; moreover, the numbers of virtual clusters and data-intensive applications are both too small. If there are then many data-intensive applications processed by massive virtual clusters in a cloud data center, the difference of these approaches will be large. Because the service response time and the quality of service have a direct relationship, the longer the response time, the worse the quality of service. Therefore, TVCR would be better from the quality of service perspective as well compared with the related approaches. 4.2.3. Comparison of Total Network Resource Consumption As shown in Figs. 5–8, TVCR consumes the least amount of network resources. This finding is observed because TVCR exploits the BBO-based approach to allocate the reallocated VMs to some healthy PMs. Therefore, the VMs of the virtual cluster which includes the reallocated VMs are allocated as compact as possible, e.g. in the same subnet. On the contrary, other approaches allocate these reallocated VMs more dispersedly. Meanwhile, according to the characteristics of the fat-tree, when one packet is routed by the core switches, it will also be transferred by the related aggregation and edge switches. Therefore, there are more core switches to be exploited. Further, the core link easily becomes a bottleneck. Thus, we should exploit our approach to reduce the number of used core switches. Figure 5. View largeDownload slide The number of edge switch packets processed by the all approaches. Figure 5. View largeDownload slide The number of edge switch packets processed by the all approaches. Figure 6. View largeDownload slide The number of aggregation switch packets processed by all approaches. Figure 6. View largeDownload slide The number of aggregation switch packets processed by all approaches. Figure 7. View largeDownload slide The number of root switch packets processed by all approaches. Figure 7. View largeDownload slide The number of root switch packets processed by all approaches. Figure 8. View largeDownload slide The total number of switch packets processed by all approaches. Figure 8. View largeDownload slide The total number of switch packets processed by all approaches. Based on the above analysis of the experimental results, TVCR outperforms six other approaches. Next, we further analyze the value of these metrics by varying the number of data-intensive applications (as shown in Figs. 9–11) and VMs (as shown in Figs. 12–14), respectively. Finally, we also analyze the impact of the number of reallocated VMs on these metrics (as shown in Figs. 15–17). Figure 9. View largeDownload slide The impact of different numbers of data-intensive applications on total transmission cost for all approaches. Figure 9. View largeDownload slide The impact of different numbers of data-intensive applications on total transmission cost for all approaches. Figure 10. View largeDownload slide The impact of different numbers of data-intensive applications on total processing time for TVCR. Figure 10. View largeDownload slide The impact of different numbers of data-intensive applications on total processing time for TVCR. Figure 11. View largeDownload slide The impact of different numbers of data-intensive applications on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 11. View largeDownload slide The impact of different numbers of data-intensive applications on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 12 View largeDownload slide The impact of different numbers of VMs on total transmission cost for all approaches. Figure 12 View largeDownload slide The impact of different numbers of VMs on total transmission cost for all approaches. Figure 13. View largeDownload slide The impact of different numbers of VMs on total processing time for TVCR. Figure 13. View largeDownload slide The impact of different numbers of VMs on total processing time for TVCR. Figure 14. View largeDownload slide The impact of different numbers of VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 14. View largeDownload slide The impact of different numbers of VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 15 View largeDownload slide The impact of different numbers of reallocated VMs on the total transmission cost for all approaches. Figure 15 View largeDownload slide The impact of different numbers of reallocated VMs on the total transmission cost for all approaches. Figure 16. View largeDownload slide The impact of different numbers of reallocated VMs on total processing time for TVCR. Figure 16. View largeDownload slide The impact of different numbers of reallocated VMs on total processing time for TVCR. Figure 17. View largeDownload slide The impact of different numbers of reallocated VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). Figure 17. View largeDownload slide The impact of different numbers of reallocated VMs on the total network resource consumption for all approaches including the RFF, TVCR, PSO, FF, BF, MBFD and IVCA approaches (read from left to right). 4.2.4. Impact of the Number of Data-Intensive Applications Figures 9–11 show the impact of the number of data-intensive applications on three metrics (i.e. total transmission cost, total processing time and total network resource consumption) for all approaches. In these experiments, the virtual cluster size, number of VMs and number of reallocated VMs were set at 3, 4000 and 9, respectively. Meanwhile, the number of data-intensive applications was varied from 100 to 1000 with a step value of 100. These figures indicate that the total transmission cost of all approaches was not substantially affected by the number of data-intensive applications. However, the total processing time of TVCR and the total network resource consumption of all approaches steadily increased with the increase in the number of data-intensive applications. Then, for each number of data-intensive applications, the total transmission cost and the total network resource consumption of TVCR were both the smallest of all approaches. 4.2.5. Impact of the Number of VMs Figures 12–14 show the impact of the number of VMs on all approaches. In this experiment, the virtual cluster size, number of VMs and number of reallocated data-intensive applications were set at 3, 9 and 1000, respectively. Meanwhile, we varied the number of VMs from 1000 to 5500, with a step value of 500. These figures show that the total transmission cost and the total network resource consumption for all approaches were not substantially affected by the number of VMs. However, the total processing time of TVCR fluctuates as the number of VMs varies. This is due to the fact that although the total transmission cost was not substantially affected by the change of the number of VMs, the routing path of each packet was diverse. Therefore, it is very likely that some packets need to be forwarded by the same switches; as a result, some forwarded packets need to line up to wait for the same switches. Therefore, there exists a time fluctuation under different numbers of VMs. 4.2.6. Impact of the Number of Reallocated VMs Figures 15–17 show the impact of the number of reallocated VMs on three metrics (i.e. total processing time, total transmission cost and total network resource consumption) of the all approaches. In this experiment, the virtual cluster size, number of data-intensive applications and number of VMs were set at 3, 1000 and 4000, respectively. Meanwhile, the number of reallocated VMs was varied from 2 to 9, with a step value of 1. These figures indicate that the total transmission cost of all approaches and the total processing time of TVCR increased as the number of reallocated VMs increased. However, the total network consumption of all approaches was not substantially affected by the number of reallocated VMs. 5. CONCLUSIONS AND FUTURE WORK In this work, we proposed TVCR to reallocate VMs hosted on a deteriorating PM to healthy PMs. To implement TVCR, we exploited a three-step process, in which we first exploited an IVCA approach to initially allocate a set of VMs, then exploited the PM fault prediction module to anticipate a deteriorating PM, and finally identified some healthy PMs to host these VMs on the deteriorating PM. Next, we performed the thorough performance evaluation of TVCR in term of total transmission cost, total processing time and total network resource consumption. The experimental results showed that TVCR outperforms six existed approaches. In our experimentations, we will exploit TVCR to initially allocate virtual clusters to a set of PMs to minimize the bandwidth usage in network core and the risk of violating the reliability requirement of a data-intensive application. ACKNOWLEDGEMENTS The work presented in this paper is supported by the NSFC (61472047 and 61602054), and Beijing Natural Science Foundation (4174100). REFERENCES 1 Foster , I. , Zhao , Y. , Raicu , I. and Lu , S. , ( 2008 ) Cloud computing and grid computing 360-degree compared. Proc. GCE 08, Austin, TX, USA, 12–16 November, pp.1–10, IEEE, New York. 2 Barham , P. , Dragovic , B. , Fraser , K. , Hand , S. , Harris , T.L. , Ho , A. , Neugebauer , R. , Pratt , I. and Warfield , A. , ( 2003 ) Xen and the Art of Virtualization. Proc. SOSP 03, Bolton Landing, New York, USA, 19–22 October, pp. 164–177, ACM, New York. 3 Middleton , A. ( 2010 ) Data-Intensive Technologies for Cloud Computing. Borko Furht and Armando Escalante. In Handbook of Cloud Computing . Springer , US, New York . 4 Buyya , R. , Yeo , C.S. , Venugopal , S. , Broberg , J. and Brandic , I. ( 2009 ) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility . Future Gener. Comput. Syst. , 25 , 599 – 616 . Google Scholar CrossRef Search ADS 5 Skillicorn , D. and Talia , D. ( 1998 ) Models and languages for parallel computation . ACM Comput. Sruveys , 30 , 123 – 169 . Google Scholar CrossRef Search ADS 6 Liang , Q. , Zhang , J. , Zhang , Y. and Liang , J. ( 2014 ) The placement method of resources and applications based on request prediction in cloud data center . Inf. Sci. , 279 , 735 – 745 . Google Scholar CrossRef Search ADS 7 Ho , J. , Hsiu , P. and Chen , M. , ( 2015 ) Improving Serviceability for Virtual Clusters in Bandwidth-Constrained Datacenters. Proc. CLOUD 15, New York, USA, 27 June–2 July, pp. 710–717, IEEE, New York. 8 Chowdhury , M. , Zaharia , M. , Ma , J. , Jordan , M.I. and Stoica , I. , ( 2011 ) Managing Data Transfers in Computer Clusters with Orchestra. Proc. SIGCOMM 11, Toronto, Ontario, Canada, 15–19 August, pp. 98–109, ACM, New York. 9 Networking , C.V. ( 2017 ) Cisco Global Cloud Index: Forecast and Methodology, 2015–20, Cisco Systems, California. 10 Breitgand , D. and Epstein , A. , ( 2012 ) Improving Consolidation of Virtual Machines with Risk-Aware Bandwidth Oversubscription in Compute Clouds. Proc. INFOCOM 12, Orlando, FL, USA, 25–30 March, pp. 2861–65, IEEE, New York. 11 Al-Fares , M. , Loukissas , A. and Vahdat , A. , ( 2008 ) A Scalable, Commodity Data Center Network Architecture. Proc. SIGCOMM 08, Seattle, WA, USA, 17–22 August, pp. 63–74, ACM, New York. 12 Simon , D. ( 2008 ) Biogeography-based optimization . IEEE Trans. Evol. Comput , 12 , 702 – 713 . Google Scholar CrossRef Search ADS 13 Duan , J. , Guo , Z. and Yang , Y. , ( 2015 ) Cost Efficient and Performance Guaranteed Virtual Network Embedding in Multicast Fat-Tree DCNs. Proc. INFOCOM 15, Kowloon, Hong Kong, 26 April–1 May, pp.136–144, IEEE, New York. 14 Meng , X. , Papps , V. and Zhang , L. , ( 2010 ) Improving the Scalability of Data Center Networks with Traffic-Aware Virtual Machine Placement. Proc. INFOCOM 10, San Diego, CA, USA, 14–19 March, pp.1–9, IEEE, New York. 15 Dias , D. and Costa , L. , ( 2012 ) Online Traffic-Aware Virtual Machine Placement in Data Center Networks. Proc. GIIS 12, Choroni, Venezuela, 17–19 December, pp. 1–8, IEEE, New York. 16 Breitgand , D. , Epstein , A. and Glikson , A. , ( 2013 ) Network Aware Virtual Machine and image Placement in a Cloud. Proc. CNSM 13, Zurich, Switzerland, 14–18 October, pp. 9–17, IEEE, New York. 17 You , K. , Tang , B. and Ding , F. , ( 2013 ) Near-Optimal Virtual Machine Placement with Product Traffic Pattern in Data Centers. Proc. ICC 13, Budapest, Hungary, 9–13 June, pp. 3705–3709, IEEE, New York. 18 Biran , O. , Corradi , A. , Fanelli , M. , Foschini , L. , Nus , A. , Raz , D. and Silvera , E. , ( 2012 ) A Stable Network-Aware VM Placement for Cloud Systems. Proc. CCGRID 12, Ottawa, Canada, 13–16 May, pp. 498–506, IEEE, New York. 19 Cohen , R. , Eytan , L. , Naor , J. and Raz , D. , ( 2013 ) Almost Optimal Virtual Machine Placement for Traffic Intense Data Centers. Proc. INFOCOM 13, Turin, Italy, 14–19 April, pp. 355–359, IEEE, New York. 20 Ilkhechi , A. , Korpeoglu , I. and Ulusoy , O. ( 2015 ) Network-aware virtual machine placement in cloud data centers with multiple traffic-intensive components . Comput. Netw. , 91 , 508 – 527 . Google Scholar CrossRef Search ADS 21 Kuo , J. , Yang , H. and Tsai , M. , ( 2014 ) Optimal Approximation Algorithm of Virtual Machine Placement for Data Latency Minimization in Cloud Systems. Proc. INFOCOM 14, Toronto, ON, Canada, 27 April–2 May, pp. 1303–1311, IEEE, New York. 22 Alicherry , M. and Lakshman , T. , ( 2013 ) Optimizing Data Access Latencies in Cloud Systems by Intelligent Virtual Machine Placement. Proc. INFOCOM 13, Turin, Italy, 14–19 April, pp. 647–655, IEEE, New York. 23 Jiang , J. , Lan , T. , Ha , S. and Chen , M. , ( 2012 ) Joint VM Placement and Routing for Data Center Traffic Engineering. Proc. INFOCOM 12, Orlando, FL, USA, 25–30 March, pp. 2876–2880, IEEE, New York. 24 Li , X. , Wu , J. , Tang , S. and Lu , S. , ( 2014 ) Let’s Stay Together: Towards Traffic Aware Virtual Machine Placement in Data Centers. Proc. INFOCOM 14, Toronto, ON, Canada, 27 April−2 May, pp. 1842–1850, IEEE, New York. 25 Fukunaga , T. , Hirahara , S. and Yoshikawa , H. , ( 2015 ) Virtual Machine Placement for Minimizing Connection Cost in Data Center Networks. Proc. INFOCOM WKSHPS 15, Hong Kong, China, 26 April− May, pp. 486–491,IEEE, New York. 26 Zhu , J. , Li , D. , Wu , J. and Liu , H. , ( 2012 ) Towards Bandwidth Guarantee in Multi-tenancy Cloud Computing Networks. Proc. ICNP 12, Austin, TX, USA, 30 October−2 November, pp. 1–10, IEEE, New York. 27 Jin , H. , Pan , D. , Xu , J. and Pissinou , N. , ( 2012 ) Efficient VM Placement with Multiple Deterministic and Stochastic Resources in Data Centers. Proc. GLOBECOM 12, Anaheim, CA, USA, 3–7 December, pp. 2505–2510, IEEE, New York. 28 Kandula , S. , Sengupta , S. , Greenberg , A. , Patel , P. and Chaiken , R. , ( 2009 ) The Nature of Data Center Traffic: Measurements & Analysis. Proc. IMC 09, Chicago, IL, USA, 04–06 November, pp. 202–208, ACM, New York. 29 Wang , M. , Meng , X. and Zhang , L. , ( 2011 ) Consolidating Virtual Machines with Dynamic Bandwidth Demand in Data Centers. Proc. INFOCOM 11, Shanghai, China, 10–15 April, pp. 71–75, IEEE, New York. 30 da Silva , R.A.C. and da Fonseca , N.L.S. , ( 2015 ) Algorithm for the Placement of Groups of Virtual Machines in Data Centers. Proc. ICC 15, London, UK, 8–12 June, pp. 6080–6085, IEEE, New York. 31 Liu , J. , Wang , S. , Zhou , A. , Kumar , S. , Yang , F. and Buyya , R. ( 2016 ) Using proactive fault-tolerance approach to enhance cloud service reliability . IEEE Trans. Cloud Comput. , doi:10.1109/TCC.2016.2567392. 32 Kenndy , J. and Eberhart , R. , ( 1995 ) Particle Swarm Optimization. Proc. ICNN 95, Perth, Western Australia, 27 November−1 December, pp. 1942–1948, IEEE, New York. 33 Ma , H. , Ni , S. and Sun , M. , ( 2009 ) Equilibrium Species Counts and Migration Model Tradeoffs for Biogeography-Based Optimization. Proc. CDC/CCC 09, Shanghai, China, 15–18 December, pp. 3306–3310, IEEE, New York. 34 Simon , D. , Ergezer , M. and Du , D. , ( 2009 ) Population Distributions in Biogeography-Based Optimization Algorithms with Elitism. Proc. SMC 09, San Antonio,TX, USA, 11–14 October, pp. 991–996, IEEE, New York. 35 Calheiros , R.N. , Ranjan , R. , Beloglazov , A. , De Rose , C.A. and Buyya , R. ( 2011 ) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms . Softw. Pract. Exper. , 41 , 23 – 50 . Google Scholar CrossRef Search ADS 36 Zhou , A. , Wang , S. , Zheng , Z. , Hsu , C.-H. , Lyu , M.R. and Yang , F. ( 2016 ) On cloud service reliability enhancement with optimal resource usage . IEEE Trans. Cloud Comput. , 4 , 452 – 466 . Google Scholar CrossRef Search ADS 37 Beloglazov , A. and Buyya , R. ( 2012 ) Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers . Concur. Comput.: Pract. Exper. , 24 , 1397 – 1420 . Google Scholar CrossRef Search ADS 38 Bodík , P. , Menache , I. , Chowdhury , M. , Mani , P. , Maltz , D.A. and Stoica , I. , ( 2012 ) Surviving Failures in Bandwidth-Constrained Datacenters. Proc. SIGCOMM 12, Helsinki, Finland, 13–17 August, pp. 431–442, ACM, New York. 39 Lee , J. , Turner , Y. , Lee , M. , Popa , L. , Banerjee , S. , Kang , J.-M. and Sharma , P. , ( 2014 ) Application-Driven Bandwidth Guarantees in Datacenters. Proc. SIGCOMM 14, Chicago, IL, USA, 17–22 August, pp. 467–478, ACM, New York. 40 Garg , S.K. and Buyya , R. ( 2013 ) An environment for modeling and simulation of message-passing parallel applications for cloud computing . Softw. Pract. Exper. , 43 , 1359 – 1375 . Google Scholar CrossRef Search ADS 41 Yang , H.-c. , Dasdan , A. , Hsiao , R.-L. and Parker , D.S. , ( 2007 ) Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters. Proc. SIGMOD 07, Beijing, China, 11–14 June, pp. 1029–1040, ACM, New York. 42 Beloglazov , A. , Abawajy , J. and Buyya , R. ( 2012 ) Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing . Future Gener. Comput. Syst. , 28 , 755 – 768 . Google Scholar CrossRef Search ADS 43 Wang , S. , Liu , Z. , Zheng , Z. , Sun , Q. and Yang , F. , ( 2013 ) Particle Swarm Optimization for Energy-Aware Virtual Machine Placement Optimization in Virtualized Data Centers. Proc. ICPADS 13, Seoul, South Korea, 15–18 December, pp. 102–109, IEEE, New York. © The British Computer Society 2017. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Journal

The Computer JournalOxford University Press

Published: Sep 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off