Scalability and Accuracy in a Large-Scale Network Emulator Ken Yocum, Kevin Walsh, Amin Vahdat, Priya Mahadevan, Dejan Kostic, Jeff Chase, and David Becker ∗ Department of Computer Science, Duke University {grant,walsh,vahdat,priya,dkostic,chase,becker}@cs.duke.edu Experimental distributed services are dif cult to evaluate due to their scale and the complexity of the Internet. The lack of an accepted evaluation methodology that predicts real behavior is a key obstacle to progress in distributed services research. ModelNet is a network emulation environment for evaluating distributed services across largescale networks. A key research contribution of ModelNet is the development of a number of novel techniques for large-scale network emulation that trade accuracy for scalability. ModelNet allows researchers to deploy unmodi ed software prototypes in a con gurable Internet-like environment. Unlike live deployment, where researchers wait for extraordinary events to test a distributed application’s resilience to harsh network conditions, ModelNet can reproduce those events, emulating ‚ash crowds and network partitions. While simulation often abstracts away key details of real systems, emulation in ModelNet captures complex behavior by running unmodi ed application code and kernels at the end systems. ModelNet emulates a wide-area network on a scalable gigabit LAN cluster. Edge nodes, running user-speci ed OS’s and application software, are con gured to route their packets through a set of ModelNet core nodes, which cooperate to subject the traf c to the bandwidth, latency, and loss pro le of a target network topology. Target topologies are created from Internet traces (e.g., from CAIDA), BGP dumps, or synthetic topology generators (GT-ITM, Brite, or Inet). Of course, no cluster can capture the full scale and complexity of the Internet. A signi cant contribution of this work is an evaluation of a set of scalable techniques for approximating Internet conditions. One technique, distillation, is applied to the target topology before running the emulation. Distillation applies a set of transforms to the target topology to decrease emulation cost. Another technique, assignment, distributes the emulation across ∗ This work is supported in part by the U.S. National Science Foundation (EIA-9972879), Hewlett Packard, IBM, Intel, and Microsoft. Vahdat is also supported by an NSF CAREER award (CCR-9984328) the ModelNet cores. The goal of these techniques is to provide the highest emulation accuracy within the capacity of the available emulation hardware. ModelNet distillation lowers emulation cost by reducing the number of hops packets take in the target topology. With respect to the measured overheads of our hardware, removing an average of two hops per packet can theoretically double the emulation capacity, in packets/sec, of a single core. There are two extremes to distillation. A hop-by-hop distillation is isomorphic to the target topology and provides the highest degree of emulation accuracy. Application cross traf c and contention on all links are emulated faithfully. In contrast, an end-to-end distillation creates a fully interconnected mesh between all clients. Accuracy suffers as cross traf c is isolated from previously affected ‚ows. Distillation provides a continuum between these two extremes, trading accuracy for increasing scalability. ModelNet assignment maps pieces of the distilled topology across the available number of ModelNet cores. The ideal assignment is dependent not only on the static information available in the distilled topology (routes, link rates and delays), but also upon the runtime load generated by the application. Assignment seeks to minimize the number of packets crossing between cores. This reduces both CPU and link utilization. Our current assignment techniques, based on graph partitioning algorithms, signi cantly reduce intercore traf c, decreasing the number of cores required for the emulation. ModelNet cores are implemented as a set of loadable kernel modules for FreeBSD. Core nodes can accurately emulate 85,000 packets/sec at nearly 100% CPU utilization. At this load packets are delayed within a millisecond of link speci cations. Experiments with several large-scale distributed services demonstrate the generality and effectiveness of the infrastructure: a peer-to-peer le service, an adaptive overlay, a replicated web service, and an ad hoc wireless communication scenario. ACM SIGCOMM Computer Communications Review Volume 32, Number 3: July 2002

/lp/association-for-computing-machinery/scalability-and-accuracy-in-a-large-scale-network-emulator-QsKnLvBy7W