TY - JOUR
AU - Vatavu, Radu-Daniel
AB - Abstract Software architecture and applications for the connected car can process and share a variety of digital content, among which high-definition video and augmented reality (AR) content, toward enhanced driving assistance, navigation and infotainment systems and services. However, several technical challenges need to be overcome to make such systems and services viable and efficient, including dealing effectively with a variety of types of systems, devices and platforms, either installed inside the vehicle or represented by the personal mobile and wearable devices of the drivers and passengers. In this paper, we outline these technical challenges and propose a software solution in the form of an event-based middleware layer by modeling the smart, connected car as a specific type of a smart environment. We employ an adapted version of Euphoria, a recently introduced software architecture for general-purpose smart environments, to implement asynchronous communications among heterogeneous input/output devices inside the vehicle. We also adapt Euphoria to fit into the four-layer infrastructure model of the connected car. We conduct a technical evaluation of the request-response time performance achieved with the Euphoria middleware for streaming digital content from 1 Mbps (360p@30fps) to 32 Mbps (4K@30fps) on various devices, either integrated in the vehicle, not integrated but used inside the vehicle and devices outside the vehicle (the control condition). Our results show effective live streaming achieved for 2K content at 30fps with the 600 Mbps network (i.e., the connected car) and for 4K content at 30fps with the 1.7 Gbps network envisioned for hyper-connected vehicles. These results open up opportunities for high-definition video and AR applications in the automotive industry. RESEARCH HIGHLIGHTS Model the smart car as a specific type of a smart environment. Present technical challenges for the delivery of high-definition video and AR content inside smart cars. Report empirical results for high-definition content delivery inside smart cars. 1. INTRODUCTION The automotive industry is moving toward the concept of a software-driven car to meet the increasing consumers’ needs and demands for smart services inside connected vehicles (Burkacky et al., 2018, GreenCarCongres, 2018, Pelliccione et al., 2017). Autonomous driving and connectivity with road infrastructure, other vehicles and pedestrians as well as data sharing between the connected vehicle and the drivers’ and passengers’ smart devices (Bilius & Vatavu, 2020) represent just a few examples of applications that necessitate complex in-vehicle software architecture (Burkacky et al., 2018, Charette, 2009, Petit & Shladover, 2015, Zheng et al., 2016). The growing need for smart services, high-definition content delivery and fast communications for the connected car has led to the vision of a ‘hyper-connected vehicle’ powered by 1.7 Gbps networks (Gai & Violante, 2016), which subsumes more data being processed, more content rendered and, consequently, more complex in-vehicle software architecture that needs to run efficiently. This vision includes high-definition video and augmented reality (AR) content delivered inside the vehicle. Several application scenarios have been proposed in the scientific community for in-vehicle AR to enhance the driving and travel experience (Alhaija et al., 2018, Kim & Dey, 2016, Moniri et al., 2012, Qiu et al., 2017, Rameau et al., 2016, Yuan et al., 2018). One of the most frequent scenario has been to extend the driver’s field of view with insights about what is happening around the vehicle (Alhaija et al., 2018, Olaverri-Monreal et al., 2010, Qiu et al., 2017, Rameau et al., 2016, Yuan et al., 2018), which enables the driver to become more aware of other entities, e.g., vehicles, pedestrians, road infrastructure, etc., and, thus, to make better and more informed decisions that can increase driving safety. Other scenarios have addressed passengers by delivering more information about specific points of interest along the road (Lim et al., 2015, Schinke et al., 2010) and by enabling advanced interactions inside the vehicle and with the vehicle (Bilius & Vatavu, 2020, Gheran & Vatavu, 2020, Kim & Dey, 2016, Moniri et al., 2012). To enable such features, smart vehicles must be able to efficiently manage complex hardware and software modules and components, starting with the modules in charge of data collection and ending with the specific processing pipelines and systems that present content to users (Alhaija et al., 2018, Moniri et al., 2012, Rameau et al., 2016). Figure 1 Open in new tabDownload slide The three-step method adopted in this paper (new model for the smart car as a specific type of a smart environment, adaptation of software architecture for general-purpose smart environments to hyper-connected cars and technical evaluation) and the corresponding theoretical and practical contributions that this paper makes at each step. Figure 1 Open in new tabDownload slide The three-step method adopted in this paper (new model for the smart car as a specific type of a smart environment, adaptation of software architecture for general-purpose smart environments to hyper-connected cars and technical evaluation) and the corresponding theoretical and practical contributions that this paper makes at each step. In this context, there are several pressing technical challenges (GreenCarCongres, 2018, Pelliccione et al., 2017, Schipor & Vatavu, 2019). For example, when content is generated on-the-fly, processing pipelines need to be flexible enough to adapt to a wide range of situations, e.g., the movement of pedestrians and of other vehicles or to changes in the quality and reliability of input data when, for instance, the car video cameras become obstructed by weather conditions such as rain or snow. Moreover, the in-vehicle infrastructure integrates heterogeneous hardware and software components that employ different technology, platforms and operating systems; see Fig. 2 for an illustrative example showing the in-vehicle infotainment system and the driver’s smartwatch. In this context, an imperative design requirement is represented by loose coupling among the various modules, devices and systems from the in-vehicle environment, a paradigm that has been considered only recently in the automotive industry (Burkacky et al., 2018, Gai & Violante, 2016, GreenCarCongres, 2018, Zheng et al., 2016). How high-definition video and AR content fits into this paradigm and how practical it is to deliver such content effectively inside the vehicle, given the currently available in-vehicle processing and communications technology, remains to be understood. In this paper, we present both theoretical and practical advances in this direction (see Fig. 1 for a visual illustration of our method and contributions), as follows: (i) We present an inventory of current design and engineering challenges regarding the delivery of high-definition video and AR content inside the vehicle, and we contribute a new formalization of the in-vehicle environment where the hyper-connected car is modeled as a specific kind of a smart environment. (ii) Within the new framework of the connected car as a smart environment, we employ Euphoria (Schipor et al., 2019a), a recently introduced software architecture for general-purpose smart environments, specifically designed to deal effectively with applications in which heterogeneous input/output devices share data. We propose an extension and adaptation of Euphoria for the standard four-layer infrastructure of connected cars—the Application, Service, Devices and Electronics layers (Burkacky et al., 2018, Marisetty et al., 2009, Milosevic et al., 2018, Traub et al., 2017, Zheng et al., 2016)—for which Euphoria plays the role of middleware and handles subscriptions, notifications and message processing between software clients. (iii) We evaluate the technical performance of our software architecture for high-definition video and AR content delivered inside the vehicle. We report empirical results from a controlled experiment involving content size from 1 Mbps (e.g., 360p video at 30 fps) to 32 Mbps (2160p ultra-high-definition content at 30 fps) delivered to the infotainment system of a connected vehicle (Nissan Qashqai), which we compare against two other conditions represented by a conventional smartphone used inside the vehicle and a high-end laptop used outside the vehicle. Figure 2 Open in new tabDownload slide We model hyper-connected cars as a specific type of a smart environment co-inhabited by in-vehicle systems, drivers and passengers and mobile and wearable devices. Figure 2 Open in new tabDownload slide We model hyper-connected cars as a specific type of a smart environment co-inhabited by in-vehicle systems, drivers and passengers and mobile and wearable devices. 2. Related Work We discuss prior work on the design and engineering of systems and applications for connected cars, including Ethernet networks of sensors and devices, and we overview in-vehicle AR systems and applications. 2.1. The connected and hyper-connected car The concept of a connected car describes a vehicle engaged in asynchronous communications with other entities to enhance the driving and travel experience of the passengers (Karnouskos & Kerschbaum, 2018). One possible model to look at and analyze the connected car is to include vehicles into the Internet-of-Things (IoT) (Gerla et al., 2014), which requires cars to have dedicated hardware and software architectures (Burkacky et al., 2018, Gai & Violante, 2016, GreenCarCongres, 2018, Traub et al., 2017) that implement IoT standards and communications. Another model presents the connected car as the central entity of its informational ecosystem, able to receive data from the environment as well as to produce data for other entities (Karnouskos & Kerschbaum, 2018, Lu et al., 2014). This perspective is specific to the Vehicle-to-Everything (V2X) paradigm, which includes Vehicle-to-Infrastructure (V2I), Vehicle-to-Vehicle (V2V), Vehicle-to-Network (V2N), Vehicle-to-Pedestrian (V2P) and Vehicle-to-Device (V2D) communications. By implementing 1.7 Gbps network communications, connected cars transition to hyper-connectivity. The concept of a connected car is key for autonomous driving (Lugano, 2017). The US Transportation Security Administration has defined six levels of autonomous cars, from nonautonomous to fully autonomous vehicles (Liu et al., 2017). While most cars implement Level 2 (i.e., the driver can disengage from certain functions, such as cruise control and parking), the automotive industry is now exploring Level 3 (full automation, but the driver still needs to be present) and Level 4 autonomy (no driver needed behind the wheel on controlled routes, such as highways); see Colquitt et al. (2017). The transition from one level of autonomy to the next does not represent solely a quantitative shift (i.e., a growth in sensing and computing resources), but also a qualitative one (Markwalter, 2017, Liu et al., 2017) that requires novel architectural approaches based on loosely coupled modules and flexible processing chains (Liu et al., 2017). 2.2. Ethernet networks of heterogeneous devices for hyper-connected cars CES 2018 featured a Cisco and Hyundai demonstration of a flexible, scalable and secure platform to support configurable interactions between the software and hardware modules inside the vehicle (Corbett et al., 2016, GreenCarCongres, 2018, Traub et al., 2017, Zheng et al., 2016). Special attention was dedicated to the integration of heterogeneous components, such as legacy hardware and software libraries from different vendors, into that platform. Also, instead of having multiple one-to-one communications between the various components of the platform, the new approach proposed a decoupled architecture where modules acted as hosts within the 1.7 Gbps Ethernet network. The resulting Software-Defined Vehicle architecture (GreenCarCongres, 2018) has played a key role for the next generation of hyper-connected cars. In this context, advances in security, quality of service and scalability can be achieved by incorporating knowledge and results from the field of computer networks (Corbett et al., 2016); e.g., prior work has showed that Ethernet networks can successfully address most of the requirements of the automotive industry in terms of communications (Corbett et al., 2016). Moreover, a high-end network infrastructure enables connections with external systems, such as cloud data centers, smart city infrastructure, other vehicles and pedestrians (Traub et al., 2017, Zheng et al., 2016). A report on future mobility of Burkacky et al. (2018) envisioned Ethernet technology as ‘the backbone of the car’ to replace current approaches based on cluster data buses (Sharma et al., 2009) and to facilitate reuse of modules across different manufacturers (Staron, 2017). The report also highlighted Ethernet networks as a viable solution to handle the increasing number of on-board smart sensors. blackThe transition to Ethernet networks is equally facilitated by the standardization of testing processes, e.g., the automotive Ethernet TCP/IP stack. Also, the AUTomotive Open System ARchitecture (AUTOSAR) provides high-level specifications for testing protocols (AUTOSAR, 2016), while the OPEN Alliance (OPEN Alliance, 2016) has released the Ethernet testing specifications for Electronic Control Units (ECUs). 2.3. AR systems for hyper-connected cars AR is one type of content that will become more and more relevant for the automotive industry. Recently, there has been a growing interest in designing AR applications for in-vehicle contexts of use (Cao et al., 2018, Colley et al., 2018, 2017, Jose et al., 2016), but the photorealism and responsiveness expected from modern AR applications demand high-performing devices, large network bandwidth and high computational resources. For instance, the Augmented Reality for Enterprise Alliance (AREA, 2018) specifies a minimum on-board memory storage of 128 Gb for industry use case devices; the 2018 Ovum/Intel Report on 5G for media and entertainment forecast that 5G will unlock AR and VR applications (Ovum, 2018); while the global automotive AR/VR market is expected to reach 673 billion US dollars by 2025 (Liu, 2019). In this context, one key aspect to driver future development in this direction is to rethink the way in which the in-vehicle components and modules interact with each other to support drivers’ and passengers’ user experience with in-vehicle AR (Davies, 2018, Rao et al., 2014). These applications require the ability to handle dynamically generated content, manage a wide variety of technologies and assure proper decoupling of hardware and software modules. Moreover, understanding the performance of high-definition video and AR content delivery in hyper-connected cars would be useful to practitioners, but experimental results in this direction are limited. One way to deliver AR inside the vehicle is via virtual assistants (Lugano, 2017). In this regard, the automotive industry has adapted existing virtual assistants (e.g., Daimler with Google Assistant, BMW and Nissan with Cortana, Ford with Alexa, General Motors with OnStar Go) and also created new ones (e.g., Toyota Yui, Honda HANA, Volkswagen Sedric) (Baraniuk, 2017). The goal is to combine safety with infotainment and IoT within a flexible software architecture (Lugano, 2017) that goes beyond current in-vehicle applications that can be characterized as borderline AR, e.g., conventional parking assistance systems (Liu et al., 2017, Wang et al., 2014). Regarding the latter, hyper-connected cars can integrate data captured by external sensors and entities, such as the video cameras from the parking lot or nearby vehicles in order to achieve better collision detection performance (Hammoudi et al., 2018, Khanna & Anand, 2016, Mahendra et al., 2017). Such advanced features, however, put demands on the network bandwidth and processing power of the in-vehicle system, and the software architecture of such vehicles needs to be able to process content received from heterogeneous systems and generate AR content on the fly. 2.4. An overview of middleware for connected vehicles The increase in the complexity of electronics and software from the connected vehicle has led to many attempts to design appropriate middleware. For example, InCloud (Saini et al., 2017) is a middleware framework for infotainment application development in the Vehicular Adhoc Network. InCloud presents several advantages: content is easy to integrate, software clients are lightweight since most of the processing is performed remotely, the coupling between the interface and the processing modules is loose, and several sources of content can be combined. The framework revolves around four design requirements: fusion of data from multiple sources, context-awareness, reusability and loose coupling. Cisco and Hyundai proposed the Software Defined Vehicle architecture (GreenCarCongres, 2018) as a secure, multi-layer platform for integrating both software and hardware components; see subsection 2.2. Sadio et al. (2019) reported evaluation results for this platform, such as packet loss, throughput and delay in network communications. The in-vehicle environment hosts heterogeneous technologies (i.e., hardware and software components from different manufacturers) that must be integrated reliably. Joshi (2019) designed and implemented a service-oriented middleware to integrate various types of ECUs with the AUTOSTAR specification. The implementation employed an Ethernet network for in-vehicle communications since other approaches (e.g., CAN, LIN, or FlexRay) were not sufficient to manage the complexity of the driving assistance systems (Joshi, 2019, Gopu et al., 2016, Häckel, 2018). The middleware of Joshi (2019) relied on three types of components—providers, consumers and registry—and enabled streams, messages and remote procedure calls. Häckel (2018) extended the architecture with protocol abstraction through standardized interfaces, Internet-based technology and lightweight capability to achieve compatibility with low-end and legacy ECUs. The middleware was evaluated using the OMNeT++ simulation software (Häckel, 2018). Several approaches have been proposed to implement loose coupling between the components of the connected vehicle. According to the digital twin platform paradigm, the hardware components of a cyber-physical system have associated software counterparts (Yun et al., 2017, July); see Josifovska et al. (2019) that integrated the components of a digital twin system and highlighted their structural properties and interrelations. Interoperability and low coupling between heterogeneous components can also be achieved with component-based modeling (Hellwig et al., 2019), platform-agnostic logical models and middleware. The approach of Hellwig et al. (2019) was to propose a tag-based multi-platform code generation for such components starting from their declaration. 3. Challenges for Delivering High-Definition Content to the Hyper-Connected Vehicle In the following, we identify the main technical challenges regarding middleware for delivering content, including high-definition video and AR, in connected and hyper-connected vehicles (Schipor & Vatavu, 2019). To illustrate these challenges, we resort to a practical scenario mentioned repeatedly in prior work (Alhaija et al., 2018, Olaverri-Monreal et al., 2010, Qiu et al., 2017, Rameau et al., 2016, Yuan et al., 2018). In this scenario, the in-vehicle hardware and software modules interoperate to extend the driver’s visual field to cover other entities around the vehicle, such as pedestrians, traffic signs and other vehicles. The virtual 3D model of the space surrounding the car is generated based on a variety of input delivered by video cameras, proximity sensors, other vehicles and road infrastructure. Drivers are presented with a virtual model by means of video projections, smartglasses, head-mounted displays, or holograms. Such systems aim to reduce the risks of road accidents by assisting drivers during complex maneuvers, such as overtaking other cars or parking. The in-vehicle system can also deliver passengers with enriched entertainment experiences (Bilius & Vatavu, 2020, Lim et al., 2015). We will use this scenario throughout this section to illustrate challenges for AR systems in connected cars. 3.1. Challenge #1: production of content on the fly AR content is dynamically generated and synchronized with the physical world (Alhaija et al., 2018, Qiu et al., 2017). This feature is achieved by real-time data acquisition and processing and via a specialized chain of hardware and software modules. To respond effectively to changes from the environment, this processing chain needs to add or remove components and reconfigure the processing pipeline on the fly. The corresponding technical challenge is to ensure flexible association between the components of the AR models and the modules of the in-vehicle system. While in a classical architecture the relationships between components are defined during design, a flexible processing pipeline requires a higher level of abstraction (Sharma et al., 2009, Staron, 2017). In this case, components need to deliver results in a standardized way without any knowledge of the potential consumers (Burkacky et al., 2018, Milosevic et al., 2018). 3.2. Challenge #2: heterogeneous devices Hyper-connected cars rely on a wide variety of hardware and software technologies (Blom et al., 2016, Diaconescu et al., 2018, Pelliccione et al., 2017, Staron, 2017), an aspect that is important to consider when designing and engineering in-vehicle AR (Burkacky et al., 2018, Staron, 2017). The data collected for the generation of the virtual models require inter-vehicle communications, high-speed networks and effective and efficient transmission protocols. Furthermore, the generation of the virtual models requires robust computer vision algorithms for object detection and recognition under complex conditions, e.g., bad weather, low light and so on. Additionally, hyper-connected cars must assist drivers in specific situations, such as overtaking or parking, by presenting estimations of distances and emergency breaking, which are features that require real-time processing capabilities. An effective software architecture for the connected car should enable unobtrusive operation of such modules, interweaving engine operation, in-vehicle infotainment services, connections to smart devices and services from the cloud, and driving safety (Gai & Violante, 2016, Staron, 2017). In this context, a recent trend has gained traction in the automotive industry regarding new architectures that are highly inter-operable and that can host virtually any hardware and software technology (Burkacky et al., 2018, Milosevic et al., 2018, Sharma et al., 2009, Staron, 2017). 3.3. Challenge #3: effective decoupling of hardware and software modules Connected cars rest on the principle of high abstraction of their hardware and software components so that heterogeneous systems can be easily integrated. In fact, the automotive industry has gradually increased the level of abstraction from microcontrollers to systems-on-chip and to scalable computing platforms (Gai & Violante, 2016, Traub et al., 2017). By moving from a hardware-centered to a function-centered perspective and approach, software applications can run on a central computer (Burkacky et al., 2018, Gai & Violante, 2016, Traub et al., 2017). For example, AUTOSTAR (Gai & Violante, 2016) offers a standardized runtime environment for the Services and Applications layers; see Fig. 3, left. Moreover, AUTOSTAR can operate in conjunction with embedded virtualization technologies and leverage their capabilities to hide the complexity of the hardware. To control the data flow, such an approach requires a central information server and a broker for the data transferred through the network (Traub et al., 2017). The project “Hercules” (Marko, 2018) is another example of a high-performing architecture for low-power embedded systems. One of its goals was the implementation of a homogeneous application programming interface (API) on top of heterogeneous commercial off-the-shelf platforms. Figure 3 Open in new tabDownload slide Integration of the Euphoria software architecture (Schipor et al., 2019a), designed for general-purpose smart environments, into the standard four-layer infrastructure of the connected car at the middleware level. Figure 3 Open in new tabDownload slide Integration of the Euphoria software architecture (Schipor et al., 2019a), designed for general-purpose smart environments, into the standard four-layer infrastructure of the connected car at the middleware level. 4. The Euphoria Software Architecture for Hyper-Connected Cars A 2018 Report by McKinsey & Co. (Burkacky et al., 2018) presented a design for in-vehicle layered architecture connecting low-level components (e.g., sensors, actuators and power electronics) with high-level modules (infotainment, connectivity and cloud access) via a middleware layer; see the left part of Fig. 3. From this perspective, a significant part of the business logic of in-vehicle applications can be transferred to distinct processing layers, such as layers that implement Artificial Intelligence tasks or advanced analytics. The Applications layer can also be expanded to support multiple guest operating systems within the infotainment component and micro-kernel architectures (David & Matt, 2012). This blue-print was designed to deliver effective data redundancy mechanisms for safety-critical applications. In this section, we connect to the four-layer model of the connected car (Burkacky et al., 2018) in which we integrate Euphoria (Schipor et al., 2019a), a software architecture designed for general-purpose smart environments, to act as middleware. But, before, we introduce the concept of a hyper-connected car as a specific kind of a smart environment. 4.1. The hyper-connected car as a specific kind of a smart environment We model the hyper-connected car as a specific kind of a smart environment, where the in-vehicle modules, end users (driver and passengers) and other systems and devices (e.g., smartphones, tablets, smartwatches, etc.) co-inhabit the in-vehicle environment and create, store, process and share content. We adopt this perspective and model in order to introduce a software architecture for hyper-connected cars that (i) connects to the four layers designed by previous work for connected cars, i.e., Applications, Services, Devices and Electronics (Burkacky et al., 2018, Marisetty et al., 2009, Milosevic et al., 2018, Traub et al., 2017, Zheng et al., 2016) and (ii) is inspired from and builds on top of recent, high-performing software architecture designs for general-purpose smart environments (Schipor et al., 2019a). Next, we present the characteristics of Euphoria (Schipor et al., 2019a) that recommend its use for our purpose. 4.2. The Euphoria software architecture for general-purpose smart environments and heterogeneous input and output devices In the following, we conduct our discussion around the core engine of Euphoria (Schipor et al., 2019a), a general event-driven architecture for engineering interactions in smart environments between heterogeneous entities; see Schipor et al. (2019a) for technical details, evaluation of technical performance and examples of applications in (Popovici et al., 2019, Schipor & Vatavu, 2018, Schipor et al., 2017, 2019b, c). Euphoria rests on ten design criteria: two handling techniques to describe the processing of events (i.e., event-driven and asynchronous processing), four quality features that characterize how entities interact with each other (i.e., adaptability, modularity, flexibility and interoperability), and four contextual properties regarding specific technology (i.e., web-based, open-source, smart space and JavaScript); see Schipor et al. (2019a). Also, Euphoria consists of five layers: Producers and Consumers (i.e., actual devices and software components), Emitters and Receivers (standardized abstractions of actual producers and consumers), and the core Engine responsible for dispatching events and messages from producers to consumers. The asynchronous and event-driven design of Euphoriaassures high decoupling between its components (Schipor et al., 2019a). Moreover, the abstraction of standardized interfaces enables heterogeneous hardware and software modules to interoperate, which falls in line with recent trends in designing automotive architectures (Traub et al., 2017, Zheng et al., 2016). The informational flow is implemented by JSON messages delivered through WebSockets, and the engine is written entirely in JavaScript (Schipor et al., 2019a). Euphoria has been used for engineering applications for smart environments (Popovici et al., 2019, Schipor & Vatavu, 2018, Schipor et al., 2017), and extended to meet other requirements; see Schipor et al. (2019b, c) for discussions about integrating peripheral interaction with smart environments and AR powered by the Euphoria architecture. We chose Euphoria over other software architecture designs and platforms for smart environments (Fortino et al., 2012, Zahariadis et al., 2014, Marquardt et al., 2011, Fortino et al., 2014, Nebeling et al., 2014, Ledo et al., 2015, Roda et al., 2016, Mocanu & Schipor, 2017, Lou et al., 2017) because it implements more quality properties. For example, compared to the UbiComp middleware (Goumopoulos et al., 2009) and SPINE (Bellifemine et al., 2011), Euphoria was specifically designed for the web; compared to GS-GPE (Lou et al., 2017) and GPWS (Vatavu et al., 2012), Euphoria allows various input modalities for end users; and compared to MAPS (Aiello et al., 2011) that needs adaptations to support interactions, Euphoria implements them natively. We refer the interested reader to Table 1 from Schipor et al. (2019a) and the associated discussion for more comparisons between Euphoria and other software architectures and platforms proposed in the scientific literature in the context of smart environments. 4.3. Euphoria for the hyper-connected car Euphoria rests on common Internet standards and protocols implemented by virtually all programming languages and operating systems. This feature is useful in our application scenario, because handling high-definition content as well as specific content (video, AR) requires the integration of various types of devices, platforms and applications. Due to the standardized header of the messages transmitted through Euphoria, the processing flow can be rerouted at software level to respond dynamically to various types of events. Figure 4 Open in new tabDownload slide Two examples of JSON messages for data exchange. The header sections contain metadata about the entity and the event, i.e., the detection of a speed limit traffic sign (top) and the detection of a gesture (bottom). The body section consists of fields required to specify the context of the event, i.e., the maximum speed and the gesture command, respectively. Figure 4 Open in new tabDownload slide Two examples of JSON messages for data exchange. The header sections contain metadata about the entity and the event, i.e., the detection of a speed limit traffic sign (top) and the detection of a gesture (bottom). The body section consists of fields required to specify the context of the event, i.e., the maximum speed and the gesture command, respectively. Figure 5 Open in new tabDownload slide JavaScript code exemplifying a web client application that creates Euphoria consumers and sends messages using HTTP requests. Figure 5 Open in new tabDownload slide JavaScript code exemplifying a web client application that creates Euphoria consumers and sends messages using HTTP requests. We employ Euphoria to mediate non-critical interactions between in-vehicle systems since it is Ethernet-based, works with the abstraction of hardware and software modules and employs the JSON format for messages. Figure 3 illustrates the integration of Euphoria with the main four layers of an in-vehicle architecture. The Applications level consists of all the software modules that provide interfaces for passengers (e.g., part of the in-vehicle infotainment system and the advanced driving assistance system) (Engen et al., 2009, Okuda et al., 2014); entertainment, navigation and autonomous driving applications rely on the Services layer; and the in-vehicle hardware is addressed by the Devices and Electronics layers. While Devices relate to high-level components with which users can interact directly, Electronics comprise the processing and power circuits of the vehicle. Each layer has a dedicated adapter to interface with the Euphoria middleware. However, critical, real-time and safety-related functions benefit of direct informational shortcuts, as illustrated in Fig. 3. In order for external components to become Consumers,1 they need to register with Euphoria by using the API provided by the Subscription module; see the right part of Fig. 3. The registration process involves creation of a web socket and specifying which types of messages will be forwarded to the consumer. The consumer sends to the engine a list of entity-event associations and the Notification module will feed the consumer with messages that match those associations. Each message is encoded in the JSON format with a header section containing the entity and type of event; see Fig. 4 for examples. The Publication-Center implements the connection between the engine and Producers.2 Producers deliver messages both by HTTP requests (GET and POST) and via web sockets; see Fig. 5. To avoid the decapsulation of large messages, Euphoria allows entities and event types to be specified directly in the request string; see Fig. 6 for an example. The Message-Processing-Service is in charge with identifying the entity and event type of a message either from the request string or from the header section. This information is converted in an internal format and made available to the Notification module. Figure 6 Open in new tabDownload slide JavaScript code exemplifying a server application that registers Euphoria consumers and dispatches messages to registered consumers. Figure 6 Open in new tabDownload slide JavaScript code exemplifying a server application that registers Euphoria consumers and dispatches messages to registered consumers. Modules can act both as producers and consumers, and the HTTP web client illustrated in Fig. 5 exemplifies this aspect. The same functionality can be obtained using various programming languages and platforms. Once the request for creating a new client has been sent to the server, the subscription process is triggered; see Fig. 6. Each consumer consists of a web socket and a list of entity-event associations. Messages received by Euphoria are dispatched according to their types. 4.4. Addressing the challenges for delivering high-definition content in connected cars using the Euphoria middleware Euphoria can successfully address the challenges of handling content in hyper-connected cars (see our discussion from Section 3), as follows: (i) Euphoria can handle dynamically generated content. In Euphoria, all the entities, i.e., devices and software modules, are logically connected by the information from the headers of standardized messages. When a producer creates a message, it does not need to know which consumers will receive that message. Moreover, a consumer can dynamically select which messages to handle by sending the list of producers and event types to Euphoria. Therefore, associations between the various components of the AR model and the modules of the system can be dynamically established using the header of exchanged messages. This aspect is important when content is not static, but dynamically generated and modified by the various components of the application. (ii) Euphoria can manage a wide variety of in-vehicle technologies. Euphoria was designed to use common Internet standards and protocols for data processing and exchange, e.g., HTTP, WebSockets and JSON, implemented in virtually all operating systems and programming languages. The core components of Euphoria are written in JavaScript and, thus, can run on any device equipped with a standard web browser. Embedded systems, devices and software modules can be integrated within Euphoria by exposing their public interfaces via Internet protocols. Components become consumers by instantiating web sockets using the Subscription API. Messages are delivered to Euphoria via HTTP requests and web sockets. (iii) Euphoria assures high decoupling for the hardware and software modules. This outcome follows from a design requirement of Euphoria to integrate heterogeneous hardware and software components (Schipor et al., 2019a). The registration and notification mechanisms implemented in Euphoria treat both the hardware and software modules unitarily so that the actual implementation of a component is transparent for developers and end users. Due to the standardized interfaces (see Figs 5 and 6), changes operated within a specific module remain isolated and do not propagate to the other levels. Figure 7 Open in new tabDownload slide Apparatus employed in our experiment to measure request-response times: (1) infotainment system integrated in a a Nissan Qashqai, (2) smartphone used inside the vehicle and (3) laptop outside the vehicle (control condition). Note: the same client application (HTML and JavaScript) was used on each device and is displayed in all the photos from (1) to (3). Figure 7 Open in new tabDownload slide Apparatus employed in our experiment to measure request-response times: (1) infotainment system integrated in a a Nissan Qashqai, (2) smartphone used inside the vehicle and (3) laptop outside the vehicle (control condition). Note: the same client application (HTML and JavaScript) was used on each device and is displayed in all the photos from (1) to (3). 5. Experiment We conducted a controlled experiment to evaluate the technical performance of the Euphoria middleware for applications that stream content up to 4K ultra-high definition video. For this purpose, we implemented two JavaScript applications, which we ran on various devices and measured the request-response times for content delivered through Euphoria. One application acted as a Producer and generated data of various sizes (from 1 Mbps to 32 Mbps) and streamed that data via Euphoria to the second application that acted as the Client. 5.1. Design Our experiment was a repeated-measure, within-subjects design with the following three independent variables: (i) Content-Size: the size of the video or AR content, after compression, transmitted through Euphoria and ranging from 1 Mb to 32 Mb with the content size following a geometric progression with ratio 2, i.e., 1, 2, 4, 8, 16 and 32 Mb. We chose these conditions to correspond to one second of video streamed at resolutions and frame rates of 360p@30fps (⁠|$\approx $|1 Mbps), 480p@30fps (2 Mbps), 720p@30fps (HD, 4 Mbps), 1080p@30fps (Full HD, 8 Mbps), 1440p@30fps (2K, 16 Mbps) and 2160p@30fps (4K, 32 Mbps), respectively, according to the recommended upload encoding settings, frame rates and bit rates for YouTube videos.3 (ii) Device, ordinal variable with three conditions: the infotainment system of a Nissan Qashqai vehicle (Android) and two other devices (high-end laptop and conventional smartphone running Windows and Android, respectively), described in detail in the Apparatus section. These conditions correspond to: (1) device integrated in the vehicle; (2) device not integrated, but used inside the vehicle; and (3) high-end device outside the vehicle (the control condition). (iii) Network, ordinal variable with two conditions: 600 Mbps@2.4GHz and 1.7 Gbps@5GHz, representing the signal rate characteristics of two wireless networks connecting each device with Euphoria. Our experiment design had a total number of |$6 \times 3 \times 2 = 36$| conditions to measure our dependent variable, Request-Response-Time (in milliseconds), when streaming digital content via the Euphoria middleware. 5.2. Task We performed 100 repeated measurements (referred to as trials) for each combination of the experimental conditions. We generating random data of size ranging from 1 Mb to 32 Mb by selecting pixels at random from a high-resolution photograph. The order of the trials and the actual content that was transferred were randomized to prevent biases in time measurements due to caching effects in the wireless network out of our control. Figure 8 Open in new tabDownload slide The effect of Content-Size, Device and Network on the Request-Response-Time needed to stream one second of content of size ranging from 1 Mb to 32 Mb using the Euphoria middleware. Note: the horizontal scale is logarithmic. Figure 8 Open in new tabDownload slide The effect of Content-Size, Device and Network on the Request-Response-Time needed to stream one second of content of size ranging from 1 Mb to 32 Mb using the Euphoria middleware. Note: the horizontal scale is logarithmic. Figure 9 Open in new tabDownload slide Our empirical results demonstrate the difference in technical performance between the status-quo (600 Mbps networks for the connected car) and what is possible with 1.7 Gbps networks (for the hyper-connected car). Future work will address integration of 1.7 Gbps compatible devices in the vehicle environment (steps 1-3 in the figure and see the text for details), which will open new opportunities for new empirical studies and applications for the hyper-connected car. Figure 9 Open in new tabDownload slide Our empirical results demonstrate the difference in technical performance between the status-quo (600 Mbps networks for the connected car) and what is possible with 1.7 Gbps networks (for the hyper-connected car). Future work will address integration of 1.7 Gbps compatible devices in the vehicle environment (steps 1-3 in the figure and see the text for details), which will open new opportunities for new empirical studies and applications for the hyper-connected car. 5.3. Apparatus Euphoria ran on a Windows 10 platform powered by a Dell Inspiron 15 laptop featuring an Intel Core i7-7500U 2.7 GHz CPU with 16 GB RAM. The in-vehicle system was a custom EDOTEC 499 infotainment device, integrated in a Nissan Qashqai (production version 5D 1.2L M/T 2WD/2015),featuring an Octa-Core 1.8 GHz CPU with x86-64 architecture, Spreadtrum SC9853 chipset, 2 GB RAM and running Android 8; see Fig. 7, left. Despite being a high-end device for in-vehicle infotainment, the EDOTEC 499 system supported only the 600 Mbps 2.4 GHz wireless network connection. Therefore, we also tested Euphoria on a Huawei P10 Lite smartphone with an Octa-Core 2.1 GHz CPU, 3 GB RAM and Android 8 (a conventional smartphone) that supported the 1.7 Gbps 5 GHz wireless connection. We included the smartphone as a distinct experimental condition to understand the opportunity of using the 1.7 Gbps network inside the vehicle and, therefore, to draw implications for future designs of and requirements for in-vehicle infotainment systems. The smartphone condition also represents the situation where the driver or one of the passengers prefers to stream content directly on their personal mobile device, while inside the vehicle; see Fig. 7, middle. We also employed a high-end laptop with an Intel Core i7-4710HQ 2.5 GHz CPU, 12 GB RAM memory and Windows 10 as a control condition (Fig. 7, right) to understand how the request-response times obtained for the infotainment system and the smartphone used inside the vehicle compare to those obtained on a high-end computing device used outside the vehicle. The 600 Mbps and the 1.7 Gbps Wi-Fi networks were implemented using the same equipment (ASUS RT-AC87U wireless router) that supported both modes of operation. 5.4. Results Figure 8 illustrates the effects of Content-Size, Device and Network independent variables on the Request-Response-Time as the average performance, for each combination of the independent variables, of the 100 trials. The smallest response time (23 ms) was obtained when streaming one second of content in the condition 320p@30 fps between Euphoria and the high-end laptop (our control condition) over the 1.7 Gbps Wi-Fi network. The request-response times increased for the control condition up to 369 ms for content of size 2160@30fps and the 1.7 Gbps network and to 744 ms when the 600 Mbps network was used. This result shows that, in the control condition involving a high-end device, Euphoria can effectively deliver high-definition content. Having established the baseline, we look at the performance of the infotainment system integrated in the vehicle, for which only the 600 Mbps wireless network connection was supported. Request-response times started at 114 ms for the 320p@30 fps condition for content size and reached 837 ms for 1440p@30fps. These results show that the in-vehicle system can effectively deliver 2K content using the Euphoria middleware. However, in the most demanding condition represented by 4K content, the in-vehicle system delivered 1,954 ms, well beyond the 1 s limit. This result shows the limitations of current systems commonly available in today’s vehicles (the infotainment system was limited to 600 Mbps connections only, and we could not make it to work with the 1.7 Gbps network) and indicates that, in order to obtain better request-response times, it is necessary to shift to faster network connections, i.e., from the connected car to the hyper-connected car. In the light of the baseline and integrated system results, we want to understand the potential of the Euphoria middleware to stream high-definition content to a hyper-connected car. To this end, we look at the smartphone condition where the smartphone, used inside the vehicle, was connected to the 1.7 Gbps wireless network. The request-response time for the most demanding experimental condition (2160p@30fps) was just 407 ms; see Fig. 8. This result shows the benefits of moving toward hyper-connected cars, for which Euphoria was able to deliver the same content in just 20.8% of the time needed by the integrated infotainment system using the 600 Mbps network connection. A regression analysis showed statistically significant linear relationships (⁠|$R^2{>}.989$|⁠, |$p{<}.001$|⁠) between Request-Response-Time (denoted with T in the equations shown in Fig. 8) and Content-Size (denoted with CS) for the four Device|$\times $|Network combinations examined in our experiment. Since the average request-response time measurements for the 1.7 Gbps network were upper bounded by 1,000 ms, we can conclude that the Euphoria middleware supports live streaming of Ultra High-Definition content up to 4K@30fps. The request-response time for the in-vehicle infotainment system indicates that 2K video content can be streamed even over a 600 Mbps network, and the smartphone condition demonstrates that 4K content at 30fps is equally achievable inside the vehicle when shifting to 1.7 Gbps connections. 6. Conclusion and Future Work We presented in this paper empirical results regarding software architecture for live streaming of high-definition content inside connected and hyper-connected vehicles. Our contribution was possible by adopting a model for the hyper-connected car as a distinct type of a smart environment, which enabled us to leverage recent advances in software architecture design for general-purpose smart environments, which we adapted and applied for the in-vehicle environment. Our empirical results showed that 2K video can be streamed for 600 Mbps Wi-Fi networks using the Euphoria middleware, but also that 1.7 Gbps support is needed (i.e., moving toward the hyper-connected car in the automotive industry) in order to attain the 4K@30fps content limit inside the vehicle, which we were able to effectively demonstrate with the passenger’s smartphone experimental condition. Moreover, the smartphone and the high-end laptop (the control) showed very similar performance for the 1.7 Gbps network. While our empirical results are sufficient to highlight the difference in technical performance between the status-quo (600 Mbps networks) and what is possible with hyper-connectivity (1.7 Gbps networks), future work will look at integrating 1.7 Gbps-compatible devices in the vehicle via the dedicated interfaces (e.g., audio/video input, CanBus, radio antenna, illumination control, etc.); see Fig. 9 for an illustration of work in progress. This approach will enable more experiments and reporting of more results in this direction. Our results also open opportunities for deploying software applications for the hyper-connected car that render high-definition content to drivers and passengers, including AR applications. Future work will explore in-vehicle AR applications, supported by our architecture, such as photorealistic computer-generated graphics to assist drivers with enhanced navigation features, while providing enriched entertainment experiences to passengers. Exploring various input modalities to interact with high-definition content inside the vehicle (Bilius & Vatavu, 2020) represents an equally interesting direction for future work. Moreover, future work is needed to examine the impact of producing and delivering AR content not only in terms of the capabilities of the network, but also in terms of processing power of the CPUs/ECUs. We also believe that our method to model a specific environment as a smart environment in order to borrow and use readily available knowledge and tools could also be applied to other areas of applications for emerging technologies in the IoT domain that relate to the concept of a smart, connected car. Acknowledgments This work was supported by a grant of the Romanian Ministry of Research and Innovation, CCCDI - UEFISCDI, project no. PN-III-P1-1.2-PCCDI-2017-0917 (21PCCDI/2018), within PNCDI III. The infrastructure was provided by Ştefan cel Mare University of Suceava and was partially supported by the project ”Integrated center for research, development and innovation in Advanced Materials, Nanotechnologies and Distributed Systems for fabrication and control,” no. 671/09.04.2015, Sectoral Operational Program for Increase of the Economic Competitiveness, co-funded by the European Regional Development Fund. Original versions of the icons used to produce Fig. 1 were made by Freepik (https://www.flaticon.com/authors/freepik, ‘Miscellaneous Elements’ pack) from Flaticon (https://www.flaticon.com/), free for commercial use with attribution license. Footnotes 1 Terminology used by the Euphoria software architecture to denote applications that are the final recipients of content distributed through the architecture (created by producers); see Schipor et al. (Schipor et al., 2019a) for details and examples. 2 See the previous footnote. 3 YouTube. Recommended upload encoding settings. https://support.google.com/youtube/answer/1722171?hl=en References Alhaija , H. A. , Mustikovela , S. K., Mescheder , L., Geiger , A. and Rother , C. ( 2018 ) Augmented reality meets computer vision: Efficient data generation for urban driving scenes . Int. J. Comput. Vis. , 126 , 961 – 972 . http://doi.org/10.1007/s11263-018-1070-x. Google Scholar OpenURL Placeholder Text WorldCat Augmented Reality for Enterprise Alliance . ( 2018 ). Augmented Reality Functional Requirements . https://thearea.org/area-resources/augmented-reality-functional-requirements/ (accessed September 9, 2019) . Bilius , L.-B. and Vatavu , R.-D. ( 2020 ) A synopsis of input modalities for in-vehicle infotainment and consumption of interactive media . Proc. ACM int. conf. on interactive media experiences , 2020 . https://doi.org/10.1145/3391614.3399400 . Google Scholar OpenURL Placeholder Text WorldCat Blom , H. , Chen , D.-J., Kaijser , H., Lönn , H., Papadopoulos , Y., Reiser , M.-O., Kolagari , R. T. and Tucci , S. ( 2016 ) East-ADL: An architecture description language for automotive software-intensive systems in the light of recent use and research . Int. J. Syst. Dynam. Appl. (IJSDA) , 5 , 1 – 20 . http://doi.org/10.4018/IJSDA.2016070101 . Google Scholar OpenURL Placeholder Text WorldCat Burkacky , O. , Deichmann , J., Doll , G. and Knochenhauer , C. ( 2018 ) Rethinking car software and electronics architecture . McKinsey & Co., February , https://www.mckinsey.com/industries/automotive-and-assembly/our-insights/rethinking-car-software-and-electronics-architecture . Google Scholar OpenURL Placeholder Text WorldCat Cao , C. , Li , Z., Zhou , P. and Li , M. ( 2018 ) Amateur: augmented reality based vehicle navigation system . Proc. of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT) , 2 , 4, Article 155 (2018), 24 , 2474 – 9567 . http://doi.org/10.1145/3287033. Google Scholar OpenURL Placeholder Text WorldCat Charette , R. N. ( 2009 ) This car runs on code . IEEE Spectrum , 46 , 3 . https://spectrum.ieee.org/transportation/systems/this-car-runs-on-code . Google Scholar OpenURL Placeholder Text WorldCat Colley , A. , Häkkilä , J., Forsman , M.-T., Pfleging , B. and Alt , F. ( 2018 ) Car exterior surface displays: exploration in a real-world context . In Proc. of the 7th ACM int. symposium on pervasive displays (PerDis ’18) . ACM , NY, USA , Article 7, 8 pages 978-1-4503-5765-4 . http://doi.org/10.1145/3205873.3205880 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Colley , A. , Häkkilä , J., Pfleging , B. and Alt , F. ( 2017 ) A design space for external displays on cars . In Proc. of the 9th int. conf. on automotive user interfaces and interactive vehicular applications adjunct (AutomotiveUI ’17) , pp. 146 – 151 . 978-1-4503-5151-5 . ACM . http://doi.org/10.1145/3131726.3131760 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Corbett , C. , Schoch , E., Kargl , F. and Preussner , F. ( 2016 ) Automotive Ethernet: security opportunity or challenge . Sicherheit 2016-Sicherheit, Schutz und Zuverlässigkeit , https://dl.gi.de/20.500.12116/880 . Google Scholar OpenURL Placeholder Text WorldCat David , K. and Matt , J. ( 2012 ). In-vehicle infotainment software architecture: Genivi and beyond. https://www.eetimes.com/document.asp?doc_id=1279345 . Davies , C. ( 2018 ) Your car windshield could be an AR hologram in , 2020 . https://www.slashgear.com/porsche-hyundai-wayray-augmented-reality-/holographic-windshield-investment-18546322/ (accessed September 9, 2019) . Diaconescu , C. , Sburlan , D. and Popovici , D.-M. ( 2018 ) Augmenting selection by intention for in-vehicle control and command . Proc. of RoCHI 2018, the 15th int. conf. on human computer interaction, 2018 , 107 – 110 https://dblp.org/rec/bib/conf/rochi/DiaconescuSP18 . Google Scholar OpenURL Placeholder Text WorldCat Engen , T. , Lervag , L.-E. and Moen , T. ( 2009 ) Evaluation of IVIS/ADAS using driving simulators. Comparing Performance Measures in Different Environments . European transport conf., 2009. Association for European Transport (AET) , https://trid.trb.org/view/907831 . Google Scholar OpenURL Placeholder Text WorldCat Gai , P. and Violante , M. ( 2016 ) Automotive embedded software architecture in the multi-core age . In 2016 21th IEEE European test symposium (ETS) , pp. 1 – 8 . IEEE. http://doi.org/10.1109/ETS.2016.7519309 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Gerla , M. , Lee , E.-K., Pau , G. and Lee , U. ( 2014 ) Internet of vehicles: from intelligent grid to autonomous cars and vehicular clouds . In 2014 IEEE world forum on internet of things (WF-IoT) , pp. 241 – 246 . IEEE http://doi.org/10.1109/WF-IoT.2014.6803166 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Gheran , B.-F. and Vatavu , R.-D. ( 2020 ) From controls on the steering wheel to controls on the finger: using smart rings for in-vehicle interactions . Proc. of the ACM int. conf. on designing interactive systems (Companion) . https://doi.org/10.1145/3393914.3395851. Google Scholar OpenURL Placeholder Text WorldCat GreenCarCongres . ( 2018 ) Cisco and Hyundai to bring ”hyper-connected” car to production in 2019; software defined vehicle architecture . https://www.greencarcongress.com/2018/01/20180113-cisco.html (accessed September 9, 2019) . Jose , R. , Lee , G. A. and Billinghurst , M. ( 2016 ) A Comparative study of simulated augmented reality displays for vehicle navigation . In Proc. of the 28th Australian conf. on computer-human interaction (OzCHI ’16) , pp. 40 – 48 . ACM , NY, USA. 978-1-4503-4618-4 http://doi.org/10.1145/3010915.3010918 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Karnouskos , S. and Kerschbaum , F. ( 2018 ) Privacy and integrity considerations in hyperconnected autonomous vehicles . Proc. IEEE , 106 , 160 – 170. http://doi.org/10.1109/JPROC.2017.2725339 . Google Scholar OpenURL Placeholder Text WorldCat Kim , S. J. and Dey , A. K. ( 2016 ) Augmenting human senses to improve the user experience in cars: applying augmented reality and haptics approaches to reduce cognitive distances . Multimed. Tools Appl. , 75 , 9587 – 9607. http://doi.org/10.1007/s11042-015-2712-4 . Google Scholar OpenURL Placeholder Text WorldCat Lim , C. , Choi , J., Park , J.-I. and Park , H. ( 2015 ) Interactive augmented reality system using projector-camera system and smart phone . In 2015 int. symposium on consumer electronics (ISCE) , pp. 1 – 2 . IEEE. http://doi.org/10.1109/ISCE.2015.7177800 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Shanghong Liu . 2019 . Global AR/VR Automotive Spending 2017 and 2025. https://www.statista.com/statistics/828499/world-ar-vr-automotive-spending/ . Lu , N. , Cheng , N., Zhang , N., Shen , X. and Mark , J. W. ( 2014 ) Connected vehicles: solutions and challenges . IEEE Internet Things J. 1, 4 (2014) , 289 – 299. https://ieeexplore.ieee.org/document/6823640 . Google Scholar OpenURL Placeholder Text WorldCat Marisetty , S. , Srivastava , D., Hoffmann , J. and Starks , B. ( 2009 ) Low power Intel Architecture platform for in-vehicle infotainment . Intel Technol. J. , 13 , 1 . https://www.eetimes.com/document.asp?doc_id=1276766 . Google Scholar OpenURL Placeholder Text WorldCat Marko , B . Hercules: High-Performance Real-time Architectures for Low-Power Embedded Systems . https://hercules2020.eu/ (accessed September 9, 2019) . Milosevic , M. , Bjelica , M. Z., Maruna , T. and Teslic , N. ( 2018 ) Software platform for heterogeneous in-vehicle environments . IEEE Trans. Consumer Electron. , 64 , 213 – 221. http://doi.org/10.1109/TCE.2018.2844737 . Google Scholar OpenURL Placeholder Text WorldCat Moniri , M. M. , Feld , M. and Müller , C. ( 2012 ) Personalized in-vehicle information systems: Building an application infrastructure for smart cars in smart spaces . In 2012 8th int. conf. on intelligent environments. IEEE , pp. 379 – 382 . http://doi.org/10.1109/IE.2012.40 . Okuda , R. , Kajiwara , Y. and Terashima , K. ( 2014 ) A survey of technical trend of ADAS and autonomous driving . In Technical Papers of 2014 int. symposium on VLSI design, automation and test ,pp. 1 – 4 . IEEE. http://doi.org/10.1109/VLSI-TSA.2014.6839646 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Olaverri-Monreal , C. , Gomes , P., Fernandes , R., Vieira , F. and Ferreira , M. ( 2010 ) The See-Through System: A VANET-enabled assistant for overtaking maneuvers . In 2010 IEEE intelligent vehicles symposium , pp. 123 – 128 . IEEE . http://doi.org/10.1109/IVS.2010.5548020 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Ovum and Intel . ( 2018 ). How 5G Will Transform the Business of Media & Entertainment. https://ovum.informa.com/resources/product-content/intel-5g-ebook . Pelliccione , P. , Knauss , E., Rogardt , H., Ågren , S. M., Mallozzi , P., Alminger , A. and Borgentun , D. ( 2017 ) Automotive architecture framework: The experience of volvo cars . J. Syst. Archit. , 77 , 83 – 100. http://doi.org/10.1016/j.sysarc.2017.02.005 . Google Scholar OpenURL Placeholder Text WorldCat Petit , J. and Shladover , S. E. ( 2015 ) Potential cyberattacks on automated vehicles . IEEE Trans. Intell. Trans. Syst. , 16 , 546 – 556. http://doi.org/10.1109/TITS.2014.2342271 . Google Scholar OpenURL Placeholder Text WorldCat Popovici , I. , Schipor , O.-A. and Vatavu , R.-D. ( 2019 ) Hover: exploring cognitive maps and mid-air pointing for television control . Int. J. Hum., Comput. Stud. , 129 , 95 – 107. http://doi.org/10.1016/j.ijhcs.2019.03.012 . Google Scholar OpenURL Placeholder Text WorldCat Qiu , H. , Ahmad , F., Govindan , R., Gruteser , M., Bai , F. and Kar , G. ( 2017 ) Augmented vehicular reality: enabling extended vision for future vehicles . In Proc. of the 18th int. workshop on mobile computing systems and applications , pp. 67 – 72 . ACM. http://doi.org/10.1145/3032970.3032976 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Rameau , F. , Ha , H., Joo , K., Choi , J. and Park , K. So Kweon. (2016) A real-time augmented reality system to see-through cars . IEEE Trans. Vis. Comput. Graph. , 22 , 2395 – 2404. http://doi.org/10.1109/TVCG.2016.2593768 . PubMed OpenURL Placeholder Text WorldCat Rao , Q. , Grünler , C., Hammori , M. and Chakraborty , S. ( 2014 ) Design methods for augmented reality in-vehicle infotainment systems . In Proc. of the 51st annual design automation conf. , pp. 1 – 6 . ACM. http://doi.org/10.1145/2593069.2602973 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Schinke , T. , Henze , N. and Boll , S. ( 2010 ) Visualization of off-screen objects in mobile augmented reality . In Proc. of the 12th int. conf. on human computer interaction with mobile devices and services , pp. 313 – 316 . ACM. http://doi.org/10.1145/1851600.1851655 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Schipor , O.-A. and Vatavu , R.-D. ( 2019 ) Towards interactions with augmented reality systems in hyper-connected cars . Proc. of the 2nd workshop on charting the way towards methods and tools for advanced interactive systems , http://ceur-ws.org/Vol-2503/paper1_12.pdf . Google Scholar OpenURL Placeholder Text WorldCat Schipor , O.-A. and Vatavu , R.-D. ( 2018 ) Invisible, inaudible, and impalpable: users’ preferences and memory performance for digital content in thin air . IEEE Pervasive Comput. , 17 , 4 , 76 – 85. http://doi.org/10.1109/MPRV.2018.2873856 . Google Scholar OpenURL Placeholder Text WorldCat Ovidiu-Andrei Schipor , Radu-Daniel Vatavu, and Jean Vanderdonckt. ( 2019a ) Euphoria: a scalable, event-driven architecture for designing interactions across heterogeneous devices in smart environments . Inf. Softw. Technol. http://doi.org/10.1016/j.infsof.2019.01.006 . Schipor , O.-A. , Wu , W., Tsai , W.-T. and Vatavu , R.-D. ( 2017 ) Software architecture design for spatially-indexed media in smart environments . Advances in Electrical and Computer Engineering , 17 , 2 , 17 – 23. http://doi.org/10.4316/AECE.2017.02003 . Google Scholar OpenURL Placeholder Text WorldCat Schipor , O.-A. , Vatavu , R.-D. and Wenjun , W. ( 2019b ) Integrating Peripheral Interaction Into Augmented Reality Applications . Proc. of the 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) ( 2019 , 358 – 359 https://doi.org/10.1109/ISMAR-Adjunct.2019.00-12 . Google Scholar OpenURL Placeholder Text WorldCat Schipor , O.-A. , Vatavu , R.-D. and Wenjun , W. ( 2019c ) SAPIENS: towards software architecture to support peripheral interaction in smart environments . In Proc. of the ACM on human-computer interaction , p. 11 . EICS, http://doi.org/10.1145/3331153 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Sharma , K. K. , Sharma , H. and Ramani , A. K. ( 2009 ) mCAR: software framework architecture for in-vehicle pervasive multimedia services . Proc. of the int. multiConference of engineers and computer scientists , 1 . Google Scholar OpenURL Placeholder Text WorldCat Staron , M. ( 2017 ) Automotive Software Architectures: An Introduction . Springer. https://www.springer.com/gp/book/9783319586090 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Tilkov , S. and Vinoski , S. ( 2010 ) Node.js: Using javascript to build high-performance network programs . IEEE Internet Comput. , 14 , 80 – 83. http://doi.org/10.1109/MIC.2010.145 . Google Scholar OpenURL Placeholder Text WorldCat Traub , M. , Maier , A. and Barbehön , K. L. ( 2017 ) Future automotive architecture and the impact of IT trends . IEEE Softw. , 34 , 3 , 27 – 32. http://doi.org/10.1109/MS.2017.69 . Google Scholar OpenURL Placeholder Text WorldCat Yuan , B. , Chen , Y.-A. and Ye , S. ( 2018 ) A lightweight augmented reality system to see-through cars . In 2018 7th int. congress on advanced applied informatics (IIAI-AAI) , pp. 855 – 860 . IEEE. http://doi.org/10.1109/IIAI-AAI.2018.00174 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Zheng , B. , Liang , H., Zhu , Q., Yu , H. and Lin , C.-W. ( 2016 ) Next generation automotive architecture modeling and exploration for autonomous driving . In 2016 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) , pp. 53 – 58 . IEEE. http://doi.org/10.1109/ISVLSI.2016.126 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Markwalter , B. ( 2017 ) The path to driverless cars [cta insights] . IEEE Consumer Electronics Magazine , 6 , 125 – 126 . Google Scholar OpenURL Placeholder Text WorldCat Joshi , A. H. ( 2019 ) Ethernet communication design with service oriented architecture (SOA) . In PhD Thesis . Baraniuk , C. CES 2017: Car-makers choose virtual assistants , http://www.bbc.com/news/technology-38526807 Colquitt , J. , Dowsett , D., Gami , A., Equities , I.F., Jaysane-Darr , E., Partner , I.P.C., Manley , C., Equity , F., Shad , R., Income , I.F. ( 2017 ) Driverless cars: How innovation paves the road to investment opportunity . Gopu , G. , Kavitha , K. and Joy , J. ( 2016 ) Service oriented architecture based connectivity of automotive ecus . In 2016 int. conf. on circuit, power and computing technologies (ICCPCT) , pp. 1 – 4 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Häckel , T. ( 2018 ) A middleware solution for open and dynamic ICT architectures in future cars - hauptprojekt . Hammoudi , K. , Cabani , A., Melkemi , M., Benhabiles , H. and Windal , F. ( 2018 ) Towards a model of car parking assistance system using camera networks: Slot analysis and communication management . In 2018 IEEE 20th int. conf. on high performance computing and communications; IEEE 16th int. conf. on smart city; IEEE 4th int. conf. on data science and systems (HPCC/SmartCity/DSS) ,pp. 1248 – 1255 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Khanna , A. and Anand , R. ( 2016 ) Iot based smart parking system . In 2016 int. conf. on internet of things and applications (IOTA) ,pp. 266 – 270 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Liu , S. , Tang , J., Zhang , Z. and Gaudiot , J. L. ( 2017 ) Computer architectures for autonomous driving . Computer , 50 , 18 – 25 . Google Scholar OpenURL Placeholder Text WorldCat Lugano , G. ( 2017 ) Virtual assistants and self-driving cars . In 2017 15th int. conf. on ITS telecommunications (ITST) , pp. 1 – 5 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Mahendra , B. , Sonoli , S., Bhat , N., Raghu , T. et al. ( 2017 ) IoT based sensor enabled smart car parking for advanced driver assistance system . In 2017 2nd IEEE int. conf. on recent trends in electronics, information & communication technology (RTEICT) ,pp. 2188 – 2193 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Sadio , O. , Ngom , I. and Lishou , C. ( 2019 ) Design and prototyping of a software defined vehicular networking . IEEE Trans. Veh. Technol . Google Scholar OpenURL Placeholder Text WorldCat Saini , M. , Alam , K. M., Guo , H., Alelaiwi , A. and El Saddik , A. ( 2017 ) InCloud: a Cloud-based middleware for vehicular infotainment systems . Multimed. Tools Appl. , 76 , 11621 – 11649 . Google Scholar OpenURL Placeholder Text WorldCat Wang , W. , Song , Y., Zhang , J. and Deng , H. ( 2014 ) Automatic parking of vehicles: A review of literatures . Int. J. Automotive Technol. , 15 , 967 – 978 . Google Scholar OpenURL Placeholder Text WorldCat Yun , S. , Park , J. H. and Kim , W. T. ( 2017 ) Data-centric middleware based digital twin platform for dependable cyber-physical systems . In 2017 9th int. conf. on ubiquitous and future networks (ICUFN) , pp. 922 – 926 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Josifovska , K. , Yigitbas , E. and Engels , G. ( 2019 ) Reference framework for digital twins within cyber-physical systems . In 2019 IEEE/ACM 5th int. workshop on software engineering for smart cyber-physical systems (SEsCPS) , pp. 25 – 31 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Hellwig , A. D. , Kriebel , S., Kusmenko , E. and Rumpe , B. ( 2019 ) Component-based integration of interconnected vehicle architectures . In 2019 IEEE intelligent vehicles symposium (IV) ,pp. 153 – 158 . IEEE . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC OPEN Alliance’s Technical Committee 8 ( 2016 ) ECU and Network Test, Test Specification ECU. https://www.opensig.org/tech-committees/tc (accessed May 29, 2020) . AUTOSAR ( 2016 ) Testability Protocol and Service Primitives. https://www.autosar.org/fileadmin/user_upload/standards/tests/1-1/AUTOSAR_PRS_TestabilityProtocolAndServicePrimitives.pdf (accessed May 29, 2020) . Goumopoulos , C. , Kameas , A. and Hellas , P. ( 2009 ) Smart objects as components of ubicomp applications . Int. J. Multimed. Ubiquitous Eng. , 4 . http://www.sersc.org/journals/IJMUE/vol4_no3_2009/1.pdf . Google Scholar OpenURL Placeholder Text WorldCat Aiello , F. , Fortino , G., Gravina , R. and Guerrieri , A. ( 2011 ) A java-based agent platform for programming wireless sensor networks . Comput. J. , 54 , 439 – 454. http://doi.org/10.1093/comjnl/bxq019 . Google Scholar OpenURL Placeholder Text WorldCat Bellifemine , F. , Fortino , G., Giannantonio , R., Gravina , R., Guerrieri , A. and Sgroi , M. ( 2011 ) Spine: a domain-specific framework for rapid prototyping of wbsn applications . Softw. Pract. Exper. , 41 , 237 – 265. https://doi.org/10.1002/spe.998 . Google Scholar OpenURL Placeholder Text WorldCat Fortino , G. , Guerrieri , A., O’Hare , G. M. P. and Ruzzelli , A. ( 2012 ) A flexible building management framework based on wireless sensor and actuator networks . J. Netw. Comput. Appl. , 35 , 1934 – 1952. http://doi.org/10.1016/j.jnca.2012.07.016 . Google Scholar OpenURL Placeholder Text WorldCat Zahariadis , T. , Papadakis , A., Alvarez , F., Gonzalez , J., Lopez , F., Facca , F. and Al-Hazmi , Y. ( 2014 ) Fiware lab: managing resources and services in a cloud federation supporting future internet applications . In 2014 IEEE/ACM 7th int. conf. on utility and cloud computing (UCC) , pp. 792 – 799 . IEEE. https://doi.org/10.1109/UCC.2014.129 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Marquardt , N. , Diaz-Marino , R., Boring , S. and Greenberg , S. ( 2011 ) The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies, in: Proc. of the 24th annual ACM symposium on user interface software and technology, UIST ’11 , pp. 315 – 326 . ACM , New York, NY, USA. http://doi.acm.org/10.1145/2047196.2047238 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Crossref Fortino , G. , Parisi , D., Pirrone , V. and Di Fatta , G. ( 2014 ) Bodycloud: a saas approach for community body sensor networks . Fsuture Gener. Comput. Syst. , 35 , 62 – 79. http://doi.org/10.1016/j.future.2013.12.015 . Google Scholar OpenURL Placeholder Text WorldCat Crossref Nebeling , M. , Teunissen , E., Husmann , M. and Norrie , M. C. ( 2014 ) Xdkinect: Development framework for cross-device interaction using kinect . In Proc. of the 2014 ACM SIGCHI symposium on engineering interactive computing systems, EICS ’14 , pp. 65 – 74 . ACM , New York, NY, USA. http://doi.acm.org/10.1145/2607023.2607024 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Crossref Ledo , D. , Greenberg , S., Marquardt , N. and Boring , S. ( 2015 ) Proxemic-aware controls: Designing remote controls for ubiquitous computing ecologies . In Proc. of the 17th int. conf. on human-computer interaction with mobile devices and services, MobileHCI ’15 , pp. 187 – 198 . ACM , New York, NY, USA. http://doi.acm.org/10.1145/2785830.2785871 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Crossref Roda , C. , Rodríguez , A., Navarro , E., López-Jaquero , V. and González , P. ( 2016 ) Towards an architecture for a scalable and collaborative ami environment . In Trends in Practical Applications of Scalable Multi-Agent Systems, the PAAMS Collection , pp. 311 – 323 . Springer. https://doi.org/10.1007/978-3-319-40159-1_26 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Crossref Mocanu , I. and Schipor , O. A. ( 2017 ) A serious game for improving elderly mobility based on user emotional state . In the int. scientific conf. eLearning and software for education, Vol. 2, ”Carol I” National Defence University , pp. 487 . https://doi.org/10.12753/2066-026x-17-154 . Lou , Y. , Wu , W., Vatavu , R.-D. and Tsai , W.-T. ( 2017 ) Personalized gesture interactions for cyber-physical smart-home environments . Sci. China Inf. Sci. , 60 , 072104. https://doi.org/10.1007/s11432-015-1014-7 . Google Scholar OpenURL Placeholder Text WorldCat Vatavu , R.-D. , Chera , C.-M. and Tsai , W.-T. ( 2012 ) Gesture profile for web services: an event-driven architecture to support gestural interfaces for smart environments . In int. joint conf. on ambient intelligence , pp. 161 – 176 . Springer. https://doi.org/10.1007/978-3-642-34898-3_11 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC © The Author(s) 2021. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Empirical Results for High-definition Video and Augmented Reality Content Delivery in Hyper-connected Cars
JO - Interacting with Computers
DO - 10.1093/iwcomp/iwaa025
DA - 2021-01-05
UR - https://www.deepdyve.com/lp/oxford-university-press/empirical-results-for-high-definition-video-and-augmented-reality-IX070JPvXf
SP - 1
EP - 1
VL - Advance Article
IS - 
DP - DeepDyve
ER -