Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Communication Within Multi-FSM Based Robotic Systems

Communication Within Multi-FSM Based Robotic Systems The paper presents a robotic system design methodology based on the concept of an embodied agent decomposed into communicating subsystems, whose activities are specified in terms of FSMs invoking behaviours parameterised by transition functions and terminal conditions. In the implementation phase, this specification is transformed into a system composed of a whiteboard providing communication means and logically labelled FSMs (LLFSMs) defining the system behaviour. These concepts are used to generate the code of the robot controller. The inclusion of inter-subsystem communication model completes the resulting system design with respect to our previous work that did not account for this model. Thus communication plays a central role in this presentation. The design methodology is exemplified with a rudimentary table tennis ball-collecting robot. The presented methodology and the implementation tools are suitable and beneficial for application to the design of other robotic systems. Keywords Robotic system specification · Robotic system design methodology · Communication model 1 Introduction whiteboard for communication. Especially the decoupling achieved by the method of communication between the Good practises of software engineering require that the components of a robotic system is of interest here. We creation of systems involves a specification phase that start with a motivation for the presented research and the defines the model of what has to be created, and an structure of the paper. implementation phase that discloses how this is done [84]. This paper uses the specification method based 1.1 Motivation on the concept of an embodied agent [54, 92]. The novelty is twofold. Our first contribution is the method The design of a robotic system starts with the specification of implementation of robotics systems using the concepts of its architecture. As Andrew Tanenbaum points out of logic-labelled finite-state machines (LLFSMs) and in his seminal work [86], the term architecture pertains whiteboards. Secondly, we show the simplicity of the to: the instruction set, memory organisation, I/O, and transformation of the embodied agent based specification bus structure. This obviously refers to the architecture into an (executable) implementation using LLFSMs and the of computer systems. However current robotic systems are computer based, thus their architecture has the same foundation. The instruction set defines what the processing Cezary Zielinski ´ capabilities of the system, i.e. possible activities, are. c.zielinski@ia.pw.edu.pl The remaining elements refer to the structure, i.e. the Maksym Figat components and their interconnections [23, 55]. In the m.figat@ia.pw.edu.pl approach that we follow in our work, the structure refers ReneH ´ exel to the division of the system into subsystems and their r.hexel@griffith.edu.au interconnections, whilst the activities are specified by a set of such concepts as: Finite State Machines (FSMs), Institute of Control and Computation Engineering, behaviours, transition functions and their arguments, and Warsaw University of Technology, Nowowiejska 15/19, 00-665 Warsaw, Poland inter subsystem communication means, all of which 2 will be detailed later in the paper. However, in many School of ICT, Griffith University, 170 Kessels Road, Nathan, QLD 4111, Australia existing robotic systems, it is difficult to determine their 788 J Intell Robot Syst (2019) 93:787–805 architecture [55]. Robot control systems are inherently is kept concise to enable the reader to focus on the pre- complex, nonetheless their authors rarely formally specify sented methods rather than being forced to deal with the them. Lack of a formal system specification usually complexity of the robotic system. The last section presents results in an obscure architecture that is difficult to the conclusions derived from the presented work. modify or extend it in future projects or to integrate with other systems. The process of developing robotic systems requires both appropriate development methods 2 Communication (procedures, architectures, etc.) and tools (modelling languages, middleware and other frameworks). Many In telecommunications, the concept of the communication attempts have been made to introduce formal languages channel is central to the operation of a communication based on mathematics to determine both the controller system [44, 83], distinguishing between free propagation structure and its activities [18, 63, 90, 92, 96]. However, (broadcast, one-to-many) and guided propagation (point-to- the majority of the robot systems were developed based on point, one-to-one) communication types. Telecommunica- software engineering [16, 17, 68], downplaying the domain- tion deals with such characteristics of the communication specific information required by robotics. The proposed channel as its linearity or non-linearity, time variability or method of modelling and specifying robotic systems, based invariance, bandwidth, or power limitation. However those on the concept of the embodied agent [54, 92, 94], relies physical properties are of secondary importance and can on such fundamental concepts of robotics as: effectors, be abstracted away for communication of software imple- i.e. devices influencing the physical environment, and mented subsystems of robots. In the most rudimentary view, receptors, i.e. sensors gathering the information about the communication model consists of the communicating the state of that environment, as well as mathematical systems/subsystems and the communication channel. The concepts underlying decision making and sub-system communication channel (the transmission medium) can be behaviour. However, up till now this design methodology either based on technological means (e.g. wired or wireless) lacked a detailed representation of a vital component: or use natural means (e.g. voice communication using air) inter-subsystem communication. Hence, the present paper [44, 83]. In the latter case the environment itself can be used delves into that issue. The main purpose of this paper as a communication channel [4], then stigmergy results [12]. is therefore to present the system design methodology Both technological and natural communication channels can split into two phases, as required by good practises in contain buffers, which can store the transmitted information software engineering. The first phase produces the system for some time, or exist without a buffer. Transmission delays model specification based on the concept of an embodied should not be treated as a capacity of the channel to store agent, whilst the second one is the implementation phase, data. For communication to take place not only the transmis- which first transforms the model into the composition of sion medium is necessary, but also a common protocol has a whiteboard and LLFSMs as an interim implementation to exist, as the transmitter and the receiver of the informa- model that is subsequently transformed into executable tion must have a common understanding of the transmitted robotic system controller code. information. In robotics, diverse communication methods have 1.2 Structure of the Paper been considered. Three different types of communication between robots were discussed in [25]: two-way commu- In Section 2 the classification of communication methods nication, explicit one-way communication, and completely from the robotics perspective is introduced. Section 3 briefly implicit communication. This work also points out that there presents the concept of an embodied agent, showing both are several difficulties that should be considered in com- its structure and its activities. The embodied agent is the munication between collective robots, i.e. efficiency, fault primary concept of the robotic system design method used tolerance and cost, and that in contrast to centralised com- in this paper. The universality of an embodied agent is munication approaches, e.g. [88], communication between expounded in Section 4. Section 5 describes the interim robots should be distributed. Comparison of three types of implementation model, i.e. the elements necessary for the communication, i.e. through the environment, using trans- implementation of an embodied agent, focusing on the mission of state between the agents, and utilising the trans- communication aspect. mission of the detected goal, was introduced in [8]. Similar As ROS currently is the most popular robot programming criteria were discussed in [65], distinguishing direct (purely framework, the comparison of whiteboard and ROS com- communicative act) and indirect communication (based on munication mechanisms is presented in Section 6. Section 7 observed behaviour of other agents). Direct communica- showcases the proposed design and implementation meth- tion was further decomposed into one-to-one (peer-to-peer) ods on a robot collecting table-tennis balls. This example and one-to-many (broadcasting), based on the the number J Intell Robot Syst (2019) 93:787–805 789 of communicating entities. Another division of commu- on the number of message producers and the number of nication is direct vs indirect communication [34]. Direct consumers: one-to-one (peer-to-peer), one-to-many (e.g. communication occurs when a dedicated on-board hardware broadcast), many-to-one and many-to-many (e.g. multicast). device is utilised, whilst indirect communication occurs An example of one-to-one communication is presented in through the environment. The paper also notices that indi- [66] where a system composed of many robots executes a rect communication is particularly useful from the point of box-pushing task. Cases of one-to-many communication are view of Multirobot Systems (MRS), in contrast to direct presented in [7, 45, 65, 74, 76], where the consumers either communication, which in MRS may result in much more utilise broadcast messages at their will or ignore them. The expensive and unreliable solution. However, that paper also many-to-many communication can be organised as e.g. a points out that direct communication in Multi-Agent Sys- blackboard system [41]orbystigmergy[12]. tems (MAS) may be used to guarantee locality of inter- Another criterion worth considering is based on the actions and to avoid synchronisation procedures amongst location of the communication channel: inter-system or agents. A similar criterion based on how the information intra-system. Here the terms intra-system and inter-system may be obtained by robots was introduced in [75], where need to be disambiguated. We assume that in the case three types of communication were distinguished: implicit of intra-system communication, the transmission occurs communication through the world, passive action recog- between nodes existing on a single computer (composed nition (robots use sensors to directly observe the actions of a single or multiple processors), and in the case of of other robots) and explicit communication (robots com- inter-system communication transmission is between nodes municate directly and intentionally through an artificial residing on many computers. Thus in this case the word communication channel). Either way, care must be taken system does not refer to the robotic system as a whole, to avoid the hidden channel problem that results when but only to the communicating nodes that are within the differing temporal precedence between these means of com- robotic system, located either on a single computer or munication creates inconsistencies between the intent of the on several computers forming a network. The importance system designer and the actual timing and order in which of this distinction is due to each of these types of information is delivered [21]. communication requiring different software and hardware Criteria pertaining to the communication range, commu- means of implementation. nication topology and communication bandwidth were dis- Still another criterium pertains to whether the producer cussedin[24, 26]. However, those criteria concern mainly or consumer block until message delivery has completed. physical aspects of communication, already discussed in Blocking producer behaviour requires the consumer to [44, 85]. Yet other criteria pertain to interaction distance respond in order to unblock the producer, whilst in the (distance between the agents during the communication, i.e. non-blocking mode this is not required, i.e. after initiating direct physical contact, visual range, hearing range and long the communication, the producer can continue with other range), interaction simultaneity (period between the signal activities. Similarly the consumer can either be blocked emission and reception, i.e. immediate, long time), sig- whilst waiting for the message or not. In the latter case, nalling explicitness (explicitness of the emitter’s signalling the consumer can poll the communication channel when behaviour) and sophistication of interpretation (the com- required to find out whether there is a message for it to plexity of the interpretation process that gives meaning to attend to. Still another criterion is storage, i.e. whether the the signal) [48]. communication channel contains a buffer or not. In the The above analysis shows that whilst similar terminology latter case the communication channel does not have the is used for the purpose of communication classification, the capability to store messages. In the former case the size and meaning attributed to the same names is slightly different, type of the buffer is of relevance. The commonly used buffer making it difficult to propose a comprehensive communi- types in robotics are either queues or cyclic buffers. cation model. Some of the subdivision criteria are based on Moreover, it should be noted that communication physical aspects of communication (topology, place, types is sometimes associated with cooperation. However the of communication channels, bandwidth, etc.), neglecting former refers to the ability to transmit information between other also very important criteria (e.g. whether the commu- systems and the latter with the ability to jointly execute a nicating entities block each other whilst sending/receiving task. Cooperation can be achieved both by using implicit data). Thus, to make the presentation clear, we summarise and explicit communication. The former refers to a situation the criteria most relevant to the subject of this paper as where the consumer of the message is not clearly defined, follows. From the point of view of robot control system and thus the interpretation of the message by a consumer design, of importance is the possible number of subsys- may be difficult (as in the case of stigmergy) and in the latter case the consumer is clearly defined, and thus the tems writing to or reading from the communication channel. This leads to four communication types [39], depending interpretation of the message using the commonly known 790 J Intell Robot Syst (2019) 93:787–805 protocol is straightforward (as is the case with the majority of technologically based channels or such natural channels as air used for voice communication). In robotics, there is a vast body of literature dealing with cooperation using implicit communication, especially stigmergy. For example [4] presents a schema based control of a multi-robot system used for common object retrieval, [81] describes robots communicating using sensors for detecting infrared radiation reflected by the environment, [87] presents robots communicating using traction forces, [25] describes robots communicating indirectly using on-board visual sensing, whilst [20] reports on the system composed of 20 e-puck robots pushing three different objects based on inputs from infrared proximity sensors, [89] describes a multi-robot system where individual robots use an adaptive behaviour selection strategy based on sensoric information about the location of the transferred object and the other robots, [95] presents multi-robot box pushing based on the detection of the motions of the translocated box. On the one hand the examples of implicit communication are abundant in the literature and on the other hand this form of communication does not influence the system specification from the point of view of communication, as it is realised by the perception subsystem. Thus this paper will limit itself to explicit forms Fig. 1 Internal structure of an agent a of communication. if true, cause a state transition from the current to the state pointed at by the arc labelled by this predicate. With 3 Embodied Agent each state of an FSM F there is an associated subFSM j,v s B s F , specifying the behaviour B ,where ω names j,v,ω j,v,ω A single- or multi-robot system (a robotic system, in short) the behaviour. The subFSM template modelling a behaviour is represented by communicating agents [14, 46]. An agent B is presented in Fig. 2. Each iteration of a behaviour j,v,ω which has a physical body is called an embodied agent [14, consists of calculation of a transition function and two- 54, 94, 96] – its general structure is presented in Fig. 1. way transfer of data. The transition function f takes j,v,ω An embodied agent a (j is its designator) is decomposed as arguments the current data from the subsystem s internal into 5 types of subsystems: control subsystem c , virtual s i i memory S and its input buffers S , and calculates j,v j,v receptors r , virtual effectors e , real receptors R and j,k j,n j,l i+1 the values which are inserted into the output buffers S real effectors E ,where k, n, l and m are the designators j,m j,v s i+1 of particular subsystems of the agent a [94]. and the internal memory s . Superscripts i and i + 1 j,v Virtual receptors r aggregate data acquired from the j,k indicate the elapse of a single tick of the discrete time, environment through real receptors R . Virtual effectors j,l e transform control commands received from the control j,n subsystem c into a form acceptable by the real effectors E . An embodied agent contains a single control j,m subsystem and zero or more subsystems of each of the other types. The activity of a system as a whole depends on the individual activities of its subsystems and their interactions. The activity of each subsystem s ∈{c, e, r} is represented by a hierarchical FSM F [35], where v indicates the j,v subsystem name. An FSM is represented as a directed graph composed of nodes and directed arcs. Each node represents a state, whilst each arc represents a state transition. Arcs s B s Fig. 2 FSM F specifying the behaviour B are labelled by initial conditions, being predicates, which, j,v,ω j,v,ω J Intell Robot Syst (2019) 93:787–805 791 which determines the duration of the behaviour iteration. environment, invoked action planning, and then executed The behaviour subsequently sends the results of calculations the thus derived plan. Later, different criterions have been i+1 from the output buffers S to the inputs of the associated used for the decomposition into layers. Usually, the time j,v subsystems and acquires from them new data loading it of the execution of a single behaviour iteration of a com- into input buffers xS . Based on the obtained data an ponent within the layer is the discriminating factor. The j,v s ε associated error condition f and terminal condition higher the layer within the system, the longer the time of its j,v,β s τ f are tested (β and ξ are their designators). The single iteration execution. In some cases, task abstraction j,v,ξ duration of a single iteration of the subsystem behaviour is the criterion, i.e. a single activity at an upper layer is defines its sampling period. The iterations of the behaviour decomposed into several lower layer activities, e.g. NAS- terminate when the associated error or terminal condition REM (NASA Standard Reference Model) [60]. Usually, as is fulfilled. At this moment the main subsystem FSM F a consequence, the utilised ontology becomes more abstract j,v resumes its activity, changing its state. going up the layers. The lowest layer is behavioural and is tightly coupled with actuators and sensors. The mid- dle layer is the sequencing layer (sometimes termed as 4 Universality of the Embodied Agent executive layer [56]), responsible for selecting behaviours that will eventually realise the task. The uppermost tier The structure of the embodied agent and its ability to usually involves deliberation. In robotics, deliberation is connect to other such agents, thus producing networks, associated with planning. From what has been said, it is enables the reproduction of other well known and studied evident that planning and sequencing layers act together architectures. Hence, the embodied agent can be treated to accomplish the plan and in some systems are tightly as a general means for designing robotics systems. The bound together (e.g. CRAM [9], CLARAty [68]), hence 2T most popular architectures employed in robotics are: Sense- instead of 3T architecture results. The proposed embodied Plan-Act (SPA), multi-tier (usually either two- (2T) or agent based architecture can accommodate all of these three-tier (3T) are used), subsumption, and the ones based possibilities. By design, an embodied agent forms a two on some forms of schemata. The universality of the tier architecture: virtual entities controlling the real devices concept of an embodied agent will be briefly justified form the first tier and the control subsystem forms the here by showing its relationship to those architectures. second one. Above that, a hierarchy of computational Robotic system software reflects its architecture. Although agents (embodied agents lacking effectors and recep- some work has been done on formal specification of tors) can form supplementary tiers. A computationally robot control software, e.g. [6, 62, 63, 91, 93] and its demanding planning task can be decomposed into several formal verification, e.g. [49, 50], unfortunately this has parts, implemented as several interacting computational not gained widespread adoption [56]. Usually, architectures agents. are presented by informal text descriptions supplemented As planning is a time-consuming task, the subsumption by block diagrams, with varying level of details. Early architecture was proposed [13–15]. Its overall behaviour is approaches to formalisation of the specification of a robotic the result of the interaction of the activities of modules, system treated such systems as composed of computational which react to input from other modules and sensoric modules and followed operational semantics in describing stimulus. If lower layer reactions can not cope with their structure and activities [62, 63]. One of the few systems the task, upper layer reactions suppress or inhibit them, that employs formal specification and generation of code taking over control, thus subsuming their actions. As the for the implementation of the lowest layer of robotic system SPA architecture tends to be prohibitively slow and the en oM (Generator of Modules) [1, 36]. It is controller is G subsumption architecture, although very reactive, lacks integrated with the BIP (Behaviour Interaction Priorities) foresight, hybrid architectures emerged [5]. Those were [10]framework andtoolset,whichisusedtoformally usually designed as three tier systems (e.g. [1, 38]), specify and verify the correctness of the produced system, having: planning (deliberative layer), sequencing (usually treated as a real-time, component-based complex structure. implemented as a finite state machine governing the actions However, as the majority of architectures presented in the of the system), and device control layers (responsible for literature is not specified formally this discussion will not motor control and sensor data aggregation – a reactive layer rely on a formal description. resulting if both were combined). Behavioural systems [5, Frequently, hierarchical layered architectures are 6], based on reactivity, have gained widespread attention. designed. One of the oldest hierarchical architectures is In order to satisfy the requirement of the fastest possible the sense-plan-act (SPA) structure, used for instance by response, reactivity should be implemented at the lowest the Shakey robot in the 1960s [71]. Shakey sensed its level. The shortest possible loop starts with proprioceptors 792 J Intell Robot Syst (2019) 93:787–805 and directly influences the real effector. As the activities of decomposed into concurrently computed functions that the reactive layer rely mainly on sensor input and to a very define the computations of individual motor schemas. The limited degree on internal state (i.e. memory) it is very easy transition function composes the results produced by partial to reproduce this layer within an embodied agent, where the transition function into an outcome that is dispatched to quickest responses are organised within the virtual effector the virtual effector. A more formalised approach to Schema and are based on proprioceptive input. An example of this Theory is called Robot Schemas (RS) [63]. RS form a approach is the utilisation of force/torque sensing directly formal model of distributed computing in the robotics within the virtual effector (e.g. [97]). If a subsumption domain. The model is built of concurrent computing agents, architecture [13, 15] is to be reproduced, its modules can each being a Schema Instance (SI). SIs communicate be represented by blocks within the data flow diagram through ports, with which data types are associated. At definition of a transition function by using inhibiting and SI instantiation time, its ports are connected to the ports suppressing links (as presented in [54]). Otherwise, the of other SIs. Communication occurs when an SI writes modules can deliver the results of their computations into to one of its output ports. Basic schemas are defined by a composition unit, which will compute the final outcome, the port connections, local variables and the procedural subsequently dispatched to a real effector for execution. If behaviour section, which specifies how the schema reacts a slower reaction is permissible, exteroceptive input from to input. Basic schemas, once initiated, iteratively execute the virtual receptors can be used by the control subsystem, the behaviour section, synchronously reading or writing which will use the same scheme of operation as the to ports, performing computations on local variables and virtual effector. Thus, the same pattern will be reproduced input ports and instantiating and deinstantiating other at different levels of the control hierarchy. This can be SIs. Complex schemas, i.e. assemblages, are a recursive continued at even higher levels of the hierarchy by using composition of basic schemas into ever more complex extra computational agents [98]. ones. Assemblages do not have the behaviour definition Schema Theory was formulated for the purpose of part, as that is defined by the schemas they are composed representing systems created to study both artificial of. Only the port connection part exists. SIs form a intelligence and brain theory [2, 3, 5, 62, 63]. Schema dynamical network which grows or shrinks as the activities Theory provides guidance to building models of animal of the system take place. The task is represented as and artificial agent behaviour, stating that behaviour is the the outermost assemblage. Thus, the system is composed result of competition and cooperation between schemata, of sensory schemas, motor schemas and schemas that which are conglomerates of knowledge and methods of connect them, i.e. task schemas. In general, this reflects the processing [2]. Instances of schemas are created either structure of an embodied agent or can be reproduced as by other schemas or as a reaction to sensory stimulus. a network of embodied agents with some of their internal Perceptual schemas process sensory input, whilst motor components missing. It should be noted that the design of schemas produce action patterns. A network of schemas an RS-based system focuses on computation, however this is called an assemblage. The overall behaviour of an involves also instantiation and deinstantiation as well as agent is produced as a result of interactions between communication, all considered at the same computational schemas composing the network. Such schemas can be level. Thus, the basic level is flat and all aspects of embedded in an embodied agent, as virtual receptors reflect system activities are entangled, hence intelligibility is perceptual schemas, whilst virtual effectors reflect motor sacrificed. Hierarchy emerges from assembling lower level schemas. At a higher level, embodied agents without schemas into higher level ones, but as the design principles effectors match perceptual schemas, whilst those without of the lowest level are not expressed explicitly the receptors match motor schemas, this multi-agent networks creation of such systems is difficult. An embodied agent can also reproduce assemblages. The application of Schema introduces a hierarchy of concepts starting from the lowest Theory in robotics followed different paths. One of them level: transition function and terminal condition (purely is described in [3, 5]. In this case, motor schemas are computational), behaviour (adds communication), FSM basic units of behaviour. Motor schemas act in conjunction (selects behaviours), subsystems (distinguish perception, with embedded perceptual schemas. Concurrently acting action and task control), agents (form networks of any size). behaviours, i.e. motor schemas, compute individual results Each of those concepts can be dealt with separately, thus that are composed together into the final control outcome. it is easier for the designer to focus on a single entity at a This architecture is easily reproduced in an embodied agent. time, simplifying the process of system creation. Systems Perceptual schemas are reproduced by virtual sensors. composed of embodied agents, besides being capable of The control subsystem implements a behaviour that is reproducing the well known architectures, can be used to parameterised by a transition function which in turn is create other architectural patterns [94, 98]. J Intell Robot Syst (2019) 93:787–805 793 5 Implementation of an Embodied Agent with whisker sensors. Autonomous mobile robots dealing with hazardous material spill emergency situation used a The architecture of an embodied agent presented in blackboard system to integrate information from various Section 3 is implemented using several earlier-mentioned sensor-based and knowledge-based subsystems [73]. Those concepts. Here we shall concentrate on two important examples and the plethora of other works on the utilisation ones associated with the communication and behaviour of blackboards in robotics show that the effort has selection [31]: the whiteboard, which is used as the main concentrated on either different aspects of knowledge communication means between subsystems, and Logic- processing or on providing a balanced access to knowledge Labelled Finite State Machines (LLFSMs), which govern to all involved actors or on both. The concept of the actions of each of the subsystems present in the agent, a whiteboard is derived from blackboards, however it including the exchange of information between subsystems. focuses on information sharing, thus the communication The whiteboard is our implementation of the general aspect. The contents of the whiteboard is of secondary concept of a blackboard [42], thus whenever we refer here importance as is the purpose of information processing. to the former we refer to our implementation and for the The focus is on providing efficient communication means. general concept, the latter is referred to. A similar communication concept was introduced in The concept of a blackboard was created for the [1], namely a poster. It uses shared memory for the purpose of studying artificial intelligence (AI) [69]. The purpose of communication. A poster is a structured shared blackboard model consists of three major components [22, memory only writable by its owner and readable by 43, 70, 82]: knowledge sources (KSs) – corresponding any element of the architecture. Posters are completely to the domain experts, blackboard data structure – the independent from each other. They are not regions of a global resource where information to be shared is written, common blackboard and cannot be written to by many and control mechanism – responsible for processing the producers, thus they do not have all the properties of knowledge contained in the blackboard and ensuring that a blackboard. A whiteboard has all the communication only one domain expert has access to the blackboard abilities of a blackboard, so it is more general than at a given time. The blackboard can be any data a poster. structure accessible by all the KSs. Each KS operates in the following sequence: reads the information from 5.1 Whiteboard the blackboard, acts based on the information found and writes its conclusions and hypothesis into the blackboard. The gusimplewhiteboard (Griffith University Simple Whiteboard) is a library for organising inter-process The work on blackboards in AI concentrated on several aspects: the organisation of knowledge data structures, communication [31]. It implements a mutation of the reasoning utilising those data structures as well as providing general blackboard concept [42]. It enables the transfer of access to them. The concept of the whiteboard focuses C++11 objects through shared memory. As it is linked in, on data access. The use of data-centric communication no broker is necessary. The message transfer is idempotent, similar to blackboards has transcended AI and found its i.e., if the transfer is repeated the result does not change. No way into the wider software engineering community. The queues need to be involved, however, if necessary, they can Data Distribution Service (DDS) is a standard formulated be organised, as the whiteboard does not determine the data by the Object Management Group (OMG), implementing structures it will be storing – this is the choice of the system Data-Centric Publish/Subscribe (DCPS) communication designer using the whiteboard. model [40, 58]. The main purpose of DDS is to Shared memory communication exhibits certain advan- tages over the traditional approach taken by, e.g. ROS assure dependable communication in dynamic, networked environments between distributed producers and consumer messages and services, [47]. Producers and consumers use fulfilling real-time constraints. Whiteboard implements local instances of classes to communicate. When a producer the pull approach, whilst the DDS, which is based on sends a message, it copies an object containing the mes- Publish/Subscribe model, is based on the push approach. sage from its local store to the static instance of the class Blackboards have also been used in robotics. The in the gusimplewhiteboard. When a consumer wants following examples show only a fraction of the diversity to retrieve the message, the current instance of the object of their applications. A reconfigurable mobile robot used in the gusimplewhiteboard is transferred to the local a blackboard enabling a horizontal system decomposition memory of the consumer. When operating in a distributed allowing the cooperation between all the subsystems system, a UDP-based sharing algorithm that also makes use considered as knowledge sources executing a specific task of the Time-Triggered Architecture (TTA) [52]isusedto [72]. The CrunchBot [37] used a Bayesian blackboard to broadcast data to other receivers [61]. The implementation solve the mapping problem for a mobile robot equipped of the distributed version of the whiteboard uses efficient, 794 J Intell Robot Syst (2019) 93:787–805 fault-tolerant communication for distributed systems as pre- utilising the concept of a blackboard still relies on the sented in [11, 51, 57][52, 53]. Compared to the DCPS [58], push model. For instance, ROS topics act in push mode. the whiteboard does not offer inherent mechanisms for dis- However the blackboard provides an opportunity to base tributed reconfiguration, but offers a statically defined data the communication on the pull model. Whilst the proposed classes and O(1), fail-silent communication within bounded whiteboard can be used in both push and pull modes, the real-time constraints [31]. A more detailed discussion of the latter should be the method of choice. In the majority of networked version of the whiteboard [61]ispresentedin middleware, the structure of the contents of the messages Section 6. transferred is limited, whilst in gusimplewhiteboard any C++11 objects can be used [31]. 5.2 Communication Model 5.3 LLFSM The blackboard can be considered as a broker, which enables the producer to depose in it, in a non-blocking way, LLFSMs are FSMs, in which transitions from one state to whatever data it has to send out, whilst the consumer may another state are based on the fulfilment of logic expressions act in one of the following modes. Either it acts in the rather than events. As events are usually queued and logic push mode (similar to the DCPS), where the blackboard expressions are computed just on the basis of current invokes a specific function of the consumer-subscriber, thus data, as is more appropriate for robot control, this form notifying it of the arrival of new data, or in pull mode, of an FSM is preferred. In the implementation presented where the consumer accesses the data when it requires. The here, the LLFSMs, in principle, operate concurrently, former usually involves queuing of messages, whilst the but in reality are scheduled sequentially, thus critical latter implies the delivery of only the most current data. The sections and synchronisation points become unnecessary. push model implies close coupling of modules, where the This inherently removes the possibility of race conditions. subscriber must react to all messages issued by the producer The communication between LLFSMs uses whiteboards. [30]. In the pull model, the components are decoupled, the The utilised LLFSMs are synchronous automatons. With consumer may react in its own time, and only to some every state, three portions of code are associated: the of the produced messages. In both communication models, code that is executed on entering the state, the exit code the publisher introduces a message into the blackboard. invoked when the logic expression with one of the outgoing In the case of the push model the blackboard notifies transitions (directed arcs) is satisfied, and the code that is the consumer that the message it has subscribed to is executed when none of those logic labels is true, hence the available and the subscriber invokes a call back function execution of this code may be repeated many times. in response. The messages are queued, so the consumer The LLFSMs can be composed hierarchically. Each should somehow react to all of them. An event driven LLFSM can be suspended, thus enabling the execution architecture results. It relies on an assumption that there is of a subLLFSM. It can also be restarted from the state enough of computational power to handle the messages as it was suspended in or restarted from its initial state . they come. However, in the case of the pull model, such an The subLLFSMs act like subprograms and need not act assumption is not necessary. The consumer is not notified simultaneously – this prevents state replication [19]. The about the arrival of a new message. When the consumer higher level LLFSM becomes dormant during the actions needs the new message it retrieves form the blackboard the of the subLLFSMs, increasing the degree of determinism of most current one, neglecting the obsolete ones (in reality the system performance. usually they will even not be available). The publisher and From what has been said here, it is evident that the s B implementation of both the subsystem FSM F and subscriber are very loosely coupled, without assumptions as j,v s B to the available computational power, and thus the ability to the behaviour subFSM F (specifying the behaviour j,v,ω respond timely to the incoming stream of messages results. F ), as LLFSMs is straightforward, as the directed j,v,ω Obviously, the pull model works on the principle that we get arcs of those FSMs are labelled by predicates, i.e. logic a better response, if the system at any moment uses the most expressions, hence their equivalence is ascertained. current data. Both the publisher and the subscriber act with their own frequencies, without having to synchronise their 5.4 Scheduler activities. This way, no global timing schedule is required, fully decoupling the development of both components. The An embodied agent is composed of one control subsystem pull model is immune to messages being lost by definition, and usually several virtual effectors and receptors. Each of as it does not require response to every message, whilst the push model requires delivery guarantees, because every Incidentally, this property also enables the implementation of the message attracts attention. The majority of middleware subsumption architecture [13] by using those automatons [19]. J Intell Robot Syst (2019) 93:787–805 795 those subsystems is governed by one FSM – here imple- condition of each behaviour is always fulfilled, whilst mented as an LLFSM. The proposed solution schedules error condition is not. This implies that all behaviours sequentially the LLFSMs representing producers and con- are executed only once in each state of the FSM. The sumers, thus minimising the problems of determining the functioning of the scheduler is presented assuming that s 1 s delays in servicing the transfer of data. The majority of mid- initially FSM F is in the state S and the FSM F is j,v j,h j,v dleware, e.g. ROS, Microsoft Robotics Studio, relies on con- in the state S . The scheduler activates one state of each j,h currency, data queuing and the non-deterministic TCP/IP FSM or subFSM at a time, switching between all FSMs suite. Hence real-time performance is compromised. Deter- and subFSMs. Obviously this causes the execution of the ministic scheduling has the following advantages over the code associated with that state. In this case the scheduler 1 s 1 non-deterministic multi-threaded approach. Full concur- starts with state S of F and then switches to S of j,v j,v j,h rency, where switching between threads is operating system s 1 s 1 F .Being in S F transits to S of the subFSM j,h j,v j,v j,v,1 dependent, is much more difficult to envision and correctly executing the behaviour B , so this will be the next state j,v,1 apply by the robot control system designer, as it necessi- activated by the scheduler. The fourth state activated will be tates: thread synchronisation, vigilance of nondeterministic 1 s S of the subFSM F . The full switching sequence j,h,1 j,h,1 communication delays, managing critical sections, prob- s s for both FSMs F and F and their subFSMs executing j,v j,h lems with fairness as well as avoiding deadlocks, live-locks, behaviours is presented in Fig. 4. and starvation. Moreover, CPU context switch overhead has Whiteboard communication in conjunction with this to be taken into account. The sequential LLSFSM switching way of switching the LLFSMs by the scheduler avoids procedure enables fairly simple formal correctness veri- deadlocks, because only one FSM accesses the whiteboard fication of models, as it avoids the usual combinatorial at a time. Moreover, because the variables contained in the explosion of concurrent threads. The sequential switching whiteboard are located in cyclic buffers, always there is is time-triggered in contrast to the frequently used event- available data that can be accessed, alas not always the most triggered approach, where due to peak-load situations it is current. often impossible to assure reliability. Specifications using LLFSMs constitute executable models that can be formally 5.5 Communication Organisation verified. A more detailed treatment of this subject can be found in [29, 32, 33]. As in many cases, the example that follows requires only Each iteration of an LLFSM operation starts with reading intra-system communication between subsystems. Whilst the relevant variables from the whiteboard and it ends with the utilised whiteboard is capable of providing inter-system depositing the results of computations in it. Each iteration communication [61], this is not our focus here. It should is treated as atomic, thus no inconsistencies can arise. The be noted that the whiteboard intra-system communication LLFSMs are executed in a round-robin fashion one iteration gives real-time guaranties (when the underlying operating of each LLFSM at a time. system is real-time), whereas the inter-system one would To show how the scheduler works in conjunction with the depend on the network protocol. LLFSMs, a simple example is presented here. It is assumed The whiteboard is a table with slots. Each slot creates a that within the robotic system, only two subsystems v separate communication channel containing a cyclic buffer. and h exist, where v and h are the specific names of the Hence, buffered communication based on a cyclic buffer of subsystems. The exemplary activities of each subsystem limited size is realised. The whiteboard as a whole supports s s may be represented by hierarchical FSMs: F and F , j,v j,h concurrent many-to-many communication, but a message presented in Fig. 3. slot (buffer) with multiple producers requires locking and With each FSM state a behaviour is associated. It is is therefore not recommended. We therefore recommend represented as a subFSM based on behaviour template from one-to-one or one-to-many communication, which is Fig. 2. For simplicity it was assumed that the terminal s s Fig. 4 Sequence of state activations of FSMs F and F and their j,v j,h Fig. 3 The activities of two subsystems v and h of a j subFSMs executing particular behaviours 796 J Intell Robot Syst (2019) 93:787–805 lock-free with O(1) complexity for access operations. In necessarily convenient for the subscriber. In congested sys- the following example one-to-one communication will be tems using TCP, if the acknowledgments do not reach the utilised. The LLFSMs together with the whiteboard enable publisher on time, the packets will be retransmitted, thus the the implementation of both blocking and non-blocking subscriber will eventually be flooded with repeated packets communication modes both in the case of the producer and [77]. This means that the subscriber activity is tightly cou- the consumer. However, in the presented example both the pled with the publisher update rate. Additionally, the push producers and the consumers operate in the non-blocking approach assumes that the subscriber reacts to transmitted mode. In the experiment, a single sequential scheduler is data immediately (otherwise data is queued), thus the data is used, thus subsystem LLFSMs are executed sequentially. presumed to be fresh and as recent as possible, what might not be the case. This will result in reaction delays. The neg- ative influence of such delays on the performance of a drone 6 Comparison of Whiteboard and ROS are presented in [47]. Timeliness, sequencing and reliabil- Communication Mechanisms ity of the data are taken for granted. Such scenario is very optimistic, which in fact, does not occur in reality often. Initially, ROS was designed for large service robot control Therefore, it requires that the producer must eventually be software, which was meant for implementation on several slowed down or complex handshaking protocols must be computers connected via Ethernet [78], thus ROS was deployed, otherwise the data will be lost [30]. The push in fact designed for communication between computers. paradigm, used by topics, does not assure idempotent trans- However later it was mainly used for communication fer of messages, thus an additional effort must be done to within a single computer. Hence below we introduce ROS ensure that the data was received exactly once [30]. communication mechanisms provided for a single computer The ROS service is a simple RPC, thus the client is and compare them with the whiteboard approach. blocked until it obtains the response from the server. This ROS uses: topics, services, nodelets, and a parameter type of communication has three additional features. It server. Those communication concepts may be utilised uses only plain old data (POD) structures, collections of only for certain types of communication (based on the basic types, thus object oriented concepts using inheritance localisation of the communicating nodes), i.e. inter-process and polymorphism can hardly be modelled [27], whilst the and intra-process. The inter-process communication occurs approach based on whiteboards implements object oriented between ROS nodes (implemented as processes) utilising approach [30]. Additionally, ROS service forces client- TCP/IP server socket [80] and peer-to-peer topology, server relationship between the two communicating nodes, which requires some sort of lookup mechanism to allow possibly resulting in delays due to waiting for the server processes to find each other at runtime. This mechanism reply. Moreover, the ROS service again uses the TCP/IP is implemented as the ROS master [78]. ROS nodes may communication protocol. communicate with each other utilising topics, services and Parameter server is the last of the inter-process ROS parameter service. communication concepts for intra-system inter-process A ROS topic is a named bus implementing communi- communication. It is a shared, multi-variate dictionary cation between nodes. Communication between two nodes stored in the ROS master. It is accessible by nodes via may only occur if both nodes exist and are already regis- network APIs. However, it was not designed to be high- tered with the ROS master (utilising Remote Procedure Call performance, as it is to be used for static data such as (RPC)). Nodes communicate utilising TCP. Since a ROS configuration parameters [79]. topic uses a publish/subscribe mode of operation, thus it Nodelets are threads that use intra-process communication implements the push paradigm, which forces the subscriber of ROS. Data is exchanged between nodlets executed within to act in an event-driven fashion. Events are deposited into the same process by sharing memory. However, the develop- the middleware which relays them to call-backs from the ers have to take care to avoid contention due to concurrent subscribers. It is assumed that there will be sufficient com- access to the data, e.g. by using mutexes or by allocat- putational resources to enact all the threads generated and ing new memory whenever new data is written. This again to execute the subscriber callbacks. An additional assump- results in a large performance penalty as shown in [27]. tion is that events would occur with enough sparsity that There exist robotic systems that have special communi- the call-backs would be completed by the time they are exe- cation requirements. In some of them, both the producer and cuted again or alternatively require handling of concurrency the consumer should be able to operate in a non-blocking issues. This event-driven approach also results in coupling fashion (i.e. the communicating nodes should be decou- and has timing consequences. The push approach compels pled), as blocking implies possible lack of reaction from the the subscriber to keep up with message reading in order system, when such a reaction is necessary. The communi- not to lose any data, even if a specific update rate is not cation should be capable of operating in the time-triggered J Intell Robot Syst (2019) 93:787–805 797 mode, because in this mode of operation the system can and Manager – interacting with the whiteboards. Due to the react both to the message and its lack (usually those reac- fact that whiteboars contain cyclic buffers that are transmit- tions are different), i.e. lack of a message is also a message. ted as a whole, even if (due to the use of UDP) a transmitted It is required that the communication be low latency and low message is lost, in the next transmission period both the jitter in the case of intra-system transmission, as quick reac- most recent data and previous data will be delivered. tions to events occurring in the environment may influence the survival of the robot. Finally, communication should be implemented in the same way both for threads and pro- 7 Experiment cesses, because the imposition of single process system structure (even with many threads) is not acceptable, i.e. The robot system design methodology described in, e.g. multi-process implementation is treated as an advantage. [92], utilising hierarchical FSMs [35], has been applied None of the four ROS communication methods fulfils to the design of the control system of a mobile robot all of these requirements. The parameter server introduces collecting table-tennis balls. The robot (inspired by [64]) high latency. ROS services use RPC, thus they block both uses a suction mechanism to collect table-tennis balls and the producer and the consumer. In the case of ROS topics, a camera to detect them, as well as a sonar to avoid the consumer is activated by an event associated with obstacles. Based on the estimated ball position, the robot the producer depositing a message in the communication moves in the direction of the closest ball and vacuums it channel, thus communication is event triggered, not time into its container. This simple, yet useful, robot is used triggered. However non-blocking operation of the consumer here for the purpose of exemplifying the proposed general using ROS and C++ can be realised through a separate robotic system specification method and the implementation thread dealing with acquisition of the message and using of the resulting system. Both the system architecture and nodlets – rather a complex solution introducing significant the behavioural approach used resemble the well known communication delays. Unfortunately, nodelets can be used solutions presented in Section 4. As the purpose of the only for communicating threads, but not processes. As example is the presentation of the proposed specification none of the ROS communication mechanisms satisfies the and implementation method, a well known architectural above-mentioned requirements, the whiteboard was chosen solution and control paradigm have been used, to enable the (in fact, a ROS package for the whiteboard is available reader to focus only on the methodology. at [67]). This approach provides intra-system inter-process communication, which does not block neither the producer 7.1 Specification nor the consumer; as it operates on common memory it is very low-latency [31, 47]; by using the pull mode of The specification describes both the structure of the operation it operates in time triggered mode; and moreover designed system and its activities. For the purpose of it can be used both by threads and processes in the same briefness, only relevant excerpts from the full system fashion. specification are presented here. Only a single behaviour of If whiteboards could be used only for intra-system commu- the control subsystem is specified formally here to provide nication the proposed approach would not be scalable. How- an example – the others follow the same pattern. ever there exists a network implementation of the white- Structure It suffices to represent the structure of the ball boards [61], but here the latency introduced both by them collecting robot (bc) by a single embodied agent a (Fig. 5). bc and ROS are similar. In general, network based implementa- This agent contains: the control subsystem c and four tion of the concept of the whiteboard requires that each com- bc puter in the system instantiates its own whiteboards. One of those whiteboards contains variables for local communi- cation whilst the others reflect the variables in the remote whiteboards. The latter variables are updated periodically broadcasted to other computers in the network utilising the UDP Bridge module. The UDP Bridge module is used for all network connectivity and connects directly to the shared data structures of the whiteboard. The UDP Bridge can either be run as an independent process (for debugging purposes) or can be started as a thread by an instance of the Remote Whiteboard class. The UDP Bridge has three main components: Listener – listening for incoming pack- Fig. 5 Embodied agent representing the table-tennis ball collecting ets, Broadcaster – broadcasting packets when scheduled, robot 798 J Intell Robot Syst (2019) 93:787–805 virtual subsystems: r , r , e , e Activities The behaviour of a is defined by the activities bc,cam bc,sonar bc,body bc,vacuum bc and four corresponding real subsystems: R , R , of its subsystems and the interactions between them. The bc,cam bc,sonar E and E . The real receptor R detects activities of c are defined using a hierarchical FSM bc,body bc,vacuum bc,cam bc c c balls, whilst the real receptor R detects obstacles. The F presented in Fig. 6. Each state of the FSM F is bc,sonar bc bc real effector E controls four motors to which wheels associated with the corresponding behaviour: B bc,body bc,search are attached (a skid-steering mobile platform is used), (robot searches for the balls within the environment), c c whereas E controls the vacuum cleaner providing B (robot collects balls) and B (robot avoids bc,vacuum bc,collect bc,avoid suction. R acquires images of the environment and obstacles). Each behaviour is modelled using the general bc,cam sends them to the virtual receptor r , so that it can behaviour template presented in Fig. 2, requiring the bc,cam detect table-tennis balls and subsequently inform the control definition of a transition function, terminal condition, and subsystem c whether it detected at least one ball. If a ball the determination of the communication model. Below, bc is detected r sends to c an estimated position of the the definition of the behaviour B is presented, bc,cam bc bc,collect ball within the image space. R provides to the virtual thus transition function f and terminal condition bc,sonar bc,collect receptor r the measured time (i.e. interval between f are presented. Other behaviours are defined in a bc,sonar bc,collect the emission of the ultrasonic signal and receiving its echo), similar way. based on which it calculates the distance to the obstacle. Canonical decomposition of the transition function Next it sends the information to c whether an obstacle f , based on the components of the output buffer, is bc bc,collect was detected. The c commands both virtual effectors as follows: bc e and e using the information obtained from bc,body bc,vacuum c,e r and r . Finally e controls E , whilst f bc,cam bc,sonar bc,body bc,body bc,collect e i+1 c [cmd]= MOVE e controls E . ⎪ y bc,body bc,vacuum bc,vacuum i+1 ⎪ e r i r i c [d x]= c [x]− c [w]/2 Each subsystem has its own input, output and internal ⎨ y x x bc,body bc,cam bc,cam i+1 e r i c [vel]= calculate vel( c [r]) buffers. Below only those control subsystem c buffer y x bc,body bc,cam bc ⎪ i+1 e r i c i c [cmd]= TURN ON, for c [r]  c [r min] y x contents are introduced, which are relevant for this partial ⎪ bc,vacuum bc,cam bc i+1 r e i c i c [cmd]= TURN OFF, for c [r] < c [r min] y x presentation of the specification. bc,vacuum bc,cam bc (1) – Internal memory c : bc r min – minimum ball size triggering the vacuum The internal memory contains only constants, thus it does cleaner (constant) not require a transition function. The virtual receptors do not e have input buffers fed by the control subsystem, thus those – Output buffer c : bc,body transition functions are not needed. cmd–command for e , bc,body The partial function (1) calculates the parameters of cmd ∈{MOVE, LEFT, RIGHT, STOP}, the control commands for e and e .Inevery bc,body bc,vacuum d x – X coordinate of the location of the centre of c iteration of the behaviour, B a new desired velocity bc,collect the ball with respect to the image centre (expressed of the robot is calculated using the function calculate vel, in pixels) which takes into account the size of the detected ball vel – velocity of the robot (range: 0–100) image. Using the X coordinate of the the centre of the e ball with respect to the centre of the image, an offset is – Output buffer cbc, vacuum: computed that is used to navigates the robot towards the ball. cmd–command for e , bc,vacuum cmd ∈{TURN ON, TURN OFF}, – Input buffer cbc, sonar: obst det – signals whether an obstacle was detected or not; – Input buffer cbc, cam: ball det – ball was detected or not; x, y – X and Y coordinates of the detected ball centre (in pixels) w, h – image width and height (in pixels) r – detected ball radius (in pixels) Fig. 6 Activities of the control subsystem represented by an FSM F bc J Intell Robot Syst (2019) 93:787–805 799 The behaviour stops iterating when the terminal condition received from R . The virtual effectors: e bc,sonar bc,body c τ e e f is fulfilled: and e execute: B and B , bc,vacuum bc,body,move bc,vacuum,set bc,collect respectively. B using its transition function bc,body,move c τ r r transforms the commands received from c into PWM f  ¬ c [ball det]∨ c [obst det] bc bc,cam bc,sonar x x bc,collect values and sends them to E . B transforms bc,body bc,vacuum,set Other two behaviours are specified similarly. Behaviour the commands obtained from c to an appropriate PWM bc B is responsible for moving the robot in search for value and dispatches it to E . Only two states of the bc,search bc,vacuum the balls. It terminates when either a ball or an obstacle vacuum cleaner are considered: turned on or off, thus PWM is detected. Whenever an obstacle is detected behaviour is either of maximum value or zero. B is invoked. It uses the bug algorithm [59]for The intra-system communication between subsystems bc,avoid obstacle avoidance. of a is realised as one-to-one communication. The bc All other subsystems of a , i.e. virtual receptors and communication between c and its virtual subsystems uses bc bc virtual effectors, are governed by single state FSMs. The cyclic buffers. Both producers and consumers use non- associated behaviours execute in their endless loops the blocking mode of operation, thus subsystems must react subFSM template presented in Fig. 2 (thus the terminal appropriately to lack of new data. conditions of those behaviours are always false). The virtual executes the behaviour B .The receptor r 7.2 Implementation bc,cam bc,cam,detect transition function of this behaviour uses the RGB image loaded into the buffer r connected to R , The above defined subsystems are implemented using the bc,cam bc,cam transforms this image into the HSV space, cheques whether MiCASE modelling tool [28]. It enables the high-level pixels are in the appropriate colour range, erodes the image, specification of the activities of the software-implemented dilates it, finds contours, cheques the shape of the detected subsystem (i.e. control subsystem and virtual subsystems) contours, assuming that all circular contours represent balls, using hierarchical LLFSMs that constitute an executable so the one with the largest circumscribed area is used to model. To this end, a slight modification of the original s s calculate the estimated ball position within the image space. FSM F is required [35]. Each F is implemented as bc bc s  c Thus the transition function is defined as a composition of F . The transformation of F , presented in Fig. 6,into bc bc the above functions, as a result producing a pure function its implementation version F , presented in Fig. 7,is bc c ω (no side effects are involved). This is an intrinsic property straightforward. Each state of F , denoted as S (ω is the bc bc of all transition functions. Virtual receptor r executes designator of the behaviour associated with the considered bc,sonar r c B . Its transition function calculates the distance state of F ), is implemented as a pair of consecutive states bc,sonar,detect bc c  ω from the obstacle, based on the measured time of flight of F : the original state S (depicted as a rectangle org bc bc c c Fig. 7 Implementation of F FSM as F FSM modelling the activities of the control subsystem bc bc 800 J Intell Robot Syst (2019) 93:787–805 7.3 Robot Hardware The table-tennis ball collecting robot (Fig. 8) has four motors actuating the four wheels, an obstacle detecting sonar HC-SR04 with a 2–400 cm range, a ball detecting camera (Microsoft LifeCam Studio Webcam), a 12 V car vacuum cleaner producing suction, a ball container, a Raspberry Pi 3B on-board computer, Arduino Micro microcontroller board and a Xaxxon MALG PCB motor controller. The Pi 3B executes the software of the control subsystem and all virtual subsystems, sending commands to the real effectors: E (to the motors bc,body through the Xaxxon MALG motor controller), vacuum cleaner E (Arduino Mirco DC motor velocity bc,vacuum controller MCU-60127) and receives data from the real Fig. 8 Table-tennis ball collecting robot receptors: camera R (Microsoft LifeCam Studio bc,cam Webcam camera), sonar R (Arduino Micro and HC- bc,sonar SR04 sonar). The Raspberry Pi 3B is connected to both the with oval corners) and the supplementary state S wait bc Arduino Micro and Xaxxon MALG PCB using USB serial (depicted as a small oval). With each state S an action org bc port and to the camera by a SCCB interface (serial camera is associated, where a new submachine F is executed control bus). bc,ω by the restart at(SUBMACHINE) command. It models the c c behaviour B . The FSM F waits in the current bc,ω 7.4 Performance of the Communication bc ω B state S until the submachine F starts executing org within the Robot bc bc,ω (i.e. the Boolean expression is running at(SUBMACHINE) returns true). When the submachine F starts executing, If a real-time operating system is used, the execution bc,ω c  ω the FSM F changes its state to the state S . times of particular behaviour steps are as presented wait bc bc c  ω The FSM F being in the state S waits for the wait in Table 1. The produced system requires both intra- bc bc termination of activities of the subFSM F by testing system communication (i.e. between c and virtual bc bc,ω the Boolean expression is suspended at(SUBMACHINE) subsystems) and access to the hardware (i.e. between virtual (it returns true, if the submachine has finished its activities). subsystems and the real subsystems). Communication with The ball collecting robot system requires five FSMs, the hardware is organised as either: send operations of e e because it contains five software subsystems. Additionally, behaviours B and B , or receive bc,body,move bc,vacuum,set r r for each behaviour within each subsystem an additional operations of behaviours B and B . bc,cam,detect bc,sonar,detect subFSMs must be produced. Thus in total there are 12 Hardware access uses standard driver interfaces whilst intra- FSMs/subFSMs. Each of them is governed by the same, system communication utilises the whiteboard. The former single sequential scheduler, switching their activities. The communication uses USB serial port which introduces communication between subsystems is implemented using significant delays, hence those send/receive operations may the whiteboard [30]. take even several milliseconds. In the latter case, the Table 1 Maximum measured time [μs] required by behaviour steps: calculation of transition function f , sending and receiving data, where j,v vac is vacuum and wb is the whiteboard Behaviour f Receive Send j,v B 3wb→ c 16 c → wb 18 bc,search bc bc B 3wb → c 10 c → wb 29 bc,collect bc bc B 9wb → c 21 c → wb 14 bc,avoid bc bc B 20345 R → r 4720 r → wb 21 bc,cam,detect bc,cam bc,cam bc,cam B 2 R → r 1501 r → wb 9 bc,sonar,detect bc,sonar bc,sonar bc,sonar B 2wb → e 10 e → E 5093 bc,body,move bc,body bc,body bc,body e e B 3wb → e 9 → E 1384 bc,vacuum,set bc,wb bc,wb bc,vac J Intell Robot Syst (2019) 93:787–805 801 messages are transferred using shared memory, thus the time However for simple one-computer implementation of the required for execution a single send/receive operation is at system this is not necessary. The presented implementation the most 39μs(10μs+29μs). approach ensures that at a given time only a single The hardware out of which the simple robot was built to subsystem is active so there is no contention regarding data present both the design method and the software used for access. the implementation of the controller was inexpensive. The An embodied agent is a set of subsystems communicat- camera enables image acquisition at the rate of 30 Hz. Thus ing with the control subsystem utilising the whiteboard as a the virtual receptor r collects data every 34 ms. The communication means. The constraint imposed on the inter- bc,cam balls are detected by the camera at a maximum distance of nal structure of an embodied agent, which requires that all 2.5 m. The sonar enables the detection of obstacles in the virtual entities contact only the control subsystem, limits the range 2-400 cm, thus due to the velocity of sound in air, internal number of communication links. Each subsystem a data sample is obtained every 60 ms. As the maximum of an embodied agent is implemented as a separate process. velocity of the robot is 0.25 , the robot is able to stop An additional agent added to the system linearly increases and change the direction of motion well ahead of possible the number of utilised processes. The number of new con- collisions. The decisions as to the selection of the nearest nections required within the system is equal to the number ball to collect are also made well in advance, so the robot of the subsystems within the embodied agent and usually does not have to veer overly rapidly. The balls scattered low number of interconnections of the added agent with the between obstacles were collected efficiently. already existing agents, thus is scalable. Embodied agents can be implemented not only by using LLFSMs and the whiteboard, but also by utilising OROCOS 8 Conclusions components, however the transformation procedure is not that straightforward. OROCOS is a component based This paper extends the robotic system design methodology robotics framework, where each component encapsulates (presented in e.g. [35, 92, 96]) by inter-subsystem an executable algorithm (primary code). Within the communication model [31]. It assumes that the activities of component there exists an Execution Engine, which each subsystem and each of their behaviours is represented processes asynchronous operations, calls plug-in functions by a LLFSMs. The communication between the subsystems and executes the primary code, when the component is in takes place in specific states of the resulting hierarchic FSM, a running state. The FSM is related to the states of the being the composition of the higher level FSM defining the component (pre-operational, stopped, running, exception task of the subsystem and the lower level ones governing etc.) not the states of the executed tasks. Thus, the FSM the execution of this subsystem’s particular behaviours. The within the component is completely different from the FSM subsystems in those states access the whiteboard, providing in the embodied agent. The former changes the state of the or acquiring the necessary data. As the scheduler switches component, and only the running state is of relevance to us, the hierarchic FSMs sequentially no contentions arise. To as only in that state the task is executed. In the latter case, refer the proposed communication model to the state of the the FSM is related to the states of the task execution. With art, a classification of communication methods from the every one of those states a transition function parameterised robotics perspective has been introduced. behaviour is associated. Thus, if a subsystem would have The choice of LLFSMs and the whiteboard as an to be implemented as an OROCOS component both the implementation means for the proposed specification FSM and the behaviours would have to be recreated within method was dictated by the fact that the specification the primary code. Hence no benefit is obtained. Our multi- method utilises the concept of FSM and transition layer structure is being flattened within this code. The other function embedded in a behaviour, which organises the method of using OROCOS is to associate a component communication, i.e. the concepts into which it was easy with a behaviour. Then the primary code would represent to translate the specification. Importantly, the whiteboard a transition function. However then the subsystem FSM ensures reliable access to data. The whiteboard gives would have to be implemented by an extra component precise communication semantics in terms of bounded marshalling the other components (i.e. it would play the timing as well as atomicity. Moreover always the latest role of a scheduler). The activity of the agent would have to readings are delivered and no buffer overflow occurs be produced by still another component executing the FSM due to the use of a cyclic buffer. If only one scheduler of the control subsystem. However in such a system, not is employed the LLFSMs ensure reliable sequential only the communication between subsystems would have execution of the behaviours of the subsystems. Multi- to be organised, but also between behaviours (behaviours scheduler implementation on a multi-processor system is of subsystems of embodied agents have access to all the also possible, but then contention resolution has to be used. input, output and internal data of that subsystem). Thus, data 802 J Intell Robot Syst (2019) 93:787–805 transfers between behaviours would have to be arranged 12. Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm intelligence: From natural to artificial systems. Oxford university press, New through ports, which was not the intention of the model York (1999) employed by embodied agents. 13. Brooks, R.A.: A robust layered control system for a mobile robot. Both the utility of the design methodology and the IEEE J. Robot. Autom. 2(1), 14–23 (1986) employed communication method have been presented on 14. Brooks, R.A.: Intelligence without reason. Artif. Intell. Crit. Concepts 3, 107–163 (1991) a rudimentary robot searching for table-tennis balls. The 15. Brooks, R.A.: Intelligence without representation. Artif. Intell. system was implemented by using the MiCASE tool [28]. 47(1-3), 139–159 (1991) It enables both modelling of the activities of the designed 16. Brugali, D.: Software Engineering for Experimental Robotics. system and automatic code generation, resulting in an In: Brugali, D. (ed.): Stable Analysis Patterns for Robot Mobility. Software Engineering for Experimental Robotics, executable, high-level behaviour model. The resulting robot pp. 9–30. Springer Berlin Heidelberg, Berlin (2007). control system functions as specified. https://doi.org/10.1007/978-3-540-68951-5 2 17. Brugali, D., Agah, A., MacDonald, B., Nesnas, I.A.D., Acknowledgments This work was supported by the Erasmus Mundus Smart, W.D.: Software Engineering for Experimental Action 2 PANTHER (Pacific Atlantic Network for Technical Higher Robotics. In: Brugali, D. (ed.): Trends in robot software Education and Research) grant and by the National Science Centre, domain engineering. Software Engineering for Experimental Poland (grant number 2017/25/N/ST7/00900). Robotics, pp. 3–8. Springer Berlin Heidelberg, Berlin (2007). https://doi.org/10.1007/978-3-540-68951-5 1 18. Cabrera-Gamez, ´ J., Dom´ ınguez-Brito, A.C., Hernandez-Sosa, ´ D.: Open Access This article is distributed under the terms of the Sensor based intelligent robots. chap. CoolBOT: A Component- Creative Commons Attribution 4.0 International License (http:// Oriented Programming Framework for Robotics, pp. 282–304. creativecommons.org/licenses/by/4.0/), which permits unrestricted Springer Berlin / Heidelberg (2002) use, distribution, and reproduction in any medium, provided you give 19. Chen, D., Hexel, R., Raja, F.: Engineering Real-Time Commu- appropriate credit to the original author(s) and the source, provide a nication through Time-Triggered Subsumption: Towards Flexibil- link to the Creative Commons license, and indicate if changes were ity with Incus and Llfsms. In: Maciaszek, L., Filipe, J. (eds.) made. 11th International Conference on Evaluation of Novel Software Approaches to Software Engineering, pp. 272 – 281, Rome Publisher’s Note Springer Nature remains neutral with regard to (2016). https://doi.org/10.5220/0005915602720281 jurisdictional claims in published maps and institutional affiliations. 20. Chen, J., Gauci, M., Li, W., Kolling, A., Groß, R.: Occlusion- based cooperative transport with a swarm of miniature mobile robots. IEEE Trans. Robot. 31(2), 307–321 (2015). https://doi.org/ References 10.1109/TRO.2015.2400731 21. Cheriton, D.R., Skeen, D.: Understanding the limitations of causally and totally ordered communication. SIGOPS Oper. Syst. 1. Alami, R., Chatila, R., Fleury, S., Ghallab, M., Ingrand, F.: An Rev. 27(5), 44–57 (1993). https://doi.org/10.1145/173668.168623 architecture for autonomy. Int. J. Robot. Res. 17(4), 315–337 22. Corkill, D.: Blackboard systems. AI Expert. 6(9), 40–47 (1991) (1998) 23. Coste-Maniere, E., Simmons, R.: Architecture, the Backbone 2. Arbib, M.: Handbook of Physiology – The Nervous System II. of Robotic Systems. In: 2000. Proceedings. ICRA ’00. IEEE Motor Control, chap. Perceptual structures and distributed motor International Conference On Robotics and Automation, vol. 1, control, pp. 1449–1480. Wiley Online Library (1981) pp. 67–72 (2000). https://doi.org/10.1109/ROBOT.2000.844041 3. Arkin, R.C.: Motor schema-based mobile robot navigation. Int. J. 24. Dudek, G., Jenkin, M., Milios, E., Wilkes, D.: A taxonomy Robot. Res. 8, 92–112 (1989) for swarm robots. In: IROS ’93. Proceedings of the 1993 4. Arkin, R.C.: Cooperation without communication: Multiagent schema- IEEE/RSJ International Conference on Intelligent Robots and based robot navigation. J. Robot. Syst. 9(3), 351–364 (1992) Systems ’93, vol. 1, pp. 441–447 (1993). https://doi.org/10.1109/ 5. Arkin, R.C.: Behavior-Based Robotics. MIT press, Cambdrige (1998) IROS.1993.583135 6. Armbrust, C., Kiekbusch, L., Ropertz, T., Berns, K.: Soft Robot 25. Dudek, G., Jenkin, M., Milios, E., Wilkes, D.: Experiments Control with a Behaviour-Based Architecture. In: Soft Robotics, in Sensing and Communication for Robot Convoy Naviga- pp. 81–91. Springer (2015) tion. In: Proceedings. 1995 IEEE/RSJ International Conference 7. Asama, H., Ozaki, K., Matsumoto, A., Endo, I.: Development On Intelligent Robots and Systems 95. ’Human Robot Inter- of task assignment system using communication for multiple action and Cooperative Robots, vol. 2, pp. 268–273 (1995). autonomous robots. J. Robot. Mechatron. 4(2), 122–127 (1992) https://doi.org/10.1109/IROS.1995.526171 8. Balch, T., Arkin, R.C.: Communication in reactive multiagent 26. Dudek, G., Jenkin, M.R.M., Milios, E., Wilkes, D.: A taxonomy robotic systems. Auton. Robot. 1(1), 27–52 (1994) for multi-agent robotics. Auton. Robot. 3(4), 375–397 (1996). 9. Beetz, M., Mosenlechner, ¨ L., Tenorth, M.: CRAM – a Cognitive https://doi.org/10.1007/BF00240651 Robot Abstract Machine for Everyday Manipulation in Human 27. Einhorn, E., Langner, T., Stricker, R., Martin, C., Gross, Environments. In: IEEE/RSJ International Conference on Intel- H.M.: MIRA - Middleware for robotic applications. In: 2012 ligent Robots and Systems, IROS, October 18–22, 2010, Taipei, IEEE/RSJ International Conference on Intelligent Robots and Taiwan, pp. 1012–1017. IEEE (2010) Systems, Vilamoura, pp. 2591–2598 (2012). https://doi.org/10. 10. Bensalem, S., de Silva, L., Ingrand, F.R.Y.: A verifiable and 1109/IROS.2012.6385959. http://ieeexplore.ieee.org/stamp/stamp. correct-by-construction controller for robot functional levels. J. jsp?tp=&arnumber=6385959&isnumber=6385431 Softw. Eng. Robot. 2(1), 1–19 (2011) 28. Estivill-Castro, V., Hexel, R.: Arrangements of finite-state 11. Birman, K., Schiper, A., Stephenson, P.: Lightweight causal and machines - semantics, simulation, and model checking. atomic group multicast. ACM Trans. Comput. Syst. 9(3), 272–314 In: Proceedings of the 1st International Conference on (1991) J Intell Robot Syst (2019) 93:787–805 803 Model-Driven Engineering and Software Development - Volume 44. Haykin, S. Communication Systems, 5th edn. Wiley Publishing, 1: MODELSWARD, pp. 182–189 (2013). https://doi.org/10.5220/ Hoboken (2009) 0004317101820189 45. Huntsberger, T.L., Trebi-Ollennu, A., Aghazarian, H., Schenker, 29. Estivill-Castro, V., Hexel, R.: Correctness by Construction with P.S., Pirjanian, P., Nayar, H.D.: Distributed control of multi- Logic-Labeled Finite-State Machines – Comparison with Event- robot systems engaged in tightly coupled tasks. Auton. B. In: Zhu, L., Steel, J. (eds.) Proceedings of 23Rd Australasian Robot. 17(1), 79–92 (2004). https://doi.org/10.1023/B:AURO. Software Engineering Conference (ASWEC), pp. 38–47. IEEE 0000032939.08597.62 Computer Society Conference Publishing Services (CPS), Mil- 46. Jennings, N.R., Sycara, K., Wooldridge, M.: A roadmap of agent sons Point (2014) research and development. Auton. Agents Multi-Agent Syst. 1(1), 30. Estivill-Castro, V., Hexel, R.: Simple, Not Simplistic — the 7–38 (1998). https://doi.org/10.1023/A:1010090405266 Middleware of Behaviour Models. In: Filipe, J., Maciaszek, 47. Joukoff, D., Estivill-Castro, V., Hexel, R., Lusty, C.: Fast MAV L.A. (eds.) ENASE 10 International Conference on Evaluation Control by Control/Status OO-Messages on Shared-Memory of Novel Approaches to Software Engineering, pp. 189–196. Middleware. In: Robot Intelligence Technology and Applications INSTCC, SciTePress, Av. Dom Manuel i, 2910-592, Setubal ´ 4 - Results from the 4Th International Conference on Robot (2015) Intelligence Technology and Applications, RiTA 2015, Advances 31. Estivill-Castro, V., Hexel, R., Lusty, C.: High Performance in Intelligent Systems and Computing, Vol. 345, pp. 195–211. Relaying of C++11 Objects across Processes and Logic-Labeled Springer (2015) Finite-State Machines. In: Brugali, D., Broenink, J.F., Kroeger, T., 48. Jung, D., Zelinsky, A.: Grounded symbolic communication MacDonald, B.A. (eds.) Simulation, Modeling, and Programming between heterogeneous cooperating robots. Auton. Robot. 8(3), for Autonomous Robots - 4th International Conference, SIMPAR 269–292 (2000). https://doi.org/10.1023/A:1008929609573 2014, Lecture Notes in Computer Science, vol. 8810, pp. 182–194. 49. Kiekbusch, L., Armbrust, C., Berns, K.: Formal verification of Springer (2014) behaviour networks including sensor failures. Robot. Auton. Syst. 32. Estivill-Castro, V., Hexel, R., Rosenblueth, D.A.Wang, P. (ed.): 74, 331–339 (2015) Efficient Model Checking and FMEA Analysis with Deterministic 50. Kiekbusch, L., Armbrust, C., Berns, K.: Formal Verification of Scheduling of Transition-Labeled Finite-State Machines. IEEE Behaviour Networks including Hardware Failures. In: Intelligent Computer Society Conference Publishing Services (CPS), Wuhan Autonomous Systems 13, pp. 1571–1582. Springer (2016) (2012) 51. Kopetz, H.: Should responsive systems be event-triggered or 33. Estivill-Castro, V., Hexel, R., Rosenblueth, D.A.: Efficient time-triggered? IEICE Trans. Inf. Syst. 76(11), 1325 (1993) Modelling of Embedded Software Systems and Their Formal 52. Kopetz, H. Real-Time Systems - Design Principles for Distributed Verification. In: Leung, K.R., Muenchaisri, P. (eds.) The 19Th Embedded Applications, 2nd edn. Real-Time Systems Series. Asia-Pacific Software Engineering Conference (APSEC 2012), Springer, Berlin (2011) pp. 428–433. IEEE Computer Society, Conference Publishing 53. Kopetz, H., Bauer, G.: The time-triggered architecture. Proc. IEEE Services, Hong Kong (2012) 91(1), 112–126 (2003) 34. Farinelli, A., Iocchi, L., Nardi, D.: Multirobot systems: a classi- 54. Kornuta, T., Zielinski, ´ C.: Robot control system design exempli- fication focused on coordination. IEEE Trans. Syst. Man Cybern. fied by multi-camera visual servoing. J. Intell. Robotic Syst. 77(3– Part B (Cybern.) 34(5), 2015–2028 (2004). https://doi.org/ 4), 499–524 (2013). https://doi.org/10.1007/s10846-013-9883-x 10.1109/TSMCB.2004.832155 55. Kortenkamp, D., Simmons, R.: Robotic Systems Architectures 35. Figat, M., Zielinski, ´ C., Hexel, R.: Fsm Based Specification of and Programming. In: Khatib, O., Siciliano, B. (eds.) Springer Robot Control System Activities. In: 2017 11Th International Handbook of Robotics, pp. 187–206. Springer (2008) Workshop on Robot Motion and Control (Romoco), pp. 193–198. 56. Kortenkamp, D., Simmons, R., Brugali, D.: Robotic Systems https://doi.org/10.1109/RoMoCo.2017.8003912 (2017) Architectures and Programming. In: Siciliano, B., Khatib, en 36. Fleury, S., Herrb, M., Chatila, R.: G om: A tool for the O. (eds.) Springer Handbook of Robotics. 2nd edn., pp. 283–306. specification and the implementation of operating modules in a Springer (2016) distributed robot architecture. Proc. 1997 IEEE/RSJ Int. Conf. 57. Lamport, L.: Using time instead of timeout for fault-tolerant Intell. Robot. Syst. (IROS’97) 2, 842–849 (1997). https://doi.org/ distributed systems. ACM Trans. Progr. Lang. Syst. 6, 254–280 10.1109/IROS.1997.655108 (1984) 58. Lee, C., Hwang, J., Lee, J., Ahn, C., Suh, B., Shin, D.H., Nah, Y., 37. Fox, C., Evans, M., Pearson, M., Prescott, T.: Towards hier- Kim, D.H.: Self-Describing and Data Propagation Model for Data archical blackboard mapping on a whiskered robot. Robot. Distribution Service. In: Brinkschulte, U., Givargis, T., Russo, Auton. Syst. 60(11), 1356–1366 (2012). https://doi.org/ S. (eds.) Software Technologies for Embedded and Ubiquitous 10.1016/j.robot.2012.03.005. Towards Autonomous Robotic Systems, pp. 102–113. Springer, Berlin (2008) Systems 2011 59. Lumelsky, V.J., Stepanov, A.A.: Path-planning strategies for a 38. Gat, E.: On Three-Layer Architectures. In: Kortenkamp, D., point mobile automaton moving amidst obstacles of arbitrary Bonnasso, R.P., Murphy, R. (eds.) Artificial Intelligence and shape. Algorithmica 2, 403–430 (1987) Mobile Robots, pp. 195–210. AAAI Press, Cambridge (1998) 39. Giffin, S.d.: A taxonomy of Internet applications for project 60. Lumia, R., Fiala, J., Wavering, A.: The nasrem robot control management communication. Proj. Manag. J. 33(4), 39–47 (2002) system standard. Robot. Comput.-Integr. Manuf. 6(4), 303 – 308 40. Group, O.M.: Data distribution service (dds). https://www.omg. (1989). https://doi.org/10.1016/0736-5845(89)90120-8. Special org/spec/DDS/1.4 Issue Robots in Manufacturing 41. Hayes-Roth, B.: A blackboard architecture for control. Artif. 61. Lusty, C.: Cooperative Time-Triggered Wireless Communica- Intell. 26(3), 251–321 (1985) tion in Mobile Robotics. Honours thesis, Griffith University 42. Hayes-Roth, B.: A blackboard architecture for control. In: Bond, (2012) A.H., Gasser, L. (eds.) Distributed Artificial Intelligence, pp. 505– 62. Lyons, D.M.: Prerational intelligence, Studies in cognitive 540. Morgan Kaufmann Publishers Inc., San Francisco (1988). systems, vol. 2: Adaptive behavior and intelligent systems http://dl.acm.org/citation.cfm?id=60204.60241 without symbols and logic, chap. A Schema-Theory Approach to 43. Hayes-Roth, B., Hayes-Roth, F.: A cognitive model of planning. Specifying and Analysing the Behavior of Robotic Systems, pp. Cogn. Sci. 3(4), 275–310 (1979) 51–70 Kluwer Academic (2001) 804 J Intell Robot Syst (2019) 93:787–805 63. Lyons, D.M., Arbib, M.A.: A formal model of computation for 84. Sommerville, I. Software Engineering, 9th edn. Addison-Wesley sensory-based robotics. IEEE Trans. Robot. Autom. 5(3), 280–293 Publishing Company, USA (2010) (1989). https://doi.org/10.1109/70.34764 85. Song, D., Goldberg, K., Chong, N.Y.: Networked Robots. In: 64. Majcher, P.: Autonomous Mobile Robot for Collecting Table Tenis Khatib, O., Siciliano, B. (eds.) Springer Handbook of Robotics, Balls (Polish). Master’s thesis, Warsaw University of Technology pp. 1109–1131. Springer (2016) (2012) 86. Tanenbaum, A.S., Woodhull, A.S. Operating Systems Design and 65. Mataric, ´ M.J.: Issues and approaches in the design of collective Implementation, 3rd edn. Prentice-Hall, Inc., Upper Saddle River autonomous agents. Robot. Auton. Syst. 16(2), 321 – 331 (1995). (2005) https://doi.org/10.1016/0921-8890(95)00053-4 87. Trianni, V., Nolfi, S., Dorigo, M.: Cooperative hole avoidance in a 66. Mataric, ´ M., Nilsson, M., Simsarian, K.: Cooperative Multi-Robot swarm-bot. Robot. Auton. Syst. 54(2), 97–103 (2006). Intelligent Box-Pushing. In: Proceedings of the IEEE/RSJ International Autonomous Systems 8th Conference on Intelligent Autonomous Conference on Intelligent Robots and Systems, pp. 556–561 Systems (IAS-8) (1995) 88. Ueyama, T., Fukuda, T., Arai, F.: Configuration of communication 67. MiPal: Downloads (checked: 23 March 2018). http://mipal.net.au/ structure for distributed intelligent robotic system. Proc. - IEEE downloads.php Int. Conf. Robot. Autom. 1, 807–812 (1992) 68. Nesnas, I.A.D.: Software Engineering for Experimental Robotics. 89. Yamada, S., Saito, J.: Adaptive action selection without explicit In: Brugali, D. (ed.): The CLARAty project: Coping with communication for multirobot box-pushing. IEEE Trans. Syst. hardware and software heterogeneity, pp. 31–70. Springer Berlin Man Cybern. Part C 31(3), 398–404 (2001) 90. Zielinski, ´ C.: Description of semantics of robot programming Heidelberg, Berlin (2007). https://doi.org/10.1007/978-3-540- languages. Mechatronics 2(2), 171–198 (1992) 68951-5 3 91. Zielinski, ´ C.: A Quasi-Formal Approach to Structuring Multi- 69. Newell, A.: Some Problems of Basic Organization in Problem- Robot System Controllers. In: Second International Workshop Solving Programs. In: Yovitz, M., Jacobi, G., Goldstein, G. (eds.) on Robot Motion and Control, Romoco’01, pp. 121–128 Self-Organizing Systems, pp. 393–423. Spartan, Washington (1962) (2001) 70. Nii, H.P.: Blackboard systems, part two: Blackboard application 92. Zielinski, ´ C., Figat, M.: Robot System Design Procedure Based systems, blackboard systems from a knowledge engineering on a Formal Specification. In: Recent Advances in Automation, perspective. AI Mag. 7, 82–106 (1986) Robotics and Measuring Techniques, Advances in Intelligent 71. Nilsson, N.: Shakey the robot, technical note 323. Technical Systems and Computing (AISC), vol. 440, pp. 511–522. Springer report, AI Center, SRI International, 333 Ravenswood Ave., (2016). https://doi.org/10.1007/978-3-319-29357-8 45 Menlo Park, CA, pp. 94025 (1984) 93. Zielinski, ´ C., Kornuta, T.: Diagnostic requirements in multi-robot 72. Occello, M., Chaouiya, C., Thomas, M.C.: A Parallel Black- systems. In: Intelligent Systems in Technical and Medical Diag- board Model for Mobile Robotics control, pp. 439–447. nostics, vol. 230, pp. 345–356. Springer (2014). https://doi.org/10. Springer Netherlands, Dordrecht (1992). https://doi.org/10.1007/ 1007/978-3-642-39881-0 29 978-94-011-2526-0 51 94. Zielinski, ´ C., Kornuta, T., Winiarski, T.: A Systematic Method 73. Pang, G.K., Shen, H.C.: Intelligent control of an autonomous of Designing Control Systems for Service and Field Robots. In: mobile robot in a hazardous material spill accident — a blackboard 19-Th IEEE International Conference on Methods and Models structure approach. Robot. Auton. Syst. 6(4), 351–365 (1990) in Automation and Robotics, MMAR, pp. 1–14. IEEE (2014). 74. Parker, L.: ALLIANCE: An architecture for fault tolerant https://doi.org/10.1109/MMAR.2014.6957317 multirobot cooperation. IEEE Trans. Robot. Autom. 14(2), 220– 95. Zielinski, ´ C., Trojanek, P.: Stigmergic cooperation of autonomous 240 (1998) robots. J. Mech. Mach. Theory 44, 656–670 (2009) 75. Parker, L., Rus, D., Sukhatme, S.G.: Multiple Mobile Robot 96. Zielinski, ´ C., Winiarski, T.: General specification of multi-robot Systems. In: Khatib, O., Siciliano, B. (eds.) Springer Handbook of control system structures. Bullet. Pol. Acad. Sci. – Techn. Sci. Robotics, pp. 1335–1384. Springer (2016) 58(1), 15–28 (2010). https://doi.org/10.2478/v10175-010-0002-x 76. Parker, L.E.: Alliance: an architecture for fault tolerant, coop- 97. Zielinski, ´ C., Winiarski, T.: Motion generation in the MRROC++ erative control of heterogeneous mobile robots. In: IROS ’94. robot programming framework. Int. J. Robot. Res. 29(4), 386–413 Proceedings of the IEEE/RSJ/GI International Conference on (2010). https://doi.org/10.1177/0278364909348761 Intelligent Robots and Systems ’94. Advanced Robotic Systems 98. Zielinski, ´ C., Winiarski, T., Kornuta, T.: Agent-based structures and the Real World, vol. 2, pp. 776–783 (1994) of robot systems. In: K. J., et al. (eds.) Trends in Advanced 77. Postel, J.: Transmission Control Protocol, RFC 793. Technical Intelligent Control, Optimization and Automation, Advances in report, University of Southern California, Information Sciences Intelligent Systems and Computing, vol. 577, pp. 493–502 (2017). Institute. https://tools.ietf.org/html/rfc793 (1981) https://doi.org/10.1007/978-3-319-60699-6 48 78. Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.: ROS: an open-source Robot Operating System. In: Proceedings of the Open-Source Software workshop at the International Conference on Robotics and Automation (ICRA) (2009) Cezary Zielinski ´ M.Sc./Eng. (1982), Ph.D. (1988) and habilitation 79. ROS: Parameter Server. http://wiki.ros.org/ParameterServer (1996) all in control and robotics, from Warsaw University of 80. ROS: Technical Overview. http://wiki.ros.org/ROS/Technical Technology (WUT), full professor of WUT. Specialises in robotics, Overview heads the Robotics Group in Institute of Control and Computation 81. Rubenstein, M., Cornejo, A., Nagpal, R.: Programmable self- Engineering, working on the design of robot controllers and assembly in a thousand-robot swarm. Science 345(6198), 795–799 programming methods. His research interests focus on robotics in (2014). https://doi.org/10.1126/science.1254295 general and especially include: robot programming methods, multi- 82. Schwartz, D.G.: Cooperating heterogeneous systems. Springer robot system controllers, robot kinematics, robot force control, visual Science & Business Media, NO (1995) servo control, utilisation of sensors in robot control, design of digital 83. Shannon, C.E., Weaver, W.: A Mathematical Theory of Commu- circuits. Currently he is the vice-dean for General Affairs of the nication. University of Illinois Press, Baltimore (1963) Faculty of Electronics and Information Technology of WUT. J Intell Robot Syst (2019) 93:787–805 805 Maksym Figat M.Sc./Eng. (2013) in computer science (modelling Rene´ Hexel M.Sc./Eng. (1995), PhD (1999), has been a leading CAD/CAM systems), from Warsaw University of Technology (WUT), researcher in software engineering of safety-critical real-time systems. Faculty of Mathematics and Information Science. Since 2013, he His achievements include methodologies for modelling, model- is Ph.D. student at the Warsaw University of Technology (WUT), checking, evaluation, and validation of safety-critical real-time Faculty of Electronics and Information Technology (FEIT), Institute systems, including failure mode effects analysis (FMEA) through of Control and Computation Engineering (ICCE), Warsaw, Poland. fault injection. His research interests include Software Engineering He is a member of the Robotics Group in ICCE working on the of Complex Distributed Systems and autonomous robots as well as design of robot controllers and programming methods. His main robust and reliable safety critical systems. Rene ´ Hexel is Deputy scientific interests focus on automatic methods of robot control system Head of School of ICT at Griffith University and Co-Director of the generation based on a formal specification. His research in robotics Machine Intelligence and Pattern Analysis Lab (MiPal) at Institute for is based on Model Driven Engineering, Domain Specific Language, Integrated and Intelligent Systems (IIIS) in Brisbane, Australia. Embodied Agent approach and the formal specification for robotic control systems. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Intelligent & Robotic Systems Springer Journals

Communication Within Multi-FSM Based Robotic Systems

Loading next page...
1
 
/lp/springer_journal/communication-within-multi-fsm-based-robotic-systems-U03lG0Lfux

References (53)

Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s)
Subject
Engineering; Control, Robotics, Mechatronics; Electrical Engineering; Artificial Intelligence; Mechanical Engineering
ISSN
0921-0296
eISSN
1573-0409
DOI
10.1007/s10846-018-0869-6
Publisher site
See Article on Publisher Site

Abstract

The paper presents a robotic system design methodology based on the concept of an embodied agent decomposed into communicating subsystems, whose activities are specified in terms of FSMs invoking behaviours parameterised by transition functions and terminal conditions. In the implementation phase, this specification is transformed into a system composed of a whiteboard providing communication means and logically labelled FSMs (LLFSMs) defining the system behaviour. These concepts are used to generate the code of the robot controller. The inclusion of inter-subsystem communication model completes the resulting system design with respect to our previous work that did not account for this model. Thus communication plays a central role in this presentation. The design methodology is exemplified with a rudimentary table tennis ball-collecting robot. The presented methodology and the implementation tools are suitable and beneficial for application to the design of other robotic systems. Keywords Robotic system specification · Robotic system design methodology · Communication model 1 Introduction whiteboard for communication. Especially the decoupling achieved by the method of communication between the Good practises of software engineering require that the components of a robotic system is of interest here. We creation of systems involves a specification phase that start with a motivation for the presented research and the defines the model of what has to be created, and an structure of the paper. implementation phase that discloses how this is done [84]. This paper uses the specification method based 1.1 Motivation on the concept of an embodied agent [54, 92]. The novelty is twofold. Our first contribution is the method The design of a robotic system starts with the specification of implementation of robotics systems using the concepts of its architecture. As Andrew Tanenbaum points out of logic-labelled finite-state machines (LLFSMs) and in his seminal work [86], the term architecture pertains whiteboards. Secondly, we show the simplicity of the to: the instruction set, memory organisation, I/O, and transformation of the embodied agent based specification bus structure. This obviously refers to the architecture into an (executable) implementation using LLFSMs and the of computer systems. However current robotic systems are computer based, thus their architecture has the same foundation. The instruction set defines what the processing Cezary Zielinski ´ capabilities of the system, i.e. possible activities, are. c.zielinski@ia.pw.edu.pl The remaining elements refer to the structure, i.e. the Maksym Figat components and their interconnections [23, 55]. In the m.figat@ia.pw.edu.pl approach that we follow in our work, the structure refers ReneH ´ exel to the division of the system into subsystems and their r.hexel@griffith.edu.au interconnections, whilst the activities are specified by a set of such concepts as: Finite State Machines (FSMs), Institute of Control and Computation Engineering, behaviours, transition functions and their arguments, and Warsaw University of Technology, Nowowiejska 15/19, 00-665 Warsaw, Poland inter subsystem communication means, all of which 2 will be detailed later in the paper. However, in many School of ICT, Griffith University, 170 Kessels Road, Nathan, QLD 4111, Australia existing robotic systems, it is difficult to determine their 788 J Intell Robot Syst (2019) 93:787–805 architecture [55]. Robot control systems are inherently is kept concise to enable the reader to focus on the pre- complex, nonetheless their authors rarely formally specify sented methods rather than being forced to deal with the them. Lack of a formal system specification usually complexity of the robotic system. The last section presents results in an obscure architecture that is difficult to the conclusions derived from the presented work. modify or extend it in future projects or to integrate with other systems. The process of developing robotic systems requires both appropriate development methods 2 Communication (procedures, architectures, etc.) and tools (modelling languages, middleware and other frameworks). Many In telecommunications, the concept of the communication attempts have been made to introduce formal languages channel is central to the operation of a communication based on mathematics to determine both the controller system [44, 83], distinguishing between free propagation structure and its activities [18, 63, 90, 92, 96]. However, (broadcast, one-to-many) and guided propagation (point-to- the majority of the robot systems were developed based on point, one-to-one) communication types. Telecommunica- software engineering [16, 17, 68], downplaying the domain- tion deals with such characteristics of the communication specific information required by robotics. The proposed channel as its linearity or non-linearity, time variability or method of modelling and specifying robotic systems, based invariance, bandwidth, or power limitation. However those on the concept of the embodied agent [54, 92, 94], relies physical properties are of secondary importance and can on such fundamental concepts of robotics as: effectors, be abstracted away for communication of software imple- i.e. devices influencing the physical environment, and mented subsystems of robots. In the most rudimentary view, receptors, i.e. sensors gathering the information about the communication model consists of the communicating the state of that environment, as well as mathematical systems/subsystems and the communication channel. The concepts underlying decision making and sub-system communication channel (the transmission medium) can be behaviour. However, up till now this design methodology either based on technological means (e.g. wired or wireless) lacked a detailed representation of a vital component: or use natural means (e.g. voice communication using air) inter-subsystem communication. Hence, the present paper [44, 83]. In the latter case the environment itself can be used delves into that issue. The main purpose of this paper as a communication channel [4], then stigmergy results [12]. is therefore to present the system design methodology Both technological and natural communication channels can split into two phases, as required by good practises in contain buffers, which can store the transmitted information software engineering. The first phase produces the system for some time, or exist without a buffer. Transmission delays model specification based on the concept of an embodied should not be treated as a capacity of the channel to store agent, whilst the second one is the implementation phase, data. For communication to take place not only the transmis- which first transforms the model into the composition of sion medium is necessary, but also a common protocol has a whiteboard and LLFSMs as an interim implementation to exist, as the transmitter and the receiver of the informa- model that is subsequently transformed into executable tion must have a common understanding of the transmitted robotic system controller code. information. In robotics, diverse communication methods have 1.2 Structure of the Paper been considered. Three different types of communication between robots were discussed in [25]: two-way commu- In Section 2 the classification of communication methods nication, explicit one-way communication, and completely from the robotics perspective is introduced. Section 3 briefly implicit communication. This work also points out that there presents the concept of an embodied agent, showing both are several difficulties that should be considered in com- its structure and its activities. The embodied agent is the munication between collective robots, i.e. efficiency, fault primary concept of the robotic system design method used tolerance and cost, and that in contrast to centralised com- in this paper. The universality of an embodied agent is munication approaches, e.g. [88], communication between expounded in Section 4. Section 5 describes the interim robots should be distributed. Comparison of three types of implementation model, i.e. the elements necessary for the communication, i.e. through the environment, using trans- implementation of an embodied agent, focusing on the mission of state between the agents, and utilising the trans- communication aspect. mission of the detected goal, was introduced in [8]. Similar As ROS currently is the most popular robot programming criteria were discussed in [65], distinguishing direct (purely framework, the comparison of whiteboard and ROS com- communicative act) and indirect communication (based on munication mechanisms is presented in Section 6. Section 7 observed behaviour of other agents). Direct communica- showcases the proposed design and implementation meth- tion was further decomposed into one-to-one (peer-to-peer) ods on a robot collecting table-tennis balls. This example and one-to-many (broadcasting), based on the the number J Intell Robot Syst (2019) 93:787–805 789 of communicating entities. Another division of commu- on the number of message producers and the number of nication is direct vs indirect communication [34]. Direct consumers: one-to-one (peer-to-peer), one-to-many (e.g. communication occurs when a dedicated on-board hardware broadcast), many-to-one and many-to-many (e.g. multicast). device is utilised, whilst indirect communication occurs An example of one-to-one communication is presented in through the environment. The paper also notices that indi- [66] where a system composed of many robots executes a rect communication is particularly useful from the point of box-pushing task. Cases of one-to-many communication are view of Multirobot Systems (MRS), in contrast to direct presented in [7, 45, 65, 74, 76], where the consumers either communication, which in MRS may result in much more utilise broadcast messages at their will or ignore them. The expensive and unreliable solution. However, that paper also many-to-many communication can be organised as e.g. a points out that direct communication in Multi-Agent Sys- blackboard system [41]orbystigmergy[12]. tems (MAS) may be used to guarantee locality of inter- Another criterion worth considering is based on the actions and to avoid synchronisation procedures amongst location of the communication channel: inter-system or agents. A similar criterion based on how the information intra-system. Here the terms intra-system and inter-system may be obtained by robots was introduced in [75], where need to be disambiguated. We assume that in the case three types of communication were distinguished: implicit of intra-system communication, the transmission occurs communication through the world, passive action recog- between nodes existing on a single computer (composed nition (robots use sensors to directly observe the actions of a single or multiple processors), and in the case of of other robots) and explicit communication (robots com- inter-system communication transmission is between nodes municate directly and intentionally through an artificial residing on many computers. Thus in this case the word communication channel). Either way, care must be taken system does not refer to the robotic system as a whole, to avoid the hidden channel problem that results when but only to the communicating nodes that are within the differing temporal precedence between these means of com- robotic system, located either on a single computer or munication creates inconsistencies between the intent of the on several computers forming a network. The importance system designer and the actual timing and order in which of this distinction is due to each of these types of information is delivered [21]. communication requiring different software and hardware Criteria pertaining to the communication range, commu- means of implementation. nication topology and communication bandwidth were dis- Still another criterium pertains to whether the producer cussedin[24, 26]. However, those criteria concern mainly or consumer block until message delivery has completed. physical aspects of communication, already discussed in Blocking producer behaviour requires the consumer to [44, 85]. Yet other criteria pertain to interaction distance respond in order to unblock the producer, whilst in the (distance between the agents during the communication, i.e. non-blocking mode this is not required, i.e. after initiating direct physical contact, visual range, hearing range and long the communication, the producer can continue with other range), interaction simultaneity (period between the signal activities. Similarly the consumer can either be blocked emission and reception, i.e. immediate, long time), sig- whilst waiting for the message or not. In the latter case, nalling explicitness (explicitness of the emitter’s signalling the consumer can poll the communication channel when behaviour) and sophistication of interpretation (the com- required to find out whether there is a message for it to plexity of the interpretation process that gives meaning to attend to. Still another criterion is storage, i.e. whether the the signal) [48]. communication channel contains a buffer or not. In the The above analysis shows that whilst similar terminology latter case the communication channel does not have the is used for the purpose of communication classification, the capability to store messages. In the former case the size and meaning attributed to the same names is slightly different, type of the buffer is of relevance. The commonly used buffer making it difficult to propose a comprehensive communi- types in robotics are either queues or cyclic buffers. cation model. Some of the subdivision criteria are based on Moreover, it should be noted that communication physical aspects of communication (topology, place, types is sometimes associated with cooperation. However the of communication channels, bandwidth, etc.), neglecting former refers to the ability to transmit information between other also very important criteria (e.g. whether the commu- systems and the latter with the ability to jointly execute a nicating entities block each other whilst sending/receiving task. Cooperation can be achieved both by using implicit data). Thus, to make the presentation clear, we summarise and explicit communication. The former refers to a situation the criteria most relevant to the subject of this paper as where the consumer of the message is not clearly defined, follows. From the point of view of robot control system and thus the interpretation of the message by a consumer design, of importance is the possible number of subsys- may be difficult (as in the case of stigmergy) and in the latter case the consumer is clearly defined, and thus the tems writing to or reading from the communication channel. This leads to four communication types [39], depending interpretation of the message using the commonly known 790 J Intell Robot Syst (2019) 93:787–805 protocol is straightforward (as is the case with the majority of technologically based channels or such natural channels as air used for voice communication). In robotics, there is a vast body of literature dealing with cooperation using implicit communication, especially stigmergy. For example [4] presents a schema based control of a multi-robot system used for common object retrieval, [81] describes robots communicating using sensors for detecting infrared radiation reflected by the environment, [87] presents robots communicating using traction forces, [25] describes robots communicating indirectly using on-board visual sensing, whilst [20] reports on the system composed of 20 e-puck robots pushing three different objects based on inputs from infrared proximity sensors, [89] describes a multi-robot system where individual robots use an adaptive behaviour selection strategy based on sensoric information about the location of the transferred object and the other robots, [95] presents multi-robot box pushing based on the detection of the motions of the translocated box. On the one hand the examples of implicit communication are abundant in the literature and on the other hand this form of communication does not influence the system specification from the point of view of communication, as it is realised by the perception subsystem. Thus this paper will limit itself to explicit forms Fig. 1 Internal structure of an agent a of communication. if true, cause a state transition from the current to the state pointed at by the arc labelled by this predicate. With 3 Embodied Agent each state of an FSM F there is an associated subFSM j,v s B s F , specifying the behaviour B ,where ω names j,v,ω j,v,ω A single- or multi-robot system (a robotic system, in short) the behaviour. The subFSM template modelling a behaviour is represented by communicating agents [14, 46]. An agent B is presented in Fig. 2. Each iteration of a behaviour j,v,ω which has a physical body is called an embodied agent [14, consists of calculation of a transition function and two- 54, 94, 96] – its general structure is presented in Fig. 1. way transfer of data. The transition function f takes j,v,ω An embodied agent a (j is its designator) is decomposed as arguments the current data from the subsystem s internal into 5 types of subsystems: control subsystem c , virtual s i i memory S and its input buffers S , and calculates j,v j,v receptors r , virtual effectors e , real receptors R and j,k j,n j,l i+1 the values which are inserted into the output buffers S real effectors E ,where k, n, l and m are the designators j,m j,v s i+1 of particular subsystems of the agent a [94]. and the internal memory s . Superscripts i and i + 1 j,v Virtual receptors r aggregate data acquired from the j,k indicate the elapse of a single tick of the discrete time, environment through real receptors R . Virtual effectors j,l e transform control commands received from the control j,n subsystem c into a form acceptable by the real effectors E . An embodied agent contains a single control j,m subsystem and zero or more subsystems of each of the other types. The activity of a system as a whole depends on the individual activities of its subsystems and their interactions. The activity of each subsystem s ∈{c, e, r} is represented by a hierarchical FSM F [35], where v indicates the j,v subsystem name. An FSM is represented as a directed graph composed of nodes and directed arcs. Each node represents a state, whilst each arc represents a state transition. Arcs s B s Fig. 2 FSM F specifying the behaviour B are labelled by initial conditions, being predicates, which, j,v,ω j,v,ω J Intell Robot Syst (2019) 93:787–805 791 which determines the duration of the behaviour iteration. environment, invoked action planning, and then executed The behaviour subsequently sends the results of calculations the thus derived plan. Later, different criterions have been i+1 from the output buffers S to the inputs of the associated used for the decomposition into layers. Usually, the time j,v subsystems and acquires from them new data loading it of the execution of a single behaviour iteration of a com- into input buffers xS . Based on the obtained data an ponent within the layer is the discriminating factor. The j,v s ε associated error condition f and terminal condition higher the layer within the system, the longer the time of its j,v,β s τ f are tested (β and ξ are their designators). The single iteration execution. In some cases, task abstraction j,v,ξ duration of a single iteration of the subsystem behaviour is the criterion, i.e. a single activity at an upper layer is defines its sampling period. The iterations of the behaviour decomposed into several lower layer activities, e.g. NAS- terminate when the associated error or terminal condition REM (NASA Standard Reference Model) [60]. Usually, as is fulfilled. At this moment the main subsystem FSM F a consequence, the utilised ontology becomes more abstract j,v resumes its activity, changing its state. going up the layers. The lowest layer is behavioural and is tightly coupled with actuators and sensors. The mid- dle layer is the sequencing layer (sometimes termed as 4 Universality of the Embodied Agent executive layer [56]), responsible for selecting behaviours that will eventually realise the task. The uppermost tier The structure of the embodied agent and its ability to usually involves deliberation. In robotics, deliberation is connect to other such agents, thus producing networks, associated with planning. From what has been said, it is enables the reproduction of other well known and studied evident that planning and sequencing layers act together architectures. Hence, the embodied agent can be treated to accomplish the plan and in some systems are tightly as a general means for designing robotics systems. The bound together (e.g. CRAM [9], CLARAty [68]), hence 2T most popular architectures employed in robotics are: Sense- instead of 3T architecture results. The proposed embodied Plan-Act (SPA), multi-tier (usually either two- (2T) or agent based architecture can accommodate all of these three-tier (3T) are used), subsumption, and the ones based possibilities. By design, an embodied agent forms a two on some forms of schemata. The universality of the tier architecture: virtual entities controlling the real devices concept of an embodied agent will be briefly justified form the first tier and the control subsystem forms the here by showing its relationship to those architectures. second one. Above that, a hierarchy of computational Robotic system software reflects its architecture. Although agents (embodied agents lacking effectors and recep- some work has been done on formal specification of tors) can form supplementary tiers. A computationally robot control software, e.g. [6, 62, 63, 91, 93] and its demanding planning task can be decomposed into several formal verification, e.g. [49, 50], unfortunately this has parts, implemented as several interacting computational not gained widespread adoption [56]. Usually, architectures agents. are presented by informal text descriptions supplemented As planning is a time-consuming task, the subsumption by block diagrams, with varying level of details. Early architecture was proposed [13–15]. Its overall behaviour is approaches to formalisation of the specification of a robotic the result of the interaction of the activities of modules, system treated such systems as composed of computational which react to input from other modules and sensoric modules and followed operational semantics in describing stimulus. If lower layer reactions can not cope with their structure and activities [62, 63]. One of the few systems the task, upper layer reactions suppress or inhibit them, that employs formal specification and generation of code taking over control, thus subsuming their actions. As the for the implementation of the lowest layer of robotic system SPA architecture tends to be prohibitively slow and the en oM (Generator of Modules) [1, 36]. It is controller is G subsumption architecture, although very reactive, lacks integrated with the BIP (Behaviour Interaction Priorities) foresight, hybrid architectures emerged [5]. Those were [10]framework andtoolset,whichisusedtoformally usually designed as three tier systems (e.g. [1, 38]), specify and verify the correctness of the produced system, having: planning (deliberative layer), sequencing (usually treated as a real-time, component-based complex structure. implemented as a finite state machine governing the actions However, as the majority of architectures presented in the of the system), and device control layers (responsible for literature is not specified formally this discussion will not motor control and sensor data aggregation – a reactive layer rely on a formal description. resulting if both were combined). Behavioural systems [5, Frequently, hierarchical layered architectures are 6], based on reactivity, have gained widespread attention. designed. One of the oldest hierarchical architectures is In order to satisfy the requirement of the fastest possible the sense-plan-act (SPA) structure, used for instance by response, reactivity should be implemented at the lowest the Shakey robot in the 1960s [71]. Shakey sensed its level. The shortest possible loop starts with proprioceptors 792 J Intell Robot Syst (2019) 93:787–805 and directly influences the real effector. As the activities of decomposed into concurrently computed functions that the reactive layer rely mainly on sensor input and to a very define the computations of individual motor schemas. The limited degree on internal state (i.e. memory) it is very easy transition function composes the results produced by partial to reproduce this layer within an embodied agent, where the transition function into an outcome that is dispatched to quickest responses are organised within the virtual effector the virtual effector. A more formalised approach to Schema and are based on proprioceptive input. An example of this Theory is called Robot Schemas (RS) [63]. RS form a approach is the utilisation of force/torque sensing directly formal model of distributed computing in the robotics within the virtual effector (e.g. [97]). If a subsumption domain. The model is built of concurrent computing agents, architecture [13, 15] is to be reproduced, its modules can each being a Schema Instance (SI). SIs communicate be represented by blocks within the data flow diagram through ports, with which data types are associated. At definition of a transition function by using inhibiting and SI instantiation time, its ports are connected to the ports suppressing links (as presented in [54]). Otherwise, the of other SIs. Communication occurs when an SI writes modules can deliver the results of their computations into to one of its output ports. Basic schemas are defined by a composition unit, which will compute the final outcome, the port connections, local variables and the procedural subsequently dispatched to a real effector for execution. If behaviour section, which specifies how the schema reacts a slower reaction is permissible, exteroceptive input from to input. Basic schemas, once initiated, iteratively execute the virtual receptors can be used by the control subsystem, the behaviour section, synchronously reading or writing which will use the same scheme of operation as the to ports, performing computations on local variables and virtual effector. Thus, the same pattern will be reproduced input ports and instantiating and deinstantiating other at different levels of the control hierarchy. This can be SIs. Complex schemas, i.e. assemblages, are a recursive continued at even higher levels of the hierarchy by using composition of basic schemas into ever more complex extra computational agents [98]. ones. Assemblages do not have the behaviour definition Schema Theory was formulated for the purpose of part, as that is defined by the schemas they are composed representing systems created to study both artificial of. Only the port connection part exists. SIs form a intelligence and brain theory [2, 3, 5, 62, 63]. Schema dynamical network which grows or shrinks as the activities Theory provides guidance to building models of animal of the system take place. The task is represented as and artificial agent behaviour, stating that behaviour is the the outermost assemblage. Thus, the system is composed result of competition and cooperation between schemata, of sensory schemas, motor schemas and schemas that which are conglomerates of knowledge and methods of connect them, i.e. task schemas. In general, this reflects the processing [2]. Instances of schemas are created either structure of an embodied agent or can be reproduced as by other schemas or as a reaction to sensory stimulus. a network of embodied agents with some of their internal Perceptual schemas process sensory input, whilst motor components missing. It should be noted that the design of schemas produce action patterns. A network of schemas an RS-based system focuses on computation, however this is called an assemblage. The overall behaviour of an involves also instantiation and deinstantiation as well as agent is produced as a result of interactions between communication, all considered at the same computational schemas composing the network. Such schemas can be level. Thus, the basic level is flat and all aspects of embedded in an embodied agent, as virtual receptors reflect system activities are entangled, hence intelligibility is perceptual schemas, whilst virtual effectors reflect motor sacrificed. Hierarchy emerges from assembling lower level schemas. At a higher level, embodied agents without schemas into higher level ones, but as the design principles effectors match perceptual schemas, whilst those without of the lowest level are not expressed explicitly the receptors match motor schemas, this multi-agent networks creation of such systems is difficult. An embodied agent can also reproduce assemblages. The application of Schema introduces a hierarchy of concepts starting from the lowest Theory in robotics followed different paths. One of them level: transition function and terminal condition (purely is described in [3, 5]. In this case, motor schemas are computational), behaviour (adds communication), FSM basic units of behaviour. Motor schemas act in conjunction (selects behaviours), subsystems (distinguish perception, with embedded perceptual schemas. Concurrently acting action and task control), agents (form networks of any size). behaviours, i.e. motor schemas, compute individual results Each of those concepts can be dealt with separately, thus that are composed together into the final control outcome. it is easier for the designer to focus on a single entity at a This architecture is easily reproduced in an embodied agent. time, simplifying the process of system creation. Systems Perceptual schemas are reproduced by virtual sensors. composed of embodied agents, besides being capable of The control subsystem implements a behaviour that is reproducing the well known architectures, can be used to parameterised by a transition function which in turn is create other architectural patterns [94, 98]. J Intell Robot Syst (2019) 93:787–805 793 5 Implementation of an Embodied Agent with whisker sensors. Autonomous mobile robots dealing with hazardous material spill emergency situation used a The architecture of an embodied agent presented in blackboard system to integrate information from various Section 3 is implemented using several earlier-mentioned sensor-based and knowledge-based subsystems [73]. Those concepts. Here we shall concentrate on two important examples and the plethora of other works on the utilisation ones associated with the communication and behaviour of blackboards in robotics show that the effort has selection [31]: the whiteboard, which is used as the main concentrated on either different aspects of knowledge communication means between subsystems, and Logic- processing or on providing a balanced access to knowledge Labelled Finite State Machines (LLFSMs), which govern to all involved actors or on both. The concept of the actions of each of the subsystems present in the agent, a whiteboard is derived from blackboards, however it including the exchange of information between subsystems. focuses on information sharing, thus the communication The whiteboard is our implementation of the general aspect. The contents of the whiteboard is of secondary concept of a blackboard [42], thus whenever we refer here importance as is the purpose of information processing. to the former we refer to our implementation and for the The focus is on providing efficient communication means. general concept, the latter is referred to. A similar communication concept was introduced in The concept of a blackboard was created for the [1], namely a poster. It uses shared memory for the purpose of studying artificial intelligence (AI) [69]. The purpose of communication. A poster is a structured shared blackboard model consists of three major components [22, memory only writable by its owner and readable by 43, 70, 82]: knowledge sources (KSs) – corresponding any element of the architecture. Posters are completely to the domain experts, blackboard data structure – the independent from each other. They are not regions of a global resource where information to be shared is written, common blackboard and cannot be written to by many and control mechanism – responsible for processing the producers, thus they do not have all the properties of knowledge contained in the blackboard and ensuring that a blackboard. A whiteboard has all the communication only one domain expert has access to the blackboard abilities of a blackboard, so it is more general than at a given time. The blackboard can be any data a poster. structure accessible by all the KSs. Each KS operates in the following sequence: reads the information from 5.1 Whiteboard the blackboard, acts based on the information found and writes its conclusions and hypothesis into the blackboard. The gusimplewhiteboard (Griffith University Simple Whiteboard) is a library for organising inter-process The work on blackboards in AI concentrated on several aspects: the organisation of knowledge data structures, communication [31]. It implements a mutation of the reasoning utilising those data structures as well as providing general blackboard concept [42]. It enables the transfer of access to them. The concept of the whiteboard focuses C++11 objects through shared memory. As it is linked in, on data access. The use of data-centric communication no broker is necessary. The message transfer is idempotent, similar to blackboards has transcended AI and found its i.e., if the transfer is repeated the result does not change. No way into the wider software engineering community. The queues need to be involved, however, if necessary, they can Data Distribution Service (DDS) is a standard formulated be organised, as the whiteboard does not determine the data by the Object Management Group (OMG), implementing structures it will be storing – this is the choice of the system Data-Centric Publish/Subscribe (DCPS) communication designer using the whiteboard. model [40, 58]. The main purpose of DDS is to Shared memory communication exhibits certain advan- tages over the traditional approach taken by, e.g. ROS assure dependable communication in dynamic, networked environments between distributed producers and consumer messages and services, [47]. Producers and consumers use fulfilling real-time constraints. Whiteboard implements local instances of classes to communicate. When a producer the pull approach, whilst the DDS, which is based on sends a message, it copies an object containing the mes- Publish/Subscribe model, is based on the push approach. sage from its local store to the static instance of the class Blackboards have also been used in robotics. The in the gusimplewhiteboard. When a consumer wants following examples show only a fraction of the diversity to retrieve the message, the current instance of the object of their applications. A reconfigurable mobile robot used in the gusimplewhiteboard is transferred to the local a blackboard enabling a horizontal system decomposition memory of the consumer. When operating in a distributed allowing the cooperation between all the subsystems system, a UDP-based sharing algorithm that also makes use considered as knowledge sources executing a specific task of the Time-Triggered Architecture (TTA) [52]isusedto [72]. The CrunchBot [37] used a Bayesian blackboard to broadcast data to other receivers [61]. The implementation solve the mapping problem for a mobile robot equipped of the distributed version of the whiteboard uses efficient, 794 J Intell Robot Syst (2019) 93:787–805 fault-tolerant communication for distributed systems as pre- utilising the concept of a blackboard still relies on the sented in [11, 51, 57][52, 53]. Compared to the DCPS [58], push model. For instance, ROS topics act in push mode. the whiteboard does not offer inherent mechanisms for dis- However the blackboard provides an opportunity to base tributed reconfiguration, but offers a statically defined data the communication on the pull model. Whilst the proposed classes and O(1), fail-silent communication within bounded whiteboard can be used in both push and pull modes, the real-time constraints [31]. A more detailed discussion of the latter should be the method of choice. In the majority of networked version of the whiteboard [61]ispresentedin middleware, the structure of the contents of the messages Section 6. transferred is limited, whilst in gusimplewhiteboard any C++11 objects can be used [31]. 5.2 Communication Model 5.3 LLFSM The blackboard can be considered as a broker, which enables the producer to depose in it, in a non-blocking way, LLFSMs are FSMs, in which transitions from one state to whatever data it has to send out, whilst the consumer may another state are based on the fulfilment of logic expressions act in one of the following modes. Either it acts in the rather than events. As events are usually queued and logic push mode (similar to the DCPS), where the blackboard expressions are computed just on the basis of current invokes a specific function of the consumer-subscriber, thus data, as is more appropriate for robot control, this form notifying it of the arrival of new data, or in pull mode, of an FSM is preferred. In the implementation presented where the consumer accesses the data when it requires. The here, the LLFSMs, in principle, operate concurrently, former usually involves queuing of messages, whilst the but in reality are scheduled sequentially, thus critical latter implies the delivery of only the most current data. The sections and synchronisation points become unnecessary. push model implies close coupling of modules, where the This inherently removes the possibility of race conditions. subscriber must react to all messages issued by the producer The communication between LLFSMs uses whiteboards. [30]. In the pull model, the components are decoupled, the The utilised LLFSMs are synchronous automatons. With consumer may react in its own time, and only to some every state, three portions of code are associated: the of the produced messages. In both communication models, code that is executed on entering the state, the exit code the publisher introduces a message into the blackboard. invoked when the logic expression with one of the outgoing In the case of the push model the blackboard notifies transitions (directed arcs) is satisfied, and the code that is the consumer that the message it has subscribed to is executed when none of those logic labels is true, hence the available and the subscriber invokes a call back function execution of this code may be repeated many times. in response. The messages are queued, so the consumer The LLFSMs can be composed hierarchically. Each should somehow react to all of them. An event driven LLFSM can be suspended, thus enabling the execution architecture results. It relies on an assumption that there is of a subLLFSM. It can also be restarted from the state enough of computational power to handle the messages as it was suspended in or restarted from its initial state . they come. However, in the case of the pull model, such an The subLLFSMs act like subprograms and need not act assumption is not necessary. The consumer is not notified simultaneously – this prevents state replication [19]. The about the arrival of a new message. When the consumer higher level LLFSM becomes dormant during the actions needs the new message it retrieves form the blackboard the of the subLLFSMs, increasing the degree of determinism of most current one, neglecting the obsolete ones (in reality the system performance. usually they will even not be available). The publisher and From what has been said here, it is evident that the s B implementation of both the subsystem FSM F and subscriber are very loosely coupled, without assumptions as j,v s B to the available computational power, and thus the ability to the behaviour subFSM F (specifying the behaviour j,v,ω respond timely to the incoming stream of messages results. F ), as LLFSMs is straightforward, as the directed j,v,ω Obviously, the pull model works on the principle that we get arcs of those FSMs are labelled by predicates, i.e. logic a better response, if the system at any moment uses the most expressions, hence their equivalence is ascertained. current data. Both the publisher and the subscriber act with their own frequencies, without having to synchronise their 5.4 Scheduler activities. This way, no global timing schedule is required, fully decoupling the development of both components. The An embodied agent is composed of one control subsystem pull model is immune to messages being lost by definition, and usually several virtual effectors and receptors. Each of as it does not require response to every message, whilst the push model requires delivery guarantees, because every Incidentally, this property also enables the implementation of the message attracts attention. The majority of middleware subsumption architecture [13] by using those automatons [19]. J Intell Robot Syst (2019) 93:787–805 795 those subsystems is governed by one FSM – here imple- condition of each behaviour is always fulfilled, whilst mented as an LLFSM. The proposed solution schedules error condition is not. This implies that all behaviours sequentially the LLFSMs representing producers and con- are executed only once in each state of the FSM. The sumers, thus minimising the problems of determining the functioning of the scheduler is presented assuming that s 1 s delays in servicing the transfer of data. The majority of mid- initially FSM F is in the state S and the FSM F is j,v j,h j,v dleware, e.g. ROS, Microsoft Robotics Studio, relies on con- in the state S . The scheduler activates one state of each j,h currency, data queuing and the non-deterministic TCP/IP FSM or subFSM at a time, switching between all FSMs suite. Hence real-time performance is compromised. Deter- and subFSMs. Obviously this causes the execution of the ministic scheduling has the following advantages over the code associated with that state. In this case the scheduler 1 s 1 non-deterministic multi-threaded approach. Full concur- starts with state S of F and then switches to S of j,v j,v j,h rency, where switching between threads is operating system s 1 s 1 F .Being in S F transits to S of the subFSM j,h j,v j,v j,v,1 dependent, is much more difficult to envision and correctly executing the behaviour B , so this will be the next state j,v,1 apply by the robot control system designer, as it necessi- activated by the scheduler. The fourth state activated will be tates: thread synchronisation, vigilance of nondeterministic 1 s S of the subFSM F . The full switching sequence j,h,1 j,h,1 communication delays, managing critical sections, prob- s s for both FSMs F and F and their subFSMs executing j,v j,h lems with fairness as well as avoiding deadlocks, live-locks, behaviours is presented in Fig. 4. and starvation. Moreover, CPU context switch overhead has Whiteboard communication in conjunction with this to be taken into account. The sequential LLSFSM switching way of switching the LLFSMs by the scheduler avoids procedure enables fairly simple formal correctness veri- deadlocks, because only one FSM accesses the whiteboard fication of models, as it avoids the usual combinatorial at a time. Moreover, because the variables contained in the explosion of concurrent threads. The sequential switching whiteboard are located in cyclic buffers, always there is is time-triggered in contrast to the frequently used event- available data that can be accessed, alas not always the most triggered approach, where due to peak-load situations it is current. often impossible to assure reliability. Specifications using LLFSMs constitute executable models that can be formally 5.5 Communication Organisation verified. A more detailed treatment of this subject can be found in [29, 32, 33]. As in many cases, the example that follows requires only Each iteration of an LLFSM operation starts with reading intra-system communication between subsystems. Whilst the relevant variables from the whiteboard and it ends with the utilised whiteboard is capable of providing inter-system depositing the results of computations in it. Each iteration communication [61], this is not our focus here. It should is treated as atomic, thus no inconsistencies can arise. The be noted that the whiteboard intra-system communication LLFSMs are executed in a round-robin fashion one iteration gives real-time guaranties (when the underlying operating of each LLFSM at a time. system is real-time), whereas the inter-system one would To show how the scheduler works in conjunction with the depend on the network protocol. LLFSMs, a simple example is presented here. It is assumed The whiteboard is a table with slots. Each slot creates a that within the robotic system, only two subsystems v separate communication channel containing a cyclic buffer. and h exist, where v and h are the specific names of the Hence, buffered communication based on a cyclic buffer of subsystems. The exemplary activities of each subsystem limited size is realised. The whiteboard as a whole supports s s may be represented by hierarchical FSMs: F and F , j,v j,h concurrent many-to-many communication, but a message presented in Fig. 3. slot (buffer) with multiple producers requires locking and With each FSM state a behaviour is associated. It is is therefore not recommended. We therefore recommend represented as a subFSM based on behaviour template from one-to-one or one-to-many communication, which is Fig. 2. For simplicity it was assumed that the terminal s s Fig. 4 Sequence of state activations of FSMs F and F and their j,v j,h Fig. 3 The activities of two subsystems v and h of a j subFSMs executing particular behaviours 796 J Intell Robot Syst (2019) 93:787–805 lock-free with O(1) complexity for access operations. In necessarily convenient for the subscriber. In congested sys- the following example one-to-one communication will be tems using TCP, if the acknowledgments do not reach the utilised. The LLFSMs together with the whiteboard enable publisher on time, the packets will be retransmitted, thus the the implementation of both blocking and non-blocking subscriber will eventually be flooded with repeated packets communication modes both in the case of the producer and [77]. This means that the subscriber activity is tightly cou- the consumer. However, in the presented example both the pled with the publisher update rate. Additionally, the push producers and the consumers operate in the non-blocking approach assumes that the subscriber reacts to transmitted mode. In the experiment, a single sequential scheduler is data immediately (otherwise data is queued), thus the data is used, thus subsystem LLFSMs are executed sequentially. presumed to be fresh and as recent as possible, what might not be the case. This will result in reaction delays. The neg- ative influence of such delays on the performance of a drone 6 Comparison of Whiteboard and ROS are presented in [47]. Timeliness, sequencing and reliabil- Communication Mechanisms ity of the data are taken for granted. Such scenario is very optimistic, which in fact, does not occur in reality often. Initially, ROS was designed for large service robot control Therefore, it requires that the producer must eventually be software, which was meant for implementation on several slowed down or complex handshaking protocols must be computers connected via Ethernet [78], thus ROS was deployed, otherwise the data will be lost [30]. The push in fact designed for communication between computers. paradigm, used by topics, does not assure idempotent trans- However later it was mainly used for communication fer of messages, thus an additional effort must be done to within a single computer. Hence below we introduce ROS ensure that the data was received exactly once [30]. communication mechanisms provided for a single computer The ROS service is a simple RPC, thus the client is and compare them with the whiteboard approach. blocked until it obtains the response from the server. This ROS uses: topics, services, nodelets, and a parameter type of communication has three additional features. It server. Those communication concepts may be utilised uses only plain old data (POD) structures, collections of only for certain types of communication (based on the basic types, thus object oriented concepts using inheritance localisation of the communicating nodes), i.e. inter-process and polymorphism can hardly be modelled [27], whilst the and intra-process. The inter-process communication occurs approach based on whiteboards implements object oriented between ROS nodes (implemented as processes) utilising approach [30]. Additionally, ROS service forces client- TCP/IP server socket [80] and peer-to-peer topology, server relationship between the two communicating nodes, which requires some sort of lookup mechanism to allow possibly resulting in delays due to waiting for the server processes to find each other at runtime. This mechanism reply. Moreover, the ROS service again uses the TCP/IP is implemented as the ROS master [78]. ROS nodes may communication protocol. communicate with each other utilising topics, services and Parameter server is the last of the inter-process ROS parameter service. communication concepts for intra-system inter-process A ROS topic is a named bus implementing communi- communication. It is a shared, multi-variate dictionary cation between nodes. Communication between two nodes stored in the ROS master. It is accessible by nodes via may only occur if both nodes exist and are already regis- network APIs. However, it was not designed to be high- tered with the ROS master (utilising Remote Procedure Call performance, as it is to be used for static data such as (RPC)). Nodes communicate utilising TCP. Since a ROS configuration parameters [79]. topic uses a publish/subscribe mode of operation, thus it Nodelets are threads that use intra-process communication implements the push paradigm, which forces the subscriber of ROS. Data is exchanged between nodlets executed within to act in an event-driven fashion. Events are deposited into the same process by sharing memory. However, the develop- the middleware which relays them to call-backs from the ers have to take care to avoid contention due to concurrent subscribers. It is assumed that there will be sufficient com- access to the data, e.g. by using mutexes or by allocat- putational resources to enact all the threads generated and ing new memory whenever new data is written. This again to execute the subscriber callbacks. An additional assump- results in a large performance penalty as shown in [27]. tion is that events would occur with enough sparsity that There exist robotic systems that have special communi- the call-backs would be completed by the time they are exe- cation requirements. In some of them, both the producer and cuted again or alternatively require handling of concurrency the consumer should be able to operate in a non-blocking issues. This event-driven approach also results in coupling fashion (i.e. the communicating nodes should be decou- and has timing consequences. The push approach compels pled), as blocking implies possible lack of reaction from the the subscriber to keep up with message reading in order system, when such a reaction is necessary. The communi- not to lose any data, even if a specific update rate is not cation should be capable of operating in the time-triggered J Intell Robot Syst (2019) 93:787–805 797 mode, because in this mode of operation the system can and Manager – interacting with the whiteboards. Due to the react both to the message and its lack (usually those reac- fact that whiteboars contain cyclic buffers that are transmit- tions are different), i.e. lack of a message is also a message. ted as a whole, even if (due to the use of UDP) a transmitted It is required that the communication be low latency and low message is lost, in the next transmission period both the jitter in the case of intra-system transmission, as quick reac- most recent data and previous data will be delivered. tions to events occurring in the environment may influence the survival of the robot. Finally, communication should be implemented in the same way both for threads and pro- 7 Experiment cesses, because the imposition of single process system structure (even with many threads) is not acceptable, i.e. The robot system design methodology described in, e.g. multi-process implementation is treated as an advantage. [92], utilising hierarchical FSMs [35], has been applied None of the four ROS communication methods fulfils to the design of the control system of a mobile robot all of these requirements. The parameter server introduces collecting table-tennis balls. The robot (inspired by [64]) high latency. ROS services use RPC, thus they block both uses a suction mechanism to collect table-tennis balls and the producer and the consumer. In the case of ROS topics, a camera to detect them, as well as a sonar to avoid the consumer is activated by an event associated with obstacles. Based on the estimated ball position, the robot the producer depositing a message in the communication moves in the direction of the closest ball and vacuums it channel, thus communication is event triggered, not time into its container. This simple, yet useful, robot is used triggered. However non-blocking operation of the consumer here for the purpose of exemplifying the proposed general using ROS and C++ can be realised through a separate robotic system specification method and the implementation thread dealing with acquisition of the message and using of the resulting system. Both the system architecture and nodlets – rather a complex solution introducing significant the behavioural approach used resemble the well known communication delays. Unfortunately, nodelets can be used solutions presented in Section 4. As the purpose of the only for communicating threads, but not processes. As example is the presentation of the proposed specification none of the ROS communication mechanisms satisfies the and implementation method, a well known architectural above-mentioned requirements, the whiteboard was chosen solution and control paradigm have been used, to enable the (in fact, a ROS package for the whiteboard is available reader to focus only on the methodology. at [67]). This approach provides intra-system inter-process communication, which does not block neither the producer 7.1 Specification nor the consumer; as it operates on common memory it is very low-latency [31, 47]; by using the pull mode of The specification describes both the structure of the operation it operates in time triggered mode; and moreover designed system and its activities. For the purpose of it can be used both by threads and processes in the same briefness, only relevant excerpts from the full system fashion. specification are presented here. Only a single behaviour of If whiteboards could be used only for intra-system commu- the control subsystem is specified formally here to provide nication the proposed approach would not be scalable. How- an example – the others follow the same pattern. ever there exists a network implementation of the white- Structure It suffices to represent the structure of the ball boards [61], but here the latency introduced both by them collecting robot (bc) by a single embodied agent a (Fig. 5). bc and ROS are similar. In general, network based implementa- This agent contains: the control subsystem c and four tion of the concept of the whiteboard requires that each com- bc puter in the system instantiates its own whiteboards. One of those whiteboards contains variables for local communi- cation whilst the others reflect the variables in the remote whiteboards. The latter variables are updated periodically broadcasted to other computers in the network utilising the UDP Bridge module. The UDP Bridge module is used for all network connectivity and connects directly to the shared data structures of the whiteboard. The UDP Bridge can either be run as an independent process (for debugging purposes) or can be started as a thread by an instance of the Remote Whiteboard class. The UDP Bridge has three main components: Listener – listening for incoming pack- Fig. 5 Embodied agent representing the table-tennis ball collecting ets, Broadcaster – broadcasting packets when scheduled, robot 798 J Intell Robot Syst (2019) 93:787–805 virtual subsystems: r , r , e , e Activities The behaviour of a is defined by the activities bc,cam bc,sonar bc,body bc,vacuum bc and four corresponding real subsystems: R , R , of its subsystems and the interactions between them. The bc,cam bc,sonar E and E . The real receptor R detects activities of c are defined using a hierarchical FSM bc,body bc,vacuum bc,cam bc c c balls, whilst the real receptor R detects obstacles. The F presented in Fig. 6. Each state of the FSM F is bc,sonar bc bc real effector E controls four motors to which wheels associated with the corresponding behaviour: B bc,body bc,search are attached (a skid-steering mobile platform is used), (robot searches for the balls within the environment), c c whereas E controls the vacuum cleaner providing B (robot collects balls) and B (robot avoids bc,vacuum bc,collect bc,avoid suction. R acquires images of the environment and obstacles). Each behaviour is modelled using the general bc,cam sends them to the virtual receptor r , so that it can behaviour template presented in Fig. 2, requiring the bc,cam detect table-tennis balls and subsequently inform the control definition of a transition function, terminal condition, and subsystem c whether it detected at least one ball. If a ball the determination of the communication model. Below, bc is detected r sends to c an estimated position of the the definition of the behaviour B is presented, bc,cam bc bc,collect ball within the image space. R provides to the virtual thus transition function f and terminal condition bc,sonar bc,collect receptor r the measured time (i.e. interval between f are presented. Other behaviours are defined in a bc,sonar bc,collect the emission of the ultrasonic signal and receiving its echo), similar way. based on which it calculates the distance to the obstacle. Canonical decomposition of the transition function Next it sends the information to c whether an obstacle f , based on the components of the output buffer, is bc bc,collect was detected. The c commands both virtual effectors as follows: bc e and e using the information obtained from bc,body bc,vacuum c,e r and r . Finally e controls E , whilst f bc,cam bc,sonar bc,body bc,body bc,collect e i+1 c [cmd]= MOVE e controls E . ⎪ y bc,body bc,vacuum bc,vacuum i+1 ⎪ e r i r i c [d x]= c [x]− c [w]/2 Each subsystem has its own input, output and internal ⎨ y x x bc,body bc,cam bc,cam i+1 e r i c [vel]= calculate vel( c [r]) buffers. Below only those control subsystem c buffer y x bc,body bc,cam bc ⎪ i+1 e r i c i c [cmd]= TURN ON, for c [r]  c [r min] y x contents are introduced, which are relevant for this partial ⎪ bc,vacuum bc,cam bc i+1 r e i c i c [cmd]= TURN OFF, for c [r] < c [r min] y x presentation of the specification. bc,vacuum bc,cam bc (1) – Internal memory c : bc r min – minimum ball size triggering the vacuum The internal memory contains only constants, thus it does cleaner (constant) not require a transition function. The virtual receptors do not e have input buffers fed by the control subsystem, thus those – Output buffer c : bc,body transition functions are not needed. cmd–command for e , bc,body The partial function (1) calculates the parameters of cmd ∈{MOVE, LEFT, RIGHT, STOP}, the control commands for e and e .Inevery bc,body bc,vacuum d x – X coordinate of the location of the centre of c iteration of the behaviour, B a new desired velocity bc,collect the ball with respect to the image centre (expressed of the robot is calculated using the function calculate vel, in pixels) which takes into account the size of the detected ball vel – velocity of the robot (range: 0–100) image. Using the X coordinate of the the centre of the e ball with respect to the centre of the image, an offset is – Output buffer cbc, vacuum: computed that is used to navigates the robot towards the ball. cmd–command for e , bc,vacuum cmd ∈{TURN ON, TURN OFF}, – Input buffer cbc, sonar: obst det – signals whether an obstacle was detected or not; – Input buffer cbc, cam: ball det – ball was detected or not; x, y – X and Y coordinates of the detected ball centre (in pixels) w, h – image width and height (in pixels) r – detected ball radius (in pixels) Fig. 6 Activities of the control subsystem represented by an FSM F bc J Intell Robot Syst (2019) 93:787–805 799 The behaviour stops iterating when the terminal condition received from R . The virtual effectors: e bc,sonar bc,body c τ e e f is fulfilled: and e execute: B and B , bc,vacuum bc,body,move bc,vacuum,set bc,collect respectively. B using its transition function bc,body,move c τ r r transforms the commands received from c into PWM f  ¬ c [ball det]∨ c [obst det] bc bc,cam bc,sonar x x bc,collect values and sends them to E . B transforms bc,body bc,vacuum,set Other two behaviours are specified similarly. Behaviour the commands obtained from c to an appropriate PWM bc B is responsible for moving the robot in search for value and dispatches it to E . Only two states of the bc,search bc,vacuum the balls. It terminates when either a ball or an obstacle vacuum cleaner are considered: turned on or off, thus PWM is detected. Whenever an obstacle is detected behaviour is either of maximum value or zero. B is invoked. It uses the bug algorithm [59]for The intra-system communication between subsystems bc,avoid obstacle avoidance. of a is realised as one-to-one communication. The bc All other subsystems of a , i.e. virtual receptors and communication between c and its virtual subsystems uses bc bc virtual effectors, are governed by single state FSMs. The cyclic buffers. Both producers and consumers use non- associated behaviours execute in their endless loops the blocking mode of operation, thus subsystems must react subFSM template presented in Fig. 2 (thus the terminal appropriately to lack of new data. conditions of those behaviours are always false). The virtual executes the behaviour B .The receptor r 7.2 Implementation bc,cam bc,cam,detect transition function of this behaviour uses the RGB image loaded into the buffer r connected to R , The above defined subsystems are implemented using the bc,cam bc,cam transforms this image into the HSV space, cheques whether MiCASE modelling tool [28]. It enables the high-level pixels are in the appropriate colour range, erodes the image, specification of the activities of the software-implemented dilates it, finds contours, cheques the shape of the detected subsystem (i.e. control subsystem and virtual subsystems) contours, assuming that all circular contours represent balls, using hierarchical LLFSMs that constitute an executable so the one with the largest circumscribed area is used to model. To this end, a slight modification of the original s s calculate the estimated ball position within the image space. FSM F is required [35]. Each F is implemented as bc bc s  c Thus the transition function is defined as a composition of F . The transformation of F , presented in Fig. 6,into bc bc the above functions, as a result producing a pure function its implementation version F , presented in Fig. 7,is bc c ω (no side effects are involved). This is an intrinsic property straightforward. Each state of F , denoted as S (ω is the bc bc of all transition functions. Virtual receptor r executes designator of the behaviour associated with the considered bc,sonar r c B . Its transition function calculates the distance state of F ), is implemented as a pair of consecutive states bc,sonar,detect bc c  ω from the obstacle, based on the measured time of flight of F : the original state S (depicted as a rectangle org bc bc c c Fig. 7 Implementation of F FSM as F FSM modelling the activities of the control subsystem bc bc 800 J Intell Robot Syst (2019) 93:787–805 7.3 Robot Hardware The table-tennis ball collecting robot (Fig. 8) has four motors actuating the four wheels, an obstacle detecting sonar HC-SR04 with a 2–400 cm range, a ball detecting camera (Microsoft LifeCam Studio Webcam), a 12 V car vacuum cleaner producing suction, a ball container, a Raspberry Pi 3B on-board computer, Arduino Micro microcontroller board and a Xaxxon MALG PCB motor controller. The Pi 3B executes the software of the control subsystem and all virtual subsystems, sending commands to the real effectors: E (to the motors bc,body through the Xaxxon MALG motor controller), vacuum cleaner E (Arduino Mirco DC motor velocity bc,vacuum controller MCU-60127) and receives data from the real Fig. 8 Table-tennis ball collecting robot receptors: camera R (Microsoft LifeCam Studio bc,cam Webcam camera), sonar R (Arduino Micro and HC- bc,sonar SR04 sonar). The Raspberry Pi 3B is connected to both the with oval corners) and the supplementary state S wait bc Arduino Micro and Xaxxon MALG PCB using USB serial (depicted as a small oval). With each state S an action org bc port and to the camera by a SCCB interface (serial camera is associated, where a new submachine F is executed control bus). bc,ω by the restart at(SUBMACHINE) command. It models the c c behaviour B . The FSM F waits in the current bc,ω 7.4 Performance of the Communication bc ω B state S until the submachine F starts executing org within the Robot bc bc,ω (i.e. the Boolean expression is running at(SUBMACHINE) returns true). When the submachine F starts executing, If a real-time operating system is used, the execution bc,ω c  ω the FSM F changes its state to the state S . times of particular behaviour steps are as presented wait bc bc c  ω The FSM F being in the state S waits for the wait in Table 1. The produced system requires both intra- bc bc termination of activities of the subFSM F by testing system communication (i.e. between c and virtual bc bc,ω the Boolean expression is suspended at(SUBMACHINE) subsystems) and access to the hardware (i.e. between virtual (it returns true, if the submachine has finished its activities). subsystems and the real subsystems). Communication with The ball collecting robot system requires five FSMs, the hardware is organised as either: send operations of e e because it contains five software subsystems. Additionally, behaviours B and B , or receive bc,body,move bc,vacuum,set r r for each behaviour within each subsystem an additional operations of behaviours B and B . bc,cam,detect bc,sonar,detect subFSMs must be produced. Thus in total there are 12 Hardware access uses standard driver interfaces whilst intra- FSMs/subFSMs. Each of them is governed by the same, system communication utilises the whiteboard. The former single sequential scheduler, switching their activities. The communication uses USB serial port which introduces communication between subsystems is implemented using significant delays, hence those send/receive operations may the whiteboard [30]. take even several milliseconds. In the latter case, the Table 1 Maximum measured time [μs] required by behaviour steps: calculation of transition function f , sending and receiving data, where j,v vac is vacuum and wb is the whiteboard Behaviour f Receive Send j,v B 3wb→ c 16 c → wb 18 bc,search bc bc B 3wb → c 10 c → wb 29 bc,collect bc bc B 9wb → c 21 c → wb 14 bc,avoid bc bc B 20345 R → r 4720 r → wb 21 bc,cam,detect bc,cam bc,cam bc,cam B 2 R → r 1501 r → wb 9 bc,sonar,detect bc,sonar bc,sonar bc,sonar B 2wb → e 10 e → E 5093 bc,body,move bc,body bc,body bc,body e e B 3wb → e 9 → E 1384 bc,vacuum,set bc,wb bc,wb bc,vac J Intell Robot Syst (2019) 93:787–805 801 messages are transferred using shared memory, thus the time However for simple one-computer implementation of the required for execution a single send/receive operation is at system this is not necessary. The presented implementation the most 39μs(10μs+29μs). approach ensures that at a given time only a single The hardware out of which the simple robot was built to subsystem is active so there is no contention regarding data present both the design method and the software used for access. the implementation of the controller was inexpensive. The An embodied agent is a set of subsystems communicat- camera enables image acquisition at the rate of 30 Hz. Thus ing with the control subsystem utilising the whiteboard as a the virtual receptor r collects data every 34 ms. The communication means. The constraint imposed on the inter- bc,cam balls are detected by the camera at a maximum distance of nal structure of an embodied agent, which requires that all 2.5 m. The sonar enables the detection of obstacles in the virtual entities contact only the control subsystem, limits the range 2-400 cm, thus due to the velocity of sound in air, internal number of communication links. Each subsystem a data sample is obtained every 60 ms. As the maximum of an embodied agent is implemented as a separate process. velocity of the robot is 0.25 , the robot is able to stop An additional agent added to the system linearly increases and change the direction of motion well ahead of possible the number of utilised processes. The number of new con- collisions. The decisions as to the selection of the nearest nections required within the system is equal to the number ball to collect are also made well in advance, so the robot of the subsystems within the embodied agent and usually does not have to veer overly rapidly. The balls scattered low number of interconnections of the added agent with the between obstacles were collected efficiently. already existing agents, thus is scalable. Embodied agents can be implemented not only by using LLFSMs and the whiteboard, but also by utilising OROCOS 8 Conclusions components, however the transformation procedure is not that straightforward. OROCOS is a component based This paper extends the robotic system design methodology robotics framework, where each component encapsulates (presented in e.g. [35, 92, 96]) by inter-subsystem an executable algorithm (primary code). Within the communication model [31]. It assumes that the activities of component there exists an Execution Engine, which each subsystem and each of their behaviours is represented processes asynchronous operations, calls plug-in functions by a LLFSMs. The communication between the subsystems and executes the primary code, when the component is in takes place in specific states of the resulting hierarchic FSM, a running state. The FSM is related to the states of the being the composition of the higher level FSM defining the component (pre-operational, stopped, running, exception task of the subsystem and the lower level ones governing etc.) not the states of the executed tasks. Thus, the FSM the execution of this subsystem’s particular behaviours. The within the component is completely different from the FSM subsystems in those states access the whiteboard, providing in the embodied agent. The former changes the state of the or acquiring the necessary data. As the scheduler switches component, and only the running state is of relevance to us, the hierarchic FSMs sequentially no contentions arise. To as only in that state the task is executed. In the latter case, refer the proposed communication model to the state of the the FSM is related to the states of the task execution. With art, a classification of communication methods from the every one of those states a transition function parameterised robotics perspective has been introduced. behaviour is associated. Thus, if a subsystem would have The choice of LLFSMs and the whiteboard as an to be implemented as an OROCOS component both the implementation means for the proposed specification FSM and the behaviours would have to be recreated within method was dictated by the fact that the specification the primary code. Hence no benefit is obtained. Our multi- method utilises the concept of FSM and transition layer structure is being flattened within this code. The other function embedded in a behaviour, which organises the method of using OROCOS is to associate a component communication, i.e. the concepts into which it was easy with a behaviour. Then the primary code would represent to translate the specification. Importantly, the whiteboard a transition function. However then the subsystem FSM ensures reliable access to data. The whiteboard gives would have to be implemented by an extra component precise communication semantics in terms of bounded marshalling the other components (i.e. it would play the timing as well as atomicity. Moreover always the latest role of a scheduler). The activity of the agent would have to readings are delivered and no buffer overflow occurs be produced by still another component executing the FSM due to the use of a cyclic buffer. If only one scheduler of the control subsystem. However in such a system, not is employed the LLFSMs ensure reliable sequential only the communication between subsystems would have execution of the behaviours of the subsystems. Multi- to be organised, but also between behaviours (behaviours scheduler implementation on a multi-processor system is of subsystems of embodied agents have access to all the also possible, but then contention resolution has to be used. input, output and internal data of that subsystem). Thus, data 802 J Intell Robot Syst (2019) 93:787–805 transfers between behaviours would have to be arranged 12. Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm intelligence: From natural to artificial systems. Oxford university press, New through ports, which was not the intention of the model York (1999) employed by embodied agents. 13. Brooks, R.A.: A robust layered control system for a mobile robot. Both the utility of the design methodology and the IEEE J. Robot. Autom. 2(1), 14–23 (1986) employed communication method have been presented on 14. Brooks, R.A.: Intelligence without reason. Artif. Intell. Crit. Concepts 3, 107–163 (1991) a rudimentary robot searching for table-tennis balls. The 15. Brooks, R.A.: Intelligence without representation. Artif. Intell. system was implemented by using the MiCASE tool [28]. 47(1-3), 139–159 (1991) It enables both modelling of the activities of the designed 16. Brugali, D.: Software Engineering for Experimental Robotics. system and automatic code generation, resulting in an In: Brugali, D. (ed.): Stable Analysis Patterns for Robot Mobility. Software Engineering for Experimental Robotics, executable, high-level behaviour model. The resulting robot pp. 9–30. Springer Berlin Heidelberg, Berlin (2007). control system functions as specified. https://doi.org/10.1007/978-3-540-68951-5 2 17. Brugali, D., Agah, A., MacDonald, B., Nesnas, I.A.D., Acknowledgments This work was supported by the Erasmus Mundus Smart, W.D.: Software Engineering for Experimental Action 2 PANTHER (Pacific Atlantic Network for Technical Higher Robotics. In: Brugali, D. (ed.): Trends in robot software Education and Research) grant and by the National Science Centre, domain engineering. Software Engineering for Experimental Poland (grant number 2017/25/N/ST7/00900). Robotics, pp. 3–8. Springer Berlin Heidelberg, Berlin (2007). https://doi.org/10.1007/978-3-540-68951-5 1 18. Cabrera-Gamez, ´ J., Dom´ ınguez-Brito, A.C., Hernandez-Sosa, ´ D.: Open Access This article is distributed under the terms of the Sensor based intelligent robots. chap. CoolBOT: A Component- Creative Commons Attribution 4.0 International License (http:// Oriented Programming Framework for Robotics, pp. 282–304. creativecommons.org/licenses/by/4.0/), which permits unrestricted Springer Berlin / Heidelberg (2002) use, distribution, and reproduction in any medium, provided you give 19. Chen, D., Hexel, R., Raja, F.: Engineering Real-Time Commu- appropriate credit to the original author(s) and the source, provide a nication through Time-Triggered Subsumption: Towards Flexibil- link to the Creative Commons license, and indicate if changes were ity with Incus and Llfsms. In: Maciaszek, L., Filipe, J. (eds.) made. 11th International Conference on Evaluation of Novel Software Approaches to Software Engineering, pp. 272 – 281, Rome Publisher’s Note Springer Nature remains neutral with regard to (2016). https://doi.org/10.5220/0005915602720281 jurisdictional claims in published maps and institutional affiliations. 20. Chen, J., Gauci, M., Li, W., Kolling, A., Groß, R.: Occlusion- based cooperative transport with a swarm of miniature mobile robots. IEEE Trans. Robot. 31(2), 307–321 (2015). https://doi.org/ References 10.1109/TRO.2015.2400731 21. Cheriton, D.R., Skeen, D.: Understanding the limitations of causally and totally ordered communication. SIGOPS Oper. Syst. 1. Alami, R., Chatila, R., Fleury, S., Ghallab, M., Ingrand, F.: An Rev. 27(5), 44–57 (1993). https://doi.org/10.1145/173668.168623 architecture for autonomy. Int. J. Robot. Res. 17(4), 315–337 22. Corkill, D.: Blackboard systems. AI Expert. 6(9), 40–47 (1991) (1998) 23. Coste-Maniere, E., Simmons, R.: Architecture, the Backbone 2. Arbib, M.: Handbook of Physiology – The Nervous System II. of Robotic Systems. In: 2000. Proceedings. ICRA ’00. IEEE Motor Control, chap. Perceptual structures and distributed motor International Conference On Robotics and Automation, vol. 1, control, pp. 1449–1480. Wiley Online Library (1981) pp. 67–72 (2000). https://doi.org/10.1109/ROBOT.2000.844041 3. Arkin, R.C.: Motor schema-based mobile robot navigation. Int. J. 24. Dudek, G., Jenkin, M., Milios, E., Wilkes, D.: A taxonomy Robot. Res. 8, 92–112 (1989) for swarm robots. In: IROS ’93. Proceedings of the 1993 4. Arkin, R.C.: Cooperation without communication: Multiagent schema- IEEE/RSJ International Conference on Intelligent Robots and based robot navigation. J. Robot. Syst. 9(3), 351–364 (1992) Systems ’93, vol. 1, pp. 441–447 (1993). https://doi.org/10.1109/ 5. Arkin, R.C.: Behavior-Based Robotics. MIT press, Cambdrige (1998) IROS.1993.583135 6. Armbrust, C., Kiekbusch, L., Ropertz, T., Berns, K.: Soft Robot 25. Dudek, G., Jenkin, M., Milios, E., Wilkes, D.: Experiments Control with a Behaviour-Based Architecture. In: Soft Robotics, in Sensing and Communication for Robot Convoy Naviga- pp. 81–91. Springer (2015) tion. In: Proceedings. 1995 IEEE/RSJ International Conference 7. Asama, H., Ozaki, K., Matsumoto, A., Endo, I.: Development On Intelligent Robots and Systems 95. ’Human Robot Inter- of task assignment system using communication for multiple action and Cooperative Robots, vol. 2, pp. 268–273 (1995). autonomous robots. J. Robot. Mechatron. 4(2), 122–127 (1992) https://doi.org/10.1109/IROS.1995.526171 8. Balch, T., Arkin, R.C.: Communication in reactive multiagent 26. Dudek, G., Jenkin, M.R.M., Milios, E., Wilkes, D.: A taxonomy robotic systems. Auton. Robot. 1(1), 27–52 (1994) for multi-agent robotics. Auton. Robot. 3(4), 375–397 (1996). 9. Beetz, M., Mosenlechner, ¨ L., Tenorth, M.: CRAM – a Cognitive https://doi.org/10.1007/BF00240651 Robot Abstract Machine for Everyday Manipulation in Human 27. Einhorn, E., Langner, T., Stricker, R., Martin, C., Gross, Environments. In: IEEE/RSJ International Conference on Intel- H.M.: MIRA - Middleware for robotic applications. In: 2012 ligent Robots and Systems, IROS, October 18–22, 2010, Taipei, IEEE/RSJ International Conference on Intelligent Robots and Taiwan, pp. 1012–1017. IEEE (2010) Systems, Vilamoura, pp. 2591–2598 (2012). https://doi.org/10. 10. Bensalem, S., de Silva, L., Ingrand, F.R.Y.: A verifiable and 1109/IROS.2012.6385959. http://ieeexplore.ieee.org/stamp/stamp. correct-by-construction controller for robot functional levels. J. jsp?tp=&arnumber=6385959&isnumber=6385431 Softw. Eng. Robot. 2(1), 1–19 (2011) 28. Estivill-Castro, V., Hexel, R.: Arrangements of finite-state 11. Birman, K., Schiper, A., Stephenson, P.: Lightweight causal and machines - semantics, simulation, and model checking. atomic group multicast. ACM Trans. Comput. Syst. 9(3), 272–314 In: Proceedings of the 1st International Conference on (1991) J Intell Robot Syst (2019) 93:787–805 803 Model-Driven Engineering and Software Development - Volume 44. Haykin, S. Communication Systems, 5th edn. Wiley Publishing, 1: MODELSWARD, pp. 182–189 (2013). https://doi.org/10.5220/ Hoboken (2009) 0004317101820189 45. Huntsberger, T.L., Trebi-Ollennu, A., Aghazarian, H., Schenker, 29. Estivill-Castro, V., Hexel, R.: Correctness by Construction with P.S., Pirjanian, P., Nayar, H.D.: Distributed control of multi- Logic-Labeled Finite-State Machines – Comparison with Event- robot systems engaged in tightly coupled tasks. Auton. B. In: Zhu, L., Steel, J. (eds.) Proceedings of 23Rd Australasian Robot. 17(1), 79–92 (2004). https://doi.org/10.1023/B:AURO. Software Engineering Conference (ASWEC), pp. 38–47. IEEE 0000032939.08597.62 Computer Society Conference Publishing Services (CPS), Mil- 46. Jennings, N.R., Sycara, K., Wooldridge, M.: A roadmap of agent sons Point (2014) research and development. Auton. Agents Multi-Agent Syst. 1(1), 30. Estivill-Castro, V., Hexel, R.: Simple, Not Simplistic — the 7–38 (1998). https://doi.org/10.1023/A:1010090405266 Middleware of Behaviour Models. In: Filipe, J., Maciaszek, 47. Joukoff, D., Estivill-Castro, V., Hexel, R., Lusty, C.: Fast MAV L.A. (eds.) ENASE 10 International Conference on Evaluation Control by Control/Status OO-Messages on Shared-Memory of Novel Approaches to Software Engineering, pp. 189–196. Middleware. In: Robot Intelligence Technology and Applications INSTCC, SciTePress, Av. Dom Manuel i, 2910-592, Setubal ´ 4 - Results from the 4Th International Conference on Robot (2015) Intelligence Technology and Applications, RiTA 2015, Advances 31. Estivill-Castro, V., Hexel, R., Lusty, C.: High Performance in Intelligent Systems and Computing, Vol. 345, pp. 195–211. Relaying of C++11 Objects across Processes and Logic-Labeled Springer (2015) Finite-State Machines. In: Brugali, D., Broenink, J.F., Kroeger, T., 48. Jung, D., Zelinsky, A.: Grounded symbolic communication MacDonald, B.A. (eds.) Simulation, Modeling, and Programming between heterogeneous cooperating robots. Auton. Robot. 8(3), for Autonomous Robots - 4th International Conference, SIMPAR 269–292 (2000). https://doi.org/10.1023/A:1008929609573 2014, Lecture Notes in Computer Science, vol. 8810, pp. 182–194. 49. Kiekbusch, L., Armbrust, C., Berns, K.: Formal verification of Springer (2014) behaviour networks including sensor failures. Robot. Auton. Syst. 32. Estivill-Castro, V., Hexel, R., Rosenblueth, D.A.Wang, P. (ed.): 74, 331–339 (2015) Efficient Model Checking and FMEA Analysis with Deterministic 50. Kiekbusch, L., Armbrust, C., Berns, K.: Formal Verification of Scheduling of Transition-Labeled Finite-State Machines. IEEE Behaviour Networks including Hardware Failures. In: Intelligent Computer Society Conference Publishing Services (CPS), Wuhan Autonomous Systems 13, pp. 1571–1582. Springer (2016) (2012) 51. Kopetz, H.: Should responsive systems be event-triggered or 33. Estivill-Castro, V., Hexel, R., Rosenblueth, D.A.: Efficient time-triggered? IEICE Trans. Inf. Syst. 76(11), 1325 (1993) Modelling of Embedded Software Systems and Their Formal 52. Kopetz, H. Real-Time Systems - Design Principles for Distributed Verification. In: Leung, K.R., Muenchaisri, P. (eds.) The 19Th Embedded Applications, 2nd edn. Real-Time Systems Series. Asia-Pacific Software Engineering Conference (APSEC 2012), Springer, Berlin (2011) pp. 428–433. IEEE Computer Society, Conference Publishing 53. Kopetz, H., Bauer, G.: The time-triggered architecture. Proc. IEEE Services, Hong Kong (2012) 91(1), 112–126 (2003) 34. Farinelli, A., Iocchi, L., Nardi, D.: Multirobot systems: a classi- 54. Kornuta, T., Zielinski, ´ C.: Robot control system design exempli- fication focused on coordination. IEEE Trans. Syst. Man Cybern. fied by multi-camera visual servoing. J. Intell. Robotic Syst. 77(3– Part B (Cybern.) 34(5), 2015–2028 (2004). https://doi.org/ 4), 499–524 (2013). https://doi.org/10.1007/s10846-013-9883-x 10.1109/TSMCB.2004.832155 55. Kortenkamp, D., Simmons, R.: Robotic Systems Architectures 35. Figat, M., Zielinski, ´ C., Hexel, R.: Fsm Based Specification of and Programming. In: Khatib, O., Siciliano, B. (eds.) Springer Robot Control System Activities. In: 2017 11Th International Handbook of Robotics, pp. 187–206. Springer (2008) Workshop on Robot Motion and Control (Romoco), pp. 193–198. 56. Kortenkamp, D., Simmons, R., Brugali, D.: Robotic Systems https://doi.org/10.1109/RoMoCo.2017.8003912 (2017) Architectures and Programming. In: Siciliano, B., Khatib, en 36. Fleury, S., Herrb, M., Chatila, R.: G om: A tool for the O. (eds.) Springer Handbook of Robotics. 2nd edn., pp. 283–306. specification and the implementation of operating modules in a Springer (2016) distributed robot architecture. Proc. 1997 IEEE/RSJ Int. Conf. 57. Lamport, L.: Using time instead of timeout for fault-tolerant Intell. Robot. Syst. (IROS’97) 2, 842–849 (1997). https://doi.org/ distributed systems. ACM Trans. Progr. Lang. Syst. 6, 254–280 10.1109/IROS.1997.655108 (1984) 58. Lee, C., Hwang, J., Lee, J., Ahn, C., Suh, B., Shin, D.H., Nah, Y., 37. Fox, C., Evans, M., Pearson, M., Prescott, T.: Towards hier- Kim, D.H.: Self-Describing and Data Propagation Model for Data archical blackboard mapping on a whiskered robot. Robot. Distribution Service. In: Brinkschulte, U., Givargis, T., Russo, Auton. Syst. 60(11), 1356–1366 (2012). https://doi.org/ S. (eds.) Software Technologies for Embedded and Ubiquitous 10.1016/j.robot.2012.03.005. Towards Autonomous Robotic Systems, pp. 102–113. Springer, Berlin (2008) Systems 2011 59. Lumelsky, V.J., Stepanov, A.A.: Path-planning strategies for a 38. Gat, E.: On Three-Layer Architectures. In: Kortenkamp, D., point mobile automaton moving amidst obstacles of arbitrary Bonnasso, R.P., Murphy, R. (eds.) Artificial Intelligence and shape. Algorithmica 2, 403–430 (1987) Mobile Robots, pp. 195–210. AAAI Press, Cambridge (1998) 39. Giffin, S.d.: A taxonomy of Internet applications for project 60. Lumia, R., Fiala, J., Wavering, A.: The nasrem robot control management communication. Proj. Manag. J. 33(4), 39–47 (2002) system standard. Robot. Comput.-Integr. Manuf. 6(4), 303 – 308 40. Group, O.M.: Data distribution service (dds). https://www.omg. (1989). https://doi.org/10.1016/0736-5845(89)90120-8. Special org/spec/DDS/1.4 Issue Robots in Manufacturing 41. Hayes-Roth, B.: A blackboard architecture for control. Artif. 61. Lusty, C.: Cooperative Time-Triggered Wireless Communica- Intell. 26(3), 251–321 (1985) tion in Mobile Robotics. Honours thesis, Griffith University 42. Hayes-Roth, B.: A blackboard architecture for control. In: Bond, (2012) A.H., Gasser, L. (eds.) Distributed Artificial Intelligence, pp. 505– 62. Lyons, D.M.: Prerational intelligence, Studies in cognitive 540. Morgan Kaufmann Publishers Inc., San Francisco (1988). systems, vol. 2: Adaptive behavior and intelligent systems http://dl.acm.org/citation.cfm?id=60204.60241 without symbols and logic, chap. A Schema-Theory Approach to 43. Hayes-Roth, B., Hayes-Roth, F.: A cognitive model of planning. Specifying and Analysing the Behavior of Robotic Systems, pp. Cogn. Sci. 3(4), 275–310 (1979) 51–70 Kluwer Academic (2001) 804 J Intell Robot Syst (2019) 93:787–805 63. Lyons, D.M., Arbib, M.A.: A formal model of computation for 84. Sommerville, I. Software Engineering, 9th edn. Addison-Wesley sensory-based robotics. IEEE Trans. Robot. Autom. 5(3), 280–293 Publishing Company, USA (2010) (1989). https://doi.org/10.1109/70.34764 85. Song, D., Goldberg, K., Chong, N.Y.: Networked Robots. In: 64. Majcher, P.: Autonomous Mobile Robot for Collecting Table Tenis Khatib, O., Siciliano, B. (eds.) Springer Handbook of Robotics, Balls (Polish). Master’s thesis, Warsaw University of Technology pp. 1109–1131. Springer (2016) (2012) 86. Tanenbaum, A.S., Woodhull, A.S. Operating Systems Design and 65. Mataric, ´ M.J.: Issues and approaches in the design of collective Implementation, 3rd edn. Prentice-Hall, Inc., Upper Saddle River autonomous agents. Robot. Auton. Syst. 16(2), 321 – 331 (1995). (2005) https://doi.org/10.1016/0921-8890(95)00053-4 87. Trianni, V., Nolfi, S., Dorigo, M.: Cooperative hole avoidance in a 66. Mataric, ´ M., Nilsson, M., Simsarian, K.: Cooperative Multi-Robot swarm-bot. Robot. Auton. Syst. 54(2), 97–103 (2006). Intelligent Box-Pushing. In: Proceedings of the IEEE/RSJ International Autonomous Systems 8th Conference on Intelligent Autonomous Conference on Intelligent Robots and Systems, pp. 556–561 Systems (IAS-8) (1995) 88. Ueyama, T., Fukuda, T., Arai, F.: Configuration of communication 67. MiPal: Downloads (checked: 23 March 2018). http://mipal.net.au/ structure for distributed intelligent robotic system. Proc. - IEEE downloads.php Int. Conf. Robot. Autom. 1, 807–812 (1992) 68. Nesnas, I.A.D.: Software Engineering for Experimental Robotics. 89. Yamada, S., Saito, J.: Adaptive action selection without explicit In: Brugali, D. (ed.): The CLARAty project: Coping with communication for multirobot box-pushing. IEEE Trans. Syst. hardware and software heterogeneity, pp. 31–70. Springer Berlin Man Cybern. Part C 31(3), 398–404 (2001) 90. Zielinski, ´ C.: Description of semantics of robot programming Heidelberg, Berlin (2007). https://doi.org/10.1007/978-3-540- languages. Mechatronics 2(2), 171–198 (1992) 68951-5 3 91. Zielinski, ´ C.: A Quasi-Formal Approach to Structuring Multi- 69. Newell, A.: Some Problems of Basic Organization in Problem- Robot System Controllers. In: Second International Workshop Solving Programs. In: Yovitz, M., Jacobi, G., Goldstein, G. (eds.) on Robot Motion and Control, Romoco’01, pp. 121–128 Self-Organizing Systems, pp. 393–423. Spartan, Washington (1962) (2001) 70. Nii, H.P.: Blackboard systems, part two: Blackboard application 92. Zielinski, ´ C., Figat, M.: Robot System Design Procedure Based systems, blackboard systems from a knowledge engineering on a Formal Specification. In: Recent Advances in Automation, perspective. AI Mag. 7, 82–106 (1986) Robotics and Measuring Techniques, Advances in Intelligent 71. Nilsson, N.: Shakey the robot, technical note 323. Technical Systems and Computing (AISC), vol. 440, pp. 511–522. Springer report, AI Center, SRI International, 333 Ravenswood Ave., (2016). https://doi.org/10.1007/978-3-319-29357-8 45 Menlo Park, CA, pp. 94025 (1984) 93. Zielinski, ´ C., Kornuta, T.: Diagnostic requirements in multi-robot 72. Occello, M., Chaouiya, C., Thomas, M.C.: A Parallel Black- systems. In: Intelligent Systems in Technical and Medical Diag- board Model for Mobile Robotics control, pp. 439–447. nostics, vol. 230, pp. 345–356. Springer (2014). https://doi.org/10. Springer Netherlands, Dordrecht (1992). https://doi.org/10.1007/ 1007/978-3-642-39881-0 29 978-94-011-2526-0 51 94. Zielinski, ´ C., Kornuta, T., Winiarski, T.: A Systematic Method 73. Pang, G.K., Shen, H.C.: Intelligent control of an autonomous of Designing Control Systems for Service and Field Robots. In: mobile robot in a hazardous material spill accident — a blackboard 19-Th IEEE International Conference on Methods and Models structure approach. Robot. Auton. Syst. 6(4), 351–365 (1990) in Automation and Robotics, MMAR, pp. 1–14. IEEE (2014). 74. Parker, L.: ALLIANCE: An architecture for fault tolerant https://doi.org/10.1109/MMAR.2014.6957317 multirobot cooperation. IEEE Trans. Robot. Autom. 14(2), 220– 95. Zielinski, ´ C., Trojanek, P.: Stigmergic cooperation of autonomous 240 (1998) robots. J. Mech. Mach. Theory 44, 656–670 (2009) 75. Parker, L., Rus, D., Sukhatme, S.G.: Multiple Mobile Robot 96. Zielinski, ´ C., Winiarski, T.: General specification of multi-robot Systems. In: Khatib, O., Siciliano, B. (eds.) Springer Handbook of control system structures. Bullet. Pol. Acad. Sci. – Techn. Sci. Robotics, pp. 1335–1384. Springer (2016) 58(1), 15–28 (2010). https://doi.org/10.2478/v10175-010-0002-x 76. Parker, L.E.: Alliance: an architecture for fault tolerant, coop- 97. Zielinski, ´ C., Winiarski, T.: Motion generation in the MRROC++ erative control of heterogeneous mobile robots. In: IROS ’94. robot programming framework. Int. J. Robot. Res. 29(4), 386–413 Proceedings of the IEEE/RSJ/GI International Conference on (2010). https://doi.org/10.1177/0278364909348761 Intelligent Robots and Systems ’94. Advanced Robotic Systems 98. Zielinski, ´ C., Winiarski, T., Kornuta, T.: Agent-based structures and the Real World, vol. 2, pp. 776–783 (1994) of robot systems. In: K. J., et al. (eds.) Trends in Advanced 77. Postel, J.: Transmission Control Protocol, RFC 793. Technical Intelligent Control, Optimization and Automation, Advances in report, University of Southern California, Information Sciences Intelligent Systems and Computing, vol. 577, pp. 493–502 (2017). Institute. https://tools.ietf.org/html/rfc793 (1981) https://doi.org/10.1007/978-3-319-60699-6 48 78. Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.: ROS: an open-source Robot Operating System. In: Proceedings of the Open-Source Software workshop at the International Conference on Robotics and Automation (ICRA) (2009) Cezary Zielinski ´ M.Sc./Eng. (1982), Ph.D. (1988) and habilitation 79. ROS: Parameter Server. http://wiki.ros.org/ParameterServer (1996) all in control and robotics, from Warsaw University of 80. ROS: Technical Overview. http://wiki.ros.org/ROS/Technical Technology (WUT), full professor of WUT. Specialises in robotics, Overview heads the Robotics Group in Institute of Control and Computation 81. Rubenstein, M., Cornejo, A., Nagpal, R.: Programmable self- Engineering, working on the design of robot controllers and assembly in a thousand-robot swarm. Science 345(6198), 795–799 programming methods. His research interests focus on robotics in (2014). https://doi.org/10.1126/science.1254295 general and especially include: robot programming methods, multi- 82. Schwartz, D.G.: Cooperating heterogeneous systems. Springer robot system controllers, robot kinematics, robot force control, visual Science & Business Media, NO (1995) servo control, utilisation of sensors in robot control, design of digital 83. Shannon, C.E., Weaver, W.: A Mathematical Theory of Commu- circuits. Currently he is the vice-dean for General Affairs of the nication. University of Illinois Press, Baltimore (1963) Faculty of Electronics and Information Technology of WUT. J Intell Robot Syst (2019) 93:787–805 805 Maksym Figat M.Sc./Eng. (2013) in computer science (modelling Rene´ Hexel M.Sc./Eng. (1995), PhD (1999), has been a leading CAD/CAM systems), from Warsaw University of Technology (WUT), researcher in software engineering of safety-critical real-time systems. Faculty of Mathematics and Information Science. Since 2013, he His achievements include methodologies for modelling, model- is Ph.D. student at the Warsaw University of Technology (WUT), checking, evaluation, and validation of safety-critical real-time Faculty of Electronics and Information Technology (FEIT), Institute systems, including failure mode effects analysis (FMEA) through of Control and Computation Engineering (ICCE), Warsaw, Poland. fault injection. His research interests include Software Engineering He is a member of the Robotics Group in ICCE working on the of Complex Distributed Systems and autonomous robots as well as design of robot controllers and programming methods. His main robust and reliable safety critical systems. Rene ´ Hexel is Deputy scientific interests focus on automatic methods of robot control system Head of School of ICT at Griffith University and Co-Director of the generation based on a formal specification. His research in robotics Machine Intelligence and Pattern Analysis Lab (MiPal) at Institute for is based on Model Driven Engineering, Domain Specific Language, Integrated and Intelligent Systems (IIIS) in Brisbane, Australia. Embodied Agent approach and the formal specification for robotic control systems.

Journal

Journal of Intelligent & Robotic SystemsSpringer Journals

Published: Jun 2, 2018

There are no references for this article.