A Conceptual Architecture for an Event-based Information Aggregation Engine in Smart Logistics

AConceptual Architecture for an Event-based Information Aggregation Engine in Smart Logistics Anne Baumgrass1,Cristina Cabanillas2,Claudio Di Ciccio2 Abstract: The field of Smart Logistics is attracting interest in several areas of research, including Business Process Management. Awide range of research works are carried out to enhance the capability of monitoring the execution of ongoing logistics processes and predict their likely evolvement. In order to do this, it is crucial to have in place an IT infrastructure that provides the capability of automatically intercepting the digitalised transportation-related eventsstemming from widespread sources, along with their elaboration, interpretation and dispatching. In this context, we present here the service-oriented software architecture of such an event-based information engine. In particular, we describe the requisites that it must meet. Thereafter, we present the interfaces and subsequently the service-oriented components that are in charge of realising them. The outlined architecture is being utilised as the reference model for an ongoing European research project on Smart Logistics, namely GET Service. Keywords: Smart Logistics; Service-oriented Architectures; Complex Event Processing 1Introduction GET Service3 is aEuropean FP7 research project aiming at the realisation of adistributed service-oriented platform for the planning, execution and monitoring of smart transportation processes. The devised platform is meant to be adopted by Logistics Service Providers (LSPs) Europe-wide, in order to take advantage of apowerful infrastructure that allows the improvement of their core business processes, in terms of reduced CO2 emissions, better time scheduling, more precise service time estimates and thus, reduced costs. Against this goal, we notice that such aplatform must build upon the regular synchronisation of its real-world context-awareness. Forinstance, it is vital that the position of involved transportation means is kept under control during the shipment of goods in order to assist its run-time monitoring. Such information can be gathered by the interception, analysis and interpretation of so-called events. Events are known to be detected by different sensors and reported by several sources. Due to the dynamic nature of the context domain, such information is intrinsically meant to change overtime. Therefore, the GET Service core module that is in charge of extract-ing relevant information on the current development of transportation processes, deals with concurrent event streams stemming from various originators. The information com-ing from the collection and comparison of the events in the flow of updates has to be 1 Hasso-Plattner-Institut, University of Potsdam, Germany, [email protected] 2 Vienna University of Economics and Business, name.[particle.][email protected] 3 http://www.getservice-project.eu/ interpreted to detect and possibly foresee the development of the transportation process, givenits execution history and the context within which it is carried out. This paper aims at defining the architecture of the information aggregation and provisioning engine in the context of smart logistics; in particular,inthe scope of the GET Service software infrastructure, which is under development at the time of writing and is henceforth referred to as the platform. The remainder of this paper is structured as follows. Section 2presents background on event processing. In particular,Section 2.1 introduces the fundamental concepts of event processing networks (EPN), processing agents (EPA), source, consumer,object, and channel. Section 2.2 explains howsuch concepts come into play in the context of event processing. Furthermore, it outlines howaggregation and correlation patterns contribute to the gathering of knowledge regarding the evolution of transportation processes, out of event streams. Then, Section 3delves into the details of the functionalities that the information aggregation services must offer in the GET Service platform. The discussion is promoted to Section 4, where the architecture of the component offering those services is detailed, in conformance with the aforementioned criteria. Section 5concludes this paper. 2Background This section summarises the background on event processing as well as the requirements that are necessary to design the event aggregation engine. 2.1 Event Processing Infrastructure Events that are of importance in our context are the transportation-related events.They servethree main purposes: (i) tracing howaspecific transportation process is executed, (ii) coordinating the different parties involved, and (iii) making appropriate decisions in relation to re-planning and rescheduling. Typically,events are produced and collected by different kind of systems spanning an event processing network (EPN) in which event processing agents (EPA) are linked by event channels to exchange events [EN10], (cf. Fig. 1). Each EPAmay act as an event consumer to receive event objects,and as an event source in case it observes events and publishes them in amachine-readable form as event objects.Inthis way, an EPAreacts to its input by processing events and outputs events that can be fed to other EPAs overevent channels [Lu01]. In the context of GET Service, the GET Service Platform should act as an event consumer to gather events from several event sources (e.g. driver’smobile devices and weather stations) and process them to generate transportation-related events, which might be provided to several consumers, e.g., Logistics Service Providers (LSPs) [Ba13b]. Fig. 1: Event processing infrastructure. 2.2 Event Processing Mechanism In asmart logistics context likethe one of GET Service, it is of utmost importance that all events related to the transportation of goods and corresponding transportation plans can be observed. Furthermore, such events have to be processed to derive transportation-related information, and be published to all interested consumers. To this extent, event processing is in charge of computing operations on events, including reading, creating, transforming, or discarding [EN10]. Specifically,anEPA carries out these operations. First, an EPAcontains an event adapter,which transforms events into event objects [Lu01], e.g., for importing weather forecasts in XML format. The EPAreceivesthe events from an event channel as input in order to process them. Adapters are used to identify event objects published by several event sources, in possibly different formats. Data can be indeed encoded according to standards such as EDIFACT(United Nations/Electronic Data Interchange ForAdministration, Commerce and Transport [Be94]), MXML (Mining eXtensible Markup Language [vD05]), XES (eXtensible Event Stream, [GV14]), butalso more general-purpose ones likeCSV (Comma-Separated Values), Excel and XML (eXtensible Markup Language). Second, events are related to each other according to event relationships. Typically,events are related by time, causality,aggregation [Lu01] or correlation [ROS11]. Event patterns are used to specify these relationships and identify them in an event stream considered by an event processing system. Forexample, only if Container mounted happened before Goods loaded in acertain time windowand both event objects refer to the same container,theyare related by acorrelation relationship and can be aggregated to an event Goods ready for transportation (cf. Tab.1). In this way, acorrelation is specified through defining correlation attributes (e.g., time and container in the example) between event object types. Furthermore, event aggregation patterns can be used for recognising or Goods loaded Container mounted Goods ready for transportation Description Newgoods are loaded in Container1. Container1 is mounted onto Truck1. Truck1 is ready to start its transportation. Occurrence time January 2nd 2014 07:30 January 1st 2014 16:30 January 2nd 2014 07:30 Occurrence location Harbour of Rotterdam Harbour of Rotterdam Port of Rotterdam Originator(s) Container1 Truck1, Container1 GET Service Platform Impact Truck may start the transportation. Goods can be loaded in the container. Truck may start driving and transport the loaded goods to their destination. Target(s) Container1 Truck1 TransportationOrder1 Tab. 1: Example events in logistics. Forthe sakeofsimplicity,the example is kept simple and abstracted from the real-world. Forexample, in order to have atruck ready for transportation other events might be relevant as well (e.g., documentation on board). detecting asignificant group of events from among aset of events, and creating asingle event that summarises their significance in its data. Third, event patterns are also used to forward events to interested consumers. Forthis purpose, an event consumer subscribes to an event processing system with adefined event pattern. Events that match that pattern are sent as notifications overevent channels to the consumer.Anotification contains data describing an event and may additionally carry information describing the circumstances of the event. In [MFP06], the event processing infrastructure only represents the theoretical components and disregards the issues to be dealt with when implementing this infrastructure in practice, e.g., access control or messaging formats. We aim to use the transportation context to identify events from several event sources, process them into transportation-related events, and forward them to interested parties. 3Design of the Information Aggregation Engine The Information Aggregation Engine is amain component of the platform, which is responsible for collecting events from different sources and processing them in order to offer aunified interface to clients, planners, information providers, and other stakeholders. Thereby,the engine supports, among others, the use cases of track&trace, vessel arrivals and capacity visualisation. The specific functionalities that this service requires in transportation are described in the following sections. Such functionalities derivedfrom a preliminary requirements elicitation phase and thorough analysis of the typical use cases scenario in the context of the GET Service project [Tr13]. 3.1 Import and Export of Event Data It is crucial that the interfaces of the platform adhere to existing messaging standards and interchange formats of all services that are used by the involved stakeholders. Forthis purpose, the messaging standards of logistics were investigated in [Ve13]. Four common message types where identified: (i) EDIFACT, used by shipping lines, terminal operators, or customs; (ii) EDIFACTXML (UN/CEFACTworking groups); (iii) Business Logistics XML, used by larger Logistics Service Providers (LSPs); and (iv) Excel uploads and downloads, used by smaller Transportation Service Providers (TSPs). Furthermore, applications might use JSON as exchange format, which has less markup overhead in comparison to XML. To unify the communication in logistics for all stakeholders, the e-Freight project4 developed astandard framework that also needs to be considered in the platform. It is a standard for freight information exchange covering all transport modes and stakeholders. Each of the above mentioned message types can be transported overdifferent channels using different protocols and services, for example through SOAP web services, HTTP protocols, RPC, or FTP file transfers. Thus, to extract events from all exchanged messages and to publish transportation-related events in the aggregation engine, it requires four generic interfaces for communication: 1. An interface to import messages of events from different sources (e.g. from client devices of LSPs) provided in different formats. Based on the aforementioned message formats, the engine must be able to call external web services, connect to message queuing services, generate HTTP requests, and download files from FTP.Additionally,ithas to offer an interface, to which clients can push events contained in messages. 2. An interface for identifying the event information in these kinds of messages. By implementing adapters the aggregation engine defines where and howtoextract events from all the imported messages types. Thus, it must be possible to import events using EDIFACT, XML, Excel, JSON, and the e-Freight format. 3. An interface for submitting event patterns to be notified of the occurrence(s) of events that the stakeholders are interested in. Forthis purpose, the aggregation engine must enable all stakeholders to specify these event patterns in awell-chosen language, such as Esper5.Furthermore, the aggregation engine must be extendable to implement the functionality of deriving event patterns from transportation plans, logistic process models, and route descriptions. 4. An interface to forward events to interested targets. Thus, the aggregation engine must itself provide functionalities to publish events and provide them to the stakeholders involved in transportation. Community systems or other platforms might act as intermediate event distributors. Thus, the engine needs to implement amessage queuing service to distribute events and also forward notifications containing information on asubset of events. This forwarding may be implemented as HTTP responses or as API, butmay also be realised through emails to be shown on the mobile client devices. The format for the notifications depends on the client devices butshould at least adhere to the message standards mentioned above,including EDIFACT, XML, Excel, JSON, and the e-Freight format. 4http://www.efreightproject.eu/ 5http://esper.codehaus.org/ Each of these interfaces must provide capabilities to access and modify the functionalities in order to adapt to achanging environment. Therefore, the interfaces should provide methods to support the standard operations of Create, Read, Update, and Delete (CRUD). Forexample, partners interacting with the platform should be able to adapt their ownaggregation rules to changing plans, or set up newevent sources, butalso be able to delete rules that are no longer required. 3.2 Normalisation of Events Because events are collected from different sources that can have different formats, events need to be normalised into acommon unified format for further event processing. The normalisation needs to takeplace in the aggregation engine in order to process events. The normalisation includes the definition of the format of the normalised events, as well as the stored event properties that are available for purposes ranging from information extraction to correlation of events based on values. The different formats and their differing structure imply that the target event format needs to be extensible and general enough to allowfor incorporation of structured or unstructured information from all different sources. The transformation into the unified format can be specified by corresponding adapters. An adapter refers to acomponent that formats heterogeneous event data into asuitable input format. Forexample, an event stream in XML format can be processed by an XML parser and events can be extracted based on conversion rules, which can include mappings for different formats of dates and timestamps to the internal format. The mapping rules should be extensible and reusable, such that the task of connecting newsources can be conveniently performed. 3.3 Integration of Event Processing Once the events are made available to the aggregation engine in anormalised format, the actual event processing has to be performed in form of aggregation and correlation. Thus, the functional requirements for the event processing engine is to support the above mentioned relations between events, i.e. to detect relations based on time, causality,aggregation and correlation. These relations are stored as rules that allowtorelate and to aggregate several events. Furthermore, the aggregation engine is expected to capture alarge amount of events and needs to be able to process them within acomplexenvironment where manyactors subscribe for their respective events. The actual ComplexEvent Processing (CEP) system that is used is therefore required to be scalable. 3.4 Predictive Functionalities in Cooperation with Discriminative Classifiers The ability not only to monitor butalso to interpret the context information can be seen as one of the main objectivesofthe event processing component. Indeed, streams of transportation-related events represent atemporal snapshot overthe current development of transportation processes. As an example, the consecutive coordinates and altitude levels of an aeroplane trace its movements. The events may rise from different sources (e.g., weather conditions along the route, traffic information in the arrivalairport, etc.) and can altogether concur in the creation of adangerous situation for the regular advancement of the transportation. Therefore, it is of considerable relevance to distinguish the sequences of events that lead to adisruption from those that are safe. Evaluating queries overevent streams is abasic approach to this extent. Such queries would weigh the combination of events overtime in order to determine whether the current evolvement of facts is likely to end up in ariskysituation, or not. However, it would be impractical to predefine all such queries apriori. This is due both to the quantity of possible concurrent causes to check, and to the unfeasibility of foreseeing anypossible anomalous sequence of events. To this extent, classifiers from the field of Machine Learning [Mi97] can be of significant help. Forinstance, Support-Vector Machines (SVMs [CV95]) are supervised learning models for linear classifications, i.e., able to identify ahyperplane in the space of features that separates numeric representations of input objects in twodifferent categories. The hyperplane is determined on the basis of alearning process made on labelled historical data. In the context of transportation-related events, e.g., labelled historical data can represent the reported trajectories of aeroplanes, divided into those that were known to have landed in time and in the expected airport, and those, which were known to have been delayed or diverted. Once trained on such data, the classifier (e.g., SVM) can analyse current flights and predict whether theyshowananomalous behaviour,ornot. The input as events can be provided by aCEP system, as long as the transportation process specifies the information to be extracted from events to this extent. The learning systems can be used indeed to correlate available data, in order to detect anomalies based on previous knowledge. The selection of independent and dependent variables for the decision functions is thought to be determined apriori, since theyare strongly domain-related. Forinstance, the SVM can recognise apossible diversion of flights on the basis of features such as gained distance from the departure airport, velocity and altitude of the aeroplane. However, the input sources (e.g., flight monitoring services) as farasthe information aggregation and features extraction (e.g., from positional data to distance, velocity and altitude) are meant to be predefined. On the basis of the prediction made by the classifier,anew event raising an alert can be generated in case of anomaly detection. Therefore, it is required for the event stream to be restructured in away that makes it readable from an external classifier,onone hand. On the other hand, the classification returned as aresult has to be treated and transformed into anew event. It is worthwhile to recall here that aframework for controlling the safe execution of tasks and signalling possible misbehaviours at runtime has already been outlined in [Ca14b], and preliminary results are already applied in the context of flight diversion detections [Ca14a]. 3.5 Correlation of Events to Processes An additional function within the aggregation engine is the (semi-)automatic derivation of correlation rules on the basis of process data and transportation plans. The intention is to analyse process models, route descriptions, or transport execution plan as input that is analysed to derive correlation and aggregation rules. Forthis purpose, the components outside the aggregation engine need to provide these documents in away theycan be parsed and event patterns can be derived. In case of processes modelled with the Business Process Model and Notation (BPMN) 2.06format [Du13], the approach of [Ba13a] can be implemented to correlate events to process instances and identify whether this instance of aprocess model wasexecuted successfully. 3.6 Notification Mechanism The aim of the platform is to offer services to manyclients and hence, it needs to adhere to common notification paradigms. The publish/subscribe paradigm is very common in distributed systems [MFP06] and needs to be supported by the information aggregation engine. Using this paradigm allows clients and planners to subscribe to certain types of events or aggregated events. Forexample, aplanner might subscribe to all events that are correlated to the respective transportation plans that the planner has created. Then, if events occur during execution, the planner is notified about their occurrence and can react accordingly. Besides the publish/subscribe paradigm, regular access to events is required in the platform. That is, information providers need the option to add newevents directly into the platform via the appropriate interface (push). And additionally,the option to query for recent events from the event history should be made available in that interface (pull). 3.7 Summary The above sections point out that we aim to design appropriate filtering mechanisms at early stages, to reduce the burden on the correlation and prediction activities. Furthermore, the derivation of correlation rules based on processes range from very simplistic approaches (e.g., correlating by container id), to more sophisticated, control flow, location, and time-aware correlation mechanisms. To provide abrief overviewofthe requirements of the aggregation engine, atabular representation is giveninTab.2. 4Architectureofthe Information Aggregation Engine This section presents the architecture of the information aggregation engine, in the light of the requirements previously explained. UML component diagrams are used to visualise the logical interconnection of its internal components. 6http://www.bpmn.org/ Requirement Description R1. Heterogeneous Sources Connection to different kinds of event sources R2. Heterogeneous Formats Collect events from different message formats R3. Normalisation of Events Store events of different formats in same normalised format R4. Event Storage Store normalised events in acentral database R5. Event Processing R5.1 Event Aggregation Provide functionality to aggregate events of finer granularity to single events R5.2 Query Subscription Register queries to be informed of events of interest R5.3 Domain-Specific Query Subscription Register queries to be informed of transportation-related events of interest R5.4 Domain-Specific Event Correlation Automatically correlate events to transportation processes R6. Notification Mechanism Notify subscribed clients of respective events R7. Event Classifiers Determine criticality of an event for transportation Tab. 2: Summary of required capabilities of the event-based information aggregation engine. 4.1 Design of External Interfaces The information aggregation engine offers four interfaces to be used either by external event sources (e.g., driver, weather stations) or event consumers (e.g., planner,driver). Furthermore, it implements an interface to access the information store to request static information. All fiveinterfaces are required to provide the functionalities described in Section 3. These are shown in the component diagram in Fig. 2and summarised as follows. EventAdministrationInterface.The EventAdministrationInterface receivesthe structural description of an event type and offers further administrative tasks related to events. The communication through this interface has to be implemented in twoways. It should be either initiated by anyevent source sending the event type description of the events it publishes (push) or it can be configured inside the corresponding event source adapter (cf. EventSourceAdapter,Fig. 4). The implementation is meant to be realised by means of a web service to which the event source can push the event type description. EventSourceAdapterInterface.Through the EventSourceAdapterInterface the aggregation engine is able to receive events (resp. implementing R1 in Tab.2). Each event source is intended to be connected through aspecific adapter.This adapter then offers an interface of its ownthat can be used by the event source. Each adapter has to internally use the EventImportInterface (cf. Fig. 4),i.e., the interface through which the AggegrationService can takeasinput and process newevents. EventSubscriptionInterface The EventSubscriptionInterface is used to register subscriptions to the aggregation engine. These subscriptions can be arbitrarily complex, i.e., theymay be composed of specific event processing queries. The subscriptions should be pushed to the aggregation engine. Therefore, the aggregation engine provides an implementation of arequest-response pattern to register subscriptions in the platform. The events being imported via the EventImportInterface areforwarded to the event consumers by the aggregation engine based on registered subscriptions.