Cross Organisational Workflow Co-ordinated by Software Agents

We assert that a layer of software agents, that co-ordinate the distribution of work via negotiation, is the proper software abstraction for the development of cross-organisational workflow management systems. We support our assertion by describing: the use of software agents to provision distributed workflows; the role of agents to facilitate workflow interoperability; agent-based techniques for visualisation and verification of decentralised processes.

Our implementation includes a number of industry standard components, such as FIPA Agent Communication Language, CORBA, JAVA and the Process Interchange Format. The target domain was a ‘single front office, multiple back office’ configuration, based on a simplification of a live business process.

Introduction

Virtual and other extended enterprises have become increasingly common as organisations seek new ways of delivering value to their shareholders and customers. As cross-organisational workflow is a vital tool for supporting virtual enterprises, and virtual enterprises are becoming increasingly important, why are such tools not currently available? The answer is that there are three classes of problem that impede their development.

• Organisational issues: product and work ontologies differ from one business to another, as do business rules, organisational structures and the division of responsibilities. This makes it difficult for an organisation to successfully dictate business (sub) processes that its suppliers must use, let alone integrate foreign workflows with its own workflow systems. For this reason, hand-offs between organisations are costly, as they are difficult to establish and time consuming to monitor.
• Work item identification and management: at the level of workflow tools the removal of a work item from a workflow and its asynchronous re-insertion can cause problems. Work item audit trails are also an issue in such cases.
• System and technical problems: workflow systems may expect to be actively queried for process statistics or work movements. Alternatively they may generate events signifying the movement of work through an activity, and expect these to be caught and handled by other process elements. Either way, centralised monitoring and control of distributed workflow systems is currently problematic.

In this paper we argue that software agents are a suitable abstraction for developing systems that overcome these problems. Furthermore, it is inappropriate and unnecessary to implement a workflow management system using only agents. What is required is an agent layer to manage the interoperation of disparate workflows and add value for workflow users by supplying extra information services, whilst maintaining investment in existing workflow technology.

Agents and Workflow, Related Work

(Nwana 1996) identifies three properties of a software component that identify it as an agent.

• It should be autonomous, that is able to make decisions independently of human advice.
• Agents should be collaborative, that is able to work together in order to solve a problem
• Intelligent agents should learn from their experiences and adapt over time as their operating conditions change.

An agent based workflow system, then, is a set of software components that meet the criteria to be considered as agents and are involved in managing the flow of work through a business process.

Agent based workflow systems have been developed by a number of teams in order to provide exception handling and process management (Jennings etalet al 1996), (Harker, Ungar 1996), (Borghoff etalet al 1997), (Debenham 1998) each offering their own particular enhancements and features. In agent based workflow, the software agents take full responsibility for process provisioning, execution and compensation (see Figure 1), with each agent managing and controlling a given activity or set of activities. However, these tools do not appear to have been adopted and are not, at the time of writing, in common usage. In addition, a move to agent based workflow is likely to incur high re-engineering costs, and the resulting step change in technology will create extensive training requirements. Worse still, the process logic is embedded in the agents, rather than being explicitly represented elsewhere.

In both Figure 1 and Figure 2 arrows represent information flows and encapsulation (of an activity) represents ownership and control.

Agent Enhanced Workflow

In order to address the problems identified above, developed a software agent enhancement to a widely used third party workflow tool was developed. which We explored the possibility of adding an agent layer with the added value of providing mechanisms for handling and repairing workflow failures (Judge etalet al, 1998), while retaining the user’s confidence in the workflow tools they were familiar with. We were able to demonstrate the feasibility of constructing workflow interoperability mechanisms, based on agents as proposed by (Adams, Dworkin 1997). We were able to show that the agent layer could interface workflow systems to other software tools to provide further functionality, such as process monitoring and control.

We call our approach agent enhanced workflow, in contrast to the more common agent based workflow. Agent enhanced workflow is achieved by combining a layer of agents with a commercial workflow system (as per Figure 2). The agent layer is given responsibility for the provisioning phase of business process management, whilst the underlying workflow system handles process execution. This approach provides a number of benefits (including automatic provisioning, interoperability, support for visualisation and verification services), whilst protecting the original investment in workflow technology. The use of workflow technology ensures that the business process is explicitly represented and easily maintained.

Value added to Workflow by an Agent Layer

Our workflow agents were developed by using the Zeus agent toolkit (Nwana etal 1998). We chose Zeus because it offered us an advanced agent development environment which enabled us to construct our workflow agency more easily than would otherwise have been the case. Zeus provides the developer with. There are number of other toolkits which would enable the reader to develop similar agents to those we describe here, for instance JAT-0.4 (Frost 1997).

We have identified three groups of problems that have prevented the deployment of cross-organisation workflows.

• Organisational issues
• Work item identification and management
• System and technical problems

In the following sections we explain why we think that adding an agent layer to existing workflow management systems can provide at least partial solutions to these problems.

Organisational Issues

One key problem with transferring information between organisations is that the semantics of the information structure are frequently lost. The problems are less acute when dealing with domain-specific ontologies relating to the same domain, but synonyms, hypernyms, hononyms and other relationships specific to the domain must be approached with care (Sheth 1998).

Agent-based abstractions offer a solution to this problem by demanding that the members of the agent layer subscribe to a common ontology. The common ontology acts as a shared semantic which is then mapped to the local semantics of each of the inter-operating systems. We have used the PIF (Lee et al 1997) process definition standard as a shared ontology that our agents use to communicate process definitions.

Further, the use of FIPA standard interaction protocols (FIPA 1997) between the various workflow agency brokers provides a mechanism for managing inter-organisational hand-offs. Once an agreement has been reached between two workflow agencies, an electronic contract is exchanged which describes the responsibilities of both parties (in terms of volume and type of work, price, timescales etc.). The subsequent performance of each agency is compared with the terms of the various contracts that it is a party to, and the findings are used to support learning and predictive behaviour in itself and/or other agencies.

Work Item Identification and Management

Workflow systems themselves present some obstacles to interoperability. Amit Sheth (Sheth 1997) points out that "currently disparate technologies for workflow automation or process management…will need to be integrated".

The removal of a work item from a workflow and its asynchronous re-insertion can cause problems. As each agent manages a service centre, and the activities under its control are available to contribute to a wide range of business processes, it is natural that the removal and re-insertion of work items be managed by the agent layer, in accordance with the provisioning plan. Work item audit trails are also managed at the agent layer, by combining audit fragments from the various workflow systems into a whole.

System and Technical Problems

An agent layer can contain specialist broker agents which act as information repositories, transforming the mode of information delivery from the underlying workflow systems to the mode required by other external systems. The transformation functionality can be implemented as part of an agent (i.e. separate from the workflow system) by use of the FIPA interaction protocols (FIPA 1997).

These protocols are used to enable our agent enhanced workflow systems to communicate with external agencies that provide business process visualisation and monitoring services. The visualisation and monitoring system (VMS) agency is able to specify the type of information that is needed in order to provide the requested service, along with the mode of information delivery required.

Architecture

This section gives details of our agent enhanced workflow system architecture, which has two key elements.

• Each interoperable workflow system has a managing agency that uses a local ontology in order to obtain activity statistics, monitor the flow of work through the activity, and manage the workflow by removing work from some points in the process and introducing it into others.
• Each of the agencies requires a broker-agent that can translate between the local ontology of a workflow agency and the global ontology of the inter-operating systems. The broker also mediates the mode of information delivery between the model used by the particular workflow being managed and that required by other workflows or systems that are part of the interoperation.

Figure 3 is a schematic of this agent enhanced workflow architecture. The JAVA-based workflow agents were implemented using the Zeus agent building toolkit (Nwana et al 1999). Inter-Zeus agent communication uses a JAVA implementation of the Knowledge Query and Manipulation Language (KQML). The interface to workflow utilises a third-party WAPI written in C, which we have distributed using the JAVA Native Interface and CORBA-2. The brokers were implemented using JAVA and CORBA-2. The inter-broker communication channel uses a JAVA implementation of Foundation for Intelligent Physical Agents (FIPA) Agent Communication Language (ACL), with the Process Interchange Format (PIF) as a content language. The PIF content language was implemented in JAVA.

Provisioning

The AEW system operates as follows: External demand (e.g. an order for N widgets) is expressed as a goal (e.g. produce N widgets) for the workflow agents to achieve. The workflow agents collaborate to decompose the goal, forming a plan via backward chaining. The plan is both a process definition and a resource schedule, as resources are reserved as the plan is formed. The workflow agents execute the plan, causing work items to be created, processed, routed and destroyed in the underlying workflow systems, as and when appropriate. When a (sub) goal has been achieved by a given agency (e.g. completion of a batch of components required to build the widgets), other interested agencies are notified via its broker which converts Zeus agent constructs into a JAVA implementation of PIF entities, and transmits the PIF via FIPA ACL.

Visualisation and Verification

An end-to-end view of decentralised business processes can be provided via our Visualisation Monitoring System (VMS), an external agency that exists to provide common services to other FIPA ACL compliant agencies. The VMS uses FIPA ACL to request that each workflow agency send a description of any of its sub-processes that are contributing to the overall business process. The VMS then assembles the various sub-process definitions using the precedence information contained within each. The resultant process definition is checked for liveness, deadlocks etc. by converting the PIF into CCS (the Calculus for Communicating Systems (Milner 1995)), and running the Edinburgh Concurrency Workbench (http://www.dcs.ed.ac.uk/home/cwb).

Contract Compliance

The VMS can also perform centralised progress monitoring. It does this using the FIPA ACL communicative acts request-when and request-whenever, to obtain information from the various workflow agencies. Upon receipt of either message, the broker formulates and asserts a Zeus agent goal (e.g. ‘get the activity statistics for activity X’) which causes a Zeus agent to make one or more simple statistics calls to its underlying workflow system (in keeping with the conditions contained within the REQUEST-WHEN or REQUEST-WHENEVER message), The result is passed back to the broker and sent back to the VMS using FIPA ACL.

Interoperability

Interoperability between workflow agents is handled in a similar fashion to ‘workflow agent to VMS’ communication. Workflow information is obtained by a Zeus agent and passed to the broker, which converts it to PIF format. The information is transmitted to other brokers using FIPA ACL, and converted to one or more Zeus goals. As each goal is asserted, a plan is formed and executed by Zeus agents, resulting in some change to the state of the agency's underlying workflow..

Conclusions

In this position paper we have identified three major impediments to cross-organisational workflow: organisational issues; work item identification and management; system and technical problems.

We have described an Agent Enhanced Workflow architecture that uses industry standard middleware tools (CORBA, JAVA) to interface software agents to third-party workflow systems, and shown how broker agents (using standard interaction protocols such as FIPA-CONTRACT-NET, FIPA-REQUEST-WHEN and FIPA-REQUEST-WHENEVER) were used to facilitate workflow interoperability.

Agent Enhanced Workflow co-ordinates the distribution of work using negotiation techniques, coupled with a goal-based planning and scheduling approach. The agent layer formulates a provisioning plan, which is executed on demand by the underlying workflow systems. Agent Enhanced Workflow mitigates ontology problems through the use of a common process description language (such as PIF), and uses emerging agent standards to communicate with external agencies.

In summary, Agent Enhanced Workflow provides a workflow co-ordination mechanism for provisioning virtual processes, automating the hand-offs between organisations, and the support for visualisation and verification of virtual processes.

Bibliography

Adams, Dworkin 1997 Adams T, Dworkin S. "Workflow Interoperability Between Businesses". In: Lawerence P. (ed.) WfMC Workflow Handbook. John Wiley & Sons, New York, 1997, pp 211—221.

Borghoff et al 1997 Borghoff UM, Bottoni P, Mussio P, Pareschi R. "Reflective Agents for Adaptive Workflows". In: Proceedings of the Second International Conference on the Practical Application of Intelligent Agents and Multi-Agent Technology (PAAM97), London, UK, 1997. The Practical Application Company Ltd, pp 405-420.

Debenham 1998 Debenham J. "An Experimental Agent-based Workflow System". Proceedings of the Third International Conference on the Practical Application of Agent Technology (PAAM98), London, UK, 1998. The Practical Application Company Ltd, pp 101-109.

Harker, Ungar 1996 Harker PT, Ungar LH. "A market-based approach to workflow automation". Proceedings of NSF. Workshop on Workflows and Process Automation in Information Systems: State of the Art and Future Directions. 8-10 May 1996.

Jennings et al 1996 Jennings, N. Jennings NR, Faratin P, Johnson MJ, O'Brien P, and Wiegand ME. "Using intelligent agents to manage business processes". In: Proceedings of the First International Conference on The Practical Application of Intelligent Agents and Multi-Agent Technology (PAAM96), London, UK, 1996. The Practical Application Company Ltd, pp 345-360.

Lee et al 1997 Lee J, Gruninger M, Jin Y, Malone T, Tate A, Yost G. "PIF: Process Interchange Format v.1.2". PIF Working Group.

Milner 1995 Milner R. "Communication and Concurrency". Prentice Hall, 1995.

Nwana 1996 Nwana HS. "Software agents: an overview". The Knowledge Engineering Review, 11(3), 1996, pp 205-244.

Sheth 1997 Sheth A. "From Contemporary Workflow Process Automation to Adaptive and Dynamic Work Activity Co-ordination and Collaboration". Proceedings of the Workshop on Workflow Management in Scientific and Engineering Applications, Toulouse, France, September 1997.

Sheth 1998 Sheth A. "Changing focus on interoperability in information systems: from system, syntax, structure to semantics" In Interoperating Geographic Information Systems goodchild, M., Egenhofer M., Fegeas R., Kottman C. (eds.), Kluwer.