September 1999

Home Next Story Printer-Friendly Version

Search Topical Index Backissues SunWHERE

Subscribe, It's Free Letters to the Editor Events Calendar TechDispatch Newsletters

Technical FAQs Solaris Security Secure Programming Performance Q&A SE Toolkit

The emergence of workflow management in enterprise-class business solutions

What will workflow management mean to your system architecture?

Summary
Mike Conger, this month's IT Architect columnist, explains the role and value of workflow management in the context of large-scale, enterprise-class systems. An evolving technology, workflow management is likely to provide some compelling solutions to enterprise information systems, approaches, and strategies in the near future. (4,300 words)

	IT ARCHITECT
	By Mike Conger

Foreword
When we talk about adding workflow capabilities to an organization's technology architecture, many people wrongly assume that we are simply implementing a workflow product like Lotus Notes and little else. The reality is that workflow capabilities constitute more of a common service available at the enterprise level. That is, workflow is an enabling technology that can be used throughout the organization, and beyond the organization, to support various business functions. Orchestrating the introduction of an enterprise technology, as opposed to an single application technology, is challenging. This month, we discuss the business justification, potential usage, architectural considerations, and potential pitfalls of taking on workflow technologies. In addition, we explore how workflow can affect an organization's extended enterprise via the Internet, and how workflow relates to XML.

--Kara Kapczynski

n today's marketplace, businesses are continuously driven to gain a competitive advantage, improving not only the delivery of their products and services, but also their nature and makeup as businesses. Some of the drivers behind this revolution include:

• Globalization
• The need to be more customer-focused
• An increasingly competitive marketplace
• Delivery of complete solutions to customers
• Streamlining and automation of business processes
• Reduction in time to market for complex goods and services

To support this new global marketplace, added demands have been placed on information technology (IT) organizations. In response, the IT industry has increasingly accepted advanced technologies and solutions to fulfill these business demands. These technologies, integrated with a common vision and represented by a strategic enterprise architecture, provide an environment in which business and technology solutions can be applied to address critical needs. The application of workflow management approaches, architectures, and technologies can provide significant value in meeting these demands.

As this model has evolved, there has been a migration from function-focused applications to process-focused applications. A side effect of this change, long after many function-focused applications have been delivered, has been a host technical difficulties. The need for improved process efficiencies, which has already resulted in BPR (business process reengineering) for many companies, is now working its way through the technology arena. A shift in focus toward achieving technical integration and flexibility is now being realized in the real-world usability of process-oriented and component-based architectural solutions. This associated integration flexibility is an absolute IT imperative for the next decade.

However, there is a second IT imperative: the need to coordinate IT and business goals. While Web availability, business-to-business and business-to-consumer (B2B and B2C, respectively) technologies, XML (Extensible Markup Language), EAI (enterprise application integration), middleware, and additional related technologies all contribute to technical connectivity and interoperability, they are for the most part foreign to business. Leveraging formal business process descriptions and corresponding workflow technology, we find common ground between business and IT. Regardless of the implementation technology of workflow management, business process concepts will be a requirement for architects in the next decade.

The fact that there is a technology capable of mapping almost one-to-one onto a business-process language implies that workflow will become a powerful tool as enabling support technologies improve. These technologies are, however, preconditions to the widespread adoption of workflow management across extended enterprise environments (in contrast to successful but limited introduction within large-scale applications). Additional improvements in workflow management tools are also required for the expanded use of workflow in enterprise environments (some of which will be discussed later in this article). On the upside, the long history of such products, and the foresight that was shown in creating workflow management standards means that when these technologies are ready, they will integrate both within and across enterprises.

Overview of workflow management
Workflow management is defined by the Workflow Management Coalition (WfMC) as follows:

The automation of a business process, in whole or in part, during which documents, information, or tasks are passed from one participant to another for action, according to a set of procedural rules.

Workflow management has existed, as a set of either manual or automated activities, for quite some time. Most efforts -- such as the production of an automobile, the processing of customer orders, the creation of information products and services, or the purchasing of raw materials in a manufacturing environment -- often require a complex set of activities, as well as overall coordination and orchestration. Historically, workflow has been supported in organizations via manual, time-intensive processing, which leads to longer processing cycles and costly solutions.

Workflow management has also been closely associated with BPR, and BPR has, unfortunately, obtained a rather dubious reputation. This is due in part to naïve attempts at massive business process change in corporate environments, which are rarely conducted in a phased, controlled, and segmented manner. Moreover, the rapid pace of change, significant impacts across organizational boundaries, and occasional lack of fundamental market forces necessary for large-scale reengineering efforts have also led to questionable results. This is certainly a shame, as workflow technologies and reengineering are very much complementary.

Workflow technologies are often utilized for automation and associated cost-reduction purposes, yet BPR and supporting workflow technologies also offer significant potential in revenue generation. While cost-reduction will likely continue as a catalyst for corporate workflow initiatives, the ability to open new markets, improve revenue potential, and aid in the production of new products and services will also serve to expand workflow opportunities. The opportunity, then, is to leverage workflow management capabilities in order to gain a competitive advantage.

Workflow management also provides a medium by which business and technology solutions can be unified in a disciplined manner. All aspects of delivering solutions -- organizational alignment, corporate culture, the manner in which technology and business solutions are approached, the manner in which information systems are designed, and the formalism presented in describing and agreeing upon standard business processes -- are significantly improved by the proper use of workflow. The focus on cross-organization business process definitions, the implicit and necessary support for the free flow of information across organizational boundaries, the automation of core business processes, and the orchestration of business process activities provided by workflow management all serve as primary catalysts for change.

Traditionally, workflow has played a fundamental role in image processing and document management. Applications in this arena typically utilize forms-based processing, whereby electronic forms are captured, updated, and routed among participants. This form of workflow has long been used by the insurance industry, where document management and claims processing are naturally suited to automation. These document- and image-centric workflow solutions form the roots of workflow technology. Workflow technology has more recently been embedded in business applications and custom software, incorporated into intelligent messaging and groupware engines, and made available as pure workflow products.

Figure 1. Workflow architecture layers

Workflow management architecture
Workflow architecture can be represented by a number of layers:

• Business process models: Represent the formal definition of the business process in a manner suitable to interpretation and execution by a workflow engine. These serve as templates representing the set of activities making up the business process, along with additional rules, routing instructions, etc.

• Business cases: Represent specific instances of a business process model. For example, a business process might be defined to represent the establishment of a purchase order, whereas a business case would represent a specific occurrence (e.g., the purchase of 100,000 pounds of steel).

• Folders: Indicate a group of documents and data, which represent arbitrary combinations of data types (e.g., text, data, video, image, etc.), used to process business cases.

• Business process rules, roles, and routing: Govern the overall processing of activities, including the routing of requests, the assignment or distribution of requests to designated roles, the passing of workflow information from activity to activity, and the dependencies and relationships between business process activities.

• Data: Required to analyze workflow process instances on the bottom layer; may include raw data, text, images, etc.

Workflow routing can vary in complexity and follow three patterns: sequential routing, parallel routing, and conditional routing. Parallel routing permits some degree of concurrency by allowing multiple branches of workflow execution to occur, while conditional routing uses simple to complex conditional logic to control the flow of workflow execution. For example, a loan applicant's credit rating may be based on his or her credit history, past delinquent balances, average outstanding balance, income level, etc. In these cases, factors can be defined, then associated with the workflow process specification and other related activities to assign the applicant a credit rating.

Workflow management standards
The WfMC, which was founded in August 1993 and currently has 200 members, is an international coalition of workflow vendors, users, and industry groups. The coalition's goal is to promote the use of workflow and establish workflow management standards by establishing common terminology, reference models, and interoperability between workflow management products. The following is a brief summation of workflow management standards and architectural components, as defined by the WfMC.

• Reference model: A high-level view of the major components in the workflow environment, as illustrated in Figure 2.

  Figure 2. WfMC workflow reference model

• Process definition tools: The means by which business process specifications can be defined. The business process specification allows the activities comprising a business process to be specified, along with associated data interchange requirements needed between workflow activities or steps, relevant sequencing instructions, and routing instructions.

  The completed process specifications serve as a template to create specific instances of a business process. Process definitions also define the specifications for activities that make up the business process, along with relevant activity sequencing, routing, and dispatching instructions. An activity is defined as a work unit that forms one step in an overall workflow process. An activity could require manual intervention and processing, or it could be a fully automated software process. Workflow vendors provide graphical user interface tools to aid process specification. Workflow engines use these specifications, along with associated metadata, to control processing.

• Administration and monitoring tools: Administration involves the use of workflow services, and the configuration, management, and control of the overall workflow environment. Typically, an administrator is assigned privileges to perform these functions. Examples of workflow administration and monitoring functions include:
  • Establishment of users (usernames, passwords, role specifications)
  • Assignment of work items
  • Exception and error processing
  • Event generation and notification
  • Auditing
  • Tracking and reporting of workflow processing results
  • Tracking and reporting of aggregate statistics
  • Business process versioning and change management
  • Monitoring of workflow execution results

• Workflow client applications: These apps interact with workflow management systems to utilize workflow services. (The term client is used in this context to indicate a software component, and need not be restricted to an application on the client side of client/server architecture.) Interaction between the client application and the workflow engine occurs via an interface utilizing the concept of a worklist, a queue of work items assigned to a specific user.

• Invoked applications: A software application that is invoked or initiated by the workflow engine in order to perform the processing and automation of work items. This application can be handled in a synchronous or asynchronous manner, and can communicate with the workflow engine via an interface (interface 3 in Figure 2).

Enactment service and workflow API
Workflow enactment services provide a runtime execution environment in which business process instances are created, managed, controlled, and executed. Standards offer flexibility to workflow management vendors by supporting the enactment service in a centralized or distributed environment. In the distributed approach, numerous workflow engines operate in concert to achieve benefits in functionality and scalability, or support the interaction between execution environments or platform-specific applications and tools. In these cases, the synchronization of process and related process control information across workflow engines is necessary.

Business applications interface with an enactment service via workflow management APIs (WAPIs) and interchange formats. The WAPI enables interoperability between business applications and components of workflow management systems. Interchange formats provide support for the automated interchange of workflow information between different workflow management products. A significant improvement in workflow technology can be gained by using XML.

Workflow engines
Workflow engines provide a runtime execution environment and manage the overall processing and execution of workflow process instances. By referencing the business process specifications and rules that govern workflow processing, the workflow engine is able to orchestrate the necessary processing. Operations processing functions provided by the workflow engine include:

• Interpretation of business process specifications
• Creation of new business process instances
• Execution of business process instances, and management of that execution
• Navigation between work items (i.e., activities) of workflow process instances
• Management and control of workflow control information
• Routing of requests and communication of data interchange between workflow process instance activities

Workflow management and the extended enterprise
The use of workflow in the extended enterprise offers a particularly powerful capability. Much progress has been made, particularly in the manufacturing arena, by linking the players of this extended enterprise (for example, manufacturers, suppliers, distributors) in support of just-in-time manufacturing initiatives. Workflow techniques and automation mechanisms between trading partners -- whether supported via electronic data interchange (EDI) or custom interface approaches -- have provided a primary vehicle to more effectively manage processes, manufacture products, and reduce delivery times. The net effect: improved productivity levels and customer satisfaction, faster deliveries, and increased profitability. Applicability of the extended enterprise notion beyond manufacturing (for example, sales, marketing, customer management, etc.) is high.

While EDI techniques are successfully employed in this extended enterprise, they are also quite complex and unwieldy to manage. XML technologies will become a powerful and compelling solution in reducing this complexity. The benefit of XML integration -- a common language and set of rules of operation across workflow engines -- will also administer improved interoperability of workflow solutions across organizations and business partners. XML will facilitate the use of a common and dynamic business language and increase shared business process specifications. (See Resources for more on XML and EDI integration.)

The extended enterprise and associated expansion of the value chain model will be the basis of future growth in workflow technology. The role of workflow in supporting these cross-enterprise ecommerce initiatives will be profound, and will offer tremendous opportunities in allowing businesses to achieve the next level of efficiency, cost reduction, and revenue generation. This extended enterprise offers many benefits, including the capacity to:

• Govern and orchestrate the business processes and process/information exchanges between intercompany business partners (for example, consumers, manufacturers, distributors, suppliers of raw materials, etc.)

• Support the automation of these business processes across organizational boundaries and business partners

• Streamline processes and reduce manual paperwork and tracking costs

• Share a common knowledge base, business process set, and interchange language between trading partners

• Capture metrics and statistics that provide improved management and control, not only of business processes, but also in supporting organizational structures, forecasting and scheduling workloads, and streamlining overall operations

Figure 3 illustrates a network model depicting business partners interacting in the fulfillment of a customer order. This model assumes the use of workflow management product(s) by each business entity.

Figure 3. Manufacturing scenario

This scenario relies on intercompany workflow processes that represent an exchange of data and process information between the consumer, manufacturer, and suppliers of raw materials. It also assumes various internal workflow processes utilized by the manufacturer to control internal and raw materials acquisition and production activities. Based on business policy agreements reached between consuming and manufacturing entities, it's feasible that consumers could receive a detailed, real-time view into the activities being undertaken to fulfill their order. The benefits include:

• Enhanced and seamless automation across business partner organizations

• Reduced costs and controls of overall processes

• Up-to-the-minute order statuses for business partners and end consumers

• Reduced manual activities and associated paperwork (for example, invoices, shipping manifests, purchase orders, etc.)

The use of workflow technology to support ecommerce in the extended enterprise model presents a number of challenges, as well:

• Agreement between business partners on processes, interface protocols, and semantics is necessary

• Reliable computer systems and networks that support a necessary degree of concurrency, performance, scalability, and availability are required

• Use of interoperable workflow products by communicating partners is desired, but not required

• Adequate electronic verification schemes must be in place

• Adequate security measures that ensure overall process and data integrity while also providing appropriate authentication and authorization measures are a necessity

• Appropriate process monitoring, administration, logging, and auditing capabilities are required

• Workflow vendors must be ready to support intercompany workflow, particularly with regard to transport and security concerns

Scalability and performance
While technologies are in place today that support the enterprise-level usage of workflow management, a key to future success lies in ensuring that high performance and scalable workflow solutions are developed.

Oddly, a number of workflow vendors continue to rely on standalone and nonscalable architectures that are sometimes based on single-threaded models without support for replication or partitioning capabilities. This seems particularly strange in light of the fact that extensions to existing workflow management architectures that support dynamic load balancing, intelligent partitioning of workflow execution, and monitoring facilities to effectively manage and control distributed workflow engine execution are certainly achievable. In fact, workflow is well suited to these highly scalable environments, particularly when coupled with a network built to industry standards, good system management, and middleware tools and technologies.

Figure 4 illustrates a high-level conceptual architecture that leverages distributed workflow services, dynamic load balancing, statistical process metrics and data collection, and distributed remote management and monitoring facilities. The statistical data collection capability collects statistics for process level information, activity execution, aggregate and event statistics, maximum and average duration of processes and activities, etc. These statistics are not only critical in measuring and validating improvements made to business processes, but are also crucial in identifying and resolving performance problems. Statistical information is also used to support rules-based dynamic load balancing and partitioning schemes, and to adjust workflow engine capacity levels to meet performance and throughput requirements.

Figure 4. Workflow platform scalability

Many vendors have made progress in designing high-performance platforms, but few offer the scalability necessary to support the complex enterprise-class platforms needed for large-scale workflow deployments. The enhanced integration of the Internet and commercial middleware solutions (for example, CORBA, MOM, and DCOM) will no doubt help facilitate these improvements. Message-oriented middleware, which provides highly scalable synchronous and asynchronous communications and advanced queuing capabilities, is particularly well suited for the task.

Workflow transactions
Business transactions used to be managed on mainframe platforms to accommodate transactions that ranged from the very simple to the complex and large grained. The advent of client/server and distributed systems made business transactions manageable across multiple operating and database management systems.

As distributed client/server systems increased in usage, end users relied more on transaction monitors to ensure the overall transactional integrity of business requests. As business application complexity and the use of distributed resources increased, so too did the complexity of these large-grained transactions, sometimes to the point of degraded performance and reduced transactional integrity and application reliability; there was also the damaging side-effect of artificially constrained software reuse.

In contrast, the workflow model relies on a defined business process consisting of discrete activities subject to rules defined for the business process. Each activity is assured transactional integrity, while the set is entirely controlled by the workflow engine. This system provides added support and flexibility, as well as reuse of software assets as software components. It also reduces the need for overly large and complex transactions dependent on increasing resource needs, while providing recovery capabilities in the event of a processing failure. Moreover, formally specifying activities as a workflow-based business processes provides additional management and control benefits. At any point, an interested party can determine the exact status of an execution, including which activities have been processed and which are still pending.

Workflow management, now and in the future
In recent years, the general perception in some circles had been that productivity increases and cost gains afforded via the application of process automation, particularly in the manufacturing sector, had reached their maximum potential. However, in the past two years the explosion of the Internet, wide-scale deployment of intranets and extranets, and the expansion of ecommerce (B2B and B2C) have put these mistaken assumptions to rest. In fact, we have only just begun to reach the potential benefits available. A number of complementary technologies have also converged that apply workflow management to achieve further degrees of process automation. These complementary technologies include, but are not limited to, the following:

• XML
• More powerful, stable, and scalable middleware
• Component-based software development approaches
• More flexible software (both business and infrastructure) frameworks
• Improved understanding and capabilities to utilize object-oriented technology

Due to greater understanding and use of object-oriented approaches and technologies, as well as improved software development environments, scalable and high-performance middleware, and expanded use of the Internet as a foundation to support business solutions, we will continue to see further progress in automating business processes and reducing inefficiencies.

BPR and the use of workflow management (both principles and technologies) will also provide value in reducing the overall cost of supporting these business solutions. Furthermore, increasing demand and expanding marketspace for enterprise-level solutions, namely EAI, and the expanding role of Web-based transactional systems, will foster a further need for intelligent workflow management solutions.

Workflow marketplace

Workflow market penetration has not achieved its anticipated degree of success. In fact, only recently have signs indicated serious inroads to corporate systems. Reasons for limited market penetration are complex, but they include:

• Limited scalability of past workflow solutions

• Limited adherence to workflow standards and fragmentation of the workflow marketplace

• Lack of integration of workflow products with commercial middleware solutions

• Limited integration and packaging of workflow products with other complementary technologies

• Limited support for comprehensive workflow environment monitoring, statistical collection and reporting, distributed partitioning, dynamic load balancing facilities, and event management

• Vendors' reliance on workflow capabilities embedded within their software products

The last item has probably constrained adoption rates more than anything else. Often, corporations are forced to adopt workflow solutions and/or specific application development platforms with embedded workflow, thus limiting their ability to integrate these solutions with core enterprise architecture and deployment platforms. Further complicating the situation is a need to develop additional supporting infrastructure and integration with middleware solutions. Given that the significant area for future workflow management growth is likely to be the use of workflow with core business applications, these issues must be resolved.

The emergence of service-based architectural solutions, combined with distributed object and component-based platforms, will prove to be a significant opportunity for convergence and utilization of workflow technologies. Distributed objects, utilized in Web-based applications, and combined with highly scalable middleware platforms, provide an ideal framework upon which workflow benefits can be achieved. The excellent efforts being undertaken by the Object Management Group (OMG) and WfMC in establishing standards and frameworks in these areas is most promising.

Additional complementary technologies -- namely component-based software development (Enterprise JavaBeans, for example), EAI, and XML -- are all likely to prove beneficial in expanding the capabilities of workflow technology. These technologies will also provide broader appeal to companies requiring enterprise solutions and are likely to play a role in similar future workflow architectures. In order to achieve desired business ends, architects must be aware of these emerging technologies and how to integrate them.

In order for enterprise-level workflow to fulfill its potential, needs must be addressed in the areas of:

• Adequately training personnel

• Reducing cultural and organization barriers that limit cross-organizational and intercompany initiatives

• Divergent and sometimes incompatible technology and business objectives

• High-level management understanding

Fortunately, market demands and dynamics will continue to be the catalyst for these and other changes.

Ovum has published estimates that state that the growth rates of general purpose workflow products will reach over $1 billion by 2000, with an estimated annual growth rate of approximately 38 percent. Embedded workflow product expenditures are expected to grow to $319 million by the same year, with an estimated annual growth rate of 130 percent. Augmented by further improvements to core workflow technology, improved understanding and application of workflow technology, and better adherence to standards, we could see even more significant increases in the usage of workflow technology.

About the author Mike Conger is a senior technical architect with Cambridge Technology Partners, providing enterprise-class information and architecture services and supporting organizations to leverage advanced Internet, object-oriented, and middleware technologies in achieving business solutions. Cambridge Technology Partners, a consulting and systems integration firm, offers management consulting, process innovation, custom and package software deployment (including ERP applications), networking, and training.

	Resources and Related Links
	General workflow resources Beyond Reengineering: How the Processed-Centered Organization is Changing Our Work and Our Lives, Michael Hammer (Harper Business, 1996): http://www.amazon.com/exec/obidos/ASIN/0887307299/sunworldonlineA "Extensible Markup Language (XML)," World Wide Web Consortium (February 10, 1998): http://www.w3.org/TR/1998/REC-xml-19980210 The Object Management Group Web site: http://www.omg.org "XML: The future of EDI?" Uche Ogbuji (SunWorld, February 1999): http://www.sunworld.com/swol-02-1999/swol-02-xmledi.html Workflow integration initiatives The Workflow Imperative, Thomas Koulopoulos (John Wiley & Sons, 1997): http://www.amazon.com/exec/obidos/ASIN/0471286850/sunworldonlineA Workflow technology and practices Reengineering the Corporation, Michael Hammer and James Champy (Harper Business, 1994): http://www.amazon.com/exec/obidos/ASIN/088730687X/sunworldonlineA WfMC Workflow Handbook 1997, Peter Lawrence (John Wiley & Sons, 1997): http://www.amazon.com/exec/obidos/ASIN/0471969478/sunworldonlineA Workflow management business benefits, strategies, and standards Various workflow management resources can be found at the WfMC Workflow Management Coalition Web site: http://www.wfmc.org World Class Manufacturing: The Lessons of Simplicity Applied, Richard Schonberger (The Free Press, 1986): http://www.amazon.com/exec/obidos/ASIN/0029292700/sunworldonlineA Other SunWorld resources Full listing of previous IT Architect columns: http://www.sunworld.com/common/swol-backissues-columns.html#itarchitect The SunWorld Topical Index -- a comprehensive listing of all SunWorld articles by subject: http://www.sunworld.com/common/swol-siteindex.html Visit sunWHERE -- launchpad to hundreds of online resources for Sun users: http://www.sunworld.com/sunwhere.html Explore SunWorld's back issues: http://www.sunworld.com/common/swol-backissues.html IDG.net, your one-stop IT resource: http://www.idg.net

	Tell Us What You Thought of This Story
	-Very worth reading -Worth reading -Not worth reading -Too long -Just right -Too short -Too technical -Just right -Not technical enough Comments:   Name:   Email:   Company Name:

If you have technical problems with this magazine, contact webmaster@sunworld.com

URL: http://www.sunworld.com/swol-09-1999/swol-09-itarchitect.html
Last modified: