From conception to solution
Techniques and tips to help you architect a system from the ground up
As an IT architect, defining the components and scope of an architecture can be extremely difficult due to the sheer magnitude of the inputs, and the task at hand. This month, IT Architect Rajesh Venkatesh defines a process you can use to discover the requirements for an architecture, synthesize them, and transform them into an appropriate solution. (4,700 words)
This month we explore the thought process of how we, as IT architects, do our jobs. We provide insight as to how one should address a business need with technology. When we work on a project, it is imperative to discover the motivations, biases, opportunities, and costs associated with each problem it presents. Some organizations are more concerned about systems management and execution costs, others invest in the long-term viability of the product or vendor, and still others have specific technical requirements that must be met today -- and not with vaporware capabilities. This month's column discusses these trade-offs as they pertain to an application architecture, yet many of the concepts and processes relate to the overall enterprise architecture strategy.
rchitecture, whether for a single application, a group of applications or for the enterprise, is a key deliverable in software engineering today. Although each of these categories of architecture have similar deliverables, the goals of these efforts are quite different, as are the processes used for each.
In this article we'll focus on a process that will help you in your application architecture efforts.
An architecture overview
An architecture provides structure for the design and implementation of a system. The design and implementation of a system can be either constrained or enabled by the architecture. Architectures are rarely correct or incorrect. They are either more appropriate or less appropriate for a certain system and context.
What makes an architecture appropriate for an application? How do we know if an architecture is appropriate? When I first began working on architectures for software systems, I grappled with these questions and spent a lot of time reviewing and reworking. For those of you who are facing similar questions now, the process presented below attempts an answer.
In a nutshell, an architecture process involves three distinct steps:
Once you complete Step 1, you simply repeat Steps 2 and 3 until you are able to define an appropriate architecture, as shown in the figure below.
Figure 1. The architecture process
In the reminder of this article, I'll explain this process in greater detail. Specifically, we'll cover the concept of "drivers" for the architecture, the differences between drivers and input specifications, various drivers you need to consider, how the information on drivers can be used to create architectures, and ideas for evaluating architectures.
We've got quite a lot to work through here, so let's get started.
User application requirements act as the specifications for the entire application development effort. These requirements focus mainly on what end users expect from the application. If end users want an inventory-management application that sports an easy-to-use GUI with easy reporting capabilities, they're not going to be the least bit concerned whether the solution is based on a two-tier or three-tier architecture. But we know it's easier to change the business logic in a three-tier application than in a two-tier one, making a three-tier solution the better choice, as long as flexibility of business logic is required during the operational life of the system.
Deciding between two- and three-tier models is an architectural decision, and the information for making such decisions isn't necessarily included in the requirements for the application. This decision is based more on the context in which the application is developed and used.
Defining the correct architecture is a matter of trade-offs (both two- and three-tier models have their pros and cons) and fitting solutions to needs. How well you make trade-off decisions determines how well a particular architecture fares for the given application or need. Judgement for making trade-offs is influenced by both system specifications and usage context. This implies that the input for architecture isn't just a specification of what needs to be built, but should also include information on the context in which it is to be used. The term drivers, as it is used here, encompasses the input specifications and the context information. By capturing the drivers for the architecture and "extracting" the information relevant to the architecture, we will learn the architecture requirements.
Drivers provide requirements for the architecture, help in prioritization, and help predict the types of changes the architecture may encounter. Let's say you helped architect a system five years ago. Now, assume all the requests for changes to the system have been documented for the last five years and are made available to you. If it were possible to go back in time with this information, do you think you could have built a better architecture? In most cases, I would expect the answer to be yes.
A system's drivers are the forces that control the changes that will be requested. The goal of the architecture effort is to use the driver information to predict the changes that may be required in the future and architect the system to withstand those changes, or at least reduce the effort of incorporating them.
A number of drivers influence architecture:
Let's take a look at each of these drivers in more detail.
The business goal for the application/architecture is a statement or a set of statements that explains the objective(s) of the organization to be satisfied by the application/architecture. In some cases, additional goals may exist. This is especially true when old systems are being replaced with newer ones. Here are some example business goals:
Secondary goals could be:
We'll use the order-tracking system as our example throughout this discussion.
The business goals do not directly determine what the architectural needs are; however, they do influence other drivers such as the business scenarios, characteristics of the business, and business risks.
Key business scenarios
Based on the broader business goals, some key business scenarios should be identified. These scenarios will be very useful in setting tangible boundaries for architecture and application parameters. They can also be used to develop "test cases" for the architecture. Like business goals, they also provide the framework for other drivers such as execution scenarios and business risks.
Here are two example business scenarios:
Architectural goals and requirements
The next step is to list the goals for the architecture. Architectural goals and requirements provide input that is directly useful in defining the architecture. The goals are primarily technical forms of the business goals and help identify the types of abstractions that will be useful in the eventual solution. For example, to meet the "one-organization" business goal, an architectural goal could be to provide a single data store for customer information to all applications in the enterprise. Other examples include:
Figure 2. Architectural goals
Architectural goals for our example order management system might be as follows:
Some integration needs become apparent from the architectural requirements, but not all of them are obvious. A solid understanding of all the integration needs is required to complete the architecture. I strongly recommend that you list all the integration needs from a business and technology perspective early in the architecture-definition process. Like architectural requirements, the integration needs directly affect the architecture definition. Some of them even affect the choice of the architecture.
Here's a list of several integration considerations:
As may be obvious, the order-tracking system needs to integrate with inventory and shipment systems. Other systems that also may be affected include customer authentication systems, customer profile management systems (in case not all customers have the privilege to track orders), and even credit card systems.
Key execution scenarios
An execution scenario is the way the system is expected to work under certain conditions after the application has been implemented. A typical scenario describes a usage condition and the corresponding system behavior. Scenarios are very useful in "testing" the architecture to see if it's a good fit for the application. The information includes approximate response times, volumes, etc. There should be separate scenarios for normal conditions, exception conditions, and peak-time loads.
In our order-tracking system example, an exception scenario could address inventory systems that are unavailable, and a peak load scenario could address volumes during holiday periods or special sale periods.
Expected lifetime of the architecture
The planned lifetime of the architecture is another important driver. An architecture that is expected to a have a lifetime of 15 years has different needs than one with an expected lifetime of 2 years. The key risks for the different timeframes are quite different, as are the types of change the architecture may encounter and the scalability requirements. Determining the lifetime of the architecture helps you evaluate the probability of certain risks occurring.
For example, the risk of technology becoming obsolete or a change in the DBMS system is low for an architecture that has an expected lifetime of 2 years as compared to one that has an expected lifetime of 15 years. It is also important to note that the lifetime of the architecture can be different from that of the application itself. As applications are more and more integrated, the architectural choices of one application constrain and influence the architecture of all follow-on applications that interact with the processes or data of the original application. When setting architecture timeframes, you should recognize that the lifetime of the architecture is rarely less than that of the application, but quite often is much more. CICS, after all, was originally an architecture for a customer information system.
Characteristics of business
Business goals for the application focus on the high-level business requirements for the specific application, but these are not the only key considerations for architecture trade-offs. Consider an example: An organization has traditionally grown through acquisitions. Most business-critical systems in such an organization should be built such that they're able to quickly integrate with other systems. The typical frequency of acquisitions should provide a reference timeline for such an integration to be completed. If the business performs an acquisition every six months, the time it takes to integrate with other systems cannot be more than six months. Full integration may mean the application should be able to connect to different databases and systems easily. While one might argue that this consideration should be a requirement for the application, it isn't uncommon for such requirements to be overlooked (refer to Tom Gilb's Principles of Software Engineering Management for examples of missed requirements; a link to this book is available in the Resources section).
Spending some time and effort on understanding the characteristics of the business will help to identify these "missed" requirements. Such characteristics of the business help identify changes the architecture should withstand. These probable changes play an important role in the testing of the architecture. Long-term robustness to change is one of the most important ways by which the architecture adds value over its lifetime. Architectures that aren't robust to change lead to high maintenance costs for applications, and may require more frequent rewrites.
If growth by acquisition is a characteristic of the organization needing the order-tracking system, one of the implications is that the architecture should be capable of interfacing with additional inventory systems using different technologies for each interface. Business characteristics are similar to business context, but while a business' context is static, it's characteristics are dynamic.
Risks and probabilities
You must also consider the various risks associated with the system and the architecture. Here's a look at some of the risks you'll need to take into account when defining your architecture:
Try to attach probabilities to each of these risks. Risks help to identify additional change characteristics. Whenever the probability is high, consider the risk to be a characteristic (business or technology) and architect for the same.
Other drivers and considerations
Other drivers that influence your architecture include current technology infrastructure of the organization and the current skill sets in the organization.
While there may be overlap between some of the drivers we've discussed so far, redundancy is necessary to ensure that no key driver is ignored. How a driver is classified is not as important as the fact that it has been identified. Because architecture quality significantly affects the subsequent phases of software development, the extra effort here can really pay off down the road.
From the goals, requirements, characteristics of the business, and risks identified, it is important to list the different types of changes the system can be subjected to within its expected lifetime.
The different pieces of information collected in this process before the architecture is defined play two important roles:
Synthesizing the driver information
At this step you need to categorize the information you collected into architecture needs, change characteristics, scenarios, and risks. Architecture needs include tangibles such as volumes, performance, scalability, etc. Change characteristics are "predictions" of the types of changes you expect within the architecture's lifetime. Scenarios are execution scenarios that provide information about how the architecture and the application will be used. Scenarios are useful for testing. Risks are also useful for testing and may also be an architecture deliverable.
Creating the architecture
Now that you have a good idea of the different drivers for an architecture, we can move into our discussion of creating the architecture itself.
There is no right or wrong way to define architecture. Using the information you collected, you should be able to define several potential solution architectures in a number of ways. How you proceed is mostly a matter of style. You can start with a context diagram and known constraints that won't change. For example, for an Internet-based order-tracking system, you can assume the client will be a browser connected through the Internet to some Web server in the organization. The request has to eventually go to a database or, if there are already other systems that have the information, through an interface to external systems.
Figure 3. Initial architecture diagram
To this initial diagram, you can add different system components. You can identify the components in the following manner:
For example, let's say that the technology used for one of the interfaces in our order-tracking application can change. This leads us to abstract the connectivity function for interfaces to a separate layer and also to keep this layer distinct for each interface. This way, a change in the connection technology for one interface doesn't affect other interfaces, or the rest of the application.
Adding the layers or components to the context diagram gives a logical view of the system, as illustrated below. There can be multiple ways of accomplishing the system's needs, hence multiple logical views.
Figure 4. A more detailed architecture diagram
From a logical view, we work toward the physical by cementing the details of how each layer will interact with the other. Different connectivity options may exist for the interaction (layer-to-layer), and these options may be listed. It is also possible that in order to provide the required scalability or performance or availability, a logical layer may have multiple redundant physical implementations. For example, the order-tracking system may have HTTP-tunneled IIOP between the Web server and the browser and IIOP between the Web server and the application server and JDBC between the application server and the database. In order to provide high scalablity, it may use multiple instances of the Web server and multiple instances of an application server connected to the backend, or even multiple databases storing information on different products. In some cases, such details may mean additional requirements to the system and may require changes to the input and output needs for one or more layers of the system (a routing layer, for example). These details may also impact the logical view.
Having clear responsibilities for each layer is essential to easily understanding the purpose of each layer and also the impact of change on the architecture and application. Having one layer of the application responsible for a particular task reduces the impact of change on the architecture and the application. In our example, all data access logic and code is the responsibility of the data access layer of the application server. This isolates the impact of any data access mechanism change (ODBC to JDBC, for example) to the data access layer. Responsibility-based layering is like normalization in a data model. Problems similar to those of an unnormalized data model can occur in the architectural context if a certain responsibility is given to multiple layers.
If any part of the application other than the data access layer has data access code, a change in data access mechanism needs to be propagated to those parts from the data access layer every time the data access mechanism is modified. Although this seems like a simple task, such dependencies are the first to be missed when modifications occur a few months later. Occasionally, however, the implementation team may decide to have data access code in another layer (analogous to denormalization) for performance reasons. Again, like denormalization, this needs to be clearly documented to reduce the risk of inconsistent changes later on.
Different architectural patterns can be applied to both logical and physical views. For example, a message-oriented middleware (MOM) technology can be used to interface external systems and the order-management system. This approach is appropriate if the system needs realtime data and continues to function even if one of the external systems is temporarily unavailable (a benefit of the MOM approach). Another such pattern, the database-connection-pool mechanism, is useful in improving scalability and performance of data access.
Each component or layer affects scalability and performance of the architecture and the resulting application. It is important to ensure the conformance of each component to the scalability and performance goals for that component and/or the architecture. Take special care to ensure that architectural mismatches do not occur. For example, if the order-tracking application required an audit-trail mechanism, and if this mechanism was implemented by a layer of the application server that is single-threaded while the other layers are multithreaded, we'd have an architectural mismatch. This layer makes the rest of the application server single-threaded, which will probably result in a less scalable application.
Here's a look at our process at this point.
Figure 5. Review of the architectural process so far
Application architecture deliverables include:
Ideally, the interaction should be explained with the help of execution scenarios and include some exception scenarios, as well.
Apart from the architecture itself, you must provide the assumptions you've made and the risks associated with your solution. Example risks follow:
Evaluating the architecture
Now that you have one or more potential architecture solutions, it's time to test them. But how?
The methods available for testing are as follows:
Again, we'll cover each of these evaluation tools in detail in this section.
Architecture isn't something that can be executed directly (at least as of now), so just like most other paper deliverables, an architecture can be subjected to various reviews.
A thorough review incorporates a number of different perspectives:
Each of these perspectives should have a separate checklist of items to look for in the architecture and should use the execution scenarios that exist. For example, the architect perspective is most concerned with the consistency of the exception information flow and its handling, the performance of the different layers, the architecture's ability to handle peak volumes of data and transactions, etc. The scenarios and needs of the architecture are helpful in completing some perspective checklists.
Application of change scenarios
Another important tool for testing architectures is application of change. An architecture will be subjected to different types of change in its expected lifetime. Good architectures should be able to withstand these changes without additional impact. As part of the defining process when capturing drivers, you should have developed several change scenarios. Different change scenarios should be applied to the solution architecture set to study their impact. For example, for the order-tracking application, changes that can be applied include the following:
A number of these changes can happen in the lifetime of an architecture. The probability of occurrence of a change should drive its selection for testing the architecture. Some of these changes may have some impact on the application, but a good architecture keeps the impact of these changes to as few components as possible, or requires little effort to implement the change. The quality of abstraction used for the layers and components helps in reducing the impact. Ideally, the architecture deliverable includes a manual of what needs to be done when different types of changes (the most common) are required.
Some architectures may have significant risks associated with them. This is especially true with architectures that use newer and less-mature technologies, or combine technologies or products that have not been attempted before. In these cases, building proof-of-concept prototypes will help mitigate the risk of the application implementation failing.
Figure 6. Completed process diagram
Using the process
Just as structural architecture processes for a four-room home, a 50-bed hospital, or a 10-story office building are all different, no single process for software architecture fits all situations. This is true for software development in general -- architecture definition is no exception. A universal process would be either too rigorous for some problems or too abstract and high-level for others. The approach outlined in this article lists various aspects you should consider for architecture development. It may be worth while to spend some time up front to identify which aspects will be important for a specific effort and define a tailored process for each effort.
Documenting the process followed, the different drivers considered, and the different change characteristics applied to the solution set will help in bringing some objectivity to the entire exercise. It also helps teams understand why one architecture is chosen over another and also why an architecture may not be a good fit if one of the key driver assumptions change. Several organizations have attempted architecture development only to drop the effort when the project sponsor changed, or when the sponsor decided that not much value was being added. Hopefully following a process (and documenting it) will reduce the risk of effort being wasted.
The approach here can be made more rigorous by collecting metrics of architectural complexity, adding traceability between the solutions and drivers (or constraints), and using some of the evaluation methods "in-process" in the creation phase itself.
About the author
Rajesh Venkatesh works for Cambridge Technology Partners as a national technology architect for the firm's North America Architecture Group, and as such, he participates on a number of systems-integration/architecture projects. In addition, Rajesh is working on Cambridge's Software Engineering Improvement Initiative.
If you have technical problems with this magazine, contact firstname.lastname@example.org