How is Software Architecture Created?
The Visual Architecting Process™ (VAP) integrates what we have learned from great architects, and good as well as failed architectures, to help you create a software architecture that is:
- good—it is technically sound and clearly represented
- right—it meets the needs and objectives of key stakeholders, and
- successful—it is actually used in developing systems that deliver strategic advantage.
The Visual Architecting Process™ (VAP) covers the techniques, including architecture modeling and architecture trade-off analysis, used in creating a technically sound architecture. It covers architectural requirements and prioritization to create the right architecture, together with architecture validation to ensure that the architects and key stakeholders agree that it is indeed the right architecture. And it covers the organizational process steps that help ensure that the architecture is embraced and used informedly so that, ultimately, the architecture is successful.
The focal deliverable of the architecting process is the architecture document (set), motivating and describing the structure of the system through various views. However, though system structuring is at the heart of the architecting process, it is just one of several activities critical to the creation of a good architecture (Figure P1). Architectural requirements are needed to focus the structuring activities. Different architectural approaches tend to yield differing degrees of fit to various system requirements, and modeling and evaluating alternatives, and analyzing tradeoffs among them, are an important part of the structuring phase. Lastly, a validation phase provides early indicators of, and hence an opportunity to resolve, problems with the architecture.
Figure P1. The Visual Architecting Process™ (VAP)
Architectural requirements are a subset of the system requirements, determined by architectural relevance. The business objectives for the system, and the architecture in particular, are important to ensure that the architecture is aligned with the business agenda. The system context helps determine what is in scope and what is out of scope, what the system interface is, and what factors impinge on the architecture. The system value proposition helps establish how the system will fit the users’ agenda and top-level, high-priority goals. These goals are translated into a set of use cases, which are used to document functional requirements. The system structure fails if it does not support the services or functionality that users value, or if the qualities associated with this functionality inhibit user performance or are otherwise unsatisfactory. System qualities that have architectural significance (e.g., performance and security, but not usability at the user interface level) are therefore also important in directing architectural choices during structuring.
Of course, requirements may already have been collected by product teams. In that case, the architecture team needs to review those requirements for architectural relevance and completeness (especially with respect to non-functional requirements), and be concerned with requirements for future products that the architecture will need to support.
Lastly, for the architecture of a product line or family, architectural requirements that are unique to each product and those that are common across the product set need to be distinguished so that the structure can be designed to support both the commonality and the uniqueness in each product.
The architecture is created and documented in the system structuring phase. The architecture decision model guides early decision-making, and helps us to organize the architecture artifacts. These artifacts document (sets of) decisions in terms of models and views, descriptions, explanations, rationale, implications, alternatives considered but not chosen, white papers, etc.
First, the architectural vision is formulated, to act as a beacon guiding decisions during the rest of system structuring. It is a good practice to explicitly allocate time for research—or scavenging for ideas—in documented architectural styles, patterns, dominant designs and reference architectures, other architectures your organization, competitors, partners, or suppliers have created or you find documented in the literature, etc. Based on this study, and your and the team’s past experience, the meta-architecture is formulated. This includes the architectural style, concepts, mechanisms and principles that will guide the architecture team during the next steps of structuring.
The system is decomposed into components (structural elements) and the responsibilities of each component, and interconnections between components are identified. The intent of the conceptual architecture is to direct attention at an appropriate decomposition of the system without delving into the details of interface specification and type information. Moreover, it provides a useful vehicle for communicating the architecture to non-technical audiences, such as management, marketing, and many users.
The logical architecture is the detailed architecture specification, precisely defining the component interfaces and connection mechanisms and protocols. It also provides contextual information about the components and how they should be used in assembling systems.
The logical architecture is used by the component designers and developers, as well as component users (i.e., those that assemble components into systems).
The conceptual architecture forms the starting point for the logical architecture, and is likely to be modified as well as refined during the course of the creation of the logical architecture. Modeling the dynamic behavior of the system (at the architectural—or component—level) is a useful way to think through and refine the responsibilities and interfaces of the components. Component specifications make the architecture concrete. These should include a summary description of services the component provides, the component owner’s name, IID and version names, message signatures (IDL), a description of the operations, constraints or pre-post conditions for each operation (these may be represented in a state diagram), the concurrency model, constraints on component composition, a lifecycle model, how the component is instantiated, how it is named, a typical use scenario, a programming example, exceptions, and a test or performance suite.
An execution architecture is created for distributed or concurrent systems. It is formed by mapping the components onto the processes of the physical system. Different possible configurations are evaluated against requirements such as performance and scaling.
Architecture Tradeoff Analysis
At each step in structuring, it is worthwhile challenging the team’s creativity to expand the solution set under consideration, and then evaluating the different architecture alternatives against the prioritized architectural requirements. This is known as architecture tradeoff analysis (Barbacci et. al., 1998), and it recognizes that different approaches yield differing degrees of fit to the requirements. Selection of the best solution generally involves some compromise, but it is best to make this explicit.
During structuring, the architects obviously make their best effort to meet the requirements on the architecture. The architecture validation phase involves additional people from outside the architecting team to help provide an objective assessment of the architecture. In addition to enhancing confidence that the architecture will meet the demands placed on it, including the right participants in this phase can help create buy-in to the architecture. Architecture assessment involves "thought experiments", modeling and walking-through scenarios that exemplify requirements, as well as assessment by experts who look for gaps and weaknesses in the architecture based on their experience. Another important part of validation is the development of prototypes or proofs-of-concept. Taking a skeletal version of the architecture all the way through to implementation, for example, is a really good way to prove out aspects of the architecture.
Though described sequentially above, the architecting process is best conducted iteratively, with multiple cycles through requirements, structuring and validation. One approach is to have at least one cycle devoted to each of Meta, Conceptual, Logical, and Execution architecture phases and cycles for developing the architectural guidelines and any other materials to help in deploying the architecture (such as tutorials). At each cycle, just enough requirements are collected to proceed with the next structuring step, and validation concentrates on the architecture in its current phase of maturity and depth.
Moreover, a number of architecture teams that we have worked with have stopped at different points, leaving more detailed architecting to the product and component teams. At one end of the spectrum, a very small team of architects created the meta-architecture, and each of the product teams created their own architectures within the guidelines and constraints of the meta-architecture. Other architecture teams created the meta- and conceptual architectures, and a broader team of component owners developed the logical architecture. At the other end of the spectrum, the architecture team developed the entire architecture, all the way to its detailed logical architecture specification. This approach yields the most control over the architecture specification, but is typically fraught with organizational issues (e.g., the "NIH syndrome") that slow or even completely inhibit the use of the architecture.
Architecture projects are susceptible to three major organizational sources of failure—the project is under-resourced or cancelled prematurely by an uncommitted management; it is stalled with endless infighting or a lack of leadership; or the architecture is ignored or resisted by product developers. The organizational process helps address these pitfalls. Two phases—namely Init/Commit and Deployment—bookend the technical process. However, the principal activities in these phases, namely championing the architecture and leading/teaming in Init/Commit, and consulting in Deployment, also overlap with the technical process activities.
The Init/Commit phase focuses on initiating the architecture project on a sound footing, and gaining strong commitment from upper management. The creation of the architecture vision is central both to aligning the architecture team and gaining management sponsorship. A communication plan is also helpful in sensitizing the team to the need for frequent communication with others in the organization. A heads-down, hidden skunkworks architecture project may make quick progress—as long as it is well-led and its members act as a team. However, not listening to the needs of the management, developers, marketing, manufacturing and user communities and not paying attention to gaining and sustaining sponsorship in the management and technical leadership of the organization, or buy-in from the developer community, will lead to failure. The communication plan places attention on balancing the need for communication and isolation, as well as planning what to communicate when, and to whom.
The Deployment phase follows the technical process, and addresses the needs of the developers who are meant to use the architecture to design and implement products. These range from understanding the architecture and its rationale, to responding to the need for changes to the architecture. This entails consulting, and perhaps tutorials and demos, as well as the architects' involvement in design reviews.
It is important that at least the senior architect and the architecture project manager (if there is one) champion the architecture and gain the support of all levels of management affected by the architecture. Championing the architecture starts early, and continues throughout the life of the architecture, though attention to championing tapers off as the architecture comes to be embraced by the management and developer communities.
For the architecture team to be successful, there must be a leader and the team members must collaborate to bring their creativity and experience to bear on creating an architecture that will best serve the organization. This would seem so obvious as to not warrant being said, but unfortunately this is easier said than done. Explicit attention to developing the designated lead architect’s leadership skills, in the same way one would attend to developing these skills in management, is a worthy investment. Likewise, investing in activities aimed at developing the team as a team also has great payoff in the team’s efficacy.
Consulting with and assisting the developer community in their use of the architecture is important in facilitating its successful adoption and appropriate use. These activities are most intense during deployment. However, earlier communication and consulting helps create buy-in the developer community through participation and understanding. This allows the architecture team to understand the developers’ needs and the developers to understand the architecture (and its rationale) as it evolves through the cycles of the technical process.
Tailoring the Process To Your Project
We worked with and studied dozens of architecting projects, and distilled what we believe to be the best practices and pitfalls that would help architects successfully create and deploy their architectures. Based on this experience, we synthesized the Visual Architecting Process™ (VAP). Though this architecting process lays out the activities and guidelines that we have derived from real-world experience, no project that we studied followed exactly this process. Also, every project that we have consulted with or coached, has adapted the process. This has been true of other software development methods, such as SA/SD, OMT and Fusion (Malan, Coleman and Letsinger, 1995). It would appear that a method is not fully embraced by a project team until they have adapted it to their particular project needs. In this regard, methods are somewhat like architectures!
We do encourage architects to use the process, and to work quickly and iteratively, not getting stuck on any model or aspect of the architecture but always keeping to the discipline of moving forward quickly and coming back soon to revise, rework and mature the architecture. Early on, your cycles are through different models (with descriptions, explanations and rationale). Later on you will start to cycle through models and contained code experiments. As soon as the architecture is mature enough to begin construction, you will need to work with project managers and key developers to create the right balance between addressing technical risk and getting architectural elements in place, and launching teams focused on building feature sets or storylines so that users can be involved in construction feedback loops.
But remember, the visual architecting process is very much about bringing stakeholders into the process from the start, and about iterating and maturing the architecture. The point where construction begins and and the point where the architecture is considered stable and under change control are not necessarily the same point, and where they are set is a project-by-project decision. It depends on factors like scope of the architecture, co-location or geographical distribution of the team(s), maturity of the problem domain and experience with this kind of system, etc. There is no tablet of commandments in RUP or agile or anything else that can effectively prescribe where this point should be set for every project, regardless of its context. What your strategy is and internal capabilities are, how the organization is structured, what goals you are trying to address and what resources you have to go after those goals, all matter! Context matters, and experience and good judgment matter.
Scaling Agile with VAP
The VAP is all about being agile even when the complexity of the system, and the organizational unit(s) building it, demands just enough upfront design—for example, to launch concurrent agile teams. What VAP emphasizes and enables is parallelizing the requirements and architecture iterations, with intensive stakeholder involvement as pertinent to the quick cycles. VAP can also be applied on traditionally small agile projects, with just enough design work during each iteration. For larger, complex systems, early VAP cycles use the cheapest possible artifacts (models, prototypes, etc.) for learning quickly about stakeholder value and architectural challenge. This concurrency, together with the principle of stakeholder (including but not just limited to end users) involvement, is a major value and contribution of VAP. We talk about JEDUF and agile architecture in our Getting Past "But" paper.
The Open Group's IT Architect Certification Program Recognizes the Visual Architecting Process: Bredemeyer Consulting's Visual Architecting Process (VAP) is one of the architecture methods recognized by The Open Group; see http://www.opengroup.org/itac/cert/methods.tpl?CALLER=index.tpl
- Barbacci, M. R., S. J. Carriere, P. H. Feiler, R. Kazman, M. H. Klein, H. F. Lipson, T. A. Longstaff, and C. B. Weinstock, "Steps in an Architecture Tradeoff Analysis Method: Quality Attribute Models and Analysis", http://www.sei.cmu.edu/publications/documents/97.reports/97tr029/97tr029title.htm See also the SEI Architecture Tradeoff Analysis Initiative website, http://www.sei.cmu.edu/ata/ata_init.html
- Grace, Ron, D. Coleman, M. Ogush, S. Rhodes, "Experience with Documentation of Software Architectures", published on the HP Software Architecture web site, http://www.architecture.external.hp.com/Download/PracArchDoc.pdf
- Malan, Ruth, "Agile Architecting", TraceInTheSand blog post, September 8, 2007.
- Malan, R. A., R. Letsinger and D. Coleman. "Lessons Learned from Leading-Edge Object Technology Projects in Hewlett-Packard", Proceedings of OOPLSA'95, 1995.
- Malan, R. A., and D. Bredemeyer, "Defining Non-Functional Requirements" (nonFunctReq.pdf, 39kb), August 2001.
- Malan, R. A., and D. Bredemeyer, "Functional Requirements and Use Cases" (FunctReq.pdf, 39kb), June 1999.
- Malan, Ruth and Dana Bredemeyer, Software Architecture: Central Concerns, Key Decisions (.pdf, 124kb), May 2002.
- Malan, Ruth and Dana Bredemeyer, Architecture Strategy, April 2003.
- Malan, Ruth (with help from Ambrose Bierce), VAP Parody, November 2009.
- Ogush, M., D. Coleman, and D. Beringer, "A Template for Documenting Software Architectures", March 2000.
- Youngs, R., D. Redmond-Pyle, P. Spaas, and E. Kahan, "A Standard for Architecture Description", IBM Systems Journal, Vol 38 No 1. http://www.research.ibm.com/journal/sj/381/youngs.html
- Our Software Architecture Action Guide and Architecting Process Action Guides
- Architecture Principles: Definitions and links
- Our talk titled "Action Guides for the Enterprise Architect" given at the Enterprise Architecture Conference in San Diego, 2001
- The books and papers listed under "How: The Architecting Process" in our Bibliography.
- Our Modeling Software Architectures Using UML and Action Guides class and Software Architecture Workshop, as well as our Architecture Vision Workshop.
- Our Enterprise Architecture Workshop (4 days) and Enterprise Architecture Seminar (1 day)
- Our Consulting Services
- Our Architecture Books List.
- Our Tools and Visualization Resources List
Restrictions on Use: All material that is copyrighted Bredemeyer Consulting and published on this page and other pages of our site, may be downloaded and printed for personal use. If you wish to quote or paraphrase fragments of our work in another publication or web site, please properly acknowledge us as the source, with appropriate reference to the article or web page used. If you wish to republish any of our work, in any medium, you must get written permission from the lead author (in this case, Ruth Malan). Also, any commercial use must be authorized in writing by Bredemeyer Consulting.