|Software Architecture and
Requirements and Innovation
System Design and Requirements
In many organizations, we have created a divide between "requirements" and "design" insisting that "design" focuses on the "solution" and "requirements" focuses on the "problem." But we have to recognize that this separation is an artifice.
Requirements "capture" needs to be an active, creative collaboration between the designer of the user experience and those who can shed light on what is needed (including future users, and users of the current version of the system).
Consider these anecdotes:
When Dean Kamen (yes, the inventor of medical devices and also the Segway) asked "where you would put a third eye if you could have one?" everyone said on the back of their head; without hesitation, generally, on the back of the head. But when Dean said, "what about on the tip of your finger?" everyone agreed, without hesitation, that that was much better!
Henry Ford reportedly said “If I had asked people what they wanted, they would have said faster horses."
TWA asked Douglas Aircraft Corporation to produce an aircraft with three engines. When Douglas engineers probed for the reason for this "requirement," that found that the desire was to be able to fly safely even if one engine was disabled (this was in the 1930's when engines were not so reliable). Douglas produced the DC-1 with 2 engines, and proved that it could safely fly with just one engine, taking off from Albuquerque, NM and flying safely to Winslow, AZ. Only one DC-1 was built, but it was the forerunner to the DC-3, which helped make air travel popular and economically viable.
If the architect is to be held accountable for the system being good, right and successful, then the architect cannot be walled off from the critical aspects of the design work that has to do with creatively interpreting and innovating to achieve a compelling user experience and making decisions that will have huge consequences for the life cycle cost of the system!
Which is not to say that the architect (or architecture team) needs to do all the requirements work. But it does mean that the architect needs to be involved. The architect needs to have direct contact with stakeholders. She needs access to the business management to understand the strategic direction and lifecycle cost goals for the system. And she needs access to users, to understand the context into which the system will fit, as well as the user goals that the system will support. Further, the (lead) architect needs to own the architecturally significant requirements, and the decisions about what is architecturally significant.
For many, the necessity of the role of the architect in requirements is self-evident. The architect job descriptions in Quest for Great Architects make it clear that at least some organizations are positioning the architect's role as encompassing requirements and technical strategy setting.
But there are also many organizations that have worked hard to create and fill the business analyst function, and many architects we work with perceive that getting access to business users (and strategic management, for that matter) would be as unlikely as walking on water. The business analysts become gatekeepers of the client relationship, and immediate managers become gatekeepers guarding access to higher levels of management. And no matter how the architect job is positioned in the job ad, or by us as an advocates for what architects need to do, be and have responsibility for, the architects just have to work with what they get served over the wall at them.
This is a disabling proposition for many architects. But there are those for whom it is just part of the challenge. Architects, in this formative era, are part culture changers. In their organizations they have to influence and lead change. The best way to do this is not by being churlish and negative; on the contrary, the way to do this is by being positive and energetic, setting an example of goodwill and partnering effectively with those with overlapping responsibilities.
Requirements, Innovation and Architecture
Ford's Model T and the Douglas DC-1 changed the face of transportation because their inventors innovated beyond what people generally imagined they wanted or was possible. Though the Segway uses revolutionary technology, it has not been transformative in the way that Kamen promised investors. Not every product or application we work on will reshape the landscape of competition in the way that these innovations did, or promised to.
We don't get to work on revolutionary products, or even new products and services, all that often. But even when we are working on incremental changes to existing products and services, architects have a role to play in innovating to create compelling value and differentiating advantage. Ideas will come from a variety of sources: the strident voice of the dissatisfied customer, the eager voice of the customer on the bleeding edge, the fading voice of history and alluring whisper of the future, and the ingenuity of the design and development team.
The architect needs to hear these voices, at least some of the time, to be inspired and inspiring. They are the source of stories that will shape the vision of what is possible, and speak to and win the hearts and minds of sponsors and developers, marketing and project managers, and all the others that play a role in bringing innovation in process and product to realization and eager adoption.
As architects, we need to constantly balance strategic, long-term competitiveness with short-term pragmatics. We need to be the watchdog of complexity and risk, and we need to lead the organization into the future with differentiation where it is valued, and parity where it is not.
When our business decides to modernize IT, we have to decide how much of that initiative is about differentiation and how much is about simply being on a level playing field with our competitors. If it is "core" in the Moore sense (i.e., the edge where we compete and need to have and hold compelling competitive advantage) then we need to innovate to create distinction. If it is "context," and we only need to maintain parity with others in our industry, then we need to figure out what that means over the horizon of the technology investment so that we don't invest only to find ourselves behind and playing catch-up even as the replacement technology is being rolled out.
When we decide to replace the platform for our product line, we need to be careful not to simply create a "cleaned up" version of our current systems using more modern technology. We do have to do that. But we also have to think strategically about how to differentiate into the next era of competition in this product space. Architectures are around for several years (if not decades!), and we only get to substantially reconstruct the platform once every few years if our management team is disciplined about an investment strategy that refreshes our product capabilities every few years.
Yes, as architects in IT and as architects of products we have to think in terms of differentiating capabilities (function and property; service and quality of service; etc.) and we have to think about where we need only accept good enough to be an effective player in our industry. We need to find opportunities to innovate where it matters, and be disciplined and restrained about taking on complexity where it does not matter. We have to be strategic thinkers, and we have to translate that thinking into technical decisions, so that a clear sense of strategy shapes our choices.
We tend to want to do everything better, but some things matter more to our success. The architect needs to find the line that distinguishes what matters, from what does not. The architect, like the strategic manager, has to work toward compelling advantage not just short-shrift wins.
Differentiation and Business Imperatives
The architect needs to work on differentiating, compelling value propositions that deliver competitive advantage. And the architect has to work on resolving what the business must do, simply to be in the game. What are the business imperatives? What are the challenges involved in good enough to be a player and what are the challenges involved in creating distinction from competitors? Distinction in identity, and in value delivered.
The strategic direction and the imperatives of our day-to-day survival translate into drivers that we have to pay attention to as we create or renovate our architecture.
Constraints and Delight
To pick up the train of thought. We need to delight our customers/users, providing an experience that gives them value. We need to ensure that the total cost of ownership, including the cost of quality and in all its guises from frustration and time wasted to the cost of rework and upgrades, is minimized as best we can. And we need to do so within the constraints of our user environment and our development environment, constraints imposed by partners in the value chain, constraints that come from our technology, and so forth. These determine the parameters within which we need to work: development timeframe and cost, what computing resources we must assume, memory footprint constraints, what systems our system must interact with, what components we are obliged to (re-)use, and so on.
All these factors determine the challenges we face as architects, which we have to weigh and tradeoff, to achieve the system goals relating to the quality of user experience, the cost of ownership for the user and the lifecycle costs for the development organization.
Put like that, it seems like a tall order. That, at least, is the best we strive for. The more complex the products and solutions in our competitive space, the more ambitious we are driven to be, to create that competitive edge.
Architecting is about finding the best possible approach to addressing these challenges, recognizing that these forces interact, sometimes in unknown and even unknowable ways.
As architects, one of our most important jobs is to identify and resolve the most significant challenges for the system under design (whether it is a new design, next generation design, or design for renovation and enhancement of the current generation system).
Just being a player in our industry means we face a set of challenges because the state of practice keeps evolving, and if we stagnate we get behind. Then we have to modernize just to get back to a level playing field.
In business, in the arenas in which we actively compete, we have to do more than that—we have to differentiate along one or more dimensions. We need to provide product features or services that stand out from the plethora of other options customers could go with. It is not just about offering some features and attributes that are not yet in our competitor's products. We need to offer some real and compelling unique value; we need to "delight" our customers (see the Kano discussion on our Resources for Architects Strategy Process page).
The architect needs to balance the need to differentiate with the lifecycle cost of every feature and every aggressive quality attribute that is pursued. What we need to do, just to be in the game, constrains what we can accomplish in order to distinguish our products or services and business. For example, some level of security, scalability, disaster recovery, in many systems are threshold attributes not our avenues for differentiation. But we must develop strategies (authentication, encryption and firewalls; load balancing; failover, etc.) to address these challenges. And these strategies have to be implemented, along with every function or service. Just delivering the base level of features and qualities is challenging, given intense competition in most markets.
Others will have a say in what our opportunities are to differentiate (e.g., marketing). And others will have a say in identifying the challenges we face to build and field systems in our domain (e.g., development). The architect needs to play a role in balancing what we would like to do, with what we are able to do given our resources and capabilities.
Moreover, the architect needs to play a role in increasing what we are able to do, and increasing the value of what we attempt to do, by focusing. Focusing attention, enabling specialization and separation, understanding where outsourcing or licensing can be leveraged, reducing complexity and scope where it is not essential to our value proposition. Knowing what to focus on and knowing what to ignore—those are key to success.
Thus the architecture challenges arise from what we need to do because we are in the kind of business we are in, and from what we need to do uniquely well to create competitive distinction. These challenges will come from the environment, from our users and what they care about and are trying to do, and from our business, what we care about and are trying to accomplish and what we have to work with to get there.
Architects and Partnering
An area of skill development that we emphasize for architects is that of partnering. The architect role is relatively new in many organizations, yet the work was being done, more or less well, by people in other roles. This puts the architect in a situation where the architect's responsibilities overlap with those of others, and where they face turf-protectionism from at least three different constituencies: developers, managers and business analysts (or requirements analysts). Agile approaches try to put these back together, both in terms of the roles (eliminating hierarchy) and process (eliminating the "waterfall"), and this works just fine for small co-located teams. For bigger teams, especially those that are not co-located, we have to work harder to figure out roles and divvy up the work so that it can proceed effectively.
So the architect is taking away technical decision responsibility from developers. Hopefully, the architect is leading a consensus-oriented decision process, but even so, too many voices slows progress and not everyone can be satisfied all the time. We have to accept good-enough, or we will never be done! Reality check one!
And the architect is not taking away all technical decision responsibility. In a minimalist architecture approach, the architect defers all decisions that can be made at more local scope of responsibility to the owner at that scope (e.g., to the component owner which might be a tech lead for a complex component, or might be a developer, for a more narrowly scoped component).
References and Further Reading
Architects need to be great at strategy, and need to pay particular attention to strategy and innovation, so here are some more readings on strategy: