Introducing “Continuous Architecture In Practice”

It has been a few years since we published the first edition of “Continuous Architecture”, and we felt that some rework may be in order, as much has changed since our book was first published, especially in the technology domain. However, what started as a simple revision turned into a major rework, so the new book is very different from the original “Continuous Architecture” and not only because this book has the added benefit of Eoin Woods as an additional author.

While “Continuous Architecture” was more concerned with outlining and discussing concepts, ideas and tools, “Continuous Architecture in Practice” provides more hands-on advice. It focuses on giving guidance on how to leverage the continuous architecture approach and includes in-depth and up-to-date information on topics such as Security, Performance, Scalability, Resilience, Data and Emerging Technologies.

In our new book, we aim to revisit the role of architecture in the age of Agile, DevSecOps, Cloud, and Cloud Centric Platforms. We want to provide technologists with a practical guide on how to update classical software architecture practice, in order to meet the complex challenges of today’s applications. We also revisit some of the core topics of software architecture. This includes the role of the architect in the development team, meeting stakeholders’ quality attribute needs, and the importance of architecture in achieving key cross-cutting concerns including security, scalability, performance and resilience. For each of these areas, we provide an updated approach to making the architecture practice relevant, often building on conventional advice found in the previous generation of software architecture books, and explaining how to meet the challenges of these areas in a modern software development context.

We have grouped the new book’s chapters into four distinct parts. In the first part, we provide context, define terms and provide an overview of the Case Study (further described in the Appendix) that will be used throughout each chapter. We believe that using a consistent and well defined case study, based on a simplified trade finance (letter of credit) system, makes the book more interesting and practical. In the second part, our key ideas are laid out, providing the reader with an understanding of how to perform architecture work in today’s software development environment. In the third part, we explore a number of architecture topics that are central for developing modern applications: Data, Security, Scalability, Performance and Resilience. For each topic, we explain how software architecture, in particular the Continuous Architecture approach, can help to address the particular topic, while also working in an agile way that aims to continually deliver change to production. Finally, in the fourth part we look at what is ahead. We discuss the role of architecture in dealing with emerging technologies and conclude with the challenges of practicing architecture today in the “Agile and DevOps” era, as well as potential ways to meet those challenges.

Since you are reading this blog, we expect that you are familiar with the classical fundamentals of the field, perhaps from books like Software Architecture in Practice – SAiP (Bass, Len, Paul Clements, and Rick Kazman. Software architecture in practice. Addison-Wesley Professional, 2012) or Software Systems Architecture – SSA (Rozanski, Nick, and Eóin Woods. Software systems architecture: working with stakeholders using viewpoints and perspectives. Addison-Wesley, 2012) . However you may want to update your approach to meet the challenges of today’s fast moving software development environment. This new book may also be of interest to software engineers interested in learning about software architecture and design, and who will be attracted by our practical, delivery-oriented, focus.

To keep the scope of this book manageable, and focused on what has changed since our last book, we assume that you are familiar with the basics of mainstream technical topics such as information security, cloud computing, microservice-based architecture and common automation techniques such as automated testing and deployment pipelines. We expect that you are also familiar with the fundamental techniques of architectural design, how to create a visual model of your software, and associated techniques such as the Domain Driven Design (DDD) approach. For those who feel unsure about architectural design fundamentals we would suggest starting with a well-defined approach like the SEI’s Attribute Driven Design (Cervantes, Humberto and Rick Kazman.  Designing Software Architectures: A Practical Approach. Addison-Wesley, 2016. The AAD approach is also outlined in Chapter 17 of SAiP) or a simpler approach such as the one outlined in Chapter 7 of SSA.  Software modeling, although neglected for a few years, seems to be returning to mainstream practice.  For those who missed it first time around, Chapter 12 of SSA provides a starting point and Simon Brown’s books (Brown, Simon.  Software Architecture for Developers.  Lean Pub.  https://leanpub.com/b/software-architecture) are a more recent and very accessible introduction to it.

The other foundational architecture practice that we don’t discuss in this new book in how you assess a software architecture.  This was covered in Chapter 6 of our previous book, Chapter 14 of SSA and Chapter 21 of SAiP.  You can also find a lot of information about architectural evaluation methods like ATAM via an Internet search. 

We also assume an existing knowledge of agile development and so do not provide in-depth discussions of software development life cycle processes such as Agile, Scrum, or SAFe, or discuss software deployment and operation approaches, such as DevSecOps in any depth. We deliberately do not include details on any specific technology domain (e.g. database, security, automation). We of course refer to these topics where relevant, but we assume our reader is generally familiar with them. We covered these topics, except for technology details, in “Continuous Architecture”.

The foundations of software architecture haven’t changed in the last four years. The overall goal of architecture remains to enable early and continual delivery of business value from the software being developed. Unfortunately, this goal isn’t always prioritized or even understood by many architecture practitioners.

The three of us call ourselves architects, because we believe there is still no better explanation of what we do every day at work. Throughout our careers covering software and hardware vendors, management consultancy firms and large financial institutions we have predominantly done work that can be labeled as software and enterprise architecture. Yet, when we say we are architects we feel a need to qualify it; as if an explanation is required to separate ourselves from the stereotype of an IT architect that adds no value. Readers may be familiar with an expression that has gone something like this: “I am an architect but I also ‘deliver/write code/engage with clients…’ (Fill in with your choice of an activity that is perceived as valuable).”

No matter what the perception, we are confident that architects who exhibit the notorious qualities of abstract mad scientists, technology tinkerers or presentation junkies are a minority of practitioners. A majority of architects work effectively as part of software delivery teams, most of the time probably not even calling themselves architects. In reality, all software has an architecture (whether it is well understood or not) and most software products have a small set of senior developers that create a workable architecture whether or not they document it. So perhaps it is better to consider architecture to be a skill, rather than a role.

We believe that the pendulum has permanently swung away from conventional software architecture practices and perhaps from conventional Enterprise Architecture as well. However, based on our collective experience, we believe that there is still a need for an architectural approach that can encompass Agile, Continuous Delivery, DevSecOps and Cloud Centric computing, providing a broad architectural perspective to unite and integrate these approaches to deliver against our business priorities. The main topic of this book is to explain such an approach, that we call Continuous Architecture, and show how to effectively utilize this approach in practice.

“Continuous Architecture In Practice: Software Architecture in the Age of Agility and DevOps” https://www.amazon.com/gp/product/0136523560?pf_rd_r=W6DHS5HCKQNZYKBMWF22&pf_rd_p=5ae2c7f8-e0c6-4f35-9071-dc3240e894a8

---

Role of the Architect

 

The role of the architect is widely accepted in the world of technology, but not clearly defined.  This blog post provides a summary of the views we expressed in our book[i] as well as in an article published in IEEE Software[ii].

In general, the term architect exemplifies individuals who make key decisions about the structure and behavior of a system – be that software or other technology areas. Though software architect is the most commonly used term, there are network architects, storage architects, information and security architects. You could even say that any technology subject area has people that identify themselves as architects.

All these types of architects appear to share the following characteristics:

• Architects have deep knowledge in the domain of interest.
• They mostly do not directly implement (some architects can create executable prototypes during architecture sprints or runways), instead dealing with higher levels of abstraction by defining the structure of a system in the domain of interest.
• They have explicitly or implicitly the authority of making decisions in the domain of interest

In his book Mythical Man-Month[iii] Frederick Brooks talks about the conceptual integrity of a software product. This is a good place to start for defining the role of the architect – basically they are accountable for the ‘conceptual integrity’ of the entity that is being architected.

Architects deal with a higher level of abstraction than implementers, developers or engineers, do. Their concerns include the main components of the system and how they interface and they continuously validate the architecture against different scenarios. They need the ability to traverse different levels of abstraction between concepts, models and implementation details.

The role of the architect in agile development is hotly debated in the industry. Some Agile practitioners believe that there is no need for an overall architecture, since the architecture “emerges” through the implementation of stories and refactoring. We believe that such a view applies only to simple and fairly self-contained projects. If you are architecting a product or try to operate at a large scale you do need to think of the architecture – however, this does not mean that you need a separate architect, but the role of the architect is still crucial to success.

We recommend three focus areas for defining the role: Unit of Work, Product Focus and the Realized Architecture.

Unit of Work

An interesting way of considering architect role is to focus on their key activities. In other words, what is the key unit of work of an architect? Is it a fancy diagram, a logical model, a running prototype? We believe that the basic unit of work for an architect is an architectural decision. As a result, the most important output of any architectural activity is the set of decisions made along the software development journey. Surprisingly, little effort is spent in most organizations on arriving at and documenting architectural decisions in a consistent and understandable manner.

We can state that, the key responsibility of an architect is to drive architectural decisions to a conclusion.

We purposely used the term drive to a conclusion rather than make. The term drive to a conclusion implies that the architect engages with stakeholders and works through the options to arrive at a decision. It does not mean that an architect can abdicate responsibility for making sure that architectural decisions are made. An architect must make sure that decisions are made in a timely manner.

Product Focus

As explained in our book, we believe that architects should focus on architecting products, not just solutions for stand-alone projects. It follows that the architect should operate as part of a team – in this case a product team. This implies that communication and collaboration skills become highly relevant.

Within the context of a single product, the architect plays a key role in the product team. Her/his main responsibility is to balance the demand from the client as represented by the product manager, with the delivery focus provided by the product delivery manager. The architect has to make sure that the architecture of the product is coherent and sustainable. Especially in the initial phases of a product life cycle, there is often a “negotiation” between product management and the “art of the possible”. Managing this conversation is a key responsibility of the architect.

Realized Architecture

The real architecture of a system is represented by the code that is running on the physical infrastructure. We will call this the “realized architecture.”We believe that the days of an architecture only represented as a set of documents are over.

The role of the architect is to understand, influence, improve and communicate the Realized Architecture.

Therefore, it is important for an architect to understand the code running on the physical architecture.

Final Definition

We can summarize our views by defining the role of the architect as:

An architect is responsible for enabling the implementation of a software product by driving architectural decisions in a manner that protects the conceptual integrity of the software product.

 

 

[i] M. Erder and P. Pureur, Continuous Architecture: Sustainable Architecture in an Agile and Cloud-Centric World, Morgan Kaufman, 2015

[ii] M. Erder and P. Pureur, What’s the Architects Role in an  Agile, Cloud-Centric World, IEEE Software Vol 33, No 5, pp. 30-33

[iii] Brooks, Frederic, 1975 & 1995, The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley, ISBN 0-201-00650-2 (1975 ed.), 0-201-83595-9 (1995 ed.)

Delaying Decisions in Continuous Architecture

Delaying Design Decisions Yields Better Results!

In the previous installment of our “Continuous Architecture” blog as well as in our book, we discussed how to effectively capture requirements.

The second Principle of Continuous Architecture provides us with guidance to make architectural and design decisions to satisfy quality attribute requirements, and not to focus exclusively on functional requirements. Functional requirements often change frequently, especially if we are architecting one of those “Systems of Engagement” delivered over a mobile device whose user interface is likely to change frequently in response to changing customer needs, competitive response, and ever-evolving mobile technology.

Even quality attribute requirements are subject to change, or at least to underestimation. When a mobile or web application’s popularity goes “viral”, even the most carefully crafted applications can collapse under the unexpected load. In addition, performance and availability targets may be vaguely described as Service Levels Agreements (SLA’s) or Objectives (SLO’s) are not always clear. A common practice is to err on the side of conservatism when describing those objectives which may result in unrealistic requirements.

We recommend making design decisions based on known facts – not guesses. In our book and elsewhere in this blog, we describe how to leverage a Six-Sigma technique (Quality Function Deployment or QFD for short) to make sound architecture and design decisions. One of the advantages of the QFD process is that it encourages architects to document the rationale behind architecture and design decisions and to base decisions on facts, not fiction.

An interesting parallel can be made between the principle of delaying design decisions and a technique called simulated annealing which is a probabilistic technique used for solving optimization problems.

Image processing is one area where this technique is used.  Basically if you want to clean up a noisy image, instead of applying a deterministic model you iterate over the image thousands of times. At each iteration you make a decision for a particular pixel based on the value of its neighbors. The interesting part is that in the early iterations you allow a high value of uncertainty in that decision – i.e. you make a probabilistic guess. As iterations evolve you restrict the uncertainty, of the probabilistic jump. So the image cools down, just like steel cooling down when you anneal it – hence the simulated annealing term.

How does this relate to architectural decisions? Simply put if you cool down the image too quickly it cracks – just like steel cracking if you cool it down too quickly. The same concept applies for architectural decisions, if you make too many decisions early on in the process your architecture will fracture.

Below is a picture of a simulated annealing example, showing results of two parameter sets used in the algorithm (bottom left and bottom right quadrants)

As we move away from waterfall application development lifecycles involving large requirement documents and evolve toward the rapid delivery of a viable product, we need to create Minimum Viable Architectures (MVAs) to support the rapid delivery of those products. This concept is discussed in more details in Chapter 7 of our book and in one of our blog entries.

To that effect, limiting the budget spent on architecting is a good thing; it forces the team to think in terms of a Minimum Viable Architecture that starts small and is only expanded when absolutely necessary. Too often, a team will solve problems that don’t exist, and yet fail to anticipate a crippling challenge that kills the application. Getting to an executable architecture quickly, and then evolving it, is essential for modern applications.

 

Please check our blog at https://pgppgp.wordpress.com/ and our “Continuous Architecture” book (http://www.store.elsevier.com/9780128032848) for more information about Continuous Architecture

Requirements in Continuous Architecture: Let’s Clarify!

Clarifying Requirements is Important

In the previous installment of our “Continuous Architecture” blog as well as in our book, we discussed an effective approach for capturing requirements. We suggested that thinking in terms of testable hypotheses instead of traditional requirements enables teams following the Continuous Architecture approach to quickly deliver systems that evolve in response to users’ feedback and meet or even exceed users’ expectations. But are we sure that we always understand what the system stakeholders want, even when they are able to precisely quantify those requirements?

Philippe Kruchten tells the following story about the importance of clarifying requirements. Back in 1992, Philippe was leading the Architecture team for the Canadian Air Traffic Control System (CAATS), and the team had a requirement of “125 ms time to process a new position message from the Radar Processing System, from its arrival entry in the Area Control Center till all displays are up-to-date”.

Here is how Philippe tells the story: “After trying very hard to meet the 125 ms for several months, I was hiking one day, looking at a secondary radar slowly rotating (I think it was the one on the top of Mount Parke on Mayne island, just across from Vancouver Airport). I thought … “mmm, there is already a 12-20 second lag in the position of the plane, why would they bother with 125ms…?”

Note: primary radar uses an echo of an object. Secondary radar sends a message “who are you?” and the aircraft respond automatically with its ID and its altitude (http://en.wikipedia.org/wiki/Secondary_surveillance_radar). It looks like this: https://www.youtube.com/watch?v=Z0mpzIBWVG

I knew all this because I am myself a pilot…

Then I thought:  “in 125ms, how far across the screen can an aircraft go, assuming a supersonic jet… and full magnification”. Some back of the envelope computation gave me…about 1/2 pixel! When I had located the author of the requirement, he told me: “mmm, I allocated 15 seconds to the radar itself (rotation), 1 second for the radar processing system, 4 seconds for transmission, through various microwave equipment, that left 1 second in the ACC, breaking this down between all equipment in there, router, front end, etc.., it left 125ms for your processing, guys, updating and displaying the position…” These may not have been his exact words as this happened a long time ago, but this was the general line… Before agile methodologies made it a “standard”, it was useful for the architects to have direct access to the customer, often on site… And in this case being able to speak the same language (French)”.

Philippe’s story clearly stresses that it is important for an Architect to question everything, and not to assume that requirements as stated are absolute.

 

Please check our blog at https://pgppgp.wordpress.com/ and our “Continuous Architecture” book (http://www.store.elsevier.com/9780128032848) for more information about Continuous Architecture

Continuous Architecture and Requirements

Source: Continuous Architecture and Requirements

Continuous Architecture and Requirements

Don’t Think Functional Requirements, Think Faster Time to Feedback

As we discussed in Chapter 1 of the “Continuous Architecture” book and elsewhere in this blog, capturing and managing requirements accurately and timely  – especially Quality Attribute Requirements – is an essential part of the Continuous Architecture approach. Traditionally, the approach to capturing and managing requirements has been based on conducting interviews of subject matter experts (SMEs) in order to document  requirements that the system must satisfy, often documented in voluminous documents which are hard to read and analyze. Ideally, the interviewees should be actual or prospective users of the system being developed, but in practice the interviewers often have to settle for representatives from the business who believe that they are familiar with the way the system is (or will be) used.

Collecting requirements from user representatives who attempt to guess the needs of real users as well as the best ways to satisfy those needs often result in systems that fall short of real users’  expectations. In addition there is often a significant time interval between the interviews and the delivery of the system, and this may lead to further disappointment as requirements  may change due to evolving business conditions.

As Forrester Research Principal Analyst Kurt Bittner states, “The problem is the SME paradigm itself. No one person can represent the needs of all users, no matter how hard they try. The problem goes deeper: The conscious mind often cannot express what is really needed, and only knows what it doesn’t like when it sees it. As a result, the surest path to success is to put something out there that minimally satisfies some need, sometimes called a minimum viable product, and then improve upon that in rapid cycles.”

The objective of the Continuous Architecture approach is to enable the rapid delivery of a minimum viable product that may be designed to satisfy some need or validate some hypothesis, and will continuously evolve as feedback from the users is received. As we describe in the book and in this blog, we achieve this objective by creating a “minimum viable architecture” that also continuously evolve as user feedback is received, and enables the delivery of a system that meets or even exceeds its users’ expectations.

Please check our blog at https://pgppgp.wordpress.com/ and our “Continuous Architecture” book (http://www.store.elsevier.com/9780128032848) for more information about Continuous Architecture.

When Should Continuous Architecture Be Used?

Source: When Should Continuous Architecture Be Used?

When Should Continuous Architecture Be Used?

With respect to release cycles and deployment mechanisms, modern architectures may need to function at multiple speeds. An IT organization is going to have a “portfolio” of applications, some of which can support high rates of change (due to their loosely coupled architectures), and some of which cannot be released very fast because they are tightly coupled and therefore tend to be brittle.

A “Continuous Architecture approach” takes this into account and is able to deal with different release rates. For example, Systems of engagement (especially mobile and cloud applications) lend themselves to a Continuous Delivery approach. Systems of Record and Systems of Operation are usually not a good fit for Continuous Delivery as traditional tightly-coupled architectures often require substantial rewrites in order to keep pace with evolving business needs and enhancements are often postponed until the need becomes critical.

As we discussed in an earlier post, Continuous Architecture is not a formal methodology. We think of it as an approach to adapt formal architecture thinking and discipline to an ever changing and evolving world. Our approach is driven by six simple principles (see figure below) and supporting tools.

As we describe in our book as part of our discussion of Principle 4 (Architect for Change – Leverage “The Power of Small”), loosely coupled architectures enable rapid changes and provide a significant advantage when “future-proofing” architectures.  However very few projects have the luxury of starting from a blank slate, and frequently architects have to deal with legacy, monolithic systems which often contain tightly coupled components. Uncoupling monolithic architectures is often a significant challenge but not an impossible one, and we offer some techniques in our book to deal with this challenge.

Using a Continuous Delivery approach, IT teams can rapidly and safely implement new requirements from the business by creating a regular, controlled, and highly automated delivery pipeline. Using that approach, an IT organization is able to move from a traditional model where business partners specify requirements to one where they switch to “testable hypothesis”. This in turn creates a new, high-speed IT function that sits alongside the legacy IT function. Using this approach, the high-speed IT function can focus on the systems of engagement for a few business areas, and provide a lot of value to the business. However we believe that the Continuous Architecture approach isn’t limited to Agile and Continuous Delivery projects. We have used it on projects using iterative and even waterfall methodologies, and it produces excellent results!

Please check other entries in this blog and our “Continuous Architecture” book (http://www.store.elsevier.com/9780128032848) for more information about Continuous Architecture

Applying Continuous Architecture in Practice

Source: Applying Continuous Architecture in Practice

Applying Continuous Architecture in Practice

 

Continuous Architecture is a set of principles and supporting tools.

We do not aim to define a detailed architecture methodology or development process. Our main objective is to share a set of core principles and tools we have seen work in real-life practice. So applying Continuous Architecture is really about understanding the principles and applying them to the context of your environment. While doing this you can also decide about the tools you would want to recommend.

We are responding to the current challenge of creating a solid architectural foundation in the world of agile and Continuous Delivery. However, that does not mean that applying Continuous Delivery is a prerequisite for adopting the Continuous Architecture approach. We realize that some companies may not be ready to adapt agile methodologies. Moreover, even if a company if fully committed to agile methodologies, there may be situations such as working with a third party software package where other approaches such as Iterative or incremental may be more appropriate.

Does this mean that Continuous Architecture would not work in this situation? Absolutely not.  This is one of the key benefits of the “Toolbox” approach. Its contents can be easily adapted to work with Iterative or Incremental instead of Agile.

Continuous Architecture also operates in two dimensions: Time and Scale

The time dimension addresses how we enable architectural practices in a world of increasingly rapid delivery cycles, while the scale dimension looks at the level we are operating at (such as project, line of business, enterprise, etc…). We believe that the Continuous Architectural principles apply consistently at all scales, but the level of focus and the tools used might vary.

Please check our blog at https://pgppgp.wordpress.com/
and our “Continuous Architecture” book (http://www.store.elsevier.com/9780128032848) for more information about Continuous Architecture