Return of Architects

What does a software architect really do?

No one would have dared to ask this question twenty years ago, so let’s look back for a moment. This story is simplified and much shortened so as to keep it interesting.

The golden age

Software architects and software architecture were highly regarded as key components of successful software project and product development.

The role of architecture was deemed so important that new methodologies arrived, new books about architecture patterns and antipatterns written, and entire communities emerged dedicated to software architecture. With the Waterfall approach, there was not just business requirements analysis, but the huge phase of the software architecture design.

The golden age resulted in tens or hundreds of pages filled with graphs and text, and sometimes entire repositories filled with them. I remember my colleague, at the time, doing an architecture project for a whole year and the combined documentation exceeded two thousand pages. The next phase was implementation which was supposed to last four additional years.

Were the systems complex? Yes, they were. Was the project successful? No, it was cancelled. Why? The requirements and technology changed so much that everything that was planned became irrelevant and unnecessary, and the entire architecture documentation ended up in the dumpster.

Of course, there were much more successful software development  projects which ended up with improved business processes, faster product development and an increase in the business agility of the organizations. I was glad to play the role of a software architect in them.

Unfortunately, some architects began to be disconnected from the reality of their enterprises and even the single software solutions. Enclosed in their own isolated world they started to draw even more diagrams to solve the problems from previous diagrams. We even joked about their creations as “crop circles”, strange alien creations that were totally irrelevant to anyone except their authors.

One day, I was asked if I wanted to change jobs and become a Senior Enterprise Architect in a team of five… documentalists. What? Then I felt it, something was wrong, absurd and in the air; this must end and I will be there to witness it.

Agile 1.0, the age of innocence

Agile, initially, didn’t recognize the role of software architects at all. It was just software developers who were supposed to figure out how they wanted to build systems; a famous team of peers with T-shaped roles, no hierarchies and a nothing set in stone architecture. And no documentation at all, because the manifesto said that it preferred human interactions over documentation. So, it was very often misinterpreted as the total freedom to let go of documentation and the software architecture was buried as part of this.

→ Explore the Agile controversy with Avenga

And then came the way of enthusiasm, sprints, scrum meetings, yellow sticky notes and large whiteboards. Architecture, as a relic of the past, was not even mentioned. Code, as the manifest told us, was the only way to verify the progress, and if it worked, who cared about the relic?

Then came the series of failed projects, especially when the project size exceeded the small team of four or five people. Sometimes in the millions and sometimes in the hundreds of millions was wasted on the investment in software initiatives. Something was definitely lost in the Agile transformation and that became a serious problem.

Return of the architects

One of the reasons for so many failures was the lack of proper software architecture and design. Some successful companies continued to do so. . . Agile, no Agile, Scrum or Kanban. Those who were pragmatic always had architects and at least some part of an up-front system design with disciplined evolutions of the architecture.

But now, in the 2020s, software architects and software architecture are definitely back. Not the same architects and not the same architectures, but the recognition of the importance of architecture is back. There are new ways and new tools to make software architecture even more successful than it was in its golden era.

So what do they really do?

Architects, when you don’t look at them and when they are focused on the problem of design, are working mostly with non-functional requirements.

Non-functional requirements – what’s in it for business?

I paraphrased it, but believe me, I’ve heard this question many times. So, I felt the urge to explain it this time, in this article, and I will probably do it again and again.

Functional requirements are much easier for businesses to understand, as they are often functionality, which are features of the system that users can see and interact with on the screen. These are the features and system behaviors that steal most of the attention when businesses collaborate with IT.

Sometimes someone mentions integrations of some data from another system that should be available in the new system, but maybe about some notifications that connect the system data with the big data lake or warehouse.

Non-functional requirements (NFR) are those that are rarely mentioned in conversations, but if the architecture fails to meet them, it usually means a catastrophe for the entire project or a severely impaired user experience. And then, when it fails, nobody has any doubts about the impact on the business’ performance, company image, losing customers and even paying millions in settlements when data leaks happen.

So, combining the right solutions for NFR within the context of business requirements and the IT context of the organization is the key for success. None of these should be treated separately, or isolated, or disconnected, as it will alway result in problems. Architecture, even when dealing with the non-functional part, should never disconnect itself from the business requirements and functionalities.

Our idea here is to show that many invisible yet difficult requirements are the responsibility of architects, so business functionalities can be delivered efficiently to the customers, employees and partners.

Let me give you a few examples of these requirements, which are so obvious that somehow they weren’t even mentioned in any of the meetings.

Scalability

There is an expectation that the system will be efficient for the initial and small number of users, customers and transactions and that it will grow with the increasing number of them. Doing it requires vertical and horizontal scalability and the ability to efficiently run a system’s APIs on multiple processing nodes. How fast will it grow? How much is it going to cost? Too less and too much are both bad choices. And of course, there’s the eternal problem of coupling vs. cohesion.

→ Have a look at Generic API or Back-End for Front-End? You can have both.

Security

Of course, it must be secure as we are serious about our customers and data. Security on the surface is just another password management functionality, but it’s even smaller than the tip of the iceberg. Architecture must take into account this requirement in every aspect, service, component, API and data store. Plus, it requires proper incident tracking that enables fast security fixes for vulnerable components, and ideally without need for any interruptions nor downtime.

Resilience

Nobody wants a system to break at any time, especially when customers and business partners are trying to use its functionalities. And, it is usually solved by a distributed architecture with multiple processing nodes that check for single points of failure and redesign the system to avoid them. Additionally, such a fine grain division of the systems has multiple side effects which needs to be addressed by the architecture.

Evolvability

This is one of the key requirements for modern architecture. Embracing change means we need to create architecture so that it will be adaptable and evolve in an organized way. How adaptable? In which areas? At what cost? The Architects have to know the answer.

Performance

Another obvious one, of course is that it has to run fast, be responsive, and be pleasant to use. But, what if it is supposed to be efficient in the cloud environment and not consume too much computing power and memory? There’s another trade off and again, Architects need to figure it out.

Sustainability

This is one of the recent trends in non functional requirements that, for instance, systems should consume as little energy as possible. It adds another dimension to an already existing set of contradicting qualities of the architecture; for example, it may be lowering peak performance and scalability.

→ Read about Strategic Sustainability: What’s inside the box?

Deployability

This is one of the recent requirements tightly connected to the DevOps revolution. Businesses want to release new features faster and try new things with different customer groups. Waiting months for another big bang release is not acceptable anymore.

→Explore Feature toggles – faster digitalization by smart experimentation

Internal systems are built as digital platforms which connect with entire digital ecosystems and they are also supposed to deliver a great experience for developers.

Data integrity

Obviously, nobody wants inconsistent data, especially at the source.

→ More about data quality at the source

So let’s make it as consistent as technology allows. But… it will impair performance, deployability, scalability and many other aspects. Again, the trade off is another difficult choice to make.

Another one and another one

I could go on and on, as there’s tens of non-functional requirements for any medium or big system. I think you already see the pattern here. There’s no magic bullet to fulfill all the requirements perfectly at the same time. The life of an Architect is full of dilemmas.

Dilemmas of the architects

Architects live in a very imperfect world of trade offs and difficult choices. If they decide to give one aspect more priority than the other, well, the other one is usually impaired. This is a constant balancing act between the multiple quality attributes of the system. And this is not a static balance, but a very dynamic kind of balance with lots of moving parts.

There are tons of patterns and antipatterns, with different recommendations based on the particular context, and there’s rarely a simple yes or no. “It depends” is very true, but it’s not enough. On what does it depend? When shall we move to a different architecture paradigm? Is it too complex? Is it too simple for such requirements?

One of the key questions that a professional Architect asks themselves is “So, what? What’s in it for my system?”. Some things need to be planned in advance, while on the other hand there’s a risk of overdesigning and over-future-proofing the solution. And the future, as always, will be different than expected no matter how hard we try to predict it. How different? What is going to change? Architects need to both keep calm and not try to predict everything, but also be vigilant and prepare for what they know is coming.

FOMO Architects – avoid

This article has started to look like some kind of hero story about architects. So, let’s reveal some of the darker side of them. They tend to ask themselves different questions, such as “What’s in it for me? How can I play with trend ABC?”, and they use the system as a playground and lose focus on the boring stuff, such as critical non-functional requirements fulfillment. They are obsessed with the latest conference and just want to play, all at the expense of the architecture’s quality. Overusing microservices architecture is one key example of such a behavior and the same seems to apply to serverless recently. They are both very useful, but it doesn’t make any of them an automatic choice.

How do we distinguish an Architect who is looking for a way to modernize the architecture from a FOMO (fear of missing out) obsessed one? A very good question, but that’s for another time.

Future of architecture and architects

Architects are now much closer to software development teams, DevOps teams and business requirements. Only this combined knowledge can help them to achieve the desired results and maintain them over time. They are definitely back and it’s great news, especially for our growing needs for an even faster digital transformation of all aspects of our businesses and lives.

New trends, such as evolutionary architecture, enable them to work in agile environments with a high success rate.

Goodbye architecture, you won’t be missed, long live Agile architecture!