10 Reasons why Architecture finds itself unable to measure its success

By Ben Clark

Since its inception, Architecture, in particular Enterprise Architecture, has found itself failing to measure its success. In an industry where its very nature is nebulous and operates in the grey, it can take time to quantify success and showcase its actual value. 

Architecture as an industry has been caught in a status quo, as issues it faced yesteryear are still highly prominent today. Here are ten points I have noticed, of which many have not changed in years: 

  1. Not focusing on what is important to business stakeholders

Architects often focus on their outcomes rather than what is essential to the business stakeholders making the decisions. Architecture teams are very analytical but sometimes need better communication skills to articulate this to their stakeholders. Research from Henley Business School states that communication training is the biggest soft skill requiring development in EA teams. Abstracting up will always be far more effective than abstracting down. If you are using a measurement that is only effective for yourself and not your customers, you are on a losing streak. During their ill-fated IPO of 2019, WeWork used a wholly made-up term, “community adjusted EBITDA”, to demonstrate the supposed profitability of the company. Without going into great detail, by forgetting their stakeholders’ needs, and focusing only on what they thought was good, WeWork’s IPO collapsed. 


2. EA Maturity does not resonate with business stakeholders

The effectiveness of Enterprise Architecture is often based on a maturity score, but this rarely resonates with business stakeholders, who often see it as nothing more than an academic exercise. If in doubt, please visit any survey from any architecture tools vendor whose focus is on the maturity of the practice and its processes above all else.   

3. What is there to ‘hang’ from?

Businesses rarely publish a strategy worthy of its name, so the strategic architecture has very little from which to “hangs”. Less than 15% of clients have ever provided what would be considered a comprehensive strategy document in which we can anchor our work. When we start an assignment with a new client, this is one of the questions we will ask before beginning the project. 


4. Data alone does not tell the story

To effectively measure Enterprise Architecture, huge subsets of data need to be extracted from architecture tools… that’s if the data exists! That is a time-consuming practice in its own right, but using architecture tools outside of architecture practices isn’t an effective way to use time, as they just do not provide the required benefits. I know of three of our clients who have ditched their architecture tools in the last month alone. The data extracted from these tools do not tell a story that a business stakeholder can understand. 

5. Not showcasing your value with predicted long-term benefits

Where architecture can showcase its value is through the long-term benefits it has provided or, in most cases, is predicted to provide. However, the average job tenure of a CxO is usually under five years. Architecture functions may then find themselves in a pickle as if you joined in said CxO’s third year or beyond; good luck getting them interested in what you have to say.


6. One size does not fit all

Every organisation, and every stakeholder, has a different view on what is essential based on their objectives. No methodology, tool or framework can take every eventuality into account.  

7. Frequent changes are likely to happen

We have specific clients that will say: “a week is a long time at X”, as their objectives and measurements change over time. What was set out at the beginning often becomes irrelevant, leading to changes frequently. 


8. Qualitative benefits translate to white noise

Architects can sometimes struggle to “feel” what benefits they can provide. Most architects are very analytical by nature, so they find comfort in definitive facts and figures, which are often impossible to be specific on.  When architects are not sure, the result can often be an overly verbose response where the only value given is qualitative. When talking to “hard-headed” business people, purely qualitative benefits are translated into white noise.

9. Consensus building is not considered

Enterprise Architecture and Business Architecture play a critical role in alignment and improving the understanding of business issues, or as we call it at Konvergent – “consensus building”. It’s completely intangible, but it is critical in moving a transformation forward or deciding on a strategy. How can you measure a decision being made, for example, ten stakeholders all being on the same page when previously they could not agree?


10. Quantitative data is memorable but not used

Architects often need to remember that their key stakeholders may have low attention spans, as they are only interested in their own goals and the result, not the intricate detail they are usually presented with. Research from Microsoft suggests that humans now have attention spans of just 8 seconds, which is allegedly less than a goldfish. However, if you give people something new, like an ROI on consensus building, they are more likely to remember it as it will be further information. 

How have you found yourself measuring the effectiveness of architecture? Have you found yourself using alternative methods to quantify your success? What has been working for you?  Have you ever managed to convert a qualitative value into a tangible one? What could the industry do to improve? I’d love to hear, so please do let me know via LinkedIn:

More insights