Measuring Agile Maturity with Assessments

Stephen Gristock is a PMO Leader and Lean Agile Advisory Consultant for Eliassen Group, a Scaled Agile Partner. In this blog, he explores both the rationale and potential approaches for assessing levels of Agility within an organization.

A Quick Preamble

For every organization that embarks upon it, the road to Agile adoption can be long and fraught with challenges. Depending on the scope of the effort, it can be run as a slow-burn initiative or a more frenetic and rapid attempt to change the way we work. Either way, like any real journey, unless you know where you’re starting from, you can’t really be sure where you’re going.

Unfortunately, it’s also true that we see many organizations go through multiple attempts to “be Agile” and fail. Often, this is caused by a lack of understanding of the current state or a conviction that, “we can get Agile out of a box.” This is where an Agile Assessment can really help, by providing a new baseline that can act as a starting point for planning or even just provide sufficient information to adjust our course. 

What’s in a Word?

We often hear the refrain that “words matter.” Clearly, that is true. But sometimes humans have a tendency to over complicate matters by relabeling things that they aren’t comfortable with. One example of this within the Agile community is our reluctance to use the term “Assessment.” To many Agilists, this simple word has a negative connotation. As a result, we often see alternative phrases used such as “Discovery,” “Health-check,” or “Review.” Perhaps it’s the uncomfortable proximity to the word “Audit” that sends shivers down our spines! Regardless, the Merriam-Webster dictionary defines “assessment” as:

“the act of making a judgment about something”

What’s so negative about that? Isn’t that exactly what we’re striving to do? By providing a snapshot of how the existing organization compares against an industry-standard Agile framework, an Assessment can provide valuable insight into what is working well, and what needs to change.

The Influence of the Agile Manifesto

When the Agile movement was in its infancy, thought leaders sought to encapsulate the key traits of true agility within the Agile Manifesto. One of the principles of the manifesto places emphasis on the importance of favoring:

“At regular intervals, the Team reflects on how to become more effective, then tunes and adjusts its behavior accordingly”

Of course, this is key to driving a persistent focus on improvement. In Scrum this most obviously manifests itself in the Retrospective event. But improvement should span all our activities. If used appropriately, an Agile Assessment may be a very effective way of providing us with a platform to identify broad sets of opportunities and improvements. 

Establishing a Frame of Reference

Just like Agile Transformations themselves, all Assessments need to start with a frame of reference upon which to shape the associated steps of scoping, exploration, analysis, and findings. Otherwise, the whole endeavor is just likely to reflect the subjective views and perspectives of Assessor(s), rather than a representation of the organization’s maturity against a collection of proven best practices. We need to ensure that our Assessments leverage an accepted framework against which to measure our target organization. So, the selected framework provides us with a common set of concepts, practices, roles, and terminology that everyone within the organization understands. Simply put, we need a benchmark model against which to gauge maturity.

Assessment Principles

In the world of Lean and Agile, intent is everything. To realize its true purpose, an Assessment should be conducted in observance with the following overriding core principles:

  • Confidentiality: all results are owned by the target organization
  • Non-attribution: findings are aggregated at an organizational level, avoiding reference to individuals or sub-groups
  • Collaboration: the event will be imbued with a spirit of openness and partnership- this is not an audit
  • Action-oriented: the results should provide actionable items that contribute toward building a roadmap for change

Also, in order to minimize distraction and disruption, they are often intended to be lightweight and minimally invasive.

Assessment Approaches

It goes without saying that Assessments need to be tailored to fit the needs of the organization. In general, there are some common themes and patterns that we use to plan and perform them. The process for an archetypal Assessment event will often encompass these main activities:

  • Scoping and planning (sampling, scheduling)
  • Discovery/Info gathering (reviewing artifacts, observing events, interviews)
  • Analysis/Findings (synthesizing observations into findings)
  • Recommendations (heatmap, report, debrief)
  • Actions/Roadmap

Overall, the event focuses on taking a sample-based snapshot of an organization to establish its level of Agile Maturity relative to a predefined (Agile) scale. Often, findings and observations are collected or presented in a Maturity Matrix which acts as a tool for generating an Agile heatmap. Along with a detailed Report and Executive Summary, this is often one of the key deliverables which is used as a primary input to feed the organization’s transformation Roadmap.

Modes of Assessment

Not all Assessments need to be big affairs that require major planning and scheduling. In fact, once a robust baseline has been established, it often makes more sense to institute periodic cycles of lighter-weight snapshots. Here are some simple examples of the three primary Assessment modes:

  • Self-Assessment: have teams perform periodic self-assessments to track progress against goals
  • Peer Assessments: institute reciprocal peer reviews across teams to provide objective snapshots
  • Full Assessment: establish a baseline profile and/or deeper interim progress measurement

Focus on People—Not Process and Tools

Many organizations can get seduced into thinking that off-the-shelf solutions are the answer to all our Agile needs. However, even though a plethora of methods, techniques, and tools exist for assessing, one of the most important components is the Assessor. Given the complexities of human organizations, the key to any successful assessment is the ability to discern patterns, analyze, and make appropriate observations and recommendations. This requires that our Assessor is technically experienced, knowledgeable, objective, collaborative, and above all, exercises common sense. Like almost everything else in Agile, the required skills are acquired through experience. So, choosing the right Assessor is a major consideration.

Go Forth and Assess!­

In closing, most organizations that are undergoing an Agile transformation recognize the value of performing a snapshot assessment of their organization against their chosen model or framework. By providing a repeatable and consistent measurement capability, an Assessment complements and supports ongoing Continuous Improvement, while also acting as a mechanism for the exchange and promotion of best practices.

We hope that this simple tour of Assessments has got you thinking. So what are you waiting for? Get out there and assess!

For more information on assessments in the SAFe world, we recommend checking out the Measure and Grow article.

About Stephen Gristock

Stephen Gristock

Specializing in Agile-based transformation techniques, Stephen has a background in technology, project delivery and strategic transformations acquired as a consultant, coach, practitioner, and implementation leader. Having managed several large Agile transformation initiatives (with the scars to prove it), he firmly believes in the ethos of “doing it, before you teach/coach it.” He currently leads Eliassen Group’s Agile advisory and training services in the NY Metro region.