Death by a thousand cuts – problems with using unstructured requirements

Products requirements managed using an unstructured approach

In our previous blog, we talked about taking on your most strategic challenges at the organization level using an Agile mindset. As the next blog in our series, we will uncover the problems associated with using an unstructured approach for documenting product requirements and design. Product requirements play a critical role in the product delivery process – as they are the common language between the business teams and engineering. Product requirements play a vital role also because they are the first step in the product development process – where value creation is highest. Any imperfections in the product requirements get amplified as they travel down the process – making it more costly to handle these issues further downstream in the process.  Like most companies that we interact with, you too are probably using unstructured requirements – using Word and Excel documents. This ad-hoc approach for managing unstructured requirements is potentially fraught with numerous challenges. Technical design is typically also documented using such documents, including diagramming tools such as Visio. In this blog we will take a deep dive and uncover the resulting challenges with such an approach.

Unclear and Ambiguous Requirements

Unclear and ambiguous requirements

Using Word and Excel documents to manage requirements forces authors to describe product requirements using textual descriptions. Complex and visual concepts are described using text and this is problematic because it is just the wrong medium for describing these complex requirements. Often times, images are embedded in these documents that describe the visual aspects of the software product being developed. Sometimes the text and image data don’t correctly align with each other and can get out of sync quickly.  There is a huge potential for details “lost in translation”. Unclear and ambiguous requirements are very dangerous because they take development down the wrong path, often leading to re-work down the road. When unclear and unstructured requirements are interpreted incorrectly by the test team, it can lead to “spurious” defects that end up burning a lot of time from the entire team. An even bigger problem arises when unclear requirements result in sub-optimal features being released to production only to be discovered by the customers.  This is a drag on the overall productivity for the team.

Out of Sync and Outdated Documents

Out of sync and outdated documents

This is a fundamental side-effect of using unstructured requirements. Data sections and elements defined in multiple documents can easily get “out of sync” and outdated, because data is not re-used and shared as you would expect in a structured environment. Authors need to be aware of all the locations where data needs to be updated – this is likely to be an error prone process.  In an Agile development environment, this gets amplified further because of the frequent changes. In fact the documents can become outdated so quickly that many Agile teams prefer not to document technical design altogether. Having no technical documentation of your products can be very inefficient for the development process and new team members have to dig through thousands of lines of code to unravel the technical design – what could be accomplished more efficiently with the right level of technical design documentation. Not only does this perpetuate a culture of “tribal knowledge”, this is also a great inhibitor of team collaboration and communication.

Inefficient Process

Inefficient process

Writing long Word and Excel documents is a time consuming process, just as reading them is. Often times the authors need to write a lot of text based descriptions of product requirements, a process that takes time.  Authors also need to work with a multitude of related documents together, slowing them down even further. These are just the “wrong tools for the job” that make the process inefficient. Further complicating this process is the need to compose and embed images for UI screen mockups in these documents. These images often times also need some visual annotations (markups) to highlight certain aspects of the UI mockups. Handling changes is particularly painful because the authors need to go back the “source files” of these images and update them and re-insert them into the documents. This is not only very inefficient, but also error prone.

Unstructured requirements lead to Inconsistency

Unstructured requirements lead to inconsistency

Unstructured requirements allow inconsistency to creep in from different authors of the requirements. Individual authors are likely to write and describe product requirements using their personal styles – easily introducing variations in style and substance. While this problem can be controlled to some extent by using document templates – the variations from personal styles cannot be entirely eliminated.  Further inconsistencies can be introduced by teams working with different tools or processes. While this inconsistency isn’t completely fatal, it does add to the churn from poorly documented requirements.

Poor Visibility

Poor visibility

When working with a bunch of Excel and Word documents in a repository (such as SharePoint) poor visibility can be a vexing problem. It is difficult to navigate and locate these documents in the first place (as users of SharePoint would attest to). To make sense of the content in one document, it is often necessary to open a number of other related documents. Trying to piece together the big picture by reading a number of Word and Excel documents is not only frustrating, it is downright unproductive. Information “overload” kicks in and slows down our ability to process and digest the contents of the documents.

Limited Text Search

Limited text search

In this age of Google search, users expect to quickly locate items of interest by searching for key words. The limited ability to search for text terms in unstructured requirements documents is a major drawback. You can easily search for text terms within one document.  But even within a single document, searching for text that could be embedded in images becomes impossible. Text search within Excel documents also suffers from numerous limitations. But for true global text search, you need the ability to search for text across a selection of documents, and indeed across projects. This is particularly painful if not impossible in the document repository environments such as SharePoint. For text search to work effectively, it also needs to be fast.  Users don’t have the patience to wait for minutes to wait for the search results to come back. Truly global text search only works in tools that employ a modern text indexing and search engine.

Limited Collaboration

Limited collaboration

Collaboration is all the buzz today when it comes to getting the most out of an Agile development environment. The ability to work together and share ideas is indeed a key ingredient for innovation. Using a bunch of documents to manage product requirements and design suffers from a key challenge – limited ability for users to collaborate and work together. If users have to “check out” and lock the document for the duration of their editing activity, it really slows down collaboration as other users need to wait their turn. This can be ameliorated by using tools such as Google docs that do a far better job of merging changes from multiple users, allowing users to collaborate better. But besides the ability to work together on the product requirements, users also need the ability to post comments or conduct discussions. Users need to be able to post replies, up-vote and share ideas on discussions. For this kind of collaboration to be effective, the users need to be able to connect the context of their work with the discussions – standalone tools that offer collaboration but are disconnected from the work content, often tend to miss the mark.

What are your experiences in dealing with an unstructured approach for documenting product requirements and design? What challenges have you faced and how did you overcome them? Please let us know in the comments section below.