Azure Boards tool has a lot of options and ways to organize work. Here are discussed some of the key options in Azure Boards to ease the prioritization and implementation of analytics and data engineering solutions. In the presented setup, the so-called Spotify model of agile development with Tribes, Squads and Chapters is used as a guiding framework.
Theory
Autonomy is important
According to Lawrence M. Miller, team autonomy does matter:
- Creativity, entrepreneurship and continuous improvement are all connected to degrees of autonomy and the acceptance of responsibility.
- Mastery and self-learning are linked to autonomy.
Figure 1 above describes the differences of team maturity and self-control of a team. As M. Miller describes: the goal of organizational development is to move smoothly from 1A to 2B in the illustration. However, as every parent knows, developing a team to maturity, just like developing a child to adulthood, is rarely a straight line. There is always testing of control limits and failures to behave maturely. Rather than halt progress, these deviations should be viewed as inevitable experimentation on the way to maturity.
Management can have different roles for a team
See Figure 2 above of Hackman’s Authority Model (1981) which is cited a lot in the management and HR research. In the horizontal axis, different types of teams are described. On the vertical axis, various accountabilities are described. From the colour coding in the matrix we can see that a certain type of team has certain type of accountabilities. For example, if we think about the role of a leader for a self-managing team, the team can be helped to acquire the capabilities of ‘monitor & manage work’ and ‘execute team tasks’.
The Spotify model of agile development
According to the Atlassian, The Spotify model is a people-driven, autonomous approach for scaling agile that emphasizes the importance of culture and network. It has helped Spotify and other organizations increase innovation and productivity by focusing on autonomy, communication, accountability, and quality.
One of the key ideas of the Spotify model is Tribes and Chapters in a matrix organization (see Figure 3 above). Tribes manage a feature area independently. In analytics, this could be for example a business area like sales or a product area like electricity. Chapters are the functional families of specialists. For example in the case of analytics there could a chapter for analysts, another chapter for data engineers and a third chapter for data scientists. Like the Atlassian states it: Chapters are the family that each specialist has, helping to keep engineering standards in place across a discipline. Chapters are typically led by a senior technology lead, who may also be the manager for the team members in that Chapter.
Large Tribes can have Squads of 3-12 individuals. A Squad has a unique mission and typically from business side a product owner. For example in analytics solutions development, in the sales Tribe could be a Squad for online sales channels. Typically, a Squad has its own backlog & board to manage work. A Squad can even choose the agile methodology or framework it is using.
One can see that when comparing the Spotify model and Hackman’s Authority Model, the Spotify model in the feature development or Tribe/Squad level has elements of self-governing team. On the other hand, in the functional or chapter level, there is often a supervising team manager and engineering standards are kept in place. For example, the chapter of data engineers can follow common process, documentation, naming, and coding practices. It is followed that this commonly agreed operating model is followed and serves the continuity and growth of the business. Therefore, we could say that according to the Hackman’s Authority Model, a Chapter is more managed than a Squad.
How
Backlogs and tagging
Typically, in the Spotify model presented in the previous chapter, Squads have their own boards/backlogs and a way to manage their work. How to setup Squads and Chapters in Azure Boards? Let’s start with the Squads. Each Squad can have their own Backlog or Board but still it is possible to see all the work combined from the combined backlog (see Figure 4 above). In analytics, it is often the case that more specialized resources need to belong to multiple Squads or that there are selected projects which are shared between Squads. Then, it helps to have a company, business unit, or Tribe level combined backlog view to what is happening and how resources are shared between the Squads.
In Azure Boards, to add a new Squad backlog&board to a project, go to Project Settings -> General -> Teams -> New Team. The new team (Squad) comes an area under the main team or project in Azure Boards (see Figures 4&5 above). All the features of Azure Boards like resource planning work for the created Squads in Squad level but also in the combined project level (see the Combined backlog in Figure 4).
For Chapters, tagging of work items can be used (see Figure 5 above). It is possible to tag analytics work, data engineering work and data science work. When individuals grow, it is possible that for example an analyst also takes the responsibilities of a data engineer or a data scientist. Then, tagging works better than just following the work based on the names of individuals. Tasks also have other fields like Activity but the challenge is that those cannot be currently used as filters in the backlog or board views in Azure Boards.
Item levels, sprint length and prioritization
For example the following item levels or work structure can be used (see Figure 6 above):
- Epic (main project or roadmap item) – start and end date
- User Story (subproject) – time interval (technically sprint in Azure Boards, e.g. quarter) – (can be moved to two week sprints when a quarter starts if the Squad prefers that)
- Task (actual work assigned to a selected individual, work quantity estimated and followed) – sprint (e.g. two weeks) – (preferably split to more tasks if continued in the next sprint)
Epics of the backlog of a Squad can belong to the roadmap of the Squad or Tribe and have each an estimated start and end month. It does not make much sense to assign Epics to sprints. User stories, especially in analytics or data science, can last for example for a month: there is often waiting of data to be consumed or further clarifications from the business. It might make sense to use longer time intervals (technically sprints in Azure Boards) like a quarter for User Stories when they are first time entered to Azure Boards, to understand workloads in the long-term. When a quarter starts, it is possible to assing the User Stories to two week sprints if the Squad desires to plan User Story level implementation schedules in more detail inside the Squad. However, remember that the Epic level and a roadmap of Epic items is typically more powerful way to communicate to customers or business than sprints which are for internal implementation. Another way to set priorities instead or in addition to a roadmap of Epics is to use a ranked backlog of User Stories for a selected time interval (sprints). We have noticed that this backlog approach of User Stories works for business communication instead of a roadmap as long as there are not too many User Stories and the audience is limited.
A task should always be the responsibility of a single individual. If multiple individuals are required to complete a task, the task should be split to more tasks. A task should be completed in a week or two or split to smaller tasks which can be completed in a week or two in the next sprint. For tasks, Azure Boards Sprint View -> Task Board / Backlog can be used. It is important to know what tasks are done and when for the organization, as typically there are bottle-neck resources like data engineers or integration architects. An organization should be able to prioritize what is happening and when by business value and business level urgency based on the roadmap of Epics or a backlog of User Stories, not by the yelling competition of higher managerial influence and pressure to the data engineers.
Work management
Squads should independently manage their backlog, board, and sprints when it comes to resources which fully belong to their Squad. This promotes autonomy and reaching the roadmap goals of the Squad or Tribe. Typically, the Squad members are planning their roadmap together with the business and the product owners of the Squad which promotes self-governing and autonomy. A Squad has typically an owner from the implementing side (e.g. analytics owner) and from the business side (e.g. business or product owner) who have the most to say to the priorities and resourcing.
Analyst and data scientist work
For example, an analyst often belongs to only one Squad or Tribe. A lot of the analysing and data science projects can be managed in the user story level with tasks mainly used to estimate workloads of different task types in the User Story. These can include for example 1. Spesification and Design, 2. Data Modelling and 3. Visualization for a reporting project. There is no need for an analyst to describe his/her personal work with tens of detailed tasks related to e.g. Power BI report creation as this would be managerial overhead. Nobody is interested to read or follow those detailed tasks. An analyst is him/herself responsible for completing a reporting project or a User Story. Typically, business follows the roadmap level (Epics) or User Story level and looks from there how the reporting project is progressing (see the section above: Item levels, sprint length and prioritization).
Analysts and data scientists are responsible for creating tasks for each other and to data engineers. If one has a task for another person, that is not sent by email or Teams message but created to Azure Boards.
Data engineer work
If there are shared resources between Squads, for example data engineers, it might make sense to do the prioritization of work together with the Squads or Tribes who share the resources. There can be a common meeting utilizing the Combined backlog/board/sprints (see Figure 4 above) recarding the shared resources. There should be an agreement on place how the shared resources are shared in the long-term, e.g. percentage of worktime per Tribe (see the Capacity setup in Azure Boards, Figure 7 below) to guide the prioritization work.
