Leave a comment

Metrics That Destroy Your Organisation

In this post I’m going to show you the serious effect metrics can have on your organisation and customers. At the end of the post I’ll also offer some practical advice on how you should apply this information.

Short Term Effects of Metrics

Let’s start with a simple example of the effect of setting a sales department two different metrics:

  1. Time to sell a product
  2. Customer satisfaction

Note: I am not suggesting to just have one metric in a department this is purely an example. The negative effects are most likely to occur if you have no customer related metrics. 

MetricsImmediate

  • Time to sell a product:

We can see that the department have met the metric of time to sell products by reducing the quality of sales provided to the customer. To meet the target the department has also cut back on product customisation meaning the customer is less happy when they get the actual product.

  • Customer satisfaction:

In this case we can see that the department met the metric of customer satisfaction with a more tailored product coupled with more in depth demonstrations to the customer. This resulted in a better final outcome that made the customer happy.

When measuring revenue the focus on time to sell a product may give a a short term win only to see customers desert the company a year later, so be aware of delayed feedback even if things seem positive. There would be nothing wrong with having both of these metrics in place as long as the trade offs were understood.

Long Term Effects of Metrics

Now let’s look at the more serious effects of metrics appearing in the long term. The department has decided to invest money into improving the time to sell metric by speeding up quotes. Inadvertently this may put more pressure on the customer and actually lead to a lower satisfaction rating.

MetricsLongerTerm

The long term effects of concentrating on the wrong metrics can result in more and more detrimental changes. At the worst case the customers may disappear and the organisation collapse.

Metric Affect Processes

Over time the processes of the sales department will be aligned to meet the metric, which has become the artificial measure of success. Those processes can get more and more complex to meet targets that are actually damaging the organisation. In many cases the feedback from the people talking with customers will simply be ignored, as it does not benefit the head of department’s review.

Customer Centricity

Typically we would see each department with its own set of metrics and meeting those being the measure of success or failure. For example the sales team measure themselves on speed of receiving money, and the product delivery team on time taken on a client’s site:

MetricsCrossOrganisation

In this organisation there are major delays between sales and product delivery that are not picked up by those metrics.

SalesDelay

By shifting to customer satisfaction across the whole offering we would see this. Customer centricity moves us towards the customers view of how they interact with the whole organisation. It moves the metrics and processes optimisation to centre around the customer’s experience.

Customer Centricity = Shifting Project Funding

Here is a typical funding structure for projects where money is allocated to each department.  Localised departmental improvements are being made. These projects are affected by the departmental metrics.

MetricsDeptPortfolio

With a customer centric approach a wholesale shift occurs in project funding. Rather than localised departmental improvements the portfolio of projects is now based around the customer’s view. This may mean that some departments receive very few projects and others receive a lot:

MetricsCrossOrgPortfolio

Customer centric metrics focus on the end to end experience of the customer and their journey through the organisation.

Using This in Real Life

When seeking to invest in improvements it can be tempting to  start by going in and looking at business process efficiency or what an internal employees think of their processes. Here is what you should be doing…

Evaluate the situation:

  • What do your customers think of the whole service?
  • What are your department’s metrics that currently define success?

Act:

  • Align the project portfolio around the customer experience.
  • Align the departments and organisations metrics around the customer.
Leave a comment

Avoiding Project Disasters

I’ve been spending some time thinking about how projects start recently, and looking at the ideas that are generated to create new products. So I thought I’d write a post on this to give some hints and tips on ensuring you build the right thing.

Here are the two key stages that occur when starting projects:

StartingaProject1

So this is obvious? Well you’d think so, but we can go badly wrong with these steps because of our assumptions:

StartingaProjectAssumptions

Let’s deep dive into this…..

ProblemsUsersHave

Assumptions about Problems:

Let’s start with the assumptions made about the problems that users have. Here we’ll look at some silly examples (OK if these turn out to be the idea that becomes the next big thing just give me a cut of the profits):

  • Cyclists can be seen most of the time but not necessarily heard and need something that makes a constant noise.
  • Walkers need a mobile gaming system they can play whilst being able to see where they are going.
  • Setting up dates is a pain so what people need is a service that takes them to a random local place to meet.

Subtle Assumptions about Problems:

Some start-ups have died off because of crazy assumptions about what a user needs. However although the crazy examples are often easy to spot, things can be FAR more subtle than that:

  • Customers aren’t buying on our system because they can’t do one click credit card payments.
  • Customers love our products but are put off with the decor in the shops.
  • Our customers aren’t happy with our current paper process because they hate filling in forms.

Our assumptions can creep in all over the place and some of these can cause costly mistakes. These are hard to spot, they sound sensible and logical.

How to Avoid Assumptions about Problems:

This is where user research comes in. There are many techniques such as surveys, ethnography (going and seeing what they do in practice), interviews, group discussions etc. All of these seek to validate assumptions:

  • Customers aren’t buying on our system because they can’t do one click credit card payments. Real problem: Cost of postage is very high and when they come to pay they drop out.
  • Customers love our products but are put off with the decor in the shop. Real problem: Two large security guards at the front door are putting people off entering.
  • Our customers aren’t happy with our current paper process because they hate filling in the form. Real problem: We don’t warn them to bring ID then when they get here they have to do two trips.

Any project that runs with unvalidated users needs is at risk of failure or even making things worse. Your product owner can mask serious issues by sounding like they know what they are talking about. They are not the user, make sure you validate any opinions.

To Summarise….

Get your  users to tell you what they need, and if you are trying to improve a product ask them where their current pain points are.

HowAreWeGoingToSolveThem

Assumptions about Solutions:

Typically having validated the needs of the user we then look to create a solution. Those solutions can be completely unsuitable. Let’s use some real ones!:

  • My dog needs water
    • Create flavoured bottled water for pets
  • People like to smell nice
    • Harley Davidson cologne

OK those flopped, so let’s look at some more subtle issues where solutions appear to be very sensible:

  • A loan system where you have to use sliders to see costs. Result: Doesn’t work well on mobile devices.
  • A system that is accessed with a password on a computer keyboard. Result: Doesn’t work well in a dirty environment.
  • A tablet application for airport staff. Result: Doesn’t work well outside.

How to Avoid Assumptions about Solutions:

Again this is where the disciplines of UX design and interaction design can help. There are many techniques such as prototypes – high fidelity interactive / paper, wireframes etc.  All of these seek to validate assumptions about the solution(s). Seek rapid feedback on your solution(s). The best feedback is when the product is live!

To Summarise….

Get your users closely involved when designing your solution until they get what they want. Get your product out there as fast as you can by cutting down features, you can add more when you know you’re succeeding.

Leave a comment

Feedback Loops in Agile

Feedback loops are one of those things we all know something about, from the annual review process, to the speedometer on the car we are driving, we are receiving feedback from the world around us that we can act on it. After looking at a typical Scrum based project, I will show how feedback is a critical part of agile by mapping it to the principles. Built into the principles there are feedback loops used to…

  • find out as fast as possible if the project outcome will result in the expected benefits
  • improve the project outcome
  • ensure the software works properly as early as possible
  • ensure minimal effort is expended on re-work throughout the project
  • ensure that the software is built as efficiently as possible
  • ensure that collaboration and learning is as efficient as possible
  • improve the way the team works together

Feedback in a Scrum Based Project

Here’s a simple example of a Scrum based project set up. Within the sprint each story is worked on, then demonstrated in a show and tell at the end of the sprint. Finally there is a monthly product show and tell that gives a few key stakeholders the big picture view. Note: this picture purposefully shows the delivery stages being completed one after another to simplify the example described below:

Story Feedback

Consider the increasing cost of receiving feedback from a product owner or user as we get further through the stages. If these people were to tell you that the process you have designed was completely wrong during the Story Acceptance Criteria and UI Design stage the cost of change would be minimal. If this feedback was received as late as the monthly product show & tell we would obviously see a high cost to fix (or throw away). There is an additional hidden cost of re-learning (one of the seven wastes) when we go back to the problem and re-analyse it, which can make things much worse if the feedback is slow. Building feedback in to this model to be early and often greatly improves the situation and helps us remove the poor practice of completing stages before seeking feedback. Now let’s look at the agile principles and how they have feedback built into them:

Agile Principle: Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Projects range from high risk new ventures, typically start-ups, through to those with a definite need such as operating system upgrades. In business we typically release products into the marketplace with some degree of risk. A really dangerous situation occurs when we find ourselves assessing the risk incorrectly, we are so bought into our project or feature idea that we actually think it’s certain to succeed, until it doesn’t. We can get feedback from those people who are not customers to positively re-inforce the likelihood of success leading to more delusion. Ultimately we look to improve on uncertainty with factors like previous experience to lower the risk, however it is better to get real feedback. Agile has this principle of delivering to customers early in it to lower the risk of failure by getting feedback as soon as possible.

As a product owner you should always look to cut scope where there is risk in the market, looking for the real minimal marketable feature set. You can build on your success if it works and minimise loses if it doesn’t.

This feedback loop is used to find out as fast as possible if the project outcome will result in the expected benefits.

Agile Principle: Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Within a project people are learning as they go, trying to lay out every requirement up front in great detail is very difficult due to the amount of complexity in most projects. Rather than assuming that we just need to try harder to nail everything down Agile shifts to allowing changing requirements. Further to this there may be feedback from the marketplace as a competitor adjusts their strategy, allowing this flexibility makes us more likely to succeed.

This feedback loop is used to improve the project outcome.

Agile Principle: Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

By making sure that our software is fully working on real environments with real components we get feedback about how well the solution will work. Great disasters have been known to occur at the end of projects when a full scale environment is built for the first time and we learn too late that our solution isn’t performant. By delivering the working software frequently we can get feedback that it will work when released.

This feedback loop is used to ensure the software works properly as early as possible.

Agile Principle: Business people and developers must work together daily throughout the project.

In the past infrequent contact with business people would lead to the wrong product being built, then expensive and time consuming re-work occurs based on change control. By bringing the business and developers together we greatly shorten the feedback loops. The creation of the product owner in Scrum is a direct response to this problem.

This feedback loop is used to ensure minimal effort is expended on re-work throughout the project.

Agile Principle: Working software is the primary measure of progress.

Within waterfall projects we often see success in terms of the analysis and design stages being signed off on time, only to see disaster at the testing stage. Defects are found, users aren’t happy and the project spirals out of control. This is down to documentation being a poor substitute for feedback from working software. If you don’t have working software you are receiving poor feedback. This is an inefficient way of working.

Even using agile methods it can be very tempting to build up lots of screens that look like they are working without any real end-to-end scenarios truly being ready, this is adding great risk to a project even though it makes it look like progress is good. Using mock objects and having a lack of real data are typical culprits in this area.

This feedback loop is used to ensure that the software is built as efficiently as possible.

Agile Principle: The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Here we see a feedback loop from people talking with each other. It is very difficult to get complex and large amounts of information across to people. In these situations our assumption that we have transferred 100% of our knowledge will be incorrect. From this post you will likely get 10% as you are reading, discussion rises to 50%, so we can see face to face communication is far more effective than documentation. When working through problems together this rises to 75%.

This feedback loop is used to ensure that collaboration and learning is as efficient as possible.

Agile Principle: At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly

Here we see a feedback loop from within the team, focussed on how they are working. This helps improve efficiency and promotes a feedback culture. This is done at “regular intervals” to highlight the importance of re-assessing efficiency regularly and not just once at the end of a project.

This feedback loop is used to improve the way the team works together.

4 Comments

Protect the Team

“Protect the team” is part of the Scrum Masters role on a project. Please note that this post is not a discussion on the rights and wrongs of project setup. Let’s look at what this should and shouldn’t mean in practice….

  • The background

The Scrum master is there to maximise the productivity of the team by ensuring the Scrum processes are as effective as possible. They are there to help the team reflect on how they are working and improve.

They protect the team to help the team achieve their sprint goals and achieve this by:

  • Keeping the team focussed on their goals.
  • Protecting the team from external interruptions that will damage the sprint goal.
  • Ensuring that the goals are realistic.
  • Protecting the team from itself.

I hope you find little controversy in what I have written here!

  • How it can break down

The primary issue I have seen with Scrum masters is when they protect the team from their own delivery team. At its worst I have heard a Scrum master say, “Do you want me to get rid of him”, directed at someone involved in running the project.  The chickens and pigs sketch was a primary example of this, it suggested that the “chickens” or non-Scrum team members were not committed to delivery. This created a them and us culture, seeing anyone outside of the team as meddling with progress.

I believe one of the failings of Scrum is not to insist on a big picture or system wide view. When the goal of the sprint takes priority above everything else this can damage delivery overall. Here are some simple examples:

  • As a database administrator I need to know how much space the team are likely to need for a test database so I can build it for the next sprint.
  • As a the head of security I need to know how the payments engine interface is secured so I can get it certified to go live.
  • As a service manager I want to talk through how code quality is being addressed so I can be sure it’s maintainable.

Let’s be pragmatic! In this case I believe it is the role of the Scrum master to help understand and express the impact of the interruption. This can be light touch, it is not to patrol for anyone attempting to talk to the team and get rid of them. This closed view of a Scrum team can damage a project and introduce delays. We also need to be careful with solutions like, “I’ll talk to them and get back to you”, as this can create waste in the form of poor information hand-offs, leading to additional delays.

A good way of spotting this behaviour is when team members start to use the word “management” in a derogatory fashion to describe non-Scrum team members. This re-inforces a them and us culture. 

  • The solution

As a Scrum master you may wish to minimise interruptions, that’s fine:

  • Maybe you ask for a meeting to be scheduled for a later part of the day, if it’s a large subject.
  • You could openly accept interruptions just after the stand-up.
  • You can ask for non-team members to think carefully about the urgency of their request before interrupting.

These are simple suggestions and I’m sure you can come up with more that fit your situation. What we must avoid is a them and us culture, as a Scrum master help everyone to feel like a team!

1 Comment

Project Manager versus Scrum Master

This is a controversial topic to say the least, so I thought I’d share how I feel about it. The point I’m going to be making is as follows:

If we want to change things we should stop asking how these roles are supposed to work and start asking how the structures outside of the Scrum team should work.  

I’ve read a lot about the PM role no longer being of value, that it has been replaced by the Scrum Master, Product Owner (PO) and team. In the traditional sense of managing a PRINCE2 waterfall project there are certainly major differences, such as allocating resources to tasks and building Gantt plans. However time has moved on and now we have PM’s running Agile projects, their responsibilities have changed, some would say they have been replaced with a new set.

Why is it so difficult to remove this role? Well let’s use synthetic thinking to answer that, “The explanation for a system always exists outside of the system”. A simple example of this would be a kettle, it has no value in itself, it is used to boil water for something (or someone), i.e. myself making coffee. In other words if I look at any outputs being produced I need to understand the reason for their existence outside of the system. On a project this means that when I see a report being produced by a PM or Scrum team I need to look outside of it to see what it is used for, it has no value on its own, it is used outside of the current system for something.

Organisational Structure

Back to the subject then, why does this matter in relation to a project manager role? The structure above the Scrum teams is often a programme team, they are the reason for the project manager role. The inputs needed from the programme team line up with the outputs from the project manager. When you break this model and stop providing those outputs the programme team can no longer function. Note that I am not passing judgement on the programme structure I am simply looking to explain how the parts relate.

Now consider how a Scrum team would interface with a programme team if there was no project manager:

  • Costs: The PO is now accountable for costs in Scrum
  • Progress: The PO is now accountable for project progress in Scrum
  • Risk: No-one is directly responsible for recording risks in a Scrum team. The programme board either have to accept this or modify Scrum.
  • Quality: The team are now responsible for quality via a definition of done. Rather than the PM reporting on quality some new form of communication must take place.

What we need to understand therefore is that the programme board is a structure that is set up to work with defined inputs and outputs if we want to change these we have to do one of two things:

1) Change the way the programme team functions to work with Scrum without PM’s,

2) or change the way the Scrum team functions to work with the programme team.

The PM is trained to interface with a programme board in a certain way, they are skilled in it and are a central point of contact for their desired inputs, that is why this works so well (for the programme team). Now let’s consider the other factors that need to be considered before removing the PM:

Skills

The skills of the various parties now need to be changed, you cannot ignore this aspect. As an example asking a PO to take accountability for cost outputs means they need to be able to manage cost. A PM is trained in this area and is ready from day one. A similar situation usually occurs in Scrum where the PO is a subject matter expert but not an agile requirements specialist and a BA is brought in to facilitate, in this case we recognise the PO may not have the skills to perform the role.

Accountability 

Programme boards that are set up to work with project managers being accountable will find it very hard to find the Scrum team accountable. In reality everyone has a level of accountability but this is not easy to put in place. When treating a PM as being accountable you will drive a particular behaviour as they seek to monitor and control, because if they don’t they will be punished for it. We split our roles and responsibilities like this and rarely would consider punishing an entire team, instead saying some people were good and some bad, but probably weren’t controlled (managed) well enough. In true Scrum accountability comes down to the PO for building the right thing, having met PO’s few would hold themselves accountable in the same way that a PM would. I have never seen a situation where a PM has not been affected by team failing.

Summary 

So that’s why I believe there is a place for the Project Manager or indeed a Scrum Master who takes on the additional responsibilities and accountabilities of a PM (i.e. Agile PM) in todays world of programmes. If we want to change things we should stop asking how these roles are supposed to work and start asking how the structures outside of the Scrum team should work.  

Leave a comment

Lean Delivery Mechanisms – Programme and Product Management

This post is written to compare and contrast how most companies have been set up. Lean has a new way of thinking and is making very large inroads into the software industry.

You should read this if:

  • If your competitors are releasing products that their customers want faster than you and therefore your business is at risk
  • If you want to stop wasting money on products your customers don’t want
  • If you want to understand staff performance in your company
  • If you want to set your company up to be product and not project focused

Traditional Approach – Delivery Structure

Here we see a standard company set up for delivering projects (simplified):

Picture1DeliveryStructure

Traditional Approach – Project Focus

Picture2ProjectFocus

Lean Approach – Product Focus

Picture3ProductFocus

Traditional Approach – Typical Value Stream

Picture4TraditionalValueStream

Here is an example of the number of steps, between each of these there is a delay

  • This can be a Scrum based organisation
  • Activities before project delivery taking excessive time
  • Project delivery team put under pressure to deliver quickly
  • Mean time to project failure is around 10 months!
  • Large scale investment wasted

All delays and steps will depend on the actual organisation

Lean Approach –  Ideal Value Stream

Picture5IdealValueStream

In this case we generate ideas as experiments

  • Teams are not created based on large projects and programmes
  • Small features allow the assignment of the next most important thing to do
  • Experimental feature may be a prototype and not a fully built solution

This is an example only, feature size and set up may vary

Lean Approach –  Build Measure Learn

Picture6BuildMeasureLearn

  • Fail at a scale you can afford
  • Treat all change as a learning
  • Speed up feedback loops – fail fast

Traditional Approach – Improvements

Delivery improved at a local level

  • Project manager improves project processes
  • Development Manager improves development processes
  • Operations manager improves operations processes
  • Scrum teams improve at a local level
  • Organisational culture rewards people on component parts

Lean Approach – Improvements

Delivery improved at a system level

  • Holistic view of the system taken by all
  • Strategic improvements aimed at the system
  • Component parts understand their role in the whole
  • Reduce hand-off of information
  • Benefits realisation targeted and rewarded
  • Learning and adaptation targeted and rewarded

Systemic Approach to Failure – Demming

“I should estimate that in my experience most troubles and most possibilities for improvement add up to the proportions something like this:

  • 94% belongs to the system (responsibility of management)
  • 6% special”

Picture7SystemicApproachToSuccess

Picture9SystemicApproachToFailure

Systemic Approach to Success

In this same model we see testers rewarded for their successful defect discovery rates (not that they’re doing a bad job!)

Picture7SystemicApproachToSuccess

Picture10SystemicApproachToSuccess

Dev Ops Culture Example

Picture8DevOpsCulture

Faster feedback loops:

  • Release faster
  • Identify failure faster
  • Simplifies error correction if less has changed

“But it’s just a flexible framework!”

  • You see frameworks claiming they have solved how to deliver and effectively managed risk
  • Core principles of learning and systemic thinking are missing or a minor part of their definitions
  • This drives poor behaviour and can cripple a company
  • Agile and Lean work from principles
    • Working software over documentation – Agile example
    • Identify waste – Lean example
  • Good practice should be applied rather than enforcing “best” practice for complex problem domains
Leave a comment

User Persona’s – Some Hints and Tips

User personas are there to help us design products for groups of people. They represent the users or customers of your product. This helps you think about how they would relate to what you’re building. How often have you seen personas created, stuck on a wall and then ignored?

Here are some tips to help make them successful:

– Use a real picture that sums up how they feel, ideally relating to your product. Here we have a mum who’s struggling with too much to do. This could be very helpful if we were targeting a diary product at this group of people. Don’t use a known actor as we’re trying not to introduce preconceived ideas of who they are.

busymum

– They need to be descriptive enough to believe that person is real without so much detail they get confusing. Think of each person as an actor in a film. If you have enough detail you’ll be able to imagine them in a scene. For example, “Hey Carol, take a look at this new cup holder for your car, it’s carbon fibre and super light”. “Looks good but I rarely drink anything in the car as I only drive to work which takes 10 minutes, and why would I need it in lightweight carbon fibre?!”.

ferrari-f12-berlinetta-cupholder

– Create enough personas to capture your main users, look at around 6 as a guideline. You usually don’t design a product for everyone so think carefully about who you’re targeting.

– Get the Scrum team involved in their creation, help them to buy in to the idea. Avoid them becoming silly caricatures.

– As a BA / Agile Coach you should be talking about them regularly until they become part of the team dynamic. Asking how would Carol use this feature, will help assess it’s suitability. You can add them into stories like this:

As Carol a customer I want to store my contact information so that I can quickly retrieve peoples details when they contact me.

– Find the real users. Personas are great as a starting point but you need to review and improve them. If you assume every student is poor because they drink too much you may design the wrong product. Validate your assumptions with research. The whole point of the persona is to make the right product, if you have the wrong customer profile you probably won’t.

– Chose a primary persona (or stakeholder if you can identify a single person) to build for. Who is the person you’re most concerned about? Your product will be very different if your primary stakeholder is a client who wants a really cheap supportable solution, compared to a customer who wants all the bells and whistles. That doesn’t mean you take things to extremes but it will help guide your solution.

Follow

Get every new post delivered to your Inbox.