4 steps to great project monitoring

I’m imagining you’re building or strengthening a monitoring system. One that will track project progress, identify blockages, and gather evidence of impact.

But with so many concepts, frameworks and tools out there, it's hard to know where to focus. What’s core and what's secondary? Where should you focus your finite efforts?

This is my shot at setting out the core elements of a good project monitoring system (portfolio monitoring is another thing for another day).

I’ve tried many approaches but keep coming back to the same four steps. These steps reflect the basic structure of Paths. Paths makes things easier but many build a version of this using spreadsheets and slides.

For each project / intervention / investment:

Step 1. Develop a Theory of Change

Theories of Change (ToC) are useful for many reasons and they are particularly great for organizing project monitoring.

A ToC sets out the sequence of changes (‘outcomes’ or whatever) you expect to see. Tracking what actually happens is called 'monitoring'. ToCs are your best guess based on limited information at a particular point of time. They are incomplete, imperfect, conditional predications. That’s okay – monitoring is a process of error-correction.

Don’t aim for the perfect ToC - aim for 'good enough to start' and move forward. As we’ll discuss below, ToCs should evolve with your project. Activities and outcomes may change and you'll gain a clearer understanding of what needs to be monitored.

Importantly, what you include in a ToC is what gets monitored, so you must strike a balance. On one hand, you want ToCs to include the changes most important to your project. On the other, you don’t want too much detail, or you'll end up with a complex jumble of boxes and too much to monitor. Five to twelve boxes are usually sufficient to start. Again, you can always add as things progress.

Step 2. Develop a Monitoring Plan

A ToC reflects what we hope will happen. Now we need a way to track what happens in practice.

For each box in your ToC, choose at least one indicator. Indicator selection is a big topic so I'll just note a few important considerations.

First, indicators should act as early warnings, alerting you when things start to go off track. If you're only receiving data annually or less frequently, consider identifying indicators that offer more frequent, incremental updates.

Second, it's often wise to use a mix of quantitative and qualitative indicators. Together, they provide a comprehensive view of what, why, and how changes occur.

Third, if you're struggling to choose indicators, I find the following prompt useful: Imagine you're presenting project progress in reflection meetings (Step 4). What will the team want to know? What will inform and enliven discussion?

Write how data against each indicator will be collected. Methods include: observation, interviews, partner records, focus group discussions, and rapid and formal surveys. Your choice of method depends upon the nature of the change and how you choose to balance costs vs. robustness. Don’t plan everything upfront. Say, you’re conducting a survey in a year’s time – note key any key thoughts you have about its design, but the detail can come later.

Note when data against each indicator will be collected. Typically, you'll want to monitor project activities and changes amongst immediate partners regularly. These tend to be easy to monitor and, if things are going off track, you'll want to know ASAP! Changes higher-up your ToC tend to be harder and more costly to monitor (e.g., surveys of beneficiaries), so you'll want to measure these less frequently. As far as practical, you should consider dovetailing data collection with reflection meetings – so you have data recorded in time to those discussions.

Step 3. Collect and record data

Follow the plan!

Data collection is another big topic so I don’t want to say too much here. One for future blogs perhaps.

One thing I’d emphasize: For all its jargon, we need not be intimidated by data collection. Sure, some activities are complex and benefit from specialist expertise. However, for most indicators, particularly at the lower levels of your ToC, data collection is simple and straightforward. It can be done effectively by those involved in day-to-day implementation. For example, understanding whether an immediate partner has adopted a change is usually a matter of undertaking a site visit armed with some questions or a checklist. Get out into the World and be curious!

Step 4. Reflect, investigate and adapt

Steps 1-3 are not unusual. What’s less common are well-functioning reflection meetings and, without this fourth leg, the stool falls over.

Call these whatever you like – reflection, review, sense-making. What’s important is that those involved in a project get-together to discuss project progress.

Every four to six months is usually good. These meetings don’t replace the need for daily reflection; rather, they complement it by helping us step back from day-to-day and think more strategically.

For each project, things go something like this: Someone, usually the project manager, talks through progress in relation to the project’s ToC, referring to relevant monitoring data as they go. Some people show a visual of the ToC, augmented with callout boxes, to present key data. This is a good idea. Paths does this automatically but it can be done on slides.

A discussion ensues: Are we on track? Where are things going off-track? Why? Is there anything we need to understand better?

Importantly, reflection meetings are not primarily about data. The real value is in the discussions around the table. Intuition, tacit knowledge, and creativity are as important as data and evidence. Monitoring data informs and enlivens discussion.

Typically, there three outcomes to these meetings:

• No change – continue as is.
• Investigate – there’s a gap in our knowledge, something we need to find out. In this way, reflection meetings inform a rolling research agenda.
• Adapt – change or augment activities, partners, or something else.

Significant adjustments may warrant changes to the ToC and/or monitoring plan. The ToC and monitoring plan continue to reflect the evolving project. In this way, your monitoring system becomes the infrastructure of adaptation.

***

As simple as these steps are, the challenge is in operationalising them. You have to bring the whole team on the journey and it can take time. But keep at it and you’ll do some great project monitoring!