The Royal National Institute of Blind People (RNIB) is one of the UK’s leading sight loss charities and forms the largest community of blind and partially sighted people. The organisation was looking to relaunch its brand and put in place a new strategy focussed on achieving a world with no barriers to people with sight loss.
There were many areas at that time where RNIB was actively involved in pressing for social change and many others where they had previously acted. Within the management team there was a recognition that the new strategy should focus on a smaller number of priority areas, enabling them to focus their resources on initiatives where they could maximise their impact. The difficulty the organisation faced was in deciding what those areas should be.
RNIB approached the Operational Research (OR) Society and asked if it could provide consultancy support (via the Pro Bono scheme) to facilitate a prioritisation exercise to address this. A Pro Bono project profile was subsequently issued, and it was at this point that I became involved.
Having applied to provide the support, I was fortunate enough to be selected by RNIB to help them develop and implement their prioritisation approach and headed into their offices in London to discuss their requirements in a little more detail. From the Project Details, I had in mind that this would be a relatively straight-forward, workshop-based, multi-criteria decision analysis (MCDA) exercise and I would need to:
It became clear during the meeting that my expectations were a bit simplistic and that there were two main strands to the prioritisation exercise:
1. To determine what priorities blind and partially sighted people placed on barriers (established from previous RNIB work) to their full participation in society by surveying as large and representative a sample as possible
2. To provide an initial indication of how well those barriers were a ‘fit’ with RNIB strategy and capability, making use of the more usual workshop-based approach. This would feed into the planned Theory of Change exercise later in RNIB’s strategic planning process, where options for action were to be developed further.
Achieving these objectives would enable us to present the overall results ‘one of those quadrant plots much loved by consultants’, in this case showing Barrier Priority vs Fit.
During email and telephone conversations over the next week we discussed further the best ways of achieving these objectives and completed a Pro Bono Project Proposal form that effectively acted as the ‘contract’ for the engagement. It became clear while completing the proposal that the amount of support required from me was likely to exceed the 5-7day estimate in the original Project Details.
Our discussions focused initially on Barrier Prioritisation, which RNIB felt to be the most important of the two areas. It became evident that RNIB really liked the prospect of using a pairwise comparison approach (as used to establish priorities within the Analytic Hierarchy Process (AHP) method) to determine not only the rankings of the perceived barriers but how much more of a priority one barrier was over another. For those unfamiliar with the approach, the box gives an idea of how pairwise comparisons work.
It was decided to survey blind and partially sighted people using an online questionnaire that collected various demographic information and that solicited views on priorities using a series of pairwise comparison questions. To reduce the number of questions asked of respondents, and to ensure that problems with inconsistent judgements did not materialise, the minimum number of comparisons required to calculate priorities were included. Barrier 1 was compared to Barrier 2, Barrier 2 to Barrier 3, Barrier 3 to Barrier 4, etc. Identifying and dealing with inconsistent judgements requires the use of specialist software such as Transparent Choice and, often, facilitation, neither of which were available via the survey.
To ensure accessibility, the survey questionnaire had to be implemented by RNIB in their preferred software (SurveyMonkey). This meant that I needed to write software to store the survey results in a database, enable all calculations and analyses to be performed, and results presented. In addition:
• A help line was to be set up to enable people to ring in and provide their survey responses over the telephone
• YouGov were asked to provide results for a similar survey (hosted by them) from a panel of respondents diagnosed as visually impaired.
It was decided that assessing the ‘Fit to RNIB’ of the barriers would be performed using a relatively simple MCDA approach.
405 survey responses were received via SurveyMonkey and a further 251 from the YouGov panel. Their pairwise comparisons were aggregated and relative priorities for the barriers calculated. It is worth highlighting that those priorities were relative, we did not assess how serious a barrier was in absolute terms as part of the survey. Previous RNIB work had established that the barriers were significant and worthy of further attention. The highest priority barrier thus ‘scored’ 100 and the other barriers were allocated values that reflected their relative priorities. The results are shown below:
Dan Fisher, Head of Strategy and Performance at RNIB commented on these results:
“Contrary to expectations, by far the most significant barrier to blind and partially sighted peoples’ lives was deemed to be public attitudes (which, when correlated with other ongoing RNIB research, were confirmed as being due to lack of awareness rather than prejudice). This insight led to a shift in RNIB’s strategy towards increasing investment in ‘social change’ and brand activities – i.e., the things which tackle the root causes of barriers in mainstream society, rather than continuing to focus on ameliorating the symptoms through services.”
As we had collected a range of demographic data in the survey we were also able to present results for different segments of the sample, though in all cases ‘Facing low awareness of, and negative attitudes to, sight loss from the general public’ was found to be the highest priority.
The ‘Fit’ of the 12 barriers with RNIB’s strategy and capabilities was assessed using a very simple AHP ratings model implemented in the Transparent Choice software, with the ratings assessments determined at a workshop with members of the RNIB Executive Leadership Team, Trustees and other senior managers.
The workshop attendees were split into three groups and each group came to a consensus assessment of:
• How well addressing each barrier fitted with RNIB’s strategy
• RNIB’s capability to make a difference in addressing each barrier
Each of these assessments was made on a scale of High through to None as illustrated below:
Average assessment scores across the groups were then combined with the criteria weights to arrive at the overall ratings:
Of course, the ultimate test of success was whether we ended up with ‘one of those quadrant plots much loved by consultants’! The figure below demonstrates that we did:
But seriously, the ultimate test of success was whether the process that was followed and the results that were obtained were of value to RNIB.
In the words of Dan Fisher:
“The pro bono scheme gives charities like RNIB much-needed access to professional analytical skills which we struggle to be able to afford on the open market. More than that, though, it enables us to engage in partnerships with volunteers wanting to ‘give something back,’ which is inspiring for us, and hopefully useful and interesting for the volunteers who learn about a new topic or a new way of looking at the world.”
“In our case, the piece of work that Stewart Williams from Hartley McMaster undertook for us has become central to RNIB’s strategy and definition of itself. The results of the exercise were particularly interesting in two ways:
1. As I highlighted earlier, by far the most significant barrier to blind and partially sighted peoples’ lives was deemed to be public attitudes, an insight that led to a shift in RNIB’s strategy.
2. Discussions with customers and staff during the process led to the crystallisation of a belief in an ‘asset model’ of sight loss, whereby we believe that people and society benefit from understanding how people with a visual impairment ‘see’ a world which is fundamentally designed for people with sight. Achieving a world which is fully accessible and inclusive for blind and partially sighted people is, we now believe, good for everyone – i.e. society benefits both practically and psychologically.”
“Both insights are now core to RNIB’s approach, and the whole exercise kick-started further review and development of our service portfolio, shifting it service by service towards concentration on interventions that will achieve the greatest societal impact for the best social returns.”
“The work that Stewart undertook for us was absolutely critical to this, and we are extremely grateful to the OR Society for enabling this level of support. The pro bono scheme gives charities like RNIB much-needed access to professional analytical skills which we struggle to be able to afford.”
It was a pleasure to work with the RNIB Research team during this project. They put in all the hard work necessary to (among many other things) implement the surveys, provide me with data and organise meetings and workshops. Many thanks to Stuart Easton at Transparent Choice for providing free access to their web based AHP decision support software during this work.
This article was originally published in Issue 11 (Spring 2020) of the OR Society’s Impact Magazine.
Tom Dewar: Mar 2023
If you're an RStudio user using Posit Workbench and want to use GitHub for source control (you should), this is the guide for you. There are two ways...
Tom Dewar: Mar 2023
Many companies investing in data analytics struggle to achieve the full value of their investment, perhaps even becoming disillusioned. To understand...
Tom Dewar: Oct 2022
We are cursed to live in interesting times. As I write this, a war in Ukraine rumbles on, we sit on the tail of a pandemic and at the jaws of a...
Stewart Williams