• Editors' Summary
• Credits

1. Overview
2. Getting Started
3. Define Desired Performance
4. Describe Actual Performance
5. Measure/Describe Performance Gaps
6. Find the Root Causes
7. Select Interventions
8. Implement Interventions
9. Monitor and Evaluate Performance
• Did the Project Close Performance Gaps?
• Did Provinces Differ?
• Did Facilities Differ?
10. Managing Change

• Figures
• Tables
• Side-bars
• Bibliography

• PI approach offers many benefits
• Pilot project holds lessons
• Helping people do their best
• Burkina Faso: Assessing needs
• India: PI helps private providers
• Using the “Why Tree” in Ghana

• POPLINE
• Media/Materials Clearinghouse
• Other issues
• To order copies
• CCP Home Page

Published by the Population Information Program, Center for Communication Programs, The Johns Hopkins University Bloomberg School of Public Health, 111 Market Place, Suite 310, Baltimore, Maryland 21202, USA

Monitor and Evaluate Performance

Staff members or consultants monitor the solutions to ensure that they are carried out as planned, and they evaluate the solutions to assess results. Monitoring allows staff members to respond to unexpected problems or take advantage of unexpected opportunities. Among the monitoring tasks are checking that all stakeholders are involved, that top management is publicly supportive, and that the staff members whose performance is being analyzed are participating and accepting the solutions.

The program monitors notify other team members of problems or changes in schedule that affect other deadlines (102). If results fall short, midcourse adjustments can be made. In a training-of-trainers program, for example, monitors can observe classes taught by trainers, take note of any weaknesses, and suggest changes to the curriculum (126).

To evaluate solutions, staff members or consultants measure actual performance after the solutions take effect and compare it with the desired performance agreed to by stakeholders. The evaluators use the same performance indicators that were used to measure the initial performance gap. Data come from observations, interviews or surveys of staff and clients, self-assessment questionnaires, or clinic records.

Few reproductive health care organizations have evaluated their use of the PI process. Only the pilot project carried out by the Dominican Social Security Institute (IDSS) has documented its evaluation. For the IDSS, consultants measured actual performance in three provinces, San Cristóbal, La Romana, and La Vega. They carried out a baseline survey in March/April 1999 and follow-up surveys in August 1999, six weeks after the solutions were carried out, and in July/August 2000. Three questions were addressed:

• Did the project close performance gaps? The evaluation team analyzed performance over time in one province, San Cristóbal, where the IDSS carried out a full set of solutions addressing expectations, performance appraisal, and knowledge and skills.
• Did provinces differ? The evaluation team compared results in San Cristóbal with those in La Romana, where providers worked on expectations and feedback but were not specially trained, and in La Vega, the control province where no solutions were carried out.
• Did facilities differ? The evaluation team compared performance gaps for staff members in the three types of health care facilities that participated in the project—hospitals, clinics, and doctors’ offices.

The IDSS evaluation measured performance gaps in considerate treatment of clients and providers’ knowledge of reproductive health services offered by the IDSS.

Considerate treatment of clients. The evaluation had two parts: clients were interviewed after they used reproductive health services, and observers watched providers as they cared for clients. Interviewers and observers filled out a questionnaire that measured indicators of considerate treatment of clients. The questionnaires assessed the four areas of counseling: courtesy (Did the provider greet you and call you by your name?); privacy (Did the provider ensure that the consultation would be as comfortable and private as possible?); information (Did the provider give information that answered your questions or needs?); and problem-solving (Did the provider help you to reach a decision that resolved a problem?). The maximum score for desired performance based on the questionnaire was 12.

In San Cristóbal the performance gap closed significantly according to both clients and observers. According to clients, the gap decreased from 5.2 at baseline to 4.7 (10% difference from baseline) at the first evaluation survey and to 3.9 (25% difference) at the second survey. According to observers, the performance gap decreased from 7.9 to 4.3 (46%) at the first survey and then increased to 5.6 (29%) at the second survey (120).

Knowledge of reproductive health services. PI facilitators interviewed approximately 80 providers to assess their knowledge of the full range of reproductive health services offered by the IDSS. Facilitators asked providers three questions and graded them on the number of services they mentioned in their answers: (1) For you what is meant by reproductive health services? (2) What are the reproductive health services offered at this facility? and (3) For which reproductive health services can you refer clients? In San Cristóbal the performance gap decreased by 32% at the first follow-up survey but then increased by 4% at the second survey compared with the gap at the baseline survey, probably because of staff turnover (120).