Improving Development Evaluation

David Wand
  • 0,0 (0)
  • Phi lợi nhuận
  • Hai tuần một lần

Join Dr. David Wand as he takes you on a provocative, investigative journey challenging Canadian & American international development organizations as well as international organizations to improve their performance measurement frameworks (PMFs) or Activity Monitoring and Evaluation (MEL) Plans. In Season 1, part 1 of the episode, Dr. Wand will introduce you to an international development organization and one of its projects where flaws in their PMF will be identified. In part 2 of the episode, the international development organization and evaluation experts will be invited to discuss Dr. Wand's critique and offer solutions to improving the evaluation of the project. For Season 2, Dr. Wand expands the scope of his podcast to include USA government funded international development projects by acquiring Activity Monitoring Evaluation Learning Plans (MEL Plans) or their equivalent as well as continue acquiring Canada government funded international development projects that require PMFs or their equivalent. For Season 2 the focus of the podcast has also been improved whereby several projects from the same international development organization are covered in the same episode exposing how the international development organization fails to properly measure and evaluate whether its expected outcomes have been achieved. This is despite these organizations falsely claiming on their websites that they have achieved these outcomes at the expense of the taxpayer.

  1. 9 giờ trước

    GCERF in Mali fails to support its claims of its projects increasing government engagement and integration among youth and ex-prisoners

    In this episode, Dr. Wand explains how a poorly designed evaluation framework with no control or comparision groups fails to support GCERF claims that the project services they have funded were responsible for the measured percentage increases from baseline to endline for projects in Mali. Dr. Wand reviews the evaluation report produced by the Global Community Engagement Resilience Fund (GCERF) available at https://www.gcerf.org/wp-content/uploads/2015/12/Mali-Results-Paper-Round-3-Funding-2020-22.pdf Dr. Wand's review covers the following findings with detailed examples: GCERF claims a percentage increase of 12% from baseline to endline of 92% at endline of women and youth reporting trust in other members in their community after the introduction of a GCERF-funded trust program in that community. This claim is false due the absence of a control or comparison group where women and youth did not receive the trust program.GCERF claims a percentage increase of 30% from baseline to endline at 96% at endline for youth participants reporting having engaged with local leaders after the introduction of a GCERF-funded intervention involving 'mobilised dialogues'. This claim is false due the absence of a control or comparison group where youth did not receive the GCERF-funded program involving 'mobilised dialogues'.GCERF reported '67% of ex-detainees reported feeling accepted by members of their community' with this percentage being due to a GCERF-funded program that 'assisted 1,074 former prisoners in Mali to reintegrate into their communities'. However, GCERF fails to mention the presence of any control groups where former prisoners did not receive this GCERF-funded program but were also measured on the percentage 'feeling accepted by members of their community'. Hence, the GCERF claim is false.Recommendations include introducing control or comparison groups and using psychometric approachs to measuring their expected outcomes (e.g. increases in trust, social cohesion, tolerance, vocational skills, critical thinking skills specific to preventing recruitment by extremist groups) rather than resorting to flawed outcome harvesting.To receive discounts for the purchase of sunglasses or polo shirts or travel bags/backpacks/wallets designed for the Global South use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

    GCERF in Mali fails to support its claims of its projects increasing government engagement and integration among youth and ex-prisoners

  2. 24 thg 6

    GCERF post evaluation fails to show effectiveness in countering violent extremism in Nigeria

    In this episode, Dr. Wand explains how a poorly designed evaluation framework with invalid outcome indicators leads to a post-project evaluation that fails to demonstrate effectiveness in countering violent extremism in Nigeria. Dr. Wand reviews the evaluation report produced by the Global Community Engagement Resilience Fund (GCERF) available at https://www.gcerf.org/wp-content/uploads/2015/12/Nigeria-Results-Paper-Round-2-Funding-2019-21.pdf. Dr. Wand's review covers the following findings with detailed examples: Expected outcomes cannot be claimed to be achieved as GCERF claims as the outcome indicators depend on biased self-reporting rather than objective measures. Examples include GCERF claims of Community Action Response Teams being 'fully functionable' but with no objective measures of functionality.GCERF claims of high levels of confidence by beneficiaries are questionable as methods of measuring these confidence levels are not provided.Outcome indicators measuring skill acquisition for concrete skills such as the ability to mediate or manage conflicts or the ability to identify and report early warning signs of conflict are absent yet GCERF assumes without data that this skill acquisition occurred and that conflict mediation or management was successful.GCERF makes 'impact' claims beyond the project despite no evidence such as reductions in violence and alcohol in specific communities.Recommendations to include data sources and methods of data collection and analysis are provided to enable GCERF to support their claims that currently cannot be supported as well as ensuring that outcome indicators actually measure the expected outcomes. To receive discounts for the purchase of sunglasses or polo shirts or travel bags/backpacks/wallets designed for the Global South use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

    GCERF post evaluation fails to show effectiveness in countering violent extremism in Nigeria

  3. 19 thg 6

    Humber Polytechnic International Development Institute fails to properly evaluate its $2 Million Kenya & Ethiopia STEM Education for Empowerment project

    In this episode, Dr. Wand reviews the Humber Polytechnic International Development Institute (https://humber.ca/global/international-development-institute-idi) $2 Million Kenya & Ethiopia STEM Education for Empowerment project's Performance Measurement Framework (PMF). He concludes that Humber Polytechnic International Development Institute cannot make the claim that it has achieved 8 of the 9 expected Outcomes for the project since 18 out of the 19 outcome indicators are not valid measures of the Outcomes, or for the 1 valid outcome indicator fail to show the necessary statistical analysis that would support the project's claim that the outcome was achieved statistically significantly rather than due to chance. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p007925001 The PMF and Dr. Wand's critique are available by emailing evaluatecanadaaid@gmail.com. The PMF and the critique will be shared with Humber Polytechnic along with an invitation for them to respond to the critique by attending a future podcast. It will also be shared with the Secretary of State for International Development as well as the shadow critics for all the opposition parties. To receive discounts for the purchase of sunglasses or polo shirts or travel bags/backpacks/wallets designed for the Global South use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

    Humber Polytechnic International Development Institute fails to properly evaluate its $2 Million Kenya & Ethiopia STEM Education for Empowerment project

  4. 1 thg 6

    GCERF fails to validly measure rehabilitation of Returning Foreign Terrorist Fighters - Western Balkans

    The Global Community Engagement Resilience Fund (GCERF) (gcerf.org) has been funding organizations in Bosnia & Herzegovina, Kosovo, Albania, and North Macedonia to deliver psychological services to Returning Foreign Terrorist Fighters to enable them to become first, rehabilitated, and then reintegrated into their communities. Unfortunately, GCERF has failed to provide any valid measures of rehabilitation for these foreign terrorist fighters despite making claims of achieving subsequent reintegration for these fighters as shown by the absence of any rehabilitation outcome indicators in their "Results from Round 1 and Round 2 Funding in the Western Balkans (2020-2025)" report. In this episode, Dr. Wand recommends that GCERF draw upon 4 valid measures of rehabilitation specific to Returning Foreign Terrorist Fighters, including the Violent Extremism Risk Assessment VERA and the TRAP-18 (Terrorist Radicalization Assessment Protocol) psychometric tools. He also identifies flaws in statistical analysis such as aggregating outcome indicators across different projects within a country as well as aggregating outcome indicators across different countries that hinders GCERF's ability to use evaluation results to improve service delivery at the project level. He also notes the improper use of comparing averages of percent indicators rather than using medians that fails to recognize extreme percent values that can distort the overall percent indicator. For receiving a discount on the products recommended when visiting the Global South, use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

    GCERF fails to validly measure rehabilitation of Returning Foreign Terrorist Fighters - Western Balkans

  5. 26 thg 5

    GCERF reports percentage increases but fails to do the statistical analysis to support statistical significance: Ghana Results Paper Emergency Grant

    In this episode, Dr. Wand thanks the Global Community Engagement Resilience Fund (GCERF) (gcerf.org) for 'improving development evaluation' by, six years later, finally publishing their results frameworks, including outcome statements, outcome indicators, as well as actually data on these indicators. A marked increase in data transparency!!! In this episode, Dr. Wand reviews the GCERF report Ghana Results Paper: Emergency Grant and identifies 4 areas of weakness in the results paper as follows: Poor design of Outcome Indicators that leads to invalid measures of whether the expected outcome has been achieved.Failure to describe psychometric tool used to measure the Outcome Indicators along with suggested psychometric tools to measure levels of trust.Failure to measure the expected outcome of increased vocational skill specific to tailoring, parboiling rice, and processing shea butter.Failure to show the inferential statistical analysis needed to test whether reported percentage increases in the report have achieved statistical significance or have not achieved statistical significance.For receiving a discount on the products recommended when visiting the Global South, use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

    GCERF reports percentage increases but fails to do the statistical analysis to support statistical significance: Ghana Results Paper Emergency Grant

  6. 24 thg 4

    Humber Polytechnic International Development Institute fails to properly evaluate its $4.9 Million Bhutan Education and Skills Training project

    In this episode, Dr. Wand reviews the Humber Polytechnic International Development Institute (https://humber.ca/global/international-development-institute-idi) $4.9 Million Bhutan Education and Skills Training project Performance Measurement Framework (PMF). He concludes that Humber Polytechnic cannot make the claim that it has achieved 9 of the 11 expected Outcomes for the project since the outcome indicators are either not valid measures of the Outcomes and/or contain conflicts of interest for 25 out of 29 Outcome Indicators where project staff rather than independent evaluators have been responsible for data collection on these Outcome Indicators. Even for the remaining 2 Outcomes that contain 4 valid Outcome Indicators, there is no inferential statistical analysis p.05 to support the claim that observed percentage increases on the indicators would be statistically significant to support the claim that the project was effective in achieving its outcomes. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010696001 The PMF and Dr. Wand's critique are available by emailing evaluatecanadaaid@gmail.com. The PMF and the critique will be shared with Humber Polytechnic along with an invitation for them to respond to the critique by attending a future podcast. It will also be shared with the Secretary of State for International Development as well as the shadow critics for all the opposition parties. To receive discounts for the purchase of sunglasses or polo shirts use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

    Humber Polytechnic International Development Institute fails to properly evaluate its $4.9 Million Bhutan Education and Skills Training project

  7. 31 thg 3

    World Vision Canada fails to show Impact in its $41 Million Gender Equality Nutrition program but potential to show project effectiveness is there

    In this episode we review the $41 Million World Vision Canada project (www.worldvisioncanada.ca) entitled Realizing Gender Equality, Attitudinal Change, and Transformative Systems in Nutrition. More information about this project can be found at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p011791001. A critique of the Performance Measurment Framework (PMF) for this project reveals 13 expected Outcomes for this project with 38 Outcome Indicators developed to examine whether those Outcomes were indeed achieved. This critique concludes that 10 Outcome Indicators are designed to enable to examine whether those Outcomes were achieved while the remaining 28 Outcome Indicators were considered to be poorly designed with no possibility of supporting the claim that their related Outcomes were achieved. Data on these Outcome Indicators are not available to the public which would enable the public to see if Outcomes were achieved. Flaws in Outcome Indicator design include using self-reporting bias instead of external objective measures as well as Outcome Indicators not related to the Outcome. Even for the 10 Outcome Indicators considered to well designed in measuring the achievement of its related Outcome, many of these Outcome Indicators failed to offer statistical significance analysis to show that the achievment of the Outcome could be attributed to the $41 Million project where that achievement was significantly different compared to the Outcome Indicator moving in the desired direction anyway in the absence of the $41 Million project or moving in the desired direction over time but with negligible effect (i.e. lacks significant Impact). Please sign the petition here https://www.change.org/EvaluateCanadaAid to have the Government of Canada require all Canadian taxpayer funded international development organizations release on their websites their Performance Measurement Frameworks (PMFs) or its equivalent and data for all output and outcome indicators listed in the PMF. For a 15% discount on sunglasses, and cotton polo shirts when visiting the Global South use this discount link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast

    World Vision Canada fails to show Impact in its $41 Million Gender Equality Nutrition program but potential to show project effectiveness is there

  8. 17/12/2025

    Failure of CUSO International to properly evaluate its 2 projects worth $7.5 Million

    Please sign the petition here https://www.change.org/EvaluateCanadaAid to have the Government of Canada require all Canadian taxpayer funded international development organizations release on their websites their Performance Measurement Frameworks (PMFs) or its equivalent and data for all output and outcome indicators listed in the PMF. CUSO International (www.cusointernational.org) has failed to properly measure and evaluate its 2 projects costing the Canadian taxpayer $7.5 Million. CUSO International cannot make the claim that it has achieved its outcomes for their 2 projects. This is based upon a review and critique of the Performance Measurement Frameworks (PMFs) for these 2 projects. This critique is explained for each outcome and outcome indicator from these 2 PMFs in this episode. These PMFs and PMF critique can be requested by emailing evaluatecanadaaid@gmail.com. Details on these 2 CUSO International projects is available here: Blueprints for Gender Equality https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010478001 and Building Bricks Caribbean https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p013586001 For the Blueprints for Gender Equality CUSO International project, the PMF had 9 Outcomes with 13 Outcome Indicators. Only 1 of those Outcome Indicators was properly designed to support the claim that the Outcome was achieved. Even if it was achieved, no data was provided to support that claim. The remaining 12 poorly designed Outcome Indicators were poorly designed due to either self-reporting bias or conflict of interest when an external consultant should have been used instead to objectively measure whether the outcome was achieved. They used an external consultant for the 1 good Outcome Indicator but oddly refused to do it for the other Outcome Indicators!! Other poorly designed Outcome Indicators were due to measures that had nothing to do with whether the Outcome was achieved. For the Building Bricks CUSO International project, the PMF had 8 Outcomes and 15 Outcome Indicators. 4 of those Outcome Indicators were properly designed to support the claim that some of the Outcomes were achieved. Even if they were achieved, no data was provided to support that claim. The remaining 11 poorly designed Outcome Indicators were poorly designed due to either self-reporting bias or due to measures that had nothing to do with whether the Outcome was achieved. CUSO International has been invited to attend the podcast to provide a response to this critique of their project PMFs. The PMF critique and PMFs have been sent to CUSO International as well as the Secretary of State for International Development and the shadow critics for International Development. For a 20/15% discount on sunhats, sunglasses, and cotton polo shirts when visiting the Global South use this discount link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast

    Failure of CUSO International to properly evaluate its 2 projects worth $7.5 Million
Xem tất cả (32)

Trailers

Giới Thiệu

Join Dr. David Wand as he takes you on a provocative, investigative journey challenging Canadian & American international development organizations as well as international organizations to improve their performance measurement frameworks (PMFs) or Activity Monitoring and Evaluation (MEL) Plans. In Season 1, part 1 of the episode, Dr. Wand will introduce you to an international development organization and one of its projects where flaws in their PMF will be identified. In part 2 of the episode, the international development organization and evaluation experts will be invited to discuss Dr. Wand's critique and offer solutions to improving the evaluation of the project. For Season 2, Dr. Wand expands the scope of his podcast to include USA government funded international development projects by acquiring Activity Monitoring Evaluation Learning Plans (MEL Plans) or their equivalent as well as continue acquiring Canada government funded international development projects that require PMFs or their equivalent. For Season 2 the focus of the podcast has also been improved whereby several projects from the same international development organization are covered in the same episode exposing how the international development organization fails to properly measure and evaluate whether its expected outcomes have been achieved. This is despite these organizations falsely claiming on their websites that they have achieved these outcomes at the expense of the taxpayer.

Thông Tin

  • Nhà sáng tạo
    David Wand
  • Năm hoạt động
    2024 - 2026
  • Tập
    32
  • Xếp hạng
    Sạch
  • Bản quyền
    © David Wand 2023
  • Trang web chương trình
    Improving Development Evaluation