10 episodes

Program Evaluation and Research Consulting

EvalNetwork EvalNetwork

    • Business

Program Evaluation and Research Consulting

    Empowering Change: David Fetterman on Using Evaluation to Build a Better World

    Empowering Change: David Fetterman on Using Evaluation to Build a Better World

    David Fetterman is a leading expert in empowerment evaluation, an approach that emphasizes collaboration, participation, and capacity building. He has written extensively on the topic, and his work has been used in a wide range of settings, including government agencies, non-profit organizations, and businesses.

    David’s work focuses on helping people evaluate their programs and initiatives and use the findings to improve their work. He believes that empowerment evaluation is essential for building a more just and equitable society. David is known for his innovative work, commitment to social justice, and ability to make complex ideas accessible to a wide audience.

    Understanding Empowerment Evaluation: A Shift in Traditional Evaluation Models

    Empowerment evaluation revolutionized the way we view and conduct evaluations. David explains how this approach places the evaluated individuals in control, transforming them from subjects to leaders of the evaluation process. This shift not only uncovers real, often overlooked issues but also ensures more sustainable results by fostering a sense of ownership and self-efficacy among the participants.

    Integration of Empowerment Evaluation in Organizations: A Case Study

    David discusses the seamless integration of empowerment evaluation within organizational structures. He highlights a case where a school system successfully internalized evaluation, making it a natural and credible part of their daily operations and program implementation. This integration allowed for timely and data-driven mid-course corrections, ensuring the longevity and effectiveness of the evaluative practices within the organization.

    Empowering Communities through Participatory Evaluation: An Oregon Case Study

    This episode further explores the impact of empowerment evaluation on community development. Fetterman shares an inspiring story from Oregon, where children, armed with collected cigarette butts, advocated for smoke-free policies. This participatory approach, Fetterman notes, empowers communities to set and achieve their own goals, fostering independent growth and development.

    Microskills for Facilitating Empowerment Evaluation Meetings

    David delves into the essential microskills required for effectively facilitating evaluation meetings. From personal and communication skills to technical advising, these microskills ensure that the evaluation process is respectful, inclusive, and constructive, laying the foundation for meaningful and lasting change.

    Navigating Cultural and Political Realities in Project Facilitation

    The episode also touches on the challenges of navigating diverse cultural and political landscapes in project facilitation. David shares personal experiences, emphasizing the importance of listening and adapting to local perspectives to ensure the success and acceptance of the projects abroad.

    Utilizing Evaluation and Empowerment for Positive Community Change

    David highlights the transformative power of evaluation and empowerment in enacting positive community change. He shares a compelling story of a community member’s journey from opposition to advocacy for a teenage pregnancy prevention program, underscoring the importance of understanding and addressing underlying concerns within the community.

    Maintaining Self-Awareness and Reflection in Project Work

    In the podcast, David also discusses the critical role of self-awareness and reflection in project work. He shares insights on receiving and learning from feedback, working closely with community partners, and ensuring the focus remains on strengthening the community rather than asserting control.

    Making Evaluation Principles Explicit for Project Success

    He explains the significance of making evaluation principles explicit to ensure project success. He recounts a project experience in East Palo ...

    From Lecture Halls to Real-World Calls: Tiffany Berry’s Evaluation Insights

    From Lecture Halls to Real-World Calls: Tiffany Berry’s Evaluation Insights

    Whether you're an educator, a student, or simply someone passionate about youth development and educational programs, this podcast episode with Tiffany Berry, PhD., promises to give you insights into the complex world of evaluation. She is the Dean and a full research professor in the Division of Behavioral & Organizational Sciences at Claremont Graduate University, where she also received her Ph.D.



    Her research interests include Educational Program Evaluation, Educational Curricula, Comprehensive School Reform, after-school services, social-emotional learning, and other areas. She has published over 75 technical evaluation reports and peer-reviewed articles in leading evaluation and youth development journals.



    See XXXXX for a detailed description of the interview.



    OUTLINE:

    0:00 Overview of episode

    2:07 Evaluation models and conceptual models she uses

    8:56 Common challenges Tiffany encountered evaluating educational programs

    11:59 Challenges in measuring youth development outcomes

    17:36 Changing her evaluation approach due to unforeseen circumstances or changes in the program

    23:00 How to stay current in the field of evaluation

    28:10 Advice for those interested in becoming an evaluator

    32:41 Advantages and disadvantages of getting a graduate degree in evaluation versus other disciplines

    38:15 Credentialing in evaluation in the US

    44:32 The uniqueness of Claremont’s evaluation program

    49:40 Claremont’ Doctoral program in evaluation

    55:09 Building a professional network in the evaluation field



    CONNECT WITH JAMES:

    - Subscribe to this YouTube channel: https://www.youtube.com/user/jamesmpa...

    - LinkedIn: https://www.linkedin.com/in/pannjames/



    Please reach out with comments and questions. Thanks!

    Mindfulness Meets Evaluation: Insights from Jim McDavid

    Mindfulness Meets Evaluation: Insights from Jim McDavid

    In this episode, I talk with Jim McDavid, Ph.D., about his experience with mindfulness and meditation practice, how it has influenced him, and how it affects how he views and practices evaluation. Our conversation also covers practical wisdom, Jim’s interest in the environment, and challenges associated with determining cause and effect in evaluation.



    Jim is Professor Emeritus at the School of Public Administration, University of Victoria, which he joined as faculty in 1980. He was a recipient of the University of Victoria Alumni Teaching Award, and he received the University of Victoria’s highest academic honor, the UVic Distinguished Professorship Award. He has contributed significantly to the field of evaluation, and the reason I reached out to him related to his work related to mindfulness and evaluation.



    00:00 Introduction

    01:57 Jim’s definition of mindfulness

    05:20 The ethical dimension of mindfulness

    08:49 How Jim’s mindfulness practice has evolved over time

    11:46 Difference between Transcendental Meditation and Vipassana

    15:33 The spiritual dimension of meditation

    22:10 Subjectivity of cause and effect

    29:34 How mindfulness opens up the mind

    35:52 Mindfulness can support evaluation practice

    40:32 Connection to practical wisdom

    44:45 Importance of being present and really listening in evaluation contexts

    49:20 Judgment and evaluation

    50:46 Starting a mindfulness practice



    For more go to: https://evalnetwork.com/mindfulness-meets-evaluation/

    Maximize Your Survey Response Rates: Expert Insights from Sheila Robinson

    Maximize Your Survey Response Rates: Expert Insights from Sheila Robinson

    In this episode, James Pann, Ph.D., interviews Sheila Robinson, Ed.D., about the topic of surveys and response rates. We focus on the significance of response rates in surveys and the steps that can be taken to maximize them.



    Sheila is a career educator and professional learning designer with experience in K-12 public education and higher education, a certified program evaluator, and Certified Presentation Specialist (CPS™) with particular interests in survey design, data visualization, and presentation design.



    Read more here: https://evalnetwork.com/maximize-your-survey-response-rates



    Timeline

    00:40 Why Sheila is interested in survey research

    03:50 Why survey response rate is critical

    05:15 The optimal response rate

    07:37 Ensuring accurate demographic representation of the sample

    09:30 How to improve response rate

    12:01 The survey invitation message is critical

    13:26 Transparency in the length of time to complete the survey is important

    15:19 Survey reminders can be used tactically to improve the response rate

    16:43 Incentives should be used carefully

    20:46 Timing of incentives and the principle of reciprocity

    23:08 Building survey completion time into program activities can increase response rate

    26:17 Importance of piloting the survey prior to use

    28:15 Who sends the survey is important

    33:20 How to reach Sheila



    EPISODE LINKS:

    - Sheila's website: https://www.sheilabrobinson.com/

    - Sheila's Linkedin: https://www.linkedin.com/in/sheilabrobinson/



    CONNECT WITH JAMES:

    - Subscribe to this YoutTube channel: https://www.youtube.com/user/jamesmpann?sub_confirmation=1

    - LinkedIn: https://www.linkedin.com/in/pannjames/



    Please reach out with comments and questions. Thanks!

    Young Adult Cancer Survivors Increase Mindfulness and Connection During Nature Treks with David Victorson

    Young Adult Cancer Survivors Increase Mindfulness and Connection During Nature Treks with David Victorson

    /*! elementor - v3.21.0 - 15-04-2024 */

    .elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block} James Pann, Ph.D., interviews David Victorson, Ph.D., of True North Treks, a nonprofit organization whose mission is to empower young adults and caregivers affected by cancer to “find direction through connection” and mindfulness.As a child, David grew up surrounded by nature and its many restorative benefits. Therefore, when he went on to complete his postdoctoral fellowship in psychosocial oncology as a psychologist, he saw an opportunity to bring nature’s gifts to the young adult cancer patients he was seeing.In 2008, he co-founded True North Treks to fill some of the unmet needs of these cancer survivors and their caretakers and help them get their lives back on track. The reconnecting power of nature, coupled with mindfulness and meditation laid the basis for these restorative journeys.David goes on to discuss one of the most reported unmet needs: isolation. Many of the young cancer patients/survivors feel like they don’t know anybody like them. These treks allow the opportunity for deep social connection with others going through the same or similar experiences. These needs and solutions developed into three key points.







    /*! elementor - v3.21.0 - 15-04-2024 */

    .elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}True North Treks 3 Crucial Connections







    1) Connection with nature (after going through something as unnatural as cancer treatment);2) Connection with peers who get it and have walked a similar path;3) Connection with oneself through mindful awareness practices, such as meditation and yoga.While it may sound like a therapy session at first, David emphasizes the lack of an explicit group therapy aspect. The guides are trained never to question the participants about their cancer and instead simply sit back and allow them to speak their minds. Often, the participants will immediately start talking about their cancer experience on their own.The guides, primarily mental health professionals, are taught to be themselves and simply bring mindfulness coaching. The participants benefit from the mindfulness and yoga experience and being with each other in the outdoors. That said, a “therapeutic” aspect tends to emerge on its own when the participants find themselves with several others just like themselves.Being one of the 3 Crucial Connections, David defines what mindfulness means on the treks.

    The CIPP Evaluation Framework with Guili Zhang

    The CIPP Evaluation Framework with Guili Zhang

    James Pann interviews Guili Zhang about the Context, Input, Process, and Product (CIPP) evaluation model and other evaluation related areas. Dr. Zhang is Department Chair and Professor of Research and Evaluation at East Carolina University. She received a Ph.D. in Research and Evaluation Methodology from the University of Florida and postdoctoral advanced training in large scale data analysis from Stanford University. She has presented and published extensively, and led the evaluation of many programs and projects, funded by agencies such as the U.S. Department of Education and the National Science Foundation. Guili’s book, The CIPP Evaluation Model, coauthored with Daniel Stufflebeam, is the authoritative book on the CIPP Model, one of the most influential and widely used evaluation frameworks. Dr. Zhang is very active in the American Evaluation Association and is currently a Board of Directors Member-at-Large.Timeline00:00 – Introduction00:18 – How she got involved with the CIPP model01:43 – When she sent her evaluation report that used the CIPP model to Dan Stufflebeam02:38 – Dan asks Guili to write a book about the CIPP model with him05:53 – Collaborating on the writing of the book at a distance06:46 – Guili’s concise explanation of the CIPP model09:01 – CIPP model as an effective way to teach about evaluation in general10:30 – Unique advantages of using the CIPP model12:20 – A common sense approach to evaluation that can be used by many13:21 – CIPP model as a living, evolving framework14:44 – Updated CIPP model related checklists linked to in the book15:56 – What non-evaluation students can bring to their future work by learning evaluation18:22 – The best way to learn how to do evaluation19:35 – Evaluation resources she suggests21:50 – How evaluation can be used to improve our world23:41 – What Guili would like to accomplish while an AEA board member25:28 – Books she likes to give as a gift to friends and colleaguesEpisode LinksThe CIPP Evaluation Model: How to Evaluate for Improvement and Accountability. Western Michigan University, The Evaluation Center American Evaluation AssociationAmerican Family Education InstituteGuili’s LinkedinGuili’s TwitterConnect with JamesSubscribe to YouTube channelLinkedInTwitter Please reach out with comments and questions.Thanks!

Top Podcasts In Business

The Diary Of A CEO with Steven Bartlett
DOAC
The Property Academy Podcast
Opes Partners
Cooking the Books with Frances Cook
BusinessDesk and NZ Herald
Where's My Money?
rova
Girls That Invest
Girls That Invest
The Tim Ferriss Show
Tim Ferriss: Bestselling Author, Human Guinea Pig