REvaluation Podcast

fteval

REvaluation Podcast über die Zukunft der Forschungspolitik - und ihrer Evaluierung. Präsentiert von Tobias Dudenbostel und Isabella Wagner für die Österreichische Plattform für Forschungs- und Technologiepolitikevaluierung (fteval). REvaluation Podcast about the future of research policy - and its evaluation. Presented by Tobias Dudenbostel and Isabella Wagner for the Austrian Platform for Research and Technology Policy Evaluation (fteval).

  1. Folge 35 - "Critical Friends" und wie man sie findet: Ein Erfahrungsaustausch zu Panelevaluierung

    11/13/2025

    Folge 35 - "Critical Friends" und wie man sie findet: Ein Erfahrungsaustausch zu Panelevaluierung

    Am 4. November 2025 luden der Wiener Wissenschafts-, Forschungs- und Technologiefonds (WWTF) und die Plattform fteval zu einer Veranstaltung über Panelevaluierung, um die Erfahrungen in der Community in einen Austausch zu bringen. Für diese Podcastfolge haben wir die Diskussion für unsere Zuhörenden eingefangen: Magdalena Wicher und Michael Strassnig vom WWTF fassen mit Katharina Warta, Leiterin des österreichischen Büros von Technopolis Group, die wichtigsten Punkte zusammen und kommen am Ende sogar noch in eine richtige Kontroverse! Für die Beteiligten ist klar, was die Vorteile von Panels sind, weil man beispielsweise rasch die kritischen Punkte zum Evaluationsobjekt identifiziert werden können und die Rolle der "Critical Friends" sehr wertvoll sein kann. Gleichzeitig ist es herausfordernd und ressourcenintensiv, ein ausgewogenes und hochkarätiges Panels zu etablieren und Ownership über den Evaluationsgegenstand und Vertrauen zu generieren. Die Bedeutung von persönlichen Treffen wird besonders herausgestrichen. Katharina teilt ihre Erfahrung aus der Gruppendynamik und erklärt anhand des Johari-Fensters, wie man blinde Flecken, Unbekanntes und Geheimes aufdecken kann.  Links:  Foliensatz des WWTF zu Panelevaluierung mit zwei FallbeispielenFallbeispiele von Technopolis Group: Evaluation Methodology der Evaluation der wissenschaftlichen Institutionen Lettlands: https://www.izm.gov.lv/lv/media/29432/download?attachment Presentation des Evaluationsberichts für Universität Luxembourg: https://www.chd.lu/fr/meeting/13645 (ab Minute 5:50)Musik: Urbana-Metronica (wooh-yeah mix) (ft. Morusque, Jeris, CSoul, Alex Beroza) by spinningmerkaba http://ccmixter.org/people/jlbrock44; SouljaUnit Remix of DeadRobot Music’s Surfy via https://freesound.org/people/SouljaUnit/sounds/640175/ Foto: Isabella Wagner podcast(at)fteval.at - twitter.com/fteval - www.fteval.at

    47 min
  2. REvaluation Special #7: What skills does the next generation of evaluators need?

    05/30/2025

    REvaluation Special #7: What skills does the next generation of evaluators need?

    There is a generation shift in the community going on and in the dynamic landscape of research and innovation, the perspectives of the next generation are crucial for shaping future policies and practices. In this panel we tried to bring together younger researchers and evaluators and senior counterparts to discuss the challenge, demands and future directions of innovation research and policy evaluation. Panellists Florian Bayer, Department of Science and Technology Studies, University of ViennaLuyu Du, Center for Organization Research and Design (CORD), Arizona State University, United StatesNino Gogadze, CWTS, Leiden UniversityJakob Kofler, Austrian Institute for SME ResearchModerators Philippe Larédo & Julia Melkers  Music: Urbana-Metronica (wooh-yeah mix) (ft. Morusque, Jeris, CSoul, Alex Beroza) by spinningmerkaba http://ccmixter.org/people/jlbrock44; SouljaUnit Remix of DeadRobot Music’s Surfy via https://freesound.org/people/SouljaUnit/sounds/640175/ Photo: Martin Jordan Thanks to our partners and co-organisers that helped realising the REvaluation Conference 2024: Ministry of Innovation, Mobility and Infrastructure, Ministry of Women, Science and Research, Ministry of Economy, Energy and Tourism, Vienna Science and Technology Fund, COST Action PROFEEDBACK (CA ), Meeting Destination Vienna, FH Campus Vienna, Austrian Science Fund, Austrian Research Promotion Agency, ZSI - Centre for Social Innovation, Fraunhofer ISI, Institut Francilien Recherche, Innovation et Société, Joanneum Research, the whole fteval Community.  podcast(at)fteval.at - twitter.com/fteval - www.fteval.at

    50 min
  3. REvaluation Special #6: Are we all metascientists now, James Wilsdon?

    05/29/2025

    REvaluation Special #6: Are we all metascientists now, James Wilsdon?

    Motivated by evidence based scientists doing gut-based science policy, and by science policy maker acknowleding the faults of the system when they leave it rather than when they can still do something about it, James Wilsdon of RoRI turned his attention towards meta science in this REvaluation 2024 keynote. Meta Science, Meta Research, Science of Science, Research of Research are all the same, and not new, but new things are going on. Applying robust methods to understanding, analysing and improving the science system! He shows that he is not the only one, but more and more RoR is done out of the various discplines and sub-disciplines themselves, in particular, because there is data to do so, and that adds specific domain knowledge to the pool of RoR bottom-up. Top-down, from science funders, are at least in the UK again testing their funding in large-scale experimental settings; and are opening their funding data for the ROR community. Lastly, new institutions contribute to the debate, in Carlifornia funded by the Tech community, trying to "practice meta science".  Why does this happen? More investment, slow down of disruption and radical change, worries about too much reporting and to little impact, on-going crisis of replication, and rapdily changing landscape of scolar communication (open science, open data), pressure for excellence and AI and research funding. MetaScience is both a contribution to incremental change and a source for radical, systematic change. Concluding, Wilsdon argues for five priorities to go forward: more and better experimentation for evaluation (e.g. applicants reviewing other applicants, makes it easier to find reviewers, and to speed up the process); investing in infrastructures for doing MetaScience at scale and long-term (e.g. the publish-curate-review-platform MetaROR); Foster cross sector alliances (e.g. MetaScience Alliance); with more global conversations beyond western countries; and demonstrate the worth of MetaScience! Links: RoRI-Research on Research Institute: https://researchonresearch.org/Metascience Conference: https://metascience.info/ James' presentation slidesMusic: Urbana-Metronica (wooh-yeah mix) (ft. Morusque, Jeris, CSoul, Alex Beroza) by spinningmerkaba http://ccmixter.org/people/jlbrock44; SouljaUnit Remix of DeadRobot Music’s Surfy via https://freesound.org/people/SouljaUnit/sounds/640175/ Photo: Martin Jordan Thanks to our partners and co-organisers that helped realising the REvaluation Conference 2024: Ministry of Innovation, Mobility and Infrastructure, Ministry of Women, Science and Research, Ministry of Economy, Energy and Tourism, Vienna Science and Technology Fund, COST Action PROFEEDBACK (CA ), Meeting Destination Vienna, FH Campus Vienna, Austrian Science Fund, Austrian Research Promotion Agency, ZSI - Centre for Social Innovation, Fraunhofer ISI, Institut Francilien Recherche, Innovation et Société, Joanneum Research and the whole fteval Community.  podcast(at)fteval.at - twitter.com/fteval - www.fteval.at

    48 min
  4. REvaluation Special #5: How to fund transdisciplinarity?

    05/28/2025

    REvaluation Special #5: How to fund transdisciplinarity?

    There is momentum in the research community with regard to trandisciplinary research (TDR) and research funding, because societal transformation necessitates more complex solutions; but at the same time, the notion that research happens more and more within disciplines, and not across disciplines or in a transdisciplinary way. The panel discusses challenges, but also opportunities TDR provides and tackles questions and opportunities on reasons to fund TDR and how to achieve impact, and on related challenges. The panel touches on questions like how new is the approach, what can be done to support it further and how to provide contiunity and manage knowledge within the research system. The panelists were Christian Pohl, ETH Zürich, Co-Director Transdisciplinarity LabHelen Buckley Woods, University College London, Research on Research InstituteMarianne Penker, BOKU University, Academy of Science Moderation: Petra Biberhofer, Austrian Science Fund  Music: Urbana-Metronica (wooh-yeah mix) (ft. Morusque, Jeris, CSoul, Alex Beroza) by spinningmerkaba http://ccmixter.org/people/jlbrock44; SouljaUnit Remix of DeadRobot Music’s Surfy via https://freesound.org/people/SouljaUnit/sounds/640175/ Photo: Martin Jordan Thanks to our partners and co-organisers that helped realising the REvaluation Conference 2024: Ministry of Innovation, Mobility and Infrastructure, Ministry of Women, Science and Research, Ministry of Economy, Energy and Tourism, Vienna Science and Technology Fund, COST Action PROFEEDBACK (CA ), Meeting Destination Vienna, FH Campus Vienna, Austrian Science Fund, Austrian Research Promotion Agency, ZSI - Centre for Social Innovation, Fraunhofer ISI, Institut Francilien Recherche, Innovation et Société, Joanneum Research, the whole fteval Community.  podcast(at)fteval.at - twitter.com/fteval - www.fteval.at

    42 min

About

REvaluation Podcast über die Zukunft der Forschungspolitik - und ihrer Evaluierung. Präsentiert von Tobias Dudenbostel und Isabella Wagner für die Österreichische Plattform für Forschungs- und Technologiepolitikevaluierung (fteval). REvaluation Podcast about the future of research policy - and its evaluation. Presented by Tobias Dudenbostel and Isabella Wagner for the Austrian Platform for Research and Technology Policy Evaluation (fteval).