TWG 4 – Professionalization of Evaluation
Progress is being made by IOCE towards implementation of the third leg of the Global Evaluation Agenda (Professionalization). It now hosts a discussion forum and a document repository. Both can be accessed from the IOCE front web page:: http://www.ioce.net. John Gargani, AEA President, has nominated two representatives to work on professionalization under the IOCE umbrella. RelaC and AES plan to participate. Several other individuals have also joined. A webinar meeting of the IOCE professionalization group that includes Riitta Oksanen, EES President, will take place in April-May 2016 to agree on next steps and responsibilities.
Professionalism evokes expertise, credibility and concern for human welfare. An occupational group cannot aspire to the professional label without public trust earned through ethical safeguards; proven capabilities; occupational self management and credentials. To promote evaluation professionalism and to reach for excellence in evaluation practice this TWG aims to identify and promote collective actions needed within Europe and beyond. The challenge involves accelerating the harmonization of ethical, quality and capability standards, increasing the autonomy and independence of evaluators and helping to ensure that evaluators have the qualifications needed to perform quality work.
The ‘long march’ towards evaluation professionalization in Europe has begun. There is still a long way to go and the EES thematic working group has contributed to progress along this road. TWG members recognize that evaluation has acquired distinctive characteristics as a discipline in its own right. It has developed a well defined body of knowledge, a set of specialized skills and a host of ethical guidelines. It displays trans-disciplinary features that allow it to support all the social sciences through a well stocked tool kit of proven methods and processes. Thankfully doctrinal conflicts that held back cohesive progress in the evaluation community have largely abated and evaluation associations are growing in number and influence.
In line with its mandate the Professionalization TWG has validated an evaluation capabilities framework. It lays out the knowledge, experience and dispositions required to do good evaluation work (EES Surveys). With respect to issues of self regulation and accreditation that are critical ingredients of professional autonomy and independence the Canadian Evaluation Society has implemented a designation process for evaluators (see Martha McGuire’s article in the Connections December 2012 issue.
An evaluator qualification initiative adapted to European circumstances has also been endorsed. Originally put forward by Pam Oliver, former Convenor of the New Zealand evaluation society (ANZEA) and EES member it was validated by the EES membership. A questionnaire addressed to TWG members elicited broad support for the initiative. The EES board reviewed this positive feedback on July 3 2013 and decided to secure the views of the full membership.
The web-based survey elicited strong support for the scheme and, as follow up, EES (together with UKES) co-sponsored a workshop that took place in London on April 8 2014. The workshop chaired by Ian Davies, former EES President, recommended piloting of the VEPR scheme in the European space and the EES board approved the recommendation on May 10, 2014. It also endorsed broad based dissemination of the complementary Evaluator Capabilities Framework that had previously been validated by two membership surveys.
A working group assembled by EES is in the final stages of designing detailed specifications for the Voluntary Evaluator Peer Review (VEPR) pilot. After validation at a workshop funded by EvalPartners in London (April 2014) the VEPR concept was presented to the 11th EES Biennial Conference held in Dublin in October 2014 . The well attended session concluded that it would be highly desirable to adapt VEPR designs to diverse regional, national and thematic contexts while at the same time encouraging coherence among evaluation societies and associations interested in adopting the VEPR approach (the "Dublin Consensus").
Based on the work of this TWG your Society, under the leadership of Riitta Oksanen, has been an active participant in the EvalPartners process that led to the adoption of a Global Evaluation Agenda. The Agenda recognizes the value of the capabilities approach adopted by EES. The approach has a deliberate learning orientation. It focuses on professional development focused on knowledge, practice and dispositions. It is grounded in reflective practice principles and it complies with draft Guiding Principles displayed in the Global Evaluation Agenda document. They stress the importance of (i) voluntariness; (ii) autonomy; (iii) legitimacy; (iv) pluralism; (v) transparency; (vi) equity; and (vii) quality assurance..The next stage in the professionalization journey involves engaging other societies in principled partnerships focused on the adoption of ethical guidelines, capabilities frameworks and voluntary evaluator peer review processes consistent with the Guiding Principles. Extracts from EvalAgenda 2020 relevant to evaluation professionalization can be found in the last document on the list below.
For more information please download the attachments below.
 Robert Picciotto, The logic of evaluation professionalism, Evaluation, 17(2) 165-180, April 2011
Files to Download
Master in Policy and Planning at Leeds University, post graduate studies of Social Anthropology and Sociology, Former oficial at World Bank and USAID between 2003 and 2011. More than 15 years of working as evaluator of projects and programs for UNDP, USAID, GIZ, OXFAM US and the Internacional Red Cross among others. Responsible for the desing of the National Public Policy Evaluation System for the Secretary of Planning in Guatemala and Director of INCIDENCIA since 2011. Coordinator of the Masters in Monitorig and Evaluation Program in Guatemala. Active member of the Latin American Network of Evaluation as well as the International Development Evaluacion Association
Director of the Master in Monitoring and Evaluation of Public Policies in Guatemala, Universidad de San Carlos, INCIDENCIA and Universidad de Salamanca of Spain. The program has started as diploma but will become a Master program in 2015. Several members of IDEAS and RELAC participate as lecturers because the program was designed in order to have experiences from experts from different universities and countries. My main motivation to assist to the congress consist in enhancing my knowledge as well as update tendencies on evaluation methods and debates which can be incorporated as part of our training program.
Master in Policy and Planning at UK. Studies in Sociology. Official at the World Bank and USAID. More than 15 years conducting evaluations. Designed the National Public Policy Evaluation System in Guatemala. Coordinator of the Masters in Evaluation in Guatemala. Member of the Latin American Network of Evaluation and IDEAS.
Christian Bugnion de Moreta is a trained economist and has been working in international development for the past twenty five years. His experience of over 80 evaluations includes expertise in Security Sector Reform (SSR) and Disarmament, Demobilisation and Reintegration (DDR) as well as conflict transformation/mediation in fragile states and evaluation of child-soldiers reintegration programmes for donors, NGOs and UN agencies. His subject matter evaluation knowledge extends to the Balkans, Africa, Latin America and Asia. He is a vetted M&E expert for the UNDP Panama and Bratislava Regional Centers. In 2002 he created and manages Subur Consulting S.L., a private consulting firm based in Barcelona, dedicated to evaluation and training.
PhD in Political Science, is Associate Professor in the Department of Political Science and Public Administration at the Complutense University, Madrid (UCM), Spain. She has worked as an independent consultant for Spanish public administrations and agencies, the European Commission, and NGOs in the field of programme evaluation, and in the areas of community development, health promotion, drug dependency prevention, and gender policies. She is also the director of the Master on Evaluation of Programmes and Public Policies (UCM), and member of the Board of Director Committee of the National Agency for the Evaluation of Public Policies and the Quality of Public Services. She leads the European QUING project team at the UCM (QUING: Quality in Gender+ Equality Policies -2006-2011- www.quing.eu; a research project supported by the VI European Framework Programme). She has a number of publications on analysis and evaluation of public policies and gender equality policies.
Ian Craig Davies (FRANCE) has been a practising evaluator for over 25 years with professional experience in Europe, the Americas and Africa, a member and board member of professional evaluation organisations, i.e. EES, IDEAS, UKES, AEA, CES, and a regular presenter at annual conferences and professional events. Ian joined EES in 1996 and presented at the second annual conference in Stockholm in March 1997. He is an independent consultant in public management, performance measurement, accountability reporting and evaluation.
1983: DEGREE / NURSING at Saint Joseph University, Center of Nursing Sciences. Beirut, Lebanon.
1986: SPECIALIZATION in Anesthesia and Resuscitation. École d'Infirmiers Anesthésistes / Centre Hospitalier de Clermont - Ferrand, France.
2000: MASTER’S / SOCIOLOGY Faculty of Philosophy and Human Sciences (FAFICH), Federal University of Minas Gerais (UFMG), Belo Horizonte, Brazil. TITLE: The professional project of the nurses: Study in Intensive Care Units in Belo Horizonte. DISSERTATION SUPERVISOR: Dr. Maria Ligia de Oliveira Barbosa.
2009: Ph.D. / SOCIAL SCIENCES Research and Graduate Center on the Americas (CEPPAC), Institute of Social Sciences (ICS), University of Brasília, UnB, Brazil. 2009.TITLE: Social integration in Mercosur: Health education policies in Brazil and Argentina (1991-2006). DIRECTOR OF THESIS: Dr. Fernanda A. da Fonseca Sobral – Senior Researcher at the University of Brasília (UnB), Brazil.
2014: MASTER ON EVALUATION OF PROGRAMMES AND PUBLIC POLICIES. Higher School of Management Studies (CSEG), Complutense University of Madrid (UCM), Spain, 2013-2014. DIRECTOR OF FINAL WORK: Dr. Juan Andrés Ligero Lasa.
Weronika Felcis is the current President of the Polish Evaluation Society (2014-2016). She is a PhD candidate and a member of the Centre of Evaluation and Public Policy Analysis (CEAPP) at the Jagiellonian University in Krakow and also works as a freelance researcher/expert. Observing closely the work on VEPR at UKES and EES, she is initiating a similar debate and efforts in Poland. Weronika aims to incorporate in her PhD thesis some of the issues regarding introduction of VEPR system in Central and Eastern Europe.
Martha McGuire, M.S.W., C.E. has been involved in the evaluation field for almost 25 years. She is the founder of Cathexis Consulting, a firm specializing in program evaluation and applied research. She is a Credentialed Evaluator with the Canadian Evaluation Society (CES). She has served on the CES Council, having served two years as president. She is currently treasurer with the International Organization for Cooperation in Evaluation and on the management group of EvalPartners. She has supported the development of evaluation through her work in developing the core competencies with CES. She also teaches an evaluation course for non-profit managers at Ryerson University.
Martha has worked across a broad range of population including with people with disabilities, homeless people, children, youth and Aboriginal populations within Canada and recently began conducting evaluations within the international realm.
Robert (‘Bob’) Picciotto, (UK) Professor, Kings College (London) was Director General of the World Bank’s Independent Evaluation Group from 1992 to 2002. He previously served as Vice President, Corporate Planning and Budgeting and Director, Projects in three of the World Bank’s Regions. He currently sits on the United Kingdom Evaluation Society Council and the European Evaluation Society’s board. He serves as senior evaluation adviser to the International Fund for Agricultural Development and the Global Environment Fund. He is also a member of the International Advisory Committee on Development Impact which reports to the Secretary of State for International Development of the United Kingdom.
Former founder and director of ITAD limited. Evaluator with 30 years experience in international development working primarily on agriculture and natural resources and the evaluation of international aid institutions.
Romeo Santos is currently president of the Pilipinas Development Evaluation Association [PHILDEV], a national organization of evaluators in the Philippines. He also serves as Executive Director of the WorkLand M&E Institute, a non-profit, government-registered M&E think tank that provides training on research, organizational development, and M&E to various sectors. He completed IPDET’s development evaluation course in 2008 at Carlton University and was a Fulbright Visiting Professor and scholar in the USA in 2009. Mr. Santos finished Ph D Architectural Engineering (major in Project Management & Economics, 1995) in Japan. In 2012, he was elected Interim Vice President of the Asia Pacific Evaluation Association [APEA], a regional evaluation association headquartered in Kuala Lumpur, Malaysia.
I am an expert in the field of evaluation of, and technical assistance in, EU funded programmes, with a specific focus on agriculture and rural development. I have about 15 years of experience in this area, having worked as a consultant for specialist firms, including Ernt & Young, Ecoter. and Agriconsulting.
After a BS degree in Statistics and Economics and a Master in Development economics I have progressively focused on, and honed my skills in, the design and assessment of both quantitative and qualitative evaluation techniques, as well as maximising the effectiveness of presenting their results.
In particular, since 2002 I am project coordinator for Agriconsulting, one of the leading consultancies in this area. In this role I am, among other things, project manager and Leader Axis team coordinator in several evaluation activities, and lead country visit coordinator. Among my daily tasks are organising the evaluation activities, coordinating evaluation experts teams for the different Axes of RDPs, dealing with the clients, fostering continuous feedbacks with relevant stakeholders, and supervising and presenting reports to both specialist and non-specialist different audiences.
Since 2011 involved in the activities on evaluators' professionalization of Italian Evalutation Association.
Kate Winter, PhD is an independent consultant on research design and evaluation. She is Associate Faculty in City University of Seattle's School of Applied Leadership, where she teaches research design in the doctoral program. She also teaches research methods courses in Creighton University's Interdisciplinary Doctorate of Education Program in Leadership.
She formerly served as Senior Research Scientist for EPI International, an international organization dedicated to the study of educational opportunity, where she provided leadership on several project evaluations and policy studies. Prior to EPI, she was associate project director for the Alfred P. Sloan Foundation Projects on Faculty Career Flexibility at the American Council on Education.
She is the recipient of the 2008 Society of Women Engineers Work Life Balance Award, which "celebrates an individual who has worked to create programs that help women engineers and other employees balance the commitments of career, life and family." Her research focuses broadly on leadership and policy in education and her dissertation focused on how work-life concerns influence the career aspirations of future faculty. She received her Ph.D. and M.Ed. in Educational Leadership and Policy Studies from the University of Washington and her B.A. in English from the State University of New York at Geneseo.
Dr. Winter has presented research findings and policy recommendations nationally and is published in the Journal of Diversity in Higher Education, the Journal of the Professoriate, the Journal about Women in Higher Education, the Journal of Women and Minorities in Science and Engineering, the Department Chair, Academe, and Change, and has authored or co-authored book chapters on teaching and learning and on efforts to support faculty. She has presented several times at NSF ADVANCE meetings and was recently an invited participant in the Structural Change and Gender in Research working group meetings of genderSTE, a networking group of the European Cooperation in Science and Technology (COST), held in Crete, Greece.
Dr. Winter is a member of the American Evaluation Association (AEA), the European Evaluation Society (EES), genderSTE (the targeted networking group of the European Cooperation in Science and Technology), the American Educational Research Association (AERA), and the Association for the Study of Higher Education (ASHE).