A black text on a white background Description automatically generated

PADM 5345. Evaluation of International Programs and Projects

Current as of December 3, 2023

home button

syllabus button

resources button

participants button

Syllabus

Master Schedule of Sessions and Assignments

The in-person sessions in Spring 2024 will meet on Mondays and Wednesdays in M Van Rensselaer Hall G155 from 11:40-12:55. 1

 

 

 

Course Approach

The course examines results-based management of the programs and projects of international organizations, public and not-for-profit, with an emphasis on the evaluation aspect.  To do this, it examines how results-based management concepts and evaluation theory can be applied at the international level, takes the student through the process of planning an evaluation, examines the main analytical techniques that are usually employed and, as a final step, undertakes critical examinations of evaluations of specific projects and programs made by the main organizations at the international level.

2015 was International Year of Evaluation!

 

There will be a simulation of the planning of an evaluation in results-based management and exercises related to each of the main analytical techniques. At the Maxwell School of Syracuse University, in Spring 2011, the evaluation simulation was of the Internet Governance Forum. In Fall 2010 it was the evaluation of the Department of Economic and Social Affairs of the United Nation. In 2009 it was a simulation of an evaluation of the Internet Governance Forum and of the current United Nations climate change institutions (the Global Environment Facility, the UN Framework Convention on Climate Change's Clean Development Mechanism, UNEP and the United Nations Convention to Combat Desertification.) For Fall 2011, it was the Gender Equity Programme of the Food and Agriculture Organization of the United Nations. For Spring 2012, it was the Intergovernmental Panel on Climate Change (IPCC). For Spring 2013 at Cornell, the subject was two real evaluations, one of the UN Convention to Combat Desertification and the other of the Research Cooperation Program of the Swedish International Development Cooperation Agency. The subject for Spring 2014 and also for Spring 2016 was the Intergovernmental Panel on Climate Change. In Spring 2015, it was UN Women. For 2024, it will be projects of the Green Climate Fund. In addition, as a final assignment, each student will design an evaluation of a program or project of her or his choice and express it in an inception report.  The interactive part of the course takes place through a weekly live discussion session which is also available over the Internet.  Each student is expected to participate in either the in-person or through the on-line facility. 

Course Organization

The interactive part of the course will be a live discussions on Monday and Wednesday at 11:40 in M Van Rensselaer Hall G155 and by Zoom. Students are expected to participate in this each week. It is expected that a recording from session will be posted after the session for those who could not attend or for review by those who could. The lecture, readings and an audio or video presentation will usually be put up on the site by Sunday of the week prior to the session.

Readings

The course has one required text RealWorld Evaluation, Third Edition, by Michael Bamberger and Linda Mabry. Mike Bamberger was the chief sociologist for the World Bank for many years. The book can also be a useful reference in the future.  Other required readings will come from evaluation materials of international public and non-governmental organizations, that are mostly available over the Internet.

Responses to Session Questions

For each of the weekly sessions, there are a series of questions to be addressed. Students are expected to provide short, one-sentence answers to each of the questions and send them to the graduate assistants for the course, led by emiriana stanishja who will assemble them on Sunday before class.

Simulation

simulation logo

The first half of the course will include the simulation of planning and implementing an evaluation of a recently approved project of the Green Climate Fund. It will lead to the production of an inception report by each team.

Participants will be part of an external evaluation firm hired to undertake the evaluation for the next meeting of the organization's board in April. There will be weekly discussions of progress with the Managing Director of the firm or his designated facilitator.

Office Hours

Prof. Mathiason's office hours for Spring 2024 will be on Mondays and Wednesdays from 9:15-11:00 in MvR 3M219 (even if coming during office hours it is better to schedule an appointment). The Professor can also hold office hours on request over Zoom..

Grading

The course will involve a final paper, consisting of an evaluation design of an international program or project reflected in an inception report. Details on the assignment are found on the final paper page. The paper will be a detailed evaluation design written as an Inception Report. Please note that a first draft must be submitted on time for comments. If that is not done, the grade is reduced.

Final paper

60%

Participation in the simulation

30%

Participation in answering session questions

10%

 

Master Schedule of Sessions and Assignments

Date

Section of course/Content of the session

January 22, 24

1. The craft of result-based management (RBM) and its use in international organizations

That session will review concepts basic to all results-based management and their evaluations of public institutions and discuss why this is critical for international programs and projects. The session will also review the policies and approaches to results-based management of the main groups of international organizations: the World Bank, UNDP and other UN Funds and programs, the Specialized Agencies and the United Nations Secretariat. It will look at the standards adopted by the United Nations Evaluation Group. The emphasis will be on how concepts are modified to suit the specific environment of the international organizations.  It will also (on 26 January) show the projects that will be examined in the simulation.

The RBM process I:  This section will follow the main steps in doing strategic planning in international organizations.  The class will simulate the planning of an evaluation in an international organization.

January 29, 31

2. Determining objectives, outcomes and the output to make them happen and the role of evaluation in the UN System

The session will explore how objectives and outcomes should be defined, as well as the output to induce them. These are also the critical elements for evaluation, since they define what has to be evaluated.  It will include a focus on the logical framework (log-frame) which is the main tool used for this stage of planning, as it is applied in international organizations. It will also discuss the role of evaluation in international organizations. Problems to be addressed by the organization due on 1 February.

February 5, 7

3. Determining performance indicators and means of obtaining information

The session will explore how information on results can be obtained for subsequent monitoring and evaluation and the tradeoffs involved.  It will include planning for information acquisition. First draft of project logframe due on 9 February

February 12, 14

4. Evaluation Quality Assessment

The session will explore how information on results can be obtained for subsequent monitoring and evaluation and the tradeoffs involved.  It will include planning for information acquisition. Final draft of project logframe due on 15 February

The RBM process II:  This section will follow the main methods of information collection for monitoring and evaluation

February 19, 21

5. Obtaining information from documents

The session will begin to explore the use of content analysis techniques for drawing information from published document.. It will also look at evaluation matrices. Evaluation questions due on 22 February

February 28

6. Obtaining information from surveys

The session will explore the use of sample surveys and questionnaires to obtain information on program and project impact.

Topic of final paper due on 5 March. Send it as a Word Document giving the title and a brief statement of why you are interested in the chosen program or project to Professor Mathiason. Please format the document title with your last name, first name and the name of the program or project.

March 4, 6

7. Obtaining information through case studies

The session will explore the use of case study methodology as a means of undertaking evaluation

 

March 11, 13

8. Analyzing information

The session on 14 March will consist of an overview of main simple data analysis techniques used in international evaluations: descriptive statistics, cross tabulation and analysis of variance and correlation and regression.  Examples will be drawn from actual evaluations.

The RBM process III: These sessions will look at evaluation models in international practice:  This section will include actual cases of evaluations conducted by the main types of international organizations.  In advance of each session, the participants will be expected to make a critique of the evaluation being examined.

March 18, 20

9. Evaluation of a United Nations Program

The session will explore an evaluation of the Global Programme on Money-laundering of the United Nations Office on Drugs and Crime as well as of the Division for Economic and Social Council Support and Coordination.

Outline of final paper due by 25 March [Please format the document title with your last name, first name and the name of the program or project.

March 25, 27

10. Evaluation of World Bank projects

The session will examine critically an evaluation produced by the Bank's Independent Evaluation Group (formerly the Operations Evaluation Department). Four groups will present their Inception reports

April 8, 10

11. Evaluation of a UNDP project

The session will examine critically an evaluation produced by UNDP of an ILO project. Because of the full eclipse of the sun on 8 April, there will be no session that day. 

April 15, 17 

12. Evaluation of an NGO project 

The session will examine critically an evaluation produced by World Vision. First Draft of final paper due on April 24 but not later! Please format the document title with your last name, first name and the name of the program or project.

April 22, 24

 

13. Presentation of results 

The session will include a discussion of how results should be presented and discuss common problems and drawing of general conclusions about evaluations. Three groups will present their inception reports.

April 29, May 1

14. Summing up: The Future of Evaluation in the International Public Sector.

May 6

15. Final session with some individual presentations.

May 14

Final paper due on May 14!

John R. Mathiason.
Copyright 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024 ,John R. Mathiason. All rights reserved.
Revised: 3 December 2023