Syllabus
Master of Social Work
SSW 514 – Program Evaluation – Spring 2016
Credits - 3
Description
This course introduces students to theoretical and practical aspects of program evaluation. Students learn about the application of research skills to all phases of developing a program or service innovation, from needs assessment to analysis of findings. Students learn to appreciate how these skills can be used as practical tools for social change and promoting the strengths of diversity in professional social work settings. The content of this course heavily integrates other elements of the MSW curriculum and a liberal arts background into the program evaluation task. For example:
- Defining a social problem requires attention to the personal development of individuals in a population at risk and the social policy processes which interact with the
- Social problems are contextualized in a professional social work Therefore, students are asked to integrate an understanding of a social problem, research activities, and professional practice.
- Students learn how research skills in the evaluation process can be effective tools for social change and organizational
- Program evaluation practice requires critical thinking skills rooted in analysis of social issues and inductive and deductive Skills such as survey construction and budgeting for evaluation projects are introduced. The latter portion of the class is structured around in-‐class presentation of student evaluation proposals with feedback provided by other students in the class. The course is designed as a resource for both direct practice and macro students, because effective program evaluation frequently requires collaboration between direct and macro levels of practice.
Materials
Required Readings:
Patton, M.Q. (2008). Utilization-focused evaluation, 4th ed. Thousand Oaks: Sage Publications.
Additional Resources:
Additional readings include assigned journal articles that are accessible online or at the University of New England library. These will be presented in the Learning Modules.
Learning Objectives and Outcomes
School of Social Work Program Outcomes:
Graduates of the UNE SSWO will demonstrate knowledge, skills, and leadership in the following:
- Practice social inclusion to enable people, populations, and communities to fully participate in society, enhance human bonds in the context of cultural diversity and ensure improved quality of life and equitable resource distribution. EPAS Competencies 2 & 3
- Engage in culturally-informed relationship building, being respectful of the complexity and diversity of contexts and circumstances. EPAS Competencies 3 & 6
- Utilize theories of human behavior, social systems and social inclusion when offering interventions with people and their environments. EPAS Competency 8
- Promote ethical reflection, critical consciousness and shared decision-making based in social work values and with consideration of the broader contexts of the world in which we live. EPAS Competency 1
- Balance the roles of helpers, activists, and advocates through collaboration with communities to build healthy and sustainable resources. EPAS Competencies 2, 5, & 6
- Engage as critical consumers and producers of research as it relates to assessment, intervention and evaluation of clinical and community practices. EPAS Competencies 4, 7, 8 & 9
- Practice person-centered and collaborative community partnerships across diverse settings. EPAS Competency 6
Course Objectives
Upon successful completion of this course, students will:
- Be competent consumers and producers of knowledge that informs their practice.
- Integrate the process of research, problem solving, and
- Appreciate human diversity issues in evaluation research, including the importance of protecting human
- Appreciate the ethical and socio-‐political factors which influence the process of evaluation
- Be aware of the impact of evaluation outcomes on the future of social work services.
- Understand the process of doing a needs assessment including key informant surveys, focus groups and use of available public
- Distinguish between program goals and objectives, develop goals, and formulate measurable
- Propose and complete a program evaluation, which integrates and utilizes foundation research knowledge and skills developed in other classes and
Student Educational Outcomes
The course asks students to practice the conceptualization skills that inform program planning and evaluation. Development of these skills enhances all areas of social work practice. Educational objectives for this course are:
- Students will apply their knowledge of social work ethics and research ethics while conducting all work for this
- Students will use an array of intellectual and professional resources for information, including textbooks, journals, web resources, clients and professional
- Students will write evaluation reports that are empirically grounded in data and practice
- Students will use their course skills and knowledge to develop and conduct an evaluation project and write an evaluation report that may impact a professional social work
- Students will engage in collegial critique of the work of their
Assignments
Your course grade will be determined by the following:
Class Participation
(Mandatory for a passing grade).
- Participation: You are expected to participate in classroom discussions and demonstrate an understanding of course material and integration of course material into You are expected to provide a total of 3 posts per week: one initial post and two substantial responses (i.e. 4-5 sentences in length) to peers, following APA 6th edition formatting as appropriate.
- Critique of PowerPoint presentations of Final Paper. You will actively participate in the review of peer presentations of their final paper. Microsoft PowerPoint must be used for this task.
Annotated Bibliographies – Due Day 7 of Weeks 3, 5, and 8 (300 points total)
This course is premised on the notion that social work practice should be based as much as possible on Evidence can take many forms, but one important source of evidence is published literature (e.g., journals, books, government and research organization reports). You will be asked to complete THREE Annotated Bibliographies over the course of the semester. Each review will critique one journal article. Each paper is one hundred (100) points. In order to write an evaluation review paper you will need to select three research journal articles that are in the published within the last 5-10 years:
- One study must be relevant to quantitative methodology (i.e. outcome based evaluation)
- One study must be relevant to qualitative methodology (i.e. needs assessment)
- One study must be relevant to mixed methodology (i.e. process evaluation)
An Annotated Bibliography is a short (i.e., 2 single spaced pages) paper in which you provide the following information. Refer to the document “Writing an Annotated Bibliography” (found via the Annotated Bib Instructions and Materials link int he sidebar) for details. Fundamental rules are:
- Full bibliographic information following APA 6th edition formatting
- Summary of research methods and findings
- Critical Assessment of the Article
- Statement of the Value of the Article (i.e. How it applies to the social work profession)
Final Paper Introduction – Due Day 7 of Week 2 (100 points)
The Introduction can provide the reader context and direction of the topic to be discussed in detail. The objectives of the writer are to be concise and to set the tone of the proposal to the reader that conveys evidence-based practices and general knowledge of the topic areas being presented in the proposal. As per the Final Paper Instructions, the Introduction is approximately 1 double-spaced page in length following APA 6th edition formatting. The Introduction also has the potential to be incorporated into the Background of the Program and Statement of the Evaluation Goals section (1-2 pages in length).
When writing the Introduction, consider the following items to be discussed:
- The agency or program of focus
- The mission/vision of the agency or program
- The client population served
- The services/entitlements/programs provided
- What is lacking in the agency or program that could benefit from an evaluation?
Final Paper PowerPoint Presentation – Due Day 5 of Week 7 (100 points)
You will create a PowerPoint presentation of your final paper. Refer to the PowerPoint Presentation Rubric for the necessary items to include. Students have creative freedom (i.e., font, color scheme, number of slides, etc) on this task. It is recommended that if images are included, they be Clip Art or public domain images in an effort to respect copyright laws. Post to the Discussion board AND submit to Dropbox.
Final Paper – Due Week Day 7 of Week 7 (300 points)
For this paper you will work individually to conceptualize a program evaluation for a human service organization. The final paper should be no more than 12 pages (of content) in length following APA 6th edition formatting. You should have 5-8 scholarly references. Your paper should be submitted via Dropbox by Day 7 of Week 7. For the final paper, you are to use headings and sub-headings covering all the sections below:
- Introduction (1 page) – Submitted in Week 2 – revisions should be made based on comments and track changes provided by instructor.
- Background of the Program and Statement of the Evaluation Goals (1-2 pages)
- Describe the program, including its goals and objectives
- List the goals of your proposed evaluation
- Brief Literature Review (2-3 pages): What is known already about evaluating this type of program? Have program evaluations been conducted? What was found? What is lacking?
- Logic Model (included as an Appendix): Provide a visual model on the selected agency and key components to consider for the evaluation proposal
- Proposed Methodology (2-3 pages): How would you evaluate the program?
- Evaluation design (i.e. design is clearly stated and reasoning is provided)
- Sample (i.e. selected sampling frame and reasoning provided)
- Data collection techniques (i.e. how will data be collected?)
- Data collection instruments (i.e. what will be used to gather data?)
- Data Analysis (i.e. what is the overall process for qual or quan data?)
- Outcome measures (i.e. what are the objectives of this proposal?)
- Limitations of the study (i.e what are the proposed limitations? This will be discussed in detail in VII)
- Proposed Results (2-3 pages):
- What descriptive statistics would you use to describe the proposed sample?
- What statistical tests/analysis would you conduct?
- If implemented what could these findings mean? What could account for these findings other than the intervention?
- How could the limitations of the study (the design, instrument, sample, reliability, validity) influence findings?
- Conclusions and Recommendations (2-3 pages)
- Recommendations regarding the program: Based on your proposal, what does the program need to do?
- Recommendations for future evaluation research: What recommendations would you make to other researchers interested in evaluating this type of program? Be specific.
NOTE for sections V and VI: The objective is for students to practice forward critical thinking. Based on the discussions of the proposed analysis and proposed results – what can the student infer will be the next steps of the process? Based on the identified area of need in the Introduction, can the student assess if the program goals are appropriate? These are just some (of many) items to consider.
Grading Policy
The School of Social Work uses the following grading system for all courses with the exception of field education courses. Students are expected to maintain a “B” (3.0) average over the course of their study. Students with less than a GPA of 3.0 will be placed on academic probation. Students must have an overall GPA of 3.0 in order to receive their Master’s Degree.
Your grade in this course will be determined by the following criteria:
Grade Breakdown
Assignment | Point Value | % of Grade |
---|
Annotated Bibliographies | 100 points x 3 papers | 30% |
Participation | 200 points (25 pts x 8 weeks) | 20% |
Evaluation Proposal | 100 points | 10% |
Evaluation Report | 300 points | 30% |
Program Evaluation Presentation | 100 points | 10% |
Total | 1000 points | 100% |
Grade Scale
Grade |
Points Grade |
Point Average (GPA) |
A |
94 – 100% |
4.00 |
A- |
90 – 93% |
3.75 |
B+ |
87 – 89% |
3.50 |
B |
84 – 86% |
3.00 |
B- |
80 – 83% |
2.75 |
C+ |
77 – 79% |
2.50 |
C |
74 – 76% |
2.00 |
C- |
70 – 73% |
1.75 |
D |
64 – 69% |
1.00 |
F |
00 – 63% |
0.00 |
Schedule
Program Evaluation | Course Dates: 10/28/2015 – 12/20/2015
All assignments are to be submitted by 11:59 p.m. E.S.T. on the dates listed below. Unless otherwise specified below, all module discussions and assignments are due the last day of the module or unit week. “Getting Started” to be completed prior to starting Module 1.
MODULE 1
Program Evaluation Overview
Dates: 03/02/16 – 03/15/16
Readings:
- Textbook: Patton, Chapters 1 – 3
Multimedia:
- Watch: Utilization Focused Assessment: An Introduction
- Found in Eval Review Papers/Materials in Nav Bar:
- Watch: Donna Mertens on Research Methods
- Watch: Mixed Methods Research
Week 1 | Dates: 03/02/16 – 03/08/16
- Discussions: Week 1 Discussion (Due: Sunday)
Week 2 | Dates: 03/09/16 – 03/15/16
- Discussions: Week 2 Discussion (Due: Sunday)
- Assignment: Evaluation Proposal (Final Paper Introduction) Due
MODULE 2
Program Evaluation: Theories, Models, and Approaches
Dates: 03/16/16 – 03/29/16
Readings:
- Textbook: Patton, Chapters 4 – 6
- Web Site: Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources
- Document: Where Program Evaluation is Helpful
Multimedia:
- Watch: Michael Quinn Patton: Reflective Practice & Learning in Community Led Development
- Watch: Michael Quinn Patton: Intentional and Short Term Goals
- Watch: Road to Results Through Effective Development of Evaluations
Week 3 | Dates: 03/16/16 – 03/22/16
- Discussions: Week 3 Discussion (Due: Sunday)
- Assignment: 1st Annotated Bibliography
Week 4 | Dates: 03/23/16 – 03/29/16
- Discussions: Week 4 Discussion (Due: Sunday)
MODULE 3
Logic Models and Mixed Methods
Readings:
- Textbook: Patton, Chapters 7 – 11
- Article: W.K. Kellogg Foundation’s Logic Model Development Guide
- Library: Mulroy, E. & Lauber, H. (2004). A user friendly approach to program evaluation and effective community interventions for families at risk of homelessness. Social Work 49(4), 573.
Multimedia:
- Watch: What, How, Why: Using Logic Models in PHLR Research
- Watch: Painless Performance Indicators: Using a Visual Approach
- Watch: Mixed Methods Research
- Watch: Donna Mertens on Research Methods
Week 5 | Dates: 03/30/16 – 04/05/16
- Discussions: Module 3 Discussion Questions (Due: Sunday);
- Assignment: 2nd Annotated Bibliography
MODULE 4
Seeking Funding for Evaluation
Readings:
- Textbook: Rubin and Babbie, Chapter 19 and 22. It is recommended that you flip through the Appendices.
- Tutorial: Foundation Center: Proposal Writing Short Course
- Web Site: Ehrmann, How (Not) to Evaluate a Grant-Funded Project
- Video: Cost Analysis of Program Evaluation
- Web Site: Grants Northwest: Matching Evaluation to Your Project and Organization
Week 6 | Dates: 04/06/16 – 04/12/16
- Discussions: Module 4 Discussion Questions (Due: Sunday)
MODULE 5
Implications for Practice and Dissemination
Dates: 04/13/16 – 04/24/16
Readings:
- Article: Disseminating Research Results: The Challenge of Bridging the Gap Between Health Research and Health Action
Multimedia:
- PowerPoint: Making Sure Research is Used: Community-Generated Recommendations for Disseminating Research Via Non-Traditional Venues
- Tutorial: Creating Your First Presentation
Week 7 | Dates: 04/13/16 – 04/19/16
- Discussions: Final Project Presentation Discussion (Post Presentations by Sunday to give classmates two days for responding)
- Assignments:
- Final Paper (Due: Tuesday)
- Final Paper Presentation (Due: Tuesday)
Week 8 | Dates: 04/20/16 – 04/24/16
- Discussions: Reflective Discussion (Due: Friday)
- Assignments: Final Annotated Bibliography
Student Resources
Online Student Support
Your Student Support Specialist is a resource for you. Please don't hesitate to contact them for assistance, including, but not limited to course planning, current problems or issues in a course, technology concerns, or personal emergencies.
Questions? Visit the Student Support Social Work page
UNE Libraries:
Information Technology Services (ITS)
ITS Contact: Toll Free Help Desk 24 hours/7 days per week at 1-877-518-4673
Policies
Late Policy
You will receive a 10% deduction on work submitted late. The 10% deductions reflect the total point value of each assignment unless the instructor receives university documentation granting extensions. Documentation received by University of New England’s Office of Disabilities granting accommodations should be forwarded to the course instructor immediately. Accommodations are not retroactive and are made available upon receipt of documentation.
Weekly Participation
Each week you will be expected to participate in discussion questions and small group discussions about specific questions, cases, or abstract reviews that are presented in the Learning Modules. You are expected to review these questions and engage in a dialogue with classmates regarding the issue. That is, students will post responses to the questions posed and respond to other students’ comments and questions on a weekly basis. I will be looking at both the quantity and quality of the postings. These postings must add substantively to the discussion by building upon classmates’ ideas or posing critical questions to further the discussion. For example, a posting of “I agree with what people are saying” is not sufficient.
Attendance and participation are not the same. Participation is the one area where you cannot make up the earned points, and you participate in the discussion questions to earn points.
I will be monitoring the participation on a weekly basis, and welcome people to check in with me if they have questions regarding their participation. Keep in mind that weekly postings make up 20% of the final grade. All postings must be respectful. If at any time you are concerned with a posting, please notify me immediately.
All posts to the discussion boards should be completed by 11:59 p.m. E.S.T. of last day of the module as indicated in the “Course Schedule.” Any discussions that extend beyond that date and time will not be considered a part of the grade and/or assignment, but rather optional reading. Posts that no longer relate to the identified topic should be moved to one of the on-‐going discussion boards such as “Ask Your Instructor,” “Hallway Discussions,” or “Resources.”
Written Work
All written work should be typed and double-spaced and submitted in Microsoft Word (doc or docx) format. Work must include in-text citations and full reference information following APA 6th edition formatting. Students who are not clear about proper format are encouraged to consult the U.N.E. Learning Assistance Center or various library, online, and other resources to get this information (click on “University Resources” in the navigation menu to access these resources). Material obtained from the Internet should be referenced and limited to recognized scholarly, academic, or client/consumer-oriented sources (such as online journals and information from advocacy oriented organizations). Gender-neutral language should be used throughout all written assignments.
Extensions and Incompletes
Under normal circumstances, there will be no extensions for papers or other assignments, or incompletes for the course. If you have concerns about an assignment, please discuss with the instructor well in advance of the due date. If you have reason to believe that you will be unable to complete the work, contact the Course Instructor immediately. Refer to Late Policy information.
Essential Academic and Technical Standards
Please review the essential academic and technical standards of the University of New England School Social Work (SSW): https://online.une.edu/social-work/academic-and-technical-standards-une-online-ssw/
Technology Requirements
Please review the technical requirements for UNE Online Graduate Programs: Technical Requirements
Confidentiality Statement
Student and faculty participation in this course will be governed by standards in the NASW Code of Ethics relating to confidentiality in sharing information from their placement sites and practice experiences. Students should be aware that personal information they choose to share in class, class assignments or conversations with faculty does not have the status of privileged information.