Syllabus
Master of Public Health
GPH 753: Advanced Program Evaluation – Spring 2021
Credits - 3
Description
Students will collaborate with a public health site that they have identified to evaluate an aspect of a program as partial fulfillment for the requirements for their MPH degree. All MPH students must create a high quality written product, synthesizing public health competencies. Based on their selected topic, students will analyze data related to a real-world problem and present their findings in a high quality written document. Readings and lectures will provide further education and guidance, and all assignments are designed to guide students through the analysis and writing process. At the end of the course, students will have a final report of their program evaluation.
Materials
Learning Objectives and Outcomes
Public Health Competencies
PC 1. Synthesize and incorporate scientific evidence into professional writing
PC 2. Search databases and critically analyze peer-reviewed literature
FC 4. Interpret results of data analysis for public health research, policy or practice
FC 11. Select methods to evaluate public health programs
FC 13. Propose strategies to identify stakeholders and build coalitions and partnerships for influencing public health outcomes
Assignments
Class Discussions (5) – Weeks 1, 2, 4, 8, 9
Identifying Key Stakeholders
Utilizing a worksheet from the self-study guide as a template, identify stakeholders in the three broad categories: those affected, those involved in operations, and those who will use the evaluation results. Then identify key stakeholders needed to improve credibility, implementation, advocacy, or funding/authorization decisions.
Literature Review
Conduct a literature review focusing on sources that are related to the health program you are addressing and evaluations that are relevant to your topic.
Logic Model
Provide a logic model for your program that visually depicts the inputs, activities, outputs, outcomes, and impact. Use arrows to demonstrate the relationships between specific elements.
Summary of Stakeholder Interviews
Complete stakeholder meetings, and summarize what you learned using the worksheet from the CDC Program Evaluation Guide. Summarize in 1 – 2 paragraphs your plans to continue engaging these and other stakeholders throughout the evaluation process.
Evaluation Questions
Submit evaluation questions, including one or two overarching questions and several subquestions. Evaluation questions should be clear and precise so that they help focus your methods and analysis. They should also be realistic (answerable), relevant to key stakeholders, and open-ended (not yes/no questions).
Methods and Data Sources
Describe your proposed methods. Be sure to include the following information: your research design, how the chosen design will answer the evaluation questions, a description of any existing sources of data that you plan to use, a description of what additional data will need to be collected, a definition of the limitations of the selected methods.
Data Instruments or Samples
Submit the instruments you are using to collect data and/or a sample of the existing data that you have collected
Logistics and Protocol
Complete the Data Collection Logistics worksheet.
Draft Report
You will submit a draft of your report for feedback.
Final Report
Submit a clean draft of your final report, including an abstract, executive summary, background, methods, results, and discussion. Your report should be 4000 words plus appendices.
Grading Policy
Your grade in this course will be determined by the following criteria:
Grade Breakdown
| Class Discussions | 14 |
| 1:1 Meetings with your Instructor | 5 |
| Week 1: Identifying Key Stakeholders | 3 |
| Week 2: Literature Review | 4 |
| Week 3: Summary of Stakeholder Interviews | 4 |
| Week 3: Logic Model | 4 |
| Week 4: Evaluation Questions | 2 |
| Week 5: Methods and Data Collection | 6 |
| Week 6: Data Instruments or Samples | 4 |
| Week 7: Logistics and Protocol | 4 |
| Week 13: Draft ILE Report | 10 |
| Week 14: Abstract | 10 |
| Week 15: Final Report | 20 |
| Week 16: Poster | 10 |
| Total | 100 |
Grade Scale
| Grade | Points Grade | Point Average (GPA) |
| A | 94 – 100% | 4.00 |
| A- | 90 – 93% | 3.75 |
| B+ | 87 – 89% | 3.50 |
| B | 84 – 86% | 3.00 |
| B- | 80 – 83% | 2.75 |
| C+ | 77 – 79% | 2.50 |
| C | 74 – 76% | 2.00 |
| C- | 70 – 73% | 1.75 |
| D | 64 – 69% | 1.00 |
| F | 00 – 63% | 0.00 |
Schedule
Course Schedule:
Each week opens on Wednesday at 12:00 AM Eastern Time. Each week closes on Wednesday at 11:59 PM ET, with the exception of Week 16, which ends on Sunday.
Week 1: Jan 6 – Jan 13
Week 2: Jan 13 – Jan 20
Week 3: Jan 20 – Jan 27
Week 4: Jan 27 – Feb 3
Week 5: Feb 3 – Feb 10
Week 6: Feb 10 – Feb 17
Week 7: Feb 17 – Feb 24
Week 8: Feb 24 – Mar 3
Week 9: Mar 3 – Mar 10
Week 10: Mar 10 – Mar 17
Week 11: Mar 17 – Mar 24
Week 12: Mar 24 – Mar 31
Week 13: Mar 31 – Apr 7
Week 14: Apr 7 – Apr 14
Week 15: Apr 14 – Apr 21
Week 16: Apr 21 – Apr 25
The assignment/discussion descriptions mentioned below are summaries. Please make sure to review the full assignment prompts in Blackboard. There may be additional readings/videos that are not mentioned in this weekly summary, make sure to carefully review the modules in Blackboard.
Week 1: Course Introduction and Stakeholder Engagement
Outcomes
- Describe the steps and standards of the CDC evaluation framework
- Identify key stakeholders for the selected program evaluation
- Create a plan for continuous stakeholder engagement
Videos and Readings
- Lecture: Engaging Stakeholders
- CDC Self-Study Guide Step 1
- Rossi, P., & Wright, J. (1984). Evaluation Research: An Assessment. Annual Review of Sociology, 10, 331-352.
Discussion
- Introductions and Stakeholders
Assignments
- Identifying Key Stakeholders
- Schedule 1:1 Meetings with Your Instructor
Week 2: Literature Review
Outcomes
- Identify relevant literature that describes the public health problem
- Identify relevant literature about similar program evaluations
- Synthesize peer-reviewed literature to provide relevant context and background for readers of the final ILE report
Videos and Readings
- Lecture: Conducting a Comprehensive Literature Review
- Cochrane Library, About Cochrane Reviews
Discussion
- Meeting with a Stakeholder
Assignments
- Literature Review
- 1:1 Meeting with your Instructor
Week 3: Logic Model
Outcomes
- Develop a logic model that clearly describes a program’s inputs, activities, outputs, and short and long-term outcomes
- Use arrows in a logic model to demonstrate the relationship between all the program elements and outcomes
- Explain the importance and limitation of process (implementation) evaluation
Videos and Readings
- Lecture: Logic Models
- CDC Program Evaluation Guide, Step 2
- Issel and Wells, Chapter 9, Implementation Evaluation
Assignments
- Logic Model
Week 4: Evaluation Questions
Outcomes
- Write evaluation questions that focus on the evaluation and inform decisions related to methods and analysis
- Use peer-reviewed literature and stakeholder input to inform decisions about the focus of the evaluation
- Explain the benefits and limitations of outcome (effect) evaluations
Videos and Readings
- Lecture: Evaluation Questions
- CDC Evaluation Guide, Step 3
- Issel and Wells, Chapter 11, Planning the Intervention Effect Evaluations
Discussion
- Questions
Assignment
- Summary of Stakeholder Interviews
Week 5: Methods and Data Sources
Outcomes
- Select appropriate methods to answer different types of evaluation questions
- Identify data sources that may be useful to program evaluation
- Determine what additional data is needed to supplement existing data sources
Videos and Readings
- Lecture: Methods and Data Sources
- CDC Evaluation Guide, Step 4
Assignment
- Evaluation Questions, Methods, and Data Collection
Week 6: Data Sources and/or Data Collection Tool Creation
Outcomes
- Create effective tools for collecting quantitative and/or qualitative data
- Assess the quality of secondary data
- Develop a plan for analysis of quantitative and/or qualitative data
- Create a plan for data collection and analysis and display the plan using a Gantt chart
Videos and Readings
- Lecture: Key Informant Interviews
- Lecture: Practical Considerations for Data Collection
- Gehlbach H, Artino AR Jr. The Survey Checklist (Manifesto). Acad Med. 2018;93(3):360-366.
- Community Tool Box, Conducting Focus Groups
Assignments
- Data Instruments and/or Samples
- 1:1 Meeting with your Instructor
Week 7: Logistics and Protocols
Outcomes
- Establish protocols for data collection and/or use of existing data
Assignment
- Data Collection Logistics
Week 8: Data Collection – Keys to Success
Outcomes
- Examine and apply best practices for data collection, storage, and management
Videos and Readings
- Lecture: Data Collection
Discussion
- Case Studies
Week 9: Data Collection – Organizing and Cleaning Data
Outcomes
- Examine and apply best practices for data collection, storage, and management
- Describe common obstacles and possible solutions for data collection
- Clean raw data to prepare for analysis
- Maintain data according to appropriate security standards
Videos and Readings
- Lecture: Cleaning Data
Discussion
- Data Collection Challenges
Week 10: Qualitative Data Analysis
Outcomes
- Conduct basic analysis of qualitative data
- Identify the pros and cons of the use of qualitative data in program evaluation
Videos and Readings
- Lecture: Qualitative Data Analysis
- Issel and Wells, Chapter 15, Qualitative Data Analysis
Assignment
- 1:1 Meeting with your Instructor
Week 11: Quantitative Data Analysis
Outcomes
- Conduct basic quantitative analysis of evaluation data
Videos and Readings
- Issel and Wells, Chapter 14, Quantitative Data Analysis and Interpretation
Assignments
- There are no assignments to submit this week, but you should continue working on your data analysis.
Week 12: Report Writing
Outcomes
- Prepare a draft of a report of a program evaluation
Assignments
- There are no assignments to submit this week, but you should use this week to write.
Week 13: Submit Draft ILE Project Report
Outcomes
- Prepare a final draft of a report of a program evaluation
- Explain the benefits of clear, engaging data visualization
- Present evaluation results using a variety of data visualization techniques
Videos and Readings
- Evalu-ate, Evaluation Questions Chart Chooser
- Evalu-ate, Do’s and Don’ts: Basic Principles of Data Visualization
Assignment
- Draft ILE Project Report
Week 14: Abstract Writing
Outcomes
- Identify key components of an abstract
- Write an abstract using APHA guidelines
Videos and Readings
- Lecture: Abstract Writing
- APHA, How to Submit an Abstract
Assignments
- Abstract
- 1:1 Meeting with your Instructor
Week 15: Submit ILE Project Report
Outcomes
- Prepare a final draft of a report of a program evaluation
Assignment
- Final ILE Project Report
Week 16: Poster
Outcomes
- Compare the new APHA recommended poster format and the traditional APHA poster format
- Create a poster for your ILE report using the APHA recommended “better-poster” format
Videos and Readings
- Morrison M. How to create a better research poster in less time. 2019.
- QR Code Generator
Assignment
- Poster
Student Resources
Online Student Support
Your Student Support Specialist is a resource for you. Please don't hesitate to contact them for assistance, including, but not limited to course planning, current problems or issues in a course, technology concerns, or personal emergencies.
Questions? Visit the Student Support Public Health page
UNE Libraries:
- Library Access for all students: Your library login ID and password are the same as the ones you use to log into Brightspace.
- Public Health Databases: https://library.une.edu/library-materials/find/research-by-subject/public-health/
- Library Questions: Ask a librarian or phone library staff at (207) 602-2361 or (207) 221-4330.
UNE Student Academic Success Center
UNE's Student Academic Success Center (SASC) offers a range of free online services to support your academic achievement. Writing support, ESOL support, study strategy and learning style consultations, as well as downloadable resources, are available to all matriculating students. The SASC also offers tutoring for GPH 712 Epidemiology, GPH 716 Biostatistics, GPH 717 Applied Epidemiology, GPH 718 Biostatistics II, and GPH 719 Research Methods. To make an appointment for any of these services, go to une.tutortrac.com. For more information and to view and download writing and studying resources, please visit:
Information Technology Services (ITS)
- ITS Contact: Toll Free Help Desk 24 hours/7 days per week at 1-877-518-4673
Accommodations
Any student who would like to request, or ask any questions regarding, academic adjustments or accommodations must contact the Student Access Center at (207) 221-4438 or pcstudentaccess@une.edu. Student Access Center staff will evaluate the student's documentation and determine eligibility of accommodation(s) through the Student Access Center registration procedure.
Online Peer Support
Togetherall is a 24/7 communication and emotional support platform monitored by trained clinicians. It’s a safe place online to get things off your chest, have conversations, express yourself creatively, and learn how to manage your mental health. If sharing isn’t your thing, Togetherall has other tools and courses to help you look after yourself with plenty of resources to explore. Whether you’re struggling to cope, feeling low, or just need a place to talk, Togetherall can help you explore your feelings in a safe supportive environment. You can join Togetherall using your UNE email address.
Information Technology Services (ITS)
Students should notify their Student Support Specialist and instructor in the event of a problem relating to a course. This notification should occur promptly and proactively to support timely resolution.
ITS Contact: Toll-Free Help Desk 24 hours/7 days per week at 1-877-518-4673.
Career Ready Program
The College of Professional Studies supports its online students and alumni in their career journey!
The Career Ready Program provides tools and resources to help students explore and hone in on their career goals, search for jobs, create and improve professional documents, build professional network, learn interview skills, grow as a professional, and more. Come back often, at any time, as you move through your journey from career readiness as a student to career growth, satisfaction, and success as alumni.
Policies
AMA Writing Style Statement
The American Medical Association Manual (AMA) of Style, 11th edition is the required writing format for this course. Additional support for academic writing and AMA format is provided throughout the coursework as well as at the UNE Portal for Online Students.
Online resources: AMA Style Guide
Turnitin Originality Check and Plagiarism Detection Tool
The College of Professional Studies uses Turnitin to help deter plagiarism and to foster the proper attribution of sources. Turnitin provides comparative reports for submitted assignments that reflect similarities in other written works. This can include, but is not limited to, previously submitted assignments, internet articles, research journals, and academic databases.
Make sure to cite your sources appropriately as well as use your own words in synthesizing information from published literature. Webinars and workshops, included early in your coursework, will help guide best practices in APA citation and academic writing.
You can learn more about Turnitin in the guide on how to navigate your Similarity Report.
Technology Requirements
Please review the technical requirements for UNE Online Graduate Programs: Technical Requirements
Course Evaluation Policy
Course surveys are one of the most important tools that University of New England uses for evaluating the quality of your education, and for providing meaningful feedback to instructors on their teaching. In order to assure that the feedback is both comprehensive and precise, we need to receive it from each student for each course. Evaluation access is distributed via UNE email at the beginning of the last week of the course.
Late Policy
Students are responsible for submitting work by the date indicated in Brightspace.
Quizzes and Tests: Quizzes and tests must be completed by the due date. They will not be accepted after the due date.
Assignments: Unless otherwise specified, assignments will be accepted up to 3 days late; however, there is a 10% grade reduction (from the total points) for the late submission. After three days the assignment will not be accepted.
Discussion posts: If the initial post is submitted late, but still within the discussion board week, there will be a 10% grade reduction from the total discussion grade (e.g., a 3 point discussion will be reduced by 0.3 points). Any posts submitted after the end of the Discussion Board week will not be graded.
Please make every effort ahead of time to contact your instructor and your student support specialist if you are not able to meet an assignment deadline. Arrangements for extenuating circumstances may be considered by faculty.
Student Handbook Online - Policies and Procedures
The policies contained within this document apply to all students in the College of Graduate and Professional Studies. It is each student's responsibility to know the contents of this handbook.
UNE Course Withdrawal
Please contact your student support specialist if you are considering dropping or withdrawing from a course. The last day to drop for 100% tuition refund is the 2nd day of the course. Financial Aid charges may still apply. Students using Financial Aid should contact the Financial Aid Office prior to withdrawing from a course.
Academic Integrity
The University of New England values academic integrity in all aspects of the educational experience. Academic dishonesty in any form undermines this standard and devalues the original contributions of others. It is the responsibility of all members of the University community to actively uphold the integrity of the academy; failure to act, for any reason, is not acceptable. For information about plagiarism and academic misconduct, please visit https://www.une.edu/studentlife/plagiarism.
Academic dishonesty includes, but is not limited to the following:
- Cheating, copying, or the offering or receiving of unauthorized assistance or information.
- Fabrication or falsification of data, results, or sources for papers or reports.
- Action which destroys or alters the work of another student.
- Multiple submissions of the same paper or report for assignments in more than one course without permission of each instructor.
- Plagiarism, the appropriation of records, research, materials, ideas, or the language of other persons or writers and the submission of them as one's own.
Charges of academic dishonesty will be reviewed by the Program Director. Penalties for students found responsible for violations may depend upon the seriousness and circumstances of the violation, the degree of premeditation involved, and/or the student’s previous record of violations. Appeal of a decision may be made to the Dean whose decision will be final. Student appeals will take place through the grievance process outlined in the student handbook.
Attendance Policy
8 week: Students taking online graduate courses through the College of Professional Studies will be administratively dropped for non-participation if a graded assignment/discussion post is not submitted before Sunday at 11:59 pm ET of the first week of the term. Reinstatement is at the purview of the Dean's Office.
16 week: Students taking online graduate courses through the College of Professional Studies will be administratively dropped for non-participation if a graded assignment/discussion post is not submitted before Friday at 11:59 pm ET of the second week of the term. Reinstatement is at the purview of the Dean's Office.
Student Handbook Online - Policies and Procedures
The policies contained within this document apply to all students in the College of Professional Studies. It is each student's responsibility to know the contents of this handbook.
UNE Course Withdrawal
Please contact your student support specialist if you are considering dropping or withdrawing from a course. The last day to drop for 100% tuition refund is the 2nd day of the course. Financial Aid charges may still apply. Students using Financial Aid should contact the Financial Aid Office prior to withdrawing from a course.
Academic Integrity
The University of New England values academic integrity in all aspects of the educational experience. Academic dishonesty in any form undermines this standard and devalues the original contributions of others. It is the responsibility of all members of the University community to actively uphold the integrity of the academy; failure to act, for any reason, is not acceptable. For information about plagiarism and academic misconduct, please visit UNE Plagiarism Policies.
Academic dishonesty includes, but is not limited to the following:
- Cheating, copying, or the offering or receiving of unauthorized assistance or information.
- Fabrication or falsification of data, results, or sources for papers or reports.
- Action which destroys or alters the work of another student.
- Multiple submissions of the same paper or report for assignments in more than one course without permission of each instructor.
- Plagiarism, the appropriation of records, research, materials, ideas, or the language of other persons or writers and the submission of them as one's own.
Charges of academic dishonesty will be reviewed by the Program Director. Penalties for students found responsible for violations may depend upon the seriousness and circumstances of the violation, the degree of premeditation involved, and/or the student’s previous record of violations. Appeal of a decision may be made to the Dean whose decision will be final. Student appeals will take place through the grievance process outlined in the student handbook.