Syllabus
Master of Public Health
GPH 753 – Advanced Program Evaluation – Summer 2020
Credits - 3
Description
Students will collaborate with a public health site that they have identified to evaluate an aspect of a program as partial fulfillment fo the requirements for their MPH degree. All MPH student must create a high quality written product, synthesizing public health competencies. Based on their selected topic, students will analyze data related to a real-world problem and present their findings in a high quality written document. Readings and lectures will provide further education and guidance, and all assignments are designed to guide students through the analysis and writing process. At the end of the course, students will have a full draft of their program evaluation and be ready to enroll in the 8-week, 1-credit GPH 744 Integrative Learning Experience.
Materials
Learning Objectives and Outcomes
Public Health Competencies
PC 1. Synthesize and incorporate scientific evidence into professional writing
PC 2. Search databases and critically analyze peer-reviewed literature
FC 4. Interpret results of data analysis for public health research, policy or practice
FC 11. Select methods to evaluate public health programs
FC 13. Propose strategies to identify stakeholders and build coalitions and partnerships for influencing public health outcomes
Assignments
Class Discussions (8) – Weeks 1, 2, 3, 5, 8, 10, 14, 16
Identifying Key Stakeholders
Utilizing a worksheet from the self-study guide as a template, identify stakeholders in the three broad categories: those affected, those involved in operations, and those who will use the evaluation results. Then identify key stakeholders needed to improve credibility, implementation, advocacy, or funding/authorization decisions.
Literature Review
Conduct a literature review focusing on sources that are related to the health program you are addressing and evaluations that are relevant to your topic.
Summary of Stakeholder Interviews
Complete stakeholder meetings, and summarize what you learned using the worksheet from the CDC Program Evaluation Guide. Summarize in 1 – 2 paragraphs your plans to continue engaging these and other stakeholders throughout the evaluation process.
Logic Model
Provide a logic model for your program that visually depicts the inputs, activities, outputs, outcomes, and impact. Use arrows to demonstrate the relationships between specific elements.
Evaluation Questions
Submit evaluation questions, including one or two overarching questions and several subquestions. Evaluation questions should be clear and precise so that they help focus your methods and analysis. They should also be realistic (answerable), relevant to key stakeholders, and open-ended (not yes/no questions).
Methods and Data Collection
Describe your proposed methods. Be sure to include the following information: your research design, how the chosen design will answer the evaluation questions, a description of any existing sources of data that you plan to use, a description of what additional data will need to be collected, a definition of the limitations of the selected methods.
Data Instruments or Samples
Submit the instruments you are using to collect data and/or a sample of the existing data that you have collected
Data Collection Logistics
Complete the Data Collection Logistics worksheet.
IRB Application
Submit an application for IRB exemption.
Final Report
Submit a clean draft of your final report, including an abstract, executive summary, background, methods, results, and discussion.
Your report should be 4000 words plus appendices.
Grading Policy
Your grade in this course will be determined by the following criteria:
Grade Breakdown
| Class Discussions (8 X 3 Points each) | 24 |
| Week 1: Identifying Key Stakeholders | 5 |
| Week 2: Literature Review | 10 |
| Week 3: Summary of Stakeholder Interviews | 5 |
| Week 3: Logic Model | 5 |
| Week 4: Evaluation Questions | 4 |
| Week 5: Methods and Data Collection | 10 |
| Week 6: Data Instruments or Samples | 10 |
| Week 7: IRB Application | 6 |
| Week 15: Final Report | 15 |
| Total | 100 |
Grade Scale
| Grade | Points Grade | Point Average (GPA) |
| A | 94 – 100% | 4.00 |
| A- | 90 – 93% | 3.75 |
| B+ | 87 – 89% | 3.50 |
| B | 84 – 86% | 3.00 |
| B- | 80 – 83% | 2.75 |
| C+ | 77 – 79% | 2.50 |
| C | 74 – 76% | 2.00 |
| C- | 70 – 73% | 1.75 |
| D | 64 – 69% | 1.00 |
| F | 00 – 63% | 0.00 |
Schedule
Course Schedule:
Each week opens on Wednesday at 12:01 AM Eastern Time. Each week closes on Wednesday at 11:59 pm ET, with the exception of Week 16, which ends on Sunday.
- Week 1: Apr 29 – May 6
- Week 2: May 6 – May 13
- Week 3: May 13 – May 20
- Week 4: May 20 – May 27
- Week 5: May 27 – Jun 3
- Week 6: Jun 3 – Jun 10
- Week 7: Jun 10 – Jun 17
- Week 8: Jun 17 – Jun 24
- Week 9: Jun 24 – Jul 1
- Week 10: Jul 1 – Jul 8
- Week 11: Jul 8 – Jul 15
- Week 12: Jul 15 – Jul 22
- Week 13: Jul 22 – Jul 29
- Week 14: Jul 29 – Aug 5
- Week 15: Aug 5 – Aug 12
- Week 16: Aug 12 – Aug 16
With the exception of Week 1, which opens on a Wednesday, each week opens on Monday at 12:01 AM Eastern Time. Each week closes on Sunday at 11:59 pm ET.
The assignment/discussion descriptions mentioned below are summaries. Please make sure to review the full assignment prompts in Blackboard. There may be additional readings/videos that are not mentioned in this weekly summary, make sure to carefully review the modules in Blackboard.
Week 1: Course Introduction and Stakeholder Engagement
Outcomes:
-
Describe the steps and standards of the CDC evaluation framework
-
Identify key stakeholders for the selected program evaluation
-
Create a plan for continuous stakeholder engagement
-
Review the IRB review process and what will be required before data collection can begin
Lecture:
- Engaging Stakeholders
Readings and Multimedia:
-
CDC Self-Study Guide Step 1
-
Rossi, P., & Wright, J. (1984). Evaluation Research: An Assessment. Annual Review of Sociology, 10, 331-352.
Discussion:
- Introductions and Stakeholders
Assignment:
- Identifying Key Stakeholders
Week 2: Literature Review
Outcomes:
-
Identify relevant literature that describes the public health problem
-
Identify relevant literature about similar program evaluations
-
Synthesize peer-reviewed literature to provide relevant context and background for readers of the final report
Lecture:
- Conducting a Comprehensive Literature Review
Readings and Multimedia:
- Overview of Cochrane Reviews
Discussion:
- Meeting with a Stakeholder
Assignment:
- Literature Review
Week 3: Logic Model
Outcomes:
-
Develop a logic model that clearly describes a program’s inputs, activities, outputs, and short and long-term outcomes.
-
Use arrows in a logic model to demonstrate the relationship between all the program elements and outcomes.
-
Explain the importance and limitation of process (implementation) evaluation.
Lecture:
- Logic Models
Readings and Multimedia:
Required:
-
CDC Program Evaluation Guide, Step 2
-
Issel and Wells, Chapter 9, Implementation Evaluation
Recommended:
-
Learning to Love Your Logic Model (a CDC webinar)
-
Kellogg Guide to Logic Models
Discussion:
- Questions
Assignments:
-
Summary of Stakeholder Interviews
-
Logic Model
Week 4: Evaluation Questions
Outcomes:
-
Write evaluation questions that focus the evaluation and inform decisions related to methods and analysis.
-
Use peer-reviewed literature and stakeholder input to inform decisions about the focus of the evaluation.
-
Explain the benefits and limitations of outcome (effect) evaluations.
Lecture:
- Evaluation Questions
Readings and Multimedia:
-
CDC Evaluation Guide, Step 3
-
Issel and Wells, Chapter 11, Planning the Intervention Effect Evaluations
Assignment:
- Evaluation Questions
Week 5: Methods and Data Sources
Outcomes:
-
Select appropriate methods to answer different types of evaluation questions
-
Identify data sources that may be useful to program evaluation
-
Determine what additional data is needed to supplement existing data sources
Lecture:
- Methods and Data Sources
Readings and Multimedia:
- CDC Evaluation Guide, Step 4
Discussion:
- Methods and Data Sources
Assignment:
- Methods and Data Collection
Week 6: Data Sources and/or Data Collection Tool Creation
Outcomes:
-
Create effective tools for collecting quantitative and/or qualitative data
-
Assess the quality of secondary data
-
Develop a plan for analysis of quantitative and/or qualitative data
-
Create a plan for data collection and analysis and display the plan using a Gantt chart
Lecture:
- Key Informant Interviews
Readings and Multimedia:
-
The Survey Checklist (Manifesto)
-
Conducting Focus Groups by Community Tool Box
Activity:
- Sign up for Midpoint Check-in
Assignment:
- Data Instruments or Samples
Week 7: Logistics, Protocols, and IRB
Outcomes:
-
Establish protocols for data collection and/or use of existing data
-
Interpret IRB rules and processes
-
Submit an application for IRB review
Lecture:
- Practical Considerations for Data Collection
Readings and Multimedia:
- University of New England Policies, Procedures and Guidance on Research with Human Subjects.
Discussion:
- IRB Process
Assignments:
-
Data Collection Logistics
-
IRB Application
Week 8: Data Collection – Keys to Success
Outcomes:
-
Examine and apply best practices for data collection, storage, and management
Lecture:
- Data Collection
Activities:
-
There are no assignments to submit this week, but you should continue collecting data.
-
Meet with your instructor via phone or web conference. Reflection questions will be provided in advance for this midpoint check-in.
Week 9: Data Collection – Staying on Track
Outcomes:
-
Examine and apply best practices for data collection, storage, and management
Activities:
- This is a workweek. Continue to work on your data collection.
Week 10: Data Collection – Troubleshooting Challenges
Outcomes:
-
Examine and apply best practices for data collection, storage, and management
-
Describe common obstacles and possible solutions for data collection.
Discussion:
- Data Collection Challenges
Activities:
- Continue to work on your data collection.
Week 11: Organizing and Cleaning Data
Outcomes:
-
Examine and apply best practices for data collection, storage, and management.
-
Clean raw data to prepare for analysis.
-
Maintain data according to appropriate security standards.
Lecture:
- Cleaning Data
Activities:
-
There are no assignments to submit this week, but you should continue collecting data.
Week 12: Qualitative Data Analysis
Outcomes:
-
Conduct basic analysis of qualitative data
- Respond to common criticisms of the use of qualitative data in program evaluation
Lecture:
- Qualitative Data Analysis
Readings and Multimedia:
- Issel and Wells, Chapter 15, Qualitative Data Analysis
Activities:
-
There are no assignments to submit this week, but you should continue working on your data analysis.
Week 13: Quantitative Data Analysis
Outcomes:
- Conduct basic quantitative analysis of evaluation data.
Activities:
-
There are no assignments to submit this week, but you should continue working on your data analysis.
Week 14: Data Visualization
Outcomes:
-
Understand the benefits of clear, engaging data visualization
-
Present evaluation results using a variety of data visualization techniques
Readings and Multimedia:
- Chart Chooser by Evalu-ate
- Do’s and Don’ts for Data Visualization by Evalu-ate
Discussion:
- Data Visualization
Activities:
- There are no assignments to submit this week, but you should continue working on your data analysis. Consider also whether visual presentation of your data might be an effective communication tool.
Week 15: Synthesizing Your Results
Outcomes:
-
Prepare a final draft of a report of a program evaluation
Assignment:
- Final Report
Week 16: Reflections for Future Revisions
Outcomes:
-
Reflect on the program evaluation process
Discussion:
-
Reflection
Student Resources
Online Student Support
Your Student Support Specialist is a resource for you. Please don't hesitate to contact them for assistance, including, but not limited to course planning, current problems or issues in a course, technology concerns, or personal emergencies.
Questions? Visit the Student Support Public Health page
UNE Libraries:
- Library Access for all students: Your library login ID and password are the same as the ones you use to log into Blackboard/Brightspace.
- Public Health Databases: https://library.une.edu/library-materials/find/research-by-subject/public-health/
- Library Tutorial: Find an article
- Library Questions: Ask a librarian or phone library staff at (207) 602-2361 or (207) 221-4330.
UNE Student Academic Success Center
UNE's Student Academic Success Center (SASC) offers a range of free online services to support your academic achievement. Writing support, ESOL support, study strategy and learning style consultations, as well as downloadable resources, are available to all matriculating students. The SASC also offers tutoring for GPH 712 Epidemiology, GPH 716 Biostatistics, GPH 717 Applied Epidemiology, GPH 718 Biostatistics II, and GPH 719 Research Methods. To make an appointment for any of these services, go to une.tutortrac.com. For more information and to view and download writing and studying resources, please visit:
Information Technology Services (ITS)
- ITS Contact: Toll Free Help Desk 24 hours/7 days per week at 1-877-518-4673
Accommodations
Any student who would like to request, or ask any questions regarding, academic adjustments or accommodations must contact the Student Access Center at (207) 221-4438 or pcstudentaccess@une.edu. Student Access Center staff will evaluate the student's documentation and determine eligibility of accommodation(s) through the Student Access Center registration procedure.
Policies
AMA Writing Style Statement
The American Medical Association Manual (AMA) of Style, 11th edition is the required writing format for this course. Additional support for academic writing and AMA format is provided throughout the coursework as well as at the UNE Portal for Online Students.
Online resources: AMA Style Guide
Turnitin Originality Check and Plagiarism Detection Tool
The College of Professional Studies uses Turnitin to help deter plagiarism and to foster the proper attribution of sources. Turnitin provides comparative reports for submitted assignments that reflect similarities in other written works. This can include, but is not limited to, previously submitted assignments, internet articles, research journals, and academic databases.
Make sure to cite your sources appropriately as well as use your own words in synthesizing information from published literature. Webinars and workshops, included early in your coursework, will help guide best practices in APA citation and academic writing.
You can learn more about Turnitin in the Turnitin Student quick start guide.
Technology Requirements
Please review the technical requirements for UNE Online Graduate Programs: Technical Requirements
Course Evaluation Policy
Course surveys are one of the most important tools that University of New England uses for evaluating the quality of your education, and for providing meaningful feedback to instructors on their teaching. In order to assure that the feedback is both comprehensive and precise, we need to receive it from each student for each course. Evaluation access is distributed via UNE email at the beginning of the last week of the course.
Late Policy
Students are responsible for submitting work by the date indicated in Brightspace.
Quizzes and Tests: Quizzes and tests must be completed by the due date. They will not be accepted after the due date.
Assignments: Unless otherwise specified, assignments will be accepted up to 3 days late; however, there is a 10% grade reduction (from the total points) for the late submission. After three days the assignment will not be accepted.
Discussion posts: If the initial post is submitted late, but still within the discussion board week, there will be a 10% grade reduction from the total discussion grade (e.g., a 3 point discussion will be reduced by 0.3 points). Any posts submitted after the end of the Discussion Board week will not be graded.
Please make every effort ahead of time to contact your instructor and your student support specialist if you are not able to meet an assignment deadline. Arrangements for extenuating circumstances may be considered by faculty.
Student Handbook Online - Policies and Procedures
The policies contained within this document apply to all students in the College of Graduate and Professional Studies. It is each student's responsibility to know the contents of this handbook.
UNE Course Withdrawal
Please contact your student support specialist if you are considering dropping or withdrawing from a course. The last day to drop for 100% tuition refund is the 2nd day of the course. Financial Aid charges may still apply. Students using Financial Aid should contact the Financial Aid Office prior to withdrawing from a course.
Academic Integrity
The University of New England values academic integrity in all aspects of the educational experience. Academic dishonesty in any form undermines this standard and devalues the original contributions of others. It is the responsibility of all members of the University community to actively uphold the integrity of the academy; failure to act, for any reason, is not acceptable. For information about plagiarism and academic misconduct, please visit https://www.une.edu/studentlife/plagiarism.
Academic dishonesty includes, but is not limited to the following:
- Cheating, copying, or the offering or receiving of unauthorized assistance or information.
- Fabrication or falsification of data, results, or sources for papers or reports.
- Action which destroys or alters the work of another student.
- Multiple submissions of the same paper or report for assignments in more than one course without permission of each instructor.
- Plagiarism, the appropriation of records, research, materials, ideas, or the language of other persons or writers and the submission of them as one's own.
Charges of academic dishonesty will be reviewed by the Program Director. Penalties for students found responsible for violations may depend upon the seriousness and circumstances of the violation, the degree of premeditation involved, and/or the student’s previous record of violations. Appeal of a decision may be made to the Dean whose decision will be final. Student appeals will take place through the grievance process outlined in the student handbook.
Attendance Policy
Online students are required to submit a graded assignment/discussion prior to Sunday evening at 11:59 pm ET of the first week of the term. If a student does not submit a posting to the graded assignment/discussion prior to Sunday evening at 11:59 pm ET, the student will be automatically dropped from the course for non-participation. Review the full attendance policy.
Late Policy
Assignments: Late assignments will be accepted up to 3 days late; however, there is a 10% grade reduction (from the total points) for the late submission. After three days the assignment will not be accepted.
Discussion posts: If the initial post is submitted late, but still within the discussion board week, there will be a 10% grade reduction from the total discussion grade (e.g., a 3 point discussion will be reduced by 0.3 points). Any posts submitted after the end of the Discussion Board week will not be graded.
Please make every effort ahead of time to contact your instructor and your student support specialist if you are not able to meet an assignment deadline. Arrangements for extenuating circumstances may be considered by faculty.
Student Handbook Online - Policies and Procedures
The policies contained within this document apply to all students in the College of Graduate and Professional Studies. It is each student's responsibility to know the contents of this handbook.
UNE Course Withdrawal
Please contact your student support specialist if you are considering dropping or withdrawing from a course. The last day to drop for 100% tuition refund is the 2nd day of the course. Financial Aid charges may still apply. Students using Financial Aid should contact the Financial Aid Office prior to withdrawing from a course.
Academic Integrity
The University of New England values academic integrity in all aspects of the educational experience. Academic dishonesty in any form undermines this standard and devalues the original contributions of others. It is the responsibility of all members of the University community to actively uphold the integrity of the academy; failure to act, for any reason, is not acceptable. For information about plagiarism and academic misconduct, please visit UNE Plagiarism Policies.
Academic dishonesty includes, but is not limited to the following:
- Cheating, copying, or the offering or receiving of unauthorized assistance or information.
- Fabrication or falsification of data, results, or sources for papers or reports.
- Action which destroys or alters the work of another student.
- Multiple submissions of the same paper or report for assignments in more than one course without permission of each instructor.
- Plagiarism, the appropriation of records, research, materials, ideas, or the language of other persons or writers and the submission of them as one's own.
Charges of academic dishonesty will be reviewed by the Program Director. Penalties for students found responsible for violations may depend upon the seriousness and circumstances of the violation, the degree of premeditation involved, and/or the student’s previous record of violations. Appeal of a decision may be made to the Dean whose decision will be final. Student appeals will take place through the grievance process outlined in the student handbook.