Print

Syllabus

Master of Public Health

GPH 753 – Advanced Program Evaluation – Fall 2020

Credits - 3

Description

Students will collaborate with a public health site that they have identified to evaluate an aspect of a program as partial fulfillment fo the requirements for their MPH degree. All MPH student must create a high quality written product, synthesizing public health competencies. Based on their selected topic, students will analyze data related to a real-world problem and present their findings in a high quality written document. Readings and lectures will provide further education and guidance, and all assignments are designed to guide students through the analysis and writing process. At the end of the course, students will have a full draft of their program evaluation and be ready to enroll in the 8-week, 1-credit GPH 744 Integrative Learning Experience.

Materials

Learning Objectives and Outcomes

Public Health Competencies

PC 1.  Synthesize and incorporate scientific evidence into professional writing

PC 2.  Search databases and critically analyze peer-reviewed literature

FC 4.  Interpret results of data analysis for public health research, policy or practice

FC 11. Select methods to evaluate public health programs

FC 13. Propose strategies to identify stakeholders and build coalitions and partnerships for influencing public health outcomes

Assignments

Class Discussions (8) – Weeks 1, 2, 3, 5, 8, 10, 14, 16

Identifying Key Stakeholders

Utilizing a worksheet from the self-study guide as a template, identify stakeholders in the three broad categories: those affected, those involved in operations, and those who will use the evaluation results. Then identify key stakeholders needed to improve credibility, implementation, advocacy, or funding/authorization decisions.

Literature Review

 Conduct a literature review focusing on sources that are related to the health program you are addressing and evaluations that are relevant to your topic.

Summary of Stakeholder Interviews

Complete stakeholder meetings, and summarize what you learned using the worksheet from the CDC Program Evaluation Guide. Summarize in 1 – 2 paragraphs your plans to continue engaging these and other stakeholders throughout the evaluation process.

Logic Model

Provide a logic model for your program that visually depicts the inputs, activities, outputs, outcomes, and impact. Use arrows to demonstrate the relationships between specific elements.

Evaluation Questions

Submit evaluation questions, including one or two overarching questions and several subquestions. Evaluation questions should be clear and precise so that they help focus your methods and analysis. They should also be realistic (answerable), relevant to key stakeholders, and open-ended (not yes/no questions).

Methods and Data Collection

Describe your proposed methods. Be sure to include the following information: your research design, how the chosen design will answer the evaluation questions, a description of any existing sources of data that you plan to use, a description of what additional data will need to be collected, a definition of the limitations of the selected methods.

Data Instruments or Samples

Submit the instruments you are using to collect data and/or a sample of the existing data that you have collected

Data Collection Logistics

Complete the Data Collection Logistics worksheet. 

IRB Application

Submit an application for IRB exemption.

Final Report

Submit a clean draft of your final report, including an abstract, executive summary, background, methods, results, and discussion.
Your report should be 4000 words plus appendices.

 

Grading Policy

Your grade in this course will be determined by the following criteria:

Grade Breakdown

Class Discussions (8 X 3 Points each)24
Week 1: Identifying Key Stakeholders5
Week 2: Literature Review10
Week 3: Summary of Stakeholder Interviews5
Week 3: Logic Model 5
Week 4: Evaluation Questions4
Week 5: Methods and Data Collection10
Week 6: Data Instruments or Samples 10
Week 7: IRB Application6
Week 15: Final Report15
Total100

Grade Scale

Grade Points Grade Point Average (GPA)
A 94 – 100% 4.00
A- 90 – 93% 3.75
B+ 87 – 89% 3.50
B 84 – 86% 3.00
B- 80 – 83% 2.75
C+ 77 – 79% 2.50
C 74 – 76% 2.00
C- 70 – 73% 1.75
D 64 – 69% 1.00
F 00 – 63% 0.00

Schedule

Course Schedule:

Each week opens on Wednesday at 12:01 AM Eastern Time. Each week closes on Wednesday at 11:59 PM ET, with the exception of Week 16, which ends on Sunday. 

  • Week 1: Aug 26 – Sep 2
  • Week 2: Sep 2 – Sep 9
  • Week 3: Sep 9 – Sep 16
  • Week 4: Sep 16 – Sep 23
  • Week 5: Sep 23 – Sep 30
  • Week 6: Sep 30 – Oct 7
  • Week 7: Oct 7 – Oct 14
  • Week 8: Oct 14 – Oct 21
  • Week 9: Oct 21 – Oct 28
  • Week 10: Oct 28 – Nov 4
  • Week 11: Nov 4 – Nov 11
  • Week 12: Nov 11 – Nov 18
  • Week 13: Nov 18 – Nov 25
  • Week 14: Nov 25 – Dec 2
  • Week 15: Dec 2 – Dec 9
  • Week 16: Dec 9 – Dec 13

The assignment/discussion descriptions mentioned below are summaries. Please make sure to review the full assignment prompts in Blackboard. There may be additional readings/videos that are not mentioned in this weekly summary, make sure to carefully review the modules in Blackboard.

Week 1: Course Introduction and Stakeholder Engagement

Outcomes:

  • Describe the steps and standards of the CDC evaluation framework

  • Identify key stakeholders for the selected program evaluation

  • Create a plan for continuous stakeholder engagement

  • Review the IRB review process and what will be required before data collection can begin

Lecture:
  • Engaging Stakeholders
Readings and Multimedia:

  • CDC Self-Study Guide Step 1

  • Rossi, P., & Wright, J. (1984). Evaluation Research: An Assessment. Annual Review of Sociology, 10, 331-352.

Discussion:
  • Introductions and Stakeholders
Assignment: 
  • Identifying Key Stakeholders

Week 2: Literature Review

Outcomes:

  • Identify relevant literature that describes the public health problem

  • Identify relevant literature about similar program evaluations

  • Synthesize peer-reviewed literature to provide relevant context and background for readers of the final report

Lecture:
  • Conducting a Comprehensive Literature Review
Readings and Multimedia:
  • Overview of Cochrane Reviews
Discussion:
  • Meeting with a Stakeholder
Assignment:
  • Literature Review

Week 3: Logic Model

Outcomes:

  • Develop a logic model that clearly describes a program’s inputs, activities, outputs, and short and long-term outcomes.

  • Use arrows in a logic model to demonstrate the relationship between all the program elements and outcomes.

  • Explain the importance and limitation of process (implementation) evaluation.

Lecture:
  • Logic Models
Readings and Multimedia:
Required:

  • CDC Program Evaluation Guide, Step 2

  • Issel and Wells, Chapter 9, Implementation Evaluation

Recommended:

  • Learning to Love Your Logic Model (a CDC webinar)

  • Kellogg Guide to Logic Models

Discussion: 
  • Questions
Assignments:

  • Summary of Stakeholder Interviews

  • Logic Model

Week 4: Evaluation Questions

Outcomes: 

  • Write evaluation questions that focus the evaluation and inform decisions related to methods and analysis.

  • Use peer-reviewed literature and stakeholder input to inform decisions about the focus of the evaluation.

  • Explain the benefits and limitations of outcome (effect) evaluations.

Lecture:
  • Evaluation Questions
Readings and Multimedia:

  • CDC Evaluation Guide, Step 3

  • Issel and Wells, Chapter 11, Planning the Intervention Effect Evaluations

Assignment:
  • Evaluation Questions

Week 5: Methods and Data Sources

Outcomes:

  • Select appropriate methods to answer different types of evaluation questions

  • Identify data sources that may be useful to program evaluation

  • Determine what additional data is needed to supplement existing data sources

Lecture:
  • Methods and Data Sources
Readings and Multimedia:
  • CDC Evaluation Guide, Step 4
Discussion:
  • Methods and Data Sources
Assignment:
  • Methods and Data Collection

Week 6: Data Sources and/or Data Collection Tool Creation

Outcomes:

  • Create effective tools for collecting quantitative and/or qualitative data

  • Assess the quality of secondary data

  • Develop a plan for analysis of quantitative and/or qualitative data

  • Create a plan for data collection and analysis and display the plan using a Gantt chart

Lecture:
  • Key Informant Interviews
Readings and Multimedia:

  • The Survey Checklist (Manifesto)

  • Conducting Focus Groups by Community Tool Box 

Activity:
  • Sign up for Midpoint Check-in
Assignment: 
  • Data Instruments or Samples

Week 7: Logistics, Protocols, and IRB

Outcomes:

  • Establish protocols for data collection and/or use of existing data

  • Interpret IRB rules and processes

  • Submit an application for IRB review

Lecture:
  • Practical Considerations for Data Collection
Readings and Multimedia:
  • University of New England Policies, Procedures and Guidance on Research with Human Subjects.
Discussion:
  • IRB Process
Assignments:

  • Data Collection Logistics

  • IRB Application

Week 8: Data Collection – Keys to Success

Outcomes:

  • Examine and apply best practices for data collection, storage, and management

Lecture:
  • Data Collection
Activities:

  • There are no assignments to submit this week, but you should continue collecting data.

  • Meet with your instructor via phone or web conference. Reflection questions will be provided in advance for this midpoint check-in.

Week 9: Data Collection – Staying on Track

Outcomes:

  • Examine and apply best practices for data collection, storage, and management

Activities:
  • This is a workweek. Continue to work on your data collection.

Week 10: Data Collection – Troubleshooting Challenges

Outcomes:

  • Examine and apply best practices for data collection, storage, and management

  • Describe common obstacles and possible solutions for data collection.

Discussion:
  • Data Collection Challenges
Activities:
  • Continue to work on your data collection.

Week 11: Organizing and Cleaning Data

Outcomes:

  • Examine and apply best practices for data collection, storage, and management.

  • Clean raw data to prepare for analysis.

  • Maintain data according to appropriate security standards.

Lecture:
  • Cleaning Data
Activities:

  • There are no assignments to submit this week, but you should continue collecting data.

Week 12: Qualitative Data Analysis

Outcomes:

  • Conduct basic analysis of qualitative data 

  • Respond to common criticisms of the use of qualitative data in program evaluation
Lecture:
  • Qualitative Data Analysis
Readings and Multimedia:
  • Issel and Wells, Chapter 15, Qualitative Data Analysis
Activities:

  • There are no assignments to submit this week, but you should continue working on your data analysis.

Week 13: Quantitative Data Analysis

Outcomes:
  • Conduct basic quantitative analysis of evaluation data.
Activities:

  • There are no assignments to submit this week, but you should continue working on your data analysis.

Week 14: Data Visualization

Outcomes:

  • Understand the benefits of clear, engaging data visualization

  • Present evaluation results using a variety of data visualization techniques

Readings and Multimedia:
  • Chart Chooser by Evalu-ate
  • Do’s and Don’ts for Data Visualization by Evalu-ate
Discussion:
  • Data Visualization
Activities:
  • There are no assignments to submit this week, but you should continue working on your data analysis. Consider also whether visual presentation of your data might be an effective communication tool.

Week 15: Synthesizing Your Results

Outcomes:

  • Prepare a final draft of a report of a program evaluation

Assignment:
  • Final Report

Week 16: Reflections for Future Revisions

Outcomes:

  • Reflect on the program evaluation process

Discussion:

  • Reflection

Student Resources

Online Student Support

Your Student Support Specialist is a resource for you. Please don't hesitate to contact them for assistance, including, but not limited to course planning, current problems or issues in a course, technology concerns, or personal emergencies.

Questions? Visit the Student Support Public Health page

UNE Libraries:

UNE Student Academic Success Center

UNE's Student Academic Success Center (SASC) offers a range of free online services to support your academic achievement. Writing support, ESOL support, study strategy and learning style consultations, as well as downloadable resources, are available to all matriculating students. The SASC also offers tutoring for GPH 712 Epidemiology, GPH 716 Biostatistics, GPH 717 Applied Epidemiology, GPH 718 Biostatistics II, and GPH 719 Research Methods. To make an appointment for any of these services, go to une.tutortrac.com. For more information and to view and download writing and studying resources, please visit:

Information Technology Services (ITS)

  • ITS Contact: Toll Free Help Desk 24 hours/7 days per week at 1-877-518-4673

Accommodations

Any student who would like to request, or ask any questions regarding, academic adjustments or accommodations must contact the Student Access Center at (207) 221-4438 or pcstudentaccess@une.edu. Student Access Center staff will evaluate the student's documentation and determine eligibility of accommodation(s) through the Student Access Center registration procedure.

Online Peer Support

Togetherall is a 24/7 communication and emotional support platform monitored by trained clinicians. It’s a safe place online to get things off your chest, have conversations, express yourself creatively, and learn how to manage your mental health. If sharing isn’t your thing, Togetherall has other tools and courses to help you look after yourself with plenty of resources to explore. Whether you’re struggling to cope, feeling low, or just need a place to talk, Togetherall can help you explore your feelings in a safe supportive environment. You can join Togetherall using your UNE email address.

Information Technology Services (ITS)

Students should notify their Student Support Specialist and instructor in the event of a problem relating to a course. This notification should occur promptly and proactively to support timely resolution.

ITS Contact: Toll-Free Help Desk 24 hours/7 days per week at 1-877-518-4673.

Career Ready Program

The College of Professional Studies supports its online students and alumni in their career journey!

The Career Ready Program provides tools and resources to help students explore and hone in on their career goals, search for jobs, create and improve professional documents, build professional network, learn interview skills, grow as a professional, and more. Come back often, at any time, as you move through your journey from career readiness as a student to career growth, satisfaction, and success as alumni.

Policies

AMA Writing Style Statement

The American Medical Association Manual (AMA) of Style, 11th edition is the required writing format for this course. Additional support for academic writing and AMA format is provided throughout the coursework as well as at the UNE Portal for Online Students.

Online resources: AMA Style Guide

Turnitin Originality Check and Plagiarism Detection Tool

The College of Professional Studies uses Turnitin to help deter plagiarism and to foster the proper attribution of sources. Turnitin provides comparative reports for submitted assignments that reflect similarities in other written works. This can include, but is not limited to, previously submitted assignments, internet articles, research journals, and academic databases.

Make sure to cite your sources appropriately as well as use your own words in synthesizing information from published literature. Webinars and workshops, included early in your coursework, will help guide best practices in APA citation and academic writing.

You can learn more about Turnitin in the guide on how to navigate your Similarity Report.

Technology Requirements

Please review the technical requirements for UNE Online Graduate Programs: Technical Requirements

Course Evaluation Policy

Course surveys are one of the most important tools that University of New England uses for evaluating the quality of your education, and for providing meaningful feedback to instructors on their teaching. In order to assure that the feedback is both comprehensive and precise, we need to receive it from each student for each course. Evaluation access is distributed via UNE email at the beginning of the last week of the course.

Late Policy

Students are responsible for submitting work by the date indicated in Brightspace.

Quizzes and Tests: Quizzes and tests must be completed by the due date. They will not be accepted after the due date.

Assignments: Unless otherwise specified, assignments will be accepted up to 3 days late; however, there is a 10% grade reduction (from the total points) for the late submission. After three days the assignment will not be accepted.

Discussion posts: If the initial post is submitted late, but still within the discussion board week, there will be a 10% grade reduction from the total discussion grade (e.g., a 3 point discussion will be reduced by 0.3 points). Any posts submitted after the end of the Discussion Board week will not be graded.

Please make every effort ahead of time to contact your instructor and your student support specialist if you are not able to meet an assignment deadline. Arrangements for extenuating circumstances may be considered by faculty.

Student Handbook Online - Policies and Procedures

The policies contained within this document apply to all students in the College of Graduate and Professional Studies. It is each student's responsibility to know the contents of this handbook.

Student Handbook

UNE Course Withdrawal

Please contact your student support specialist if you are considering dropping or withdrawing from a course. The last day to drop for 100% tuition refund is the 2nd day of the course. Financial Aid charges may still apply. Students using Financial Aid should contact the Financial Aid Office prior to withdrawing from a course.

Academic Integrity

The University of New England values academic integrity in all aspects of the educational experience. Academic dishonesty in any form undermines this standard and devalues the original contributions of others. It is the responsibility of all members of the University community to actively uphold the integrity of the academy; failure to act, for any reason, is not acceptable. For information about plagiarism and academic misconduct, please visit https://www.une.edu/studentlife/plagiarism.

Academic dishonesty includes, but is not limited to the following:

  1. Cheating, copying, or the offering or receiving of unauthorized assistance or information.
  2. Fabrication or falsification of data, results, or sources for papers or reports.
  3. Action which destroys or alters the work of another student.
  4. Multiple submissions of the same paper or report for assignments in more than one course without permission of each instructor.
  5. Plagiarism, the appropriation of records, research, materials, ideas, or the language of other persons or writers and the submission of them as one's own.

Charges of academic dishonesty will be reviewed by the Program Director. Penalties for students found responsible for violations may depend upon the seriousness and circumstances of the violation, the degree of premeditation involved, and/or the student’s previous record of violations.  Appeal of a decision may be made to the Dean whose decision will be final.  Student appeals will take place through the grievance process outlined in the student handbook.

Attendance Policy

8 week: Students taking online graduate courses through the College of Professional Studies will be administratively dropped for non-participation if a graded assignment/discussion post is not submitted before Sunday at 11:59 pm ET of the first week of the term. Reinstatement is at the purview of the Dean's Office.

16 week: Students taking online graduate courses through the College of Professional Studies will be administratively dropped for non-participation if a graded assignment/discussion post is not submitted before Friday at 11:59 pm ET of the second week of the term. Reinstatement is at the purview of the Dean's Office.

Student Handbook Online - Policies and Procedures

The policies contained within this document apply to all students in the College of Professional Studies. It is each student's responsibility to know the contents of this handbook.

UNE Online Student Handbook

UNE Course Withdrawal

Please contact your student support specialist if you are considering dropping or withdrawing from a course. The last day to drop for 100% tuition refund is the 2nd day of the course. Financial Aid charges may still apply. Students using Financial Aid should contact the Financial Aid Office prior to withdrawing from a course.

Academic Integrity

The University of New England values academic integrity in all aspects of the educational experience. Academic dishonesty in any form undermines this standard and devalues the original contributions of others. It is the responsibility of all members of the University community to actively uphold the integrity of the academy; failure to act, for any reason, is not acceptable. For information about plagiarism and academic misconduct, please visit UNE Plagiarism Policies.

Academic dishonesty includes, but is not limited to the following:

  1. Cheating, copying, or the offering or receiving of unauthorized assistance or information.
  2. Fabrication or falsification of data, results, or sources for papers or reports.
  3. Action which destroys or alters the work of another student.
  4. Multiple submissions of the same paper or report for assignments in more than one course without permission of each instructor.
  5. Plagiarism, the appropriation of records, research, materials, ideas, or the language of other persons or writers and the submission of them as one's own.

Charges of academic dishonesty will be reviewed by the Program Director. Penalties for students found responsible for violations may depend upon the seriousness and circumstances of the violation, the degree of premeditation involved, and/or the student’s previous record of violations. Appeal of a decision may be made to the Dean whose decision will be final. Student appeals will take place through the grievance process outlined in the student handbook.